Get the final generated html source using c# or vb.net

Question

using VB.net or c#, How do I get the generated HTML source?

To get the html source of a page I can use this below but this wont get the generated source, it won't contain any of the html that was added dynamically by the javascript in the browser. How do I get the the final generated HTML source?

thanks

WebRequest req = WebRequest.Create("http://www.asp.net"); 
WebResponse res = req.GetResponse(); 
StreamReader sr = new StreamReader(res.GetResponseStream()); 
string html = sr.ReadToEnd();

if I try this below then it returns the document with out the JavaScript code injected

Public Class Form1

    Dim WB As WebBrowser = Nothing

    Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load

        WB = New WebBrowser()
        Me.Controls.Add(WB)
        AddHandler WB.DocumentCompleted, AddressOf WebBrowser1_DocumentCompleted


        WB.Navigate("mysite/Default.aspx")

    End Sub

    Private Sub WebBrowser1_DocumentCompleted(sender As Object, e As WebBrowserDocumentCompletedEventArgs)


        'Dim htmlcode As String = WebBrowser1.Document.Body.OuterHtml()
        Dim s As String = WB.DocumentText

    End Sub
End Class

HTML returned

<!DOCTYPE html>

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>

</head>
<body>
    <form id="form1" runat="server">
    <div id="center_text_panel">
    //test text  this text should be here
    </div>
    </form>
</body>
</html>

    <script type="text/javascript">

        document.getElementById("center_text_panel").innerText = "test text";


    </script>

Do you have an example? have you done it before as I tried that but could not get it to work? — Hello-World
– Hello-World, Commented Feb 13, 2013 at 6:25
Did you ever figure this out @Hello-World? I'm having the SAME issue and tried using the new WebView2 control from MS, but still no love! — Mike Hawk
– Mike Hawk, Commented Jul 4, 2020 at 0:59

Brian Webster · Accepted Answer · 2013-02-13 06:54:05Z

2

You can use WebKit.NET

Look here for official tutorials

This can not only grab the source, but also process javascript through the pageload event.

webKitBrowser1.Navigate(MyURL)

Then, handle the DocumentCompleted event, and:

private documentContent = webKitBrowser1.DocumentText

Edit - This might be the better open source WebKit option: http://code.google.com/p/open-webkit-sharp/

edited Feb 13, 2013 at 6:54

answered Feb 13, 2013 at 6:24

Brian Webster

31k51 gold badges158 silver badges227 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Alexei Levenkov Over a year ago

+1, also I think "need" is too strong here - built in WebBrowser or even PhantomJS can be used to do the same.

Mohanad S. Hamed Over a year ago

The webkit is giving me the source HTML and not the final generated.

KF2 · Accepted Answer · 2013-02-13 08:28:30Z

1

Just put a webbrowser control to your form and you flowing code:

 webBrowser1.Navigate("YourLink");

     private void webBrowser1_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
        {
           string htmlcode= webBrowser1.Document.Body.InnerHtml;//Or Each Filed Or element..//WebBrowser.DocumentText
        }

Edited

for getting also html code that generated dynamically by java script code you have two way:

run flowing code after webBrowser1_DocumentCompleted Event

 StringBuilder htmlcode = new StringBuilder();
            foreach (HtmlElement item in webBrowser1.Document.All)
            {
                htmlcode.Append( item.InnerHtml);
            }

write a javascript code for returning document.documentElement.innerHTML and using InvolkeScript Function To Return Result:

   var htmlcode = webBrowser1.Document.InvokeScript("javascriptcode");

edited Feb 13, 2013 at 8:28

answered Feb 13, 2013 at 6:28

KF2

10.2k8 gold badges46 silver badges77 bronze badges

4 Comments

Hello-World Over a year ago

thanks that s great but it returns the source not the generated souce

KF2 Over a year ago

For Getting dynamically Generated Code You must Using extra JavaScript Code if You if add more details of something that you want to do it will show solution(or adding some extra code)

Hello-World Over a year ago

Hi - WebBrowser.DocumentText needs to return the generated html code with the javascript injected into it. Do you think that this might need to be done as async. thanks for your help.

nguyenhoai890 Over a year ago

The string returned is just HTML before javascript runs.

Bugs · Accepted Answer · 2017-06-29 15:37:40Z

0

You can use this code:

webBrowser1.Document.Body.OuterHtml

edited Jun 29, 2017 at 15:37

Bugs

4,4799 gold badges34 silver badges41 bronze badges

answered Jun 29, 2017 at 15:36

ngochoaitn

374 bronze badges

Collectives™ on Stack Overflow

Get the final generated html source using c# or vb.net

3 Answers 3

2 Comments

4 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related