Use XPath to find all JavaScript type script elements

Question

I am trying to find all <script> elements in a series of HTML-documents by using html-agility-pack (plus LINQ and XPath). The documents have script-elements placed in header, and a Google Analytics in footer. First I am trying to target the header scripts and remove them. My Notepad++ shows me that I have 719 script-elements present, but my console appliaction only finds 55 of them.

I need som help to target them properly, so I can remove them from the document.

Source document (structure of head),

<!doctype html system "html.dtd">
<html>
<head>
<link rel="stylesheet" href="../IRstyle.css" type="text/css">
<title>Non-hierarchic document clustering</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="keywords" content="">
<meta name="VW96.objecttype" content="Document">

<script language="JavaScript" type="text/JavaScript">
//Javascript-code goes here
</script>
</head>
<body>    
<!--Body contents goes here-->

<!-- in footer -->
<script src="http://www.google-analytics.com/urchin.js" type="text/javascript">
</script>
<script type="text/javascript">
_uacct = "UA-67XXXX-X";
urchinTracker();
</script>
</body>
</html>

So far I've tried targeting the 'Language' type with JavaScript, but only get a few hits when parsing html/head. My method takes filenames from a list. For now, method prints out number of scripts collected in list, this will change to 'Scripts.Remove();' once I get the search string right.

private static void FindTagsToRemove(IEnumerable<string> files)
{
    var doc = new HtmlDocument();
    List<string> scripts = new List<string>();
    List<string> errors = new List<string>();
    try
    {
        foreach (var file in files)
        {
            doc.Load(@file);
            var head = doc.DocumentNode.SelectSingleNode("html/head");
            var nodes = new List<HtmlNode>();
            bool isScript = false;

            foreach (var node in head.ChildNodes.ToList())
            {
                if (node.NodeType == HtmlNodeType.Element && node.Name.Contains("script"))
                {
                    isScript = !isScript;
                    scripts.Add(node.OuterHtml);
                    Console.WriteLine(node.OuterHtml);
                }
                else if (isScript)
                {
                    nodes.Add(node);
                    node.Remove();
                }
            }
        }
        int nr_scripts = scripts.Count();
        Console.WriteLine("Number of scripts in collection: {0}", nr_scripts);
    }
    catch (Exception Ex)
    {
        Console.WriteLine(Ex.Message);
    }
}

If someone have a better way of targeting the JavaScripts in the head-node, it will be much appreciated. ANY help is appreciated! :)

goTo-devNull · Accepted Answer · 2018-02-15 21:17:35Z

2

If you only need <script> element nodes, use descendant-or-self (//). Sample HTML:

var html =
@"<!doctype html system 'html.dtd'>
<html>
<head>
<link rel='stylesheet' href='../IRstyle.css' type='text/css'>
<title>Non-hierarchic document clustering</title>
<meta http-equiv='Content-Type' content='text/html; charset=iso-8859-1'>
<meta name='keywords' content=''>
<meta name='VW96.objecttype' content='Document'>
<script language='JavaScript' type='text/JavaScript'>
//Javascript-code goes here
</script>
</head>
<body>    
<!--Body contents goes here-->

<!-- in footer -->
<script src='http://www.google-analytics.com/urchin.js' type='text/javascript'>
</script>
<script type='text/javascript'>
_uacct = 'UA-67XXXX-X';
urchinTracker();
</script>
</body>
</html>";

Parse sample:

var document = new HtmlDocument();
document.LoadHtml(html);
// target only <script> in <head>
// var scriptTags = document.DocumentNode.SelectNodes("//head/script");
var scriptTags = document.DocumentNode.SelectNodes("//script");

foreach (var script in scriptTags) script.Remove();    

document.Save(OUTPUT);

Output:

<!doctype html system 'html.dtd'>
<html>
<head>
<link rel='stylesheet' href='../IRstyle.css' type='text/css'>
<title>Non-hierarchic document clustering</title>
<meta http-equiv='Content-Type' content='text/html; charset=iso-8859-1'>
<meta name='keywords' content=''>
<meta name='VW96.objecttype' content='Document'>

</head>
<body>    
<!--Body contents goes here-->

<!-- in footer -->


</body>
</html>

answered Feb 15, 2018 at 21:17

goTo-devNull

9,3923 gold badges46 silver badges57 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Christer Johansson Over a year ago

Thanks for this! Much appreciated! :) Now I only need to find some proper error handling to make my program continue running on error. Some of the files can't be read and throws a NullReference. Another questions perhaps.. :D

Christer Johansson Over a year ago

Never mind, I managed the error handling through putting the try/catch inside my foreach loop. Now I only need to find out why html agility can't read the html-files.

Collectives™ on Stack Overflow

Use XPath to find all JavaScript type script elements

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related