Nokogiri Ruby - Remove the <!DOCTYPE ... > from the output html

Asked 8/9, 2014 at 17:43 Answered 12/10, 2018 at 10:44

I am parsing an html file using nokogiri and modifying it and then outputting it to a file like this:

htext= File.open(inputOpts.html_file).read
h_doc = Nokogiri::HTML(htext)
File.open(outputfile, 'w+')  do |file|
  file.write(h_doc)
end

The output file contains the first line as:

 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">

I do not want this because I am embedding the html in a different file and this tag is causing issues.

Question is how do I remove this from h_doc.

Punjab answered 8/9, 2014 at 17:43 Comment(0)

Depending on what you are trying to do, you could parse your HTML as a DocumentFragment:

h_doc = Nokogiri::HTML::DocumentFragment.parse(htext)

When calling to_s or to_html on a fragment the doctype line will be omitted, as will the <html> and <body> tags that Nokogiri adds if they aren’t already present.

Samons answered 8/9, 2014 at 21:11 Comment(0)

It depends on your needs. If all you need is the body then

h_doc.at_xpath("//body") #this will just pull the data from the <body></body> tags

If you need to collect the <head> too and just avoid the <DOCTYPE> then

#this will capture everything between the <head></head> and <body></body> tags
h_doc.xpath("//head") + h_doc.xpath("//body")

So something like this

h_doc = Nokogiri::HTML(open(input_opts.html_file))
File.open(outputfile,'w+') do |file|
  #for just <body>
  file << h_doc.at_xpath("//body").to_s
  #for <head> and <body>
  file << (h_doc.xpath("//head") + h_doc.xpath("//body")).to_s
end

Notice for body I used #at_xpath as this will return a Nokogiri::Element but when combining them I used #xpath becuase this will return a Nokogiri::XML::NodeSet. No need to worry this part is just for the combination and the html will come out the same e.g. h_doc.at_xpath("//head").to_s == h_doc.xpath("//head").to_s #=> true

Prisoner answered 8/9, 2014 at 17:57 Comment(2)

if he wants to embed it, this is probably the right solution – Nymphomania 8/9, 2014 at 18:9

I managed to solve this same challenge with help from this answer. I'll more detail in a separate answer. – Ophidian 12/10, 2018 at 10:29

You can just ignore the first line when reading the input file:

htext= File.readlines(inputOpts.html_file)[1..-1].join
h_doc = Nokogiri::HTML(htext)
File.open(outputfile, 'w+')  do |file|
  file.write(h_doc)
end

Margemargeaux answered 8/9, 2014 at 17:52 Comment(0)

I managed to work around this with both an HTML::Document and an HTML::DocumentFragment.

For background, I'm using Nokogiri to parse and modify "templates", "partials" and/or "components" of HTML. This means that the files I encounter are not valid HTML documents. They are, instead, pieces of an HTML document that gets put together by the framework I'm using.
For reference, HTML::Document adds the <!DOCTYPE> declaration and also wraps your document into <html> and <body> entities if they are not already present in your document. Similarly, HTML::DocumentFragment will wrap your fragment with <p> entity.

Rather than spending too much time digging into the Nokogiri library code to understand where these additional entities were being, I decided to accept to this opinionated implementation and work around it.

Solution

Here's how I write out my modified HTML:

html_str = doc.xpath("//body").children.to_html(encoding: 'UTF-8')
File.open(_filename, 'w') {|f| f.write(html_str)}

Final Word

This seems harder than it should be. I even tried using the SaveOptions setting save_with: Nokogiri::XML::Node::SaveOptions::NO_DECLARATION to no avail.

In any case, while this solution is a bit kludgey for my liking, it works.

Ophidian answered 12/10, 2018 at 10:44 Comment(0)

Solution

Final Word

Recommended topics

Hot tags