Removing hidden file end characters (BOM) in visual studio

Question

I'm using a batch file to concatenate all my css files

copy /b reset.css+action-plan.css+buttons.css+behaviours.css+footer.css+forms.css+header.css+home.css+layout.css+lightbox.css+print.css+questionnaire.css+typography.css+you-told-us.css main.css

I've done this numerous times before on various projects, but this project uses .NET and the files are all being edited in visual studio.

The problem I have is that there are some mysterious hidden characters being added at the end of each file, which, when concatenated, causes the resulting css to be invalid.

 126  BLOCKQUOTE, Q   Lexical error at line 119, column 1.
 Encountered: "?" (63), after : "" ??? /**** left column ****/ 

All the individual CSS files validate and the errors are only thrown in the combined file at teh points were the individual files join.

Ocko

This is more of a work around than a solution, but you may prefer this outcome. There are a couple of online utilities that will "minify" your css documents to improve performance. examples include (without endorsement) are minifycss.com or csscompressor.com. Both of these are 'cut and paste' operations that compact your css. If you paste each of your css file's content into one of these tools and minify you will end up with a single file (your goal) that also happens to be compressed and will improve your site's performance (not by a lot...but every bit counts.) – Randellrandene

If you look in main.css, what's on line 119? – Ardell

It turns out the error is a problem with the W3C validator - uploading the file rather than copying and pasting removes the errors. I'd love to integrate proper minification into the site's build but... well, if I made all the decisions it'd be part of the build process from day 1, but I don't make all the decisions – Ocko

Spoke too soon - it's now started throwing the same errors again, but with uFEFF replacing the ??? – Ocko

Answer 1

The problem is because of the byte order mark (BOM) in your files. The byte order mark is for unicode files to tell the processor the order of the bytes. You can read more about it here:

http://en.wikipedia.org/wiki/Byte_order_mark

The problem is that Visual Studio is adding those marks to your css file and when you combine them by concatenation, BOMs are ending up in the middle of the text, screwing things up.

When you go to the Save As dialog, you can expand the Save button to see the 'Save with Encoding' option. This will prompt you for a different encoding, and I think one of the Unicode options will leave out the BOM (somewhere in the list is UTF-8 without signature).

I don't know how to set Visual Studio to use a specific encoding by default.

To skirt the issues I created a program to concatenate files that would respect the BOM. I use that rather than copy, or the unix cat.

Tedie

Hmm, tried to edit my pose and it created a new one. Sorry for that. – Tedie

Answer 2

You probably want to use YUICompressor .NET, instead of scripting this yourself.

Skip

@BC: see the "Sample MsBuild output" on the YUI compressor homepage it combines several css files to one, what am I missing? – Skip

Even if you create a custom MSBuild task using that code (not provided by Yahoo), it still compresses each file individually using the actual jar executable. – Savona

Oops you're talking about the port of YUI Compressor. – Savona

Answer 3

I wrote a nifty little command line program that combines all files (i.e. css, etc) in a directory and removes the BOM (byte order mark) for you. It's about 5 lines of code and uses cssmin.js to handle the minification for you. There's also an example of how it looks in a Visual Studio post build event. Check it o

Funches