Combining files using COPY command and getting odd characters

Question

I have a bunch of small PowerShell functions, each in their own file and I am trying to combine them into a larger file.

Using the low-tech approach of the DOS copy command by doing: copy /Y /A functions\*.ps1 super.ps1 works fine however where two files are joined it is inserting: 

I'm guessing it's the line break characters showing up (difference of encoding), but how do I prevent these characters from showing up?

Answer 1

Those characters are unicode byte order marks which will be invisibly preceding the .ps1 content.

I'm not sure how you'd strip them off in plain DOS -- at this point I'd turn to a scripting language to do the work. You could write a PowerShell script to join them using the .net System.IO features.

Answer 2

How you tried /B for binary to stop OEM characterset translation

Who knows

Or open your scripts in notepad and save them as ANSI then try and join them with your normal copy command no /B required

Answer 3

Those characters are the byte order mark (BOM) as Steve Gilham said. The easiest thing you can do to remove the BOM is to open all the files in a text editor that supports utf-8, make a small edit and save again. Gearny is an ideal candidate for this. It supports utf-8, doesn't save the BOM and it's free. It's only for windows, but I'm assuming you have access to a windows box. I don't know any dos program capable of strip the BOM (in the DOS era utf-8 was only a project)

Once you stripped the BOM you can use the copy method but in binary mode (/b), otherwise the most possible outcome will be garbled text whenever a multibyte character appears.

Answer 4

Here is an interesting direction to take

http://msdn.microsoft.com/en-us/library/aa368046%28VS.85%29.aspx (can't get this sucker to link)

This has two solutions to copy your PowerShell scripts which appear to be Unicode to ANSI. One solution written in VB the other in PowerShell

Once in Ansi follow everyones recommendation