Powershell 2.0 generates nulls between characters

Question

With powershell 2.0:

write-output "abcd" >> mytext.txt

writes: a nul b nul c nul d nul

od -c shows the nul as a true binary zero, \0 , or: a \0 b \0 c \0 d \0 (and \r \0 \n \0)

I am trying to generate some SQL, so I don't think this will do. Any ideas of what's going on and how to use write-output to just get the specified chars?

Thanks!

Answer 1

This is because write-output defaults to UTF-16 text encoding, which is 2 bytes per character. When you are dealing with text that fits into the ASCII codepage range, the 2nd byte of each character will be zero.

This is controlled by the $OutputEncoding global variable, so you could set that to ASCII.

Another option is to use the cmdlet Out-File, which has an explicit encoding parameter. I would suggest you use this instead of output redirection, because that saves you from changing your environment globally (by setting the global preference variable $OutputEncoding)

Using Out-File, and setting encoding to be ASCII, your example would look like this:

"abcd" | out-file "mytext.txt" -Encoding ASCII

Do be aware that not all characters are representable in ASCII, and you should determine whether this is an appropiate encoding for your purpose. Personally I would typically go for UTF-8, since it is ASCII equivalent when characters fall in the ASCII range from 0-127, but also handles international characters. Obligatory link about text encoding.

Answer 2

Powershell works in 16 bit unicode by default, and however you're reading the file is likely in an 8 bit format. You could interpret the sql in an application that can read UTF16, or, because >> is syntactic sugar for the out-file cmdlet, you can do the following instead:

write-output "abcd" | out-file -path mytext.txt -Encoding "UTF8" -Append