The password is for the PKCS12 file itself, not for the private key. You can specify a blank password by adding "-password pass:" like this:
$ openssl pkcs12 -password pass: -export -in myprivatecert.pem -nokeys -out mycert.p12
You will still be prompted by Windows for the password, but you can leave it empty, and the import will work fine.
If all you are importing on Windows is the certificate, without the key, they you can also use the DER format like this:
$ openssl x509 -in myprivatecert.pem -outform DER -out mycert.der
One benefit of this is that when you double-click this file on Windows, it recognizes the der extension, and you can view the certificate details just before importing. Also, there will be no password prompt.