* * * Edit: Problem Solved - at least on my machine(s) * * *
I took a cue from the thread I found here and began examining what was happening with PowerBuilder when these errors occurred -- both during the import of the PBX and during runtime if you executed the sample application.
One person in the thread mentioned the issue was related to libeay32.dll and ssleay32.dll. This is not the first time OpenSSL has given me fits with these components -- my hard drive is littered with them in various application folders. These DLLs get loaded by PB when the PBWSCLIENT105.PBX extension is imported or by an app using the associated PBD at runtime. Sure enough, by using ProcMon, I was able to confirm that libeay32.dll and ssleay32.dll were being loaded from Windows\System32 as opposed to the version of the same DLLs located in the PB folder structure. Apparently, this is a problem. I renamed the two DLLs located in System32 to effectively remove them, fired up PB and the import of PBWSCLIENT105.PBX worked like a charm.
I have a similar issue with our PB 10.5 installation. Importing the PBX file gives me the error message:
---------- Import PB Extension:
Importing C:\Program Files\Sybase\Shared\PowerBuilder\pbwsclient105.pbx - Error: The file is not a valid dll or pbx file.
---------- Finished
I initially thought the issue was that the files were corrupt, etc. However, I installed PB 10.5 on a clean machine (a VM actually) along with the .Net 2.0 SDK and the import worked flawlessly; also, I was able to call a web service. Every developer in my group has the same problem on their own machines, but if they use a "clean" machine or VM with PB installed (and perhaps a few other apps), the import of the PBX file works without issue.
At this point I am trying to pin down what in the environment causes the problem, but haven't been able to yet. What's interesting, as you've noted, is that if you open the sample application, you can see the exact same imported libraries present; however, you get the error at runtime. All of this really points to something environmentally missing or incorrectly registered. So, if you have the ability to use a VM, I recommend that as a temporary work around.
One thing I'll add here: if you happen to use Vista or Windows 7 as your VM, the import of the library works fine, but I had issues generating the client-side proxy using the .Net web service engine. I finally discovered that in the background, the SDK tool WSDL.exe is being executed against the WSDL for the service in order to produce a .Net client-side proxy. Turns out that Vista/Win7 User Access Control was preventing WSDL.EXE from running. I ran PowerBuilder "as Administrator" and everything worked as expected in terms of generating the client-side proxy.
If you discover anything about why the PBX import doesn't work, I'd be interested in hearing about it!