I've observed that message before but I don't think it was the same cause - the "did not execute" is a generic RPC error which can be caused by many things.
In our particular case, it was because we were hammering WMI too hard and not cleaning up our objects. But it appears to be a different cause in your case.
I know you said you didn't want to shorten the parameter but that may be the only way forward for you. I'm having a hard time visualizing the circumstances under which you need to send 6M across an RPC call. Perhaps if you explain the reasons behind it, we can help further.
Other possibilities based on comments to date:
1/ Segmentation.
Have the RPC calls limit the amount of data that travels across the wire. This could be done by segmenting the message at the source and reconstructing it at the destination, such as having three parameters (you could have the server dole out message identifiers in another RPC call or find some other way to ensure no two clients have the same ID):
- a message identifier (for tying message segments together).
- a last flag (to begin the reconstruction process).
- a limited size segment (e.g., 1M).
2/ Compression.
Since XML is text, it's ripe for compression. The 7zip libraries are the best I've seen in terms of size reduction. Whether this would be fast enough is another matter.
3/ Possible fix by changing the registry.
Check out the HKLM/Software/Microsoft/Rpc registry area for the RpcMaxSize key. There are a couple of sites I've googled that suggest setting this to -1 will remove size limitations (globally, so be careful).
4/ Possible fix while registering your interface.
You can apparently achieve the same effect as (3) on a specific interface with RpcServerRegisterIf2()
.