This is a bit convoluted but may be more legitimate than a straight XSS solution:
You have no control over server A other than writing javascript to document A. But you are opening an iframe within document A, which suggests that you only have write-access to document A. This a bit confusing. Are you writing the js to document A or injecting it somehow?
Either way, here is what I dreamed up. It won't work if you have no access to the server which hosts the page which has the iframe.
The user hits submit on the form within the iframe. The form, once completed, most likely changes something on the server hosting that form. So you have an AJAX function on Document A which asks a server-side script to check if the form has been submitted yet. If it has, the script returns a "submitted" value to the AJAX function, which triggers another js function to close the iframe.
The above requires a few things:
The iframe needs to be on a page hosted on a server where you can write an additional server-side script (this avoids the cross-domain issue, since the AJAX is pointing to the same directory, in theory).
The server within the iframe must have some url that can be requested which will return some kind of confirmation that the form has been submitted.
The "check-for-submitted" script needs to know both the above-mentioned URL and what to look for upon loading said URL.
If you have all of the above, the ajax function calls the server-script, the server-script uses cURL to go the URL that reflects if the form is done, the server-script looks for the "form has been submitted" indicators, and, depending on what it finds, returns an answer of "not submitted" or "submitted" to the ajax function.
For example, maybe the form is for user registration. If your outer document knows what username will be entered into the form, the server-side script can go to http://example.org/username
and if it comes up with "user not found" you know the form has yet to be submitted.
Anything that goes beyond what is possible in the above example is probably outside of what is safe and secure anyway. While it would be very convenient to have the iframe close automatically when the user has submitted it, consider the possibility that I have sent you an email saying your bank account needs looking at. The email has a link to a page I have made which has an iframe of your bank's site set to fill the entire viewable part of my page. You log in as normal, because you are very trusting. If I had access to the fact that you hit submit on the page, that would imply I also had access to what you submitted or at the very least the URL that the iframe redirected to (which could have a session ID in or all sorts of other data the bank shouldn't include in a URL).
I don't mean to sound preachy at all. You should just consider that in order to know about one event, you often are given access to other data that you ought not have.
I think a slightly less elegant solution to your problem would be to have a link above the iframe that says "Finished" or "Close" that kills the iframe when the user is done with the form. This would not only close the iframe when the user has submitted the form, but also give them a chance to to say "oops! I don't want to fill out this form anyway. Nevermind!" Right now with your desired automatic solution, there is no way to get rid of the iframe unless the user hits submit.