If the question is, "should a script tag be allowed to remove itself?" I would think so. After all, a script tag can cause the browser to navigate to another page, in which case the entire page (including such script tag) is removed from memory.
The script is removing the DIV from the document DOM tree, and just so happens to remove the script declaration itself (after it has been loaded). So I think it should be allowable as a feature.
Also, the script tag will exist until it is loaded and the Javascript is interpreted and executed. At execution, the DIV (and SCRIPT) tag will be removed from the DOM tree and exist only as objects that will be garbage collected.
EDIT
I think the major point here is that a SCRIPT tag only creates the Javascript objects and places them in memory in the Javascript runtime engine (which is completely different from the DOM). Once the interpreter has finished reading the SCRIPT tag, it's no longer needed and can be discarded (removed from the DOM).
If the interpreter was instructed (by the JS code) to create references to OTHER objects in the DOM (e.g. as the handler for a button click), then the Javascript function will still be accessible. Any functions called from that event handler will ALSO be accessible. In this way, you can "walk" the call stack from the button->handler->other functions/variables.
An example where self removal can be handy is, for example, some long polling techniques that would certainly make the DOM large after a few minutes.
JavaScript frameworks add and remove script elements as part of their daily routine.
E.g. getScript() in jQuery places a script tag in the document head and removes it right after evaluation.
They do this to keep the DOM clean - else the tags would just build up unnecessarily. They are no longer needed after evaluation.
The only drawback I see with this is debugging - for example, Firefox with Firebug seems to be lost as to the whereabouts of such scripts and cannot find the source lines. (As of this writing.)