Google map not firing nested listener

Question

Using Google Maps v3, I'm trying to do the following:

1. Create a new bounds object and fit the map to this.
2. Wait 400ms before panning to one of the LatLngs that forms this bounds object.
3. Wait 400ms before zooming to a new level on this new centre.

In doing this I have no problem with the first 2 points above. However, I can't get the listener to fire after point 2. This listener (idle) is nested inside the listener for bounds_changed.

The js I'm using is

latlng_new = new google.maps.LatLng(lat, lng);
var bounds = new google.maps.LatLngBounds;
bounds.extend(map.getCenter());
bounds.extend(latlng_new);
google.maps.event.addListener(map, "bounds_changed", function() {
            google.maps.event.clearListeners(map, "bounds_changed");
            google.maps.event.addListener(map, "idle", function() { alert('finished panning'); });
            setTimeout("map.panTo(latlng_new);", 400);
        });
map.fitBounds(bounds);

The listener in question which never seems to fire is on line 7 of the above script.

google.maps.event.addListener(map, 'idle', function() { alert('finished panning'); });

Obviously if the event was firing as expected then I would replace the alert() with map.setZoom(), but because the alert never appears then I know this listener is never firing.

So my question is: Is it even possible to have nested listeners on the Google Maps API v3?