I have an idea of how they would do this but it seems very hacky way about going about it.
The problem I see from this is that youtube needs to grab gmail or google cookies.
The way I would do this would be to open another frame from the gmail domain that reads the gmail cookies and forwards this information to a youtube url. This youtube url would then authenticate based on the forwarded information. Like I said before it seems very hacky way about doing this and probably not the way they are doing it.
If anyone knows how they do it or has any other ways to go about doing this I would like to hear it.