If a Java solution is OK for you (Java hosting is usually double the price of PHP hosting), than SubEthaMail might be exactly what you look for:
http://code.google.com/p/subetha/
It is a mailing list manager, but it also has the Web UI to let the users use it instead of using the mail - just like google groups.
Another advantage is that is using SubEthaSMTP as a mailing engine, and that's very efficient at handling huge amounts of mail because of it's asynchronous processing.
They are also "eating their own dog food", by using SubEthaMail itself to develop it (as opposed to e.g. Apache JAMES - this is just a mailing solution, no Web based UI).
See their user list here, or you can look at some screenshots to see that is quite advanced, and regarding usability I would say that it's even better than google groups.