Assuming you are at least on SQL 2005...
The relevant metadata is stored in sys.database_permissions for database securables and in sys.server_permissions for server level securables. You get the list of database principals (users and roles) from sys.database_principals, the server principals (logins an server roles) from sys.server_principals.
This will get you the list of explicit permissions, but you need to take into account also the implicit permissions, that are not declared. Certain groups have implicit permission. To complicate things further you also have to deal with Windows groups memberships, that are not declared inside any SQL view but are considered when doing access checks. Finaly, the access rules are quite complicated: a principal can have a privilege through an explicit GRANT, through membership to a group that is GRANTed the privilege, but any DENY trumps all the GRANTs and that has to be taken into account, except for securable ownership that trumps any DENY. The icing on the cake is sysadmin membership that trumps all privilege rules: sysadmin has all privileges by definition.
You can verify any privilege on any securable for most principals by impersonating the principal via EXECUTE AS and checking the output of fn_my_permissions on the desired securable.