1)create a table with rights, ie delete, update, etc
2)create a three way pivot table on the rights table, whatever table you want row level access for and whatever table contains the unit of access rights (either group or user).
3) check for a relationship in the pivot table before you allow the operation to proceed.
your rights table could look like:
ID RIGHT
1 DELETE
2 UPDATE
the table that you want row level access control for could look like (say a blog for example):
ID TITLE CONTENT
1 blog entry 1 This is a blog entry
2 blog entry 2 This is another blog entry
and your user table could be:
ID NAME
1 Bob
2 Alice
Then the pivot table would be like
ID USER_ID RIGHT_ID BLOG_ID
1 1 2 1
2 2 1 1
3 2 2 1
4 2 1 2
5 2 2 2
This means that Bob can only update blog entry 1 but Alice can update or delete either blog entry
EDIT: If you want a right to come from the user or the group then you need two pivot tables for each table; one for users and one for groups. You will also have to query the database to check for user level rights and group level rights before you allow or disallow an operation
EDIT2: This is more complicated than David's solution but doesn't require you to compose permission_classes ahead of time: you can mix and match whatever group level and user level permissions you want which is what it seems like you want to do.