It's a very simple algorithm, actually (at least for reasonable numbers of users and groups).
Consider each user to be a set whose elements are the groups of which they are a member. To find the groups two users have in common, simply take the intersection of those two users' membership sets.
So, if Person A is in Group K, M, and N, and Person B is in K, N, and P, you would have the following sets:
A := {K, M, N}
B := {K, N, P}
intersect(A, B) = {K, N}
In Ruby, you can use the standard library class Set
to perform these calculations:
require 'set'
memberships_a = Set[:K, :M, :N]
memberships_b = Set[:K, :N, :P]
shared = memberships_a.intersection(memberships_b)
# you can also use the '&' operator as shorthand for 'intersection'
shared_2 = memberships_a & memberships_b