While that would work and OpenCV has template matching functions, it would probably be harder than necessary. Good results would require that the lighting is more or less unchanged between all images, and that the camera is fixed and no projective distorsions occur.
Instead, I would do something like this:
In the image, locate the die. The difficulty here will vary with regard to how the die looks and the background. If you have a white die on a lpain black (or some other color) background then finding the die will be easy.
When the die has been located, find the eyes. This can be done by simply finding all black blobs.
If necessary, make sure that the found eyes form a coherent pattern. E.g. if the side up is four you expect to find the eyes as the corners in a square, not on a straight line.
Count valid eyes. There's your side up.
This outline is quite vague as there are lots of ways of performing each step. I do however believe that everything you need is available in OpenCV.
Good luck!