Determining the edge normals in the SAT Separating Axis Theorem

Question

The SAT algorithm requires you to find the normal of each edge of each shape (essentially a vector perpendicular to the edge vector) to be used as the separating axes. This can be done very simply...

(x,y) => (-y,x)

OR

(x,y) => (y,-x)

Which should be used in the SAT algorithm? This is essentially a question of whether the left hand normal or the right hand normal should be used. Will it make a difference which is used? Should only the left or right hand normal be used? Should this change with different situations?

See http://www.codezealot.org/archives/55#sat-axes

Answer 1

It doesn't matter as long as you use the same convention for every face, because the normals are used to calculate projections, and comparing them:

  Projection p1 = shape1.project(axis);
  Projection p2 = shape2.project(axis);
  // do the projections overlap?
  if (!p1.overlap(p2)) {

The result of (!p1.overlap(p2)) will be the same with both formulas.

HTH

Answer 2

The normal should always point out from the edge. There are only two possible solutions, which you figured out. One of them is right and the other is wrong.

Which one is correct depends on two things:

• Whether your edges are clockwise or counter-clockwise.
• Your coordination system.