If the API changes, you should recompile even if you don't need to make any changes in your source code. If the API hasn't changed, you don't need to recompile.
The reason for the "even if you don't need to make any changes" is that some source-compatible changes may not be binary compatible. For instance, suppose you are currently calling:
public void foo(String x)
and in a later version this is changed to:
public void foo(Object x)
Obviously your code will still compile, but the method it resolves the call to will change.
This is a bit of an edge case, of course. Basically, so long as you know when the API changes, you should be okay.