Suppose I have a class in my package org.jake and it has a method with default access (no modifier). Then the method is visible inside the package only.
However, when someone receives the jar of my framework, what is to stop them from writing a new class, declaring its package as org.jake, and using my supposedly invisible method?
In other words, is there anything I can do to prevent this?
There is nothing you can do to prevent this. Even private members can be accessed via reflection. You should consider the access modifiers in java to be merely suggestive.
You could seal the package in your jar file. It's not bullet-proof though.
The main thing is not to rely on access modifiers etc from a security point of view to start with, really. If someone is running the code with unrestricted permissions, they're going to have access to all kinds of things. Access modifiers really just help to stop people from accidentally shooting themselves in the foot.
If someone is willing to put classes in your package to circumvent your encapsulation, they're clearly ignoring your best intentions - I say let 'em get on with it, but don't provide support for that scenario.
I'd say simply do not allow them to run code where it can call yours, i.e. in the same JVM. You could instead consider offering only a (web)service they can call externally. I'm not very up to date on the best ways to implement this though.
First off, this is the “DRM” scenario: ultimately, someone determined enough can defeat any protections you put in place by supplying a funky modified runtime or other such things. The reverse scenario – where the runtime is trusted but some of the packages are not – is tackled properly by Java through the use of suitable ClassLoader restrictions, but that can only work where there's something that can enforce the restrictions in a trusted fashion; that's why your scenario is basically doomed.
However, if we assume that the runtime itself is trustable then you could try, in your super-secret method, getting the stack trace of the currently executing stack (see stackoverflow.com/questions/1069066/… for how) and testing to see whether the caller of the current method is one that you trust to get access. A security manager would be even more suitable, but you can't trust the environment to have one of those installed that you like (it's much more clearly under the control of the attacker). Note that I have not tried the options in this paragraph!
The other alternative is to put your secrets on a service you control and only offer remote access to them. Or stop worrying about using technical mechanisms to deal with a problem that is fundamentally about business and legal issues (e.g., why are you dealing with people you can't trust?)