If you decompile an inner class (or watch it using debugger) you can see that there is generated code for accessing the instance of the outer class that was used to create them. The overhead for this is more memory for the additional pointer, more cpu for garbage collection because of additional pointer to test, and if you want to nit pick, longer compile time. Creating instances of non static inner classes is a bit more complicated because you need an instance of the outer class to create them.
Visibility of both static and non-static inner classes can be controlled. Usually they are private if their implementation is strongly connnected to internal details of the outer class, and the developer doesn't think the code can be reused. In this sense they are not better than private functions. Inner classes might be public in cases like Map.Entry, where the inner class is strongly connected to the interface exposed by the class, and the developer doesn't think that Map.Entry can be used without some kind of a Map. Both types have access to private members of the outer class and the outer class has access to private members of the inner class.
Instances of static and non-static inner classes are garbage collected like every other class. There is no special connection between the grabage collection of the outer class and the garbage collection of the inner class.
In the case of UI classes implementation like swing or android you will see static inner classes because they are treated like private function. These classes are not developed for reusability outside the outer class and are strongly connected to the internal implementation of the outer class. There is no reason to expose them and to make sure they can work in more cases than the specific context of the outer class requirements.