Static methods can only access static class variables. They cannot access instance variables because static methods are not tied to any specific instance of the class.
You don't need to create a new instance of a class in order to access a static member or variable (as long as it's public). In your example, you can call methodB like this:
String bString = MyClass.methodB();
As you can see, no new instance had to be created.
But because methodA is not static, you must create an instance of the class in order to call it, like you did in your example:
MyClass myClass = new MyClass();
String aString = myClass.methodA();