Is UML state machine diagram can be used to show the screen navigation?
UML is for modeling processes for an Object Oriented paradigm. State Machine actually deals with an object's state and has got nothing to do with UI navigation.
Your best bet here might be to represent that in Use-Case diagram.
Sure you can create a UML model of the UI as a state machine, for example:
Screens are states
State transitions i.e. screen changes occur on specific inputs or other triggers
Did you actually have some more specific question?
You can use state machines but it is also possible to use sequence diagrams for that. If you don't need to stick to the pure UML, there are plenty of approaches devoted to the specification of web systems that include the concept of web navigation models (with elements as pages, links,...). These approaches either use UML profiles or a completely different modeling language (as WebML)