The cost of creating the context is very low. However, using a new context means that you don't have any cached queries from previous contexts. You can work around this to some degree with view generation or CompiledQuery. See also Performance Considerations for Entity Framework Applications
On the other hand, keeping a context around for a long time means you are tracking increasing amounts of state information, which has a performance cost of its own.
In my opinion, however, the most significant cost of a context is code complication. Using multiple contexts tends to lead to confusing code. So I try to use one context per group of related operations, e.g. handling a single HTTP request.