It sounds to me like you need a data structure known as a multimap.
This is a collection like a hash but where keys can have multiple values. There isn't one as such in Java but it is easily constructed by using an ordinary collection with two levels: store a list in a map. The key is a unique token for the door and the list contains all the rooms to which the door connects. Usually this will be just two rooms, but perhaps the door has some level of magic to it.
Approximately the same thing is to have your own class Door, which contains anything you want plus a reference to both connecting rooms. This then only needs a normal map.
In either case, the Room class then just has door keys in it that can be looked up in the collection.
You certainly could put the topology of the rooms directly into the room objects, having door structures that directly reference other rooms, but I suspect that doors will have state of their own (open, closed, locked, ...) and in any case you have a chicken-and-egg problem of how to create all those links in the first place. Using a library collection solves all of these problems.