I don't have any specific literature to point to, but to help get you thinking about the various problems, here are a few thoughts.
You can think of the airline network as a huge graph, with cities at the nodes and flights as edges. There's an edge between two cities if there is a connecting flight. Now, you can ascribe various weights to these edges, such as the time of flight, the minimal ticket cost, the number of remaining seats, etc. and use graph algorithms like Dijkstra's or A* to optimize on your metric of choice. Since there are likely to be multiple flights connecting the same cities at different times/prices/etc., you'll likely have to iterate these algorithms to find compatible itineraries (e.g. layovers not too short/long) etc. But fundamentally, these are graph algorithms.
Pricing is a highly complex endeavour and whole branches of Operations Research are devoted to it. Unlike routing, pricing is essentially a game--the service provider (airline) is trying to maximize revenue globally (not just for single flights), and so they have to balance passenger loading vs. ticket costs. Airlines have incredibly complex pricing structure, where the cost for a given seat varies dramatically with time, and may be wildly different than the cost for the seat right next to it, particularly depending on if is sold to a business or pleasure traveler.
The big name in airline reservation systems used to be SABRE. I don't know if that's still true, but they do seem to still be a major player.
Good luck & happy reading.