Coming from this question here, http://stackoverflow.com/questions/218057/httpcontextcurrentsession-is-null-when-routing-requests, I also noticed that when routing, all my sessions were null when I have custom IHttpHandlers access it. I used the answer from that question to fix it. Which is basically to add System.Web.SessionState.SessionStateModule to the system.webServer module.
From this page http://learn.iis.net/page.aspx/496/iis-url-rewriting-and-aspnet-routing/, it looks like it's suppose to be there.