People have correctly said that the session contains a number of heavy weight objects. With enough users on your system, if you try to keep them all in the finite amount of memory that the server has, eventually you will crash the server when that memory max's out.
I worked on a project one time where a production code update had a memory leak. It was a J2EE project (yes J2EE not Java EE). When a user logged in to check their invoice at this phone company the user session was not released properly from memory (I can't remember the cause but that was definitely the issue). This bug mimics what you are asking about doing on purpose.
The server kept crashing. So we put a profiler on it. We would watch the memory use go up through the day until it topped out and shorty after the app server crashed. We added memory and increased the VM memory setting. I told them it was a memory leak but because I wasn't a $200.00/hour "server expert" people were unwilling to believe it because the people who were there still believed that the garbage collector was all powerful instead of being just very good.
Two days later (it affected the "view your invoice" system, not the main business system, i.e. it didn't have the same workload or memory requirements even though it had plenty of hardware memory in the servers), they hired a couple of $200.00 per hour consultants who after a day told them the app had the aforementioned memory leak. It was fixed and all was good... minus the consultants fees.
In any case here is the take away from this: if you don't end user sessions when users log out or close their browser (session time out), you run a real risk of maxing out your memory and crashing your servers. Especially if your site or app has any significant number of users. As mentioned by others, lightweight tokens/cookies are best.