See here http://stevenharman.net/blog/archive/2008/04/29/hacking-visual-studio-to-use-more-than-2gigabytes-of-memory.aspx
Remember to reapply this tweak after you install any Visual Studio updates.
I recently applied VS2008 SP1 and found that I hit out of memory exceptions again... the SP1 had refreshed the devenv.exe file!
mark.