When I enter a shell command via vim -- e.g., :!rake routes -- I'll get some output and then:
Press ENTER or type command to continue
If I press anything at that point, I am always taken back to the main vim UI. How do I stay in that shell mode for a bit longer, specifically I can scroll back to see all the output of the command?
For terminal vim scrolling is provided by terminal emulator (or terminal multiplexer like tmux/screen if you use it). I.e. if your terminal scrolls when using <S-PageUp> it must also scroll with Vim’s !. Same for mouse wheel (it works for me even if vim was configured to handle mouse on its own).
If have terminal vim and you need to view output after you pressed <CR> then you can use <C-z>/:susp. Both these capabilities will be spoiled with incorrect configuration of either vim or terminal (terminal multiplexer) (i.e. terminal configured not to support alternate screens (which is normally issue for terminal multiplexers rather then terminal emulators: screen requires altscreen on in .screenrc) or vim setting telling vim what to output to the terminal on startup/resume and shutdown/suspend was spoiled).
If you have GUI vim your options are very limited. It is better to follow link provided by @glts or @EricAndres advice in this case as I do not use GUI vim.
:r! [shell command] will read the output from the command into the buffer. So if you open a new buffer then use that command, you can search and scroll through it all you like. See Vimcasts episode 57 for more information.
:!rake routes \| more? That might work.