Keys in VI Don't Work as Expected

After installing Ubuntu, you may find that using vi to edit any file may not work well.
Hitting i may not display "INSERT" at the bottom of the screen, hitting the arrow keys may only cause A, B, C, or D to be typed on the screen (instead of actually moving the cursor), the Backspace key may not work, etc.

The easiest way to fix this is to make sure that you have vim installed (not just vi).

sudo apt-get install vim

If you don't want to install vim, you can make vi act more like you're expecting.
Run this command to configure it:

echo "set nocompatible" > ~/.vimrc && echo "set backspace=2" >> ~/.vimrc