What is the difference between an atom in the ground state and an atom in an excited state?

In terms of what you'd see, an atom in the ground state wouldn't emit any light because all the orbital electrons are in their lowest possible energies in the atom. 

A ground state atom has its orbiting electrons all in their lowest energy states consistent with, I think, the Pauli (nobel prize winner) exclusion principle and Hund's rules.An excited state atom has one or more electrons in higher energy states - say by thermal agitation (heating them). When these electrons "de excite" back to a lower energy level, they emit electromagnetic radiation in the form of photons. With the correct energy/frequency the photons are visible as a spectrum.