Think of sight kind of like hearing and sonar, but confined to a cone of detection that is almost 180 degrees wide, and maybe about 45 degrees up, and 45 degrees down. This field of vision is incredibly detailed at the center of focus, which is a spot around 6 degrees wide. Detail drops away rather quickly, the further away from your center of vision it gets. The visual information isn't missing, it's just not coming in at the same fidelity, like background conversation when you're listening to one person. You can get more information by shifting your gaze/focus as needed.
Imagine reading braille, but the pad of your finger is just a patch of skin in the middle of a large field of skin, like on your back. In order to read, you'd have to stick the entire page to your back, and then move that center of focus all over the page. You'd feel the braille all over your back and recognize it as such, but you'd only be reading that one spot. That's roughly what shifting your gaze is like.
Light is like loudness, in that too much of it hurts, and wipes out detail. When there is very little light, detail gets wiped out as well.
Think of the visible light spectrum like the notes of the musical scale, where the note F always sounds like an F, regardless of the octave. The colors do seem related to each other in a similar manner as musical notes; orange does look like it fits between red and yellow, the way that C sharp/D flat fits between C and D.