Rare but Real: People Who Feel, Taste and Hear Color

"When Ingrid Carey says she feels colors, she does not mean she sees red, or feels blue, or is green with envy. She really does feel them.

She can also taste them, and hear them, and smell them."

http://www.livescience.com/health/050222_synesthesia.html

I think the idea is summarized by this comment from the link:

"In this way of thinking, animals and humans are born with immature brains that are highly malleable. Connections between different sensory parts of the brain exists that later become pruned or blocked as an organism matures, Mondloch explained."

As a musician I am aware that for many people each note in a musical scale has its own feel, or colour. For some people even the absolute pitch or key has its own feel but for most it's relative pitch that's important.