It is a simple optical illusion, though a cool one.
"Negative afterimages are caused when the eye's photoreceptors, primarily those known as cone cells, adapt from the overstimulation and lose sensitivity. Normally the eye deals with this problem by rapidly moving small amounts (see: microsaccade), the motion later being "filtered out" so it is not noticeable. However if the color image is large enough that the small movements are not enough to change the color under one area of the retina, those cones will eventually tire or adapt and stop responding. The rod cells can also be affected by this.
When the eyes are then diverted to a blank space, the adapted photoreceptors send out a weak signal and those colors remain muted. However, the surrounding cones that were not being excited by that color are still "fresh", and send out a strong signal. The signal is exactly the same as if looking at the opposite color, which is how the brain interprets it."
I have a great fondness for illusions that our brains provide us. Some of the auditory ones are staggering. Search for the Shepard-Risset Glissando on Youtube and you'll see what I mean.