[accessibleimage] Re: Mental synthesis of images

Hi Chris,

> Was this a Kandinsky or some other Bauhaus painting?

Ha ha, no it was an original "Meijer", I'm afraid. <smile>
Dutch painter Pieter Mondrian might have liked my squares,
but probably not my curve. <grin>

And yes, the squares are here so small that they appear
not too different from bright specks of any shape, but
one characteristic on an upright filled square (or frontal
face of a cube) is that its sound starts and stops very
suddenly under the left to right scans, with a constant
sounding noise burst in between. A filled circle (or
sphere) starts and stops more gradually, so an early
blind person too can easily make the distinction despite
the centuries' old so-called Molyneux problem as posed by
philosopher John Locke. For those unfamiliar with this:
it was about the question if an early blind person could
distinguish a cube from a sphere in case eyesight were
suddenly restored. However, in our case the shapes need
to be large enough to notice the differences. Also,
distinguishing circles from triangles is harder because
the untrained human hearing system lacks the equivalent of
corner detectors. No one knows yet if this can and will
change through extensive experience, or to what extent age
(and decreasing brain plasticity) plays a role.

> Have you tried a 3D audio system?

Yes, but these are here a mixed blessing, because the
direction dependent spectral filtering of a good 3D
audio system now interferes with the semantics of using
pitch for height and brightness for loudness. In other
words, it can add some ambiguity. Still, registered
users of The vOICe Learning Edition software will find
an option for an experimental sound rendering labelled
"HESTER", which attempts to add some 3D audio effect
without defeating the key image to sound mapping
principles. This is an area of continued research.

Best wishes,

Peter


Seeing with Sound - The vOICe
http://www.seeingwithsound.com/winvoice.htm

Chris Hofstader wrote: 
Was this a Kandinsky or some other Bauhaus painting?

Knowing the description of the image, I can certainly synthesize a curve
from the sounds.  The "beeps" don't say "squares" to me but rather that
something in varying degrees of brightness are in those positions.  They
could just as easily be triangles or circles.  Of course, I haven't spent
more than 5 minutes with the sounds so I can't speak from an informed
position.

Have you tried a 3D audio system?  SurroundSound headphones are pretty cheap
and Microsoft has done some pretty incredible things with the latest Direct
Sound technology.  The new X-Box sounds tremendous.

Enjoy,
cdh

Other related posts: