[accessibleimage] Re: Mental synthesis of images

Hi,

Well, it sounded like Bauhaus to me...

Did "early blind people" have to use tactile clues to read cave paintings or
did Cro-Magnons provide auditory descriptions <laugh>?

In my personal research (using the incredibly unscientific sample size of
one - namely me) I find that 2 or 3D audio images are easier to interpret
when there is a sense of motion.  Your curve, once your rules are explained,
was easy as it "moved" from left to right.  The squares, after another half
hour of playing with both the fast and slow samples, remain to small and too
"still" to distinguish much more than their locations.  In Shades of Doom,
Greenwood keeps most everything "moving" (objects like walls remain still
but one perceives them by moving toward, away or beside them so motion is
involved) and most everything, after playing for a couple of hours, is
pretty easy to distinguish from everything else.  In SOD, Greenwood plays up
to 32 simultaneous audio tracks and, a lot can be understood when one first
starts playing and virtually everything can be understood after the short
initiation period.  

John's AGC seemed fairly obvious to me from the first time I heard a sample.
I think the "motion" of the curves and my previous knowledge of how such
things look was very helpful.  I think the prior knowledge of appearance
also gives SOD a "familiar" feeling to it (no, I haven't actually been
attacked by mutants but I can imagine it from memories of "Dawn of the Dead"
and other low budget horror flicks I watched in my misspent youth).  I don't
know how people who have been blind since birth or early childhood perceive
AGC or the GMA games but would be interested in hearing about them.

Recently, a friend doing work in this area sent me some DVDA recordings of
3D wire frame images.  They contained still points for intersections in a
set of "brighter" audio pixels and the lines that formed the frame were
constantly in motion.  Using the 5.1 3D effects, I could picture these
solids both through SurroundSound headphones and on my Bose home
entertainment system sitting in my living room.  I think the movement of the
sound through three dimensional space helped me understand it more readily.
I don't know why.

Finally, Car Talk had a puzzler that asked which was the only non zero value
that would come up with the same value when added to itself or multiplied by
itself.  When Ray was giving the answer, he did so using a graph example.
He said that the first equation was y = 2x and the second was y = x^2.  I
wasn't paying very close attention to the program but, as soon as he stated
the equations and started explaining the graphs, I could immediately see a
line and parabola intersecting where both x and y equaled 2.  So, in this
case, I am certain the prior knowledge of how such graphs would look were
instrumental in my ability to draw them in my mind.

I find your work incredibly interesting.  Is there somehow I can help beta
test or have something more substantial to play around with?  I am really
good at giving feedback.

Have fun,
cdh  

 

-----Original Message-----
From: accessibleimage-bounce@xxxxxxxxxxxxx
[mailto:accessibleimage-bounce@xxxxxxxxxxxxx] On Behalf Of Zorro
Sent: Wednesday, January 11, 2006 2:56 PM
To: accessibleimage@xxxxxxxxxxxxx
Subject: [accessibleimage] Re: Mental synthesis of images

Hi Chris,

 > Was this a Kandinsky or some other Bauhaus painting?

Ha ha, no it was an original "Meijer", I'm afraid. <smile>
Dutch painter Pieter Mondrian might have liked my squares,
but probably not my curve. <grin>

And yes, the squares are here so small that they appear
not too different from bright specks of any shape, but
one characteristic on an upright filled square (or frontal
face of a cube) is that its sound starts and stops very
suddenly under the left to right scans, with a constant
sounding noise burst in between. A filled circle (or
sphere) starts and stops more gradually, so an early
blind person too can easily make the distinction despite
the centuries' old so-called Molyneux problem as posed by
philosopher John Locke. For those unfamiliar with this:
it was about the question if an early blind person could
distinguish a cube from a sphere in case eyesight were
suddenly restored. However, in our case the shapes need
to be large enough to notice the differences. Also,
distinguishing circles from triangles is harder because
the untrained human hearing system lacks the equivalent of
corner detectors. No one knows yet if this can and will
change through extensive experience, or to what extent age
(and decreasing brain plasticity) plays a role.

 > Have you tried a 3D audio system?

Yes, but these are here a mixed blessing, because the
direction dependent spectral filtering of a good 3D
audio system now interferes with the semantics of using
pitch for height and brightness for loudness. In other
words, it can add some ambiguity. Still, registered
users of The vOICe Learning Edition software will find
an option for an experimental sound rendering labelled
"HESTER", which attempts to add some 3D audio effect
without defeating the key image to sound mapping
principles. This is an area of continued research.

Best wishes,

Peter


Seeing with Sound - The vOICe
http://www.seeingwithsound.com/winvoice.htm

Chris Hofstader wrote:
> Was this a Kandinsky or some other Bauhaus painting?
> 
> Knowing the description of the image, I can certainly synthesize a curve
> from the sounds.  The "beeps" don't say "squares" to me but rather that
> something in varying degrees of brightness are in those positions.  They
> could just as easily be triangles or circles.  Of course, I haven't spent
> more than 5 minutes with the sounds so I can't speak from an informed
> position.
> 
> Have you tried a 3D audio system?  SurroundSound headphones are pretty
cheap
> and Microsoft has done some pretty incredible things with the latest
Direct
> Sound technology.  The new X-Box sounds tremendous.
> 
> Enjoy,
> cdh

Other related posts: