----- Original Message ----- THE PATHWAY TO SIGHT.
by Ken O'Sullivan.In 1979 Pat Fletcher lost her sight in a horrific industrial accident. Today she has emerged as a trailblazer in the use of an amazing technology, which, in a very real sense, has allowed her to see again by using her ears. Not long ago I
had the chance to talk with Pat, who lives in Buffalo, N.Y.Her journey started unexpectedly in the late '90s, and had more to do with her wardrobe than anything else. "I wanted to discover a means where I could use my scanner to detect color ... So I went on the Internet and found the Voice. At that time I saw that it was a color detector; I wasn't interested in any of the other stuff, nor did I even understand the other capabilities of the program. So
I downloaded it, put my shirt on the scanner, and, voila!"Developed by a Dutch software engineer, Dr. Peter Meijer, the "other stuff" Pat mentions is a powerful method of converting visual images into sound. Soon Pat was exploring sample soundscapes from Meijer's web site. Although the results were intriguing enough to keep her going, she had yet to find the key to unlock
her understanding, until she took matters into her own hands. She scanned aplastic cup, and used that picture with the voice program. "I turned the program
on where I could use the arrow keys. Then while arrowing up and down ondifferent spots on the soundscape I ran my finger along the cup. I applied what
I was hearing to what I was touching, and then it made sense."Pat's principle breakthrough came when she determined to take the Voice beyond the four walls of her study. She searched the Internet for a small video camera that she could use with a notebook computer. "When I did that, that unchained
me. I took a tiny webcam and put it on the end of a ball cap, and put the notebook in a backpack, and put on a headset. I really wasn't expectinganything. I thought 'who knows what's going to happen here.' I walked out of my room, and looked down the hall, and nearly went down to my knees because I could
actually see the blinds that were hanging on the side of the wall. Now, mind you, I did know there were blinds hanging in the hall, but you forget aboutevery little aspect of your environment when you don't see it. It just brought tears to my eyes, and from that point I decided I'm going to use this stuff. I'm going to work with it and wear it. That's when I discovered that the more you work with the program, the more your body could understand and relate to what
you were seeing."To be clear, the Voice software does not produce a "voice" at all. It is neither human nor computerized speech. Those close to it say learning to use the Voice is akin to learning a new language: The meaning is there, but it doesn't jump
right out at you.While the technology may be complex, the concept is fairly simple. Here's how it works: A small video camera is used to capture an image. The camera sends that visual information to a computer. The Voice software then converts the picture into a dynamic, multi-layered stereo sound with a rather science-fiction-like quality. Pitch is used to indicate physical elevation: a higher pitch for up and a lower pitch for down. Volume is used to represent brightness: the brighter the image, the higher the relative volume. Thus, when perceiving one's front yard
through earphones, a tall oak tree on the right will create a distinctly different impression than a sunlit driveway on the left. At present, thecomputer-generated soundscape refreshes about once every second. Of course no description here can substitute for actually hearing these sounds (more on that
at the end of this article).Pat explains, "You can develop a sense of the soundscapes within two to three
weeks. Within three months or so you should start seeing the flashes of your environment where you'd be able to identify things just by looking at them."In truth, this is the essence of the seeing with sound concept. When Pat speaks of "seeing," that's exactly what she means. She's emphatic: "It is sight," she
says. "I know what sight looks like. I remember it."As controversial as this assertion may be, as incredible as it may sound, this claim isn't merely subjective or anecdotal; it's real. And it is only one facet of a growing body of knowledge known as neuroplasticity, which shows that the brain is far more malleable, far more adaptable than science has long assumed.
About seven years ago, Pat (along with another individual, blind from birth) took part in a Harvard study. By using Magnetic Resonance Imaging, a leadingresearcher in the field, Dr. Alvaro Pascual-Leone, found that when sound is used in this way it stimulates not the auditory centers of the brain but the areas of
the brain that convert sensory information into vision.Pat says, "They actually took pictures of our brains as we listened to different
soundscapes, or touched an object, or heard a sound like a whistle. What itproved, after they went through all that imaging, is that my brain lit up in the
same way that a sighted person's brain lights up when they see." To illustrate the power of the Voice system in her day-to-day life, Pat described a recent car trip. She was able to see passing cars, lines on theroad, even differences among those lines. From her place in the back seat, she could perceive the driver's hands on the steering wheel, and the appearance of
the dashboard. She explains, "After a point the soundscapes are no longer soundscapes. Youdon't see or hear soundscapes; you see vision ... actual vision. I am never not amazed when I put on the gear. At any time I could have an experience of sight. Of course I have continuous sight in areas that I know, but there are times when
I'm looking at stuff and I don't know what it is."Pat likes the fact that the Voice system is easy to set aside. Incredibly, this is something she refers to as "going blind." "If I don't want to see, if I just
want to watch TV, I can take it off. I can lay it down. Now, it takes me aminute to become blind again. I can't just take it off and ... boom. It takes my
brain a few minutes." A gadgeteer by nature, Pat has spent years mixing and matching components tomake her gear ever more inconspicuous and comfortable to wear. She's an expert. Her contacts with the scientific community and the media have established her as
a kind of ambassador for the seeing with sound concept. She is intelligent,articulate, and enthusiastic. Her optimism is palpable, and with good reason.
Seeing with sound now seems to be on the threshold of a substantial new development, one that could move it into the mainstream. A pair of computerscientists, working in Pasadena, Calif., have formed a company called MetaModal, LLC. Using the Voice as a starting point, their ultimate goal is to integrate software and hardware into a single, commercially viable unit, a sophisticated pair of sunglasses, and to train individuals in its use. Working under a grant
from the National Science Foundation, Dr. Enrico Di Bernardo and Dr. Luis Gonçalves put together a prototype. They've tapped Pat's knowledge andexperience, and are working with orientation and mobility specialists to train
Voice users and learn what works best. Di Bernardo says, "We really want to make a difference to the end user, to empower them for their independent living."Di Bernardo, who speaks in a vibrant Italian accent, says a device of the kind
they're envisioning is only made possible by technological advances over the last 10 years, especially in regard to cell phones. "We are piggybacking onevery single part of it, from the cameras to the batteries to the electronics.
If cell phones didn't push the boundaries of miniaturization, we wouldn't be able to do this." MetaModal is a term the Pasadena team borrowed from Dr. Pascual-Leone. "Hecoined this term," Di Bernardo says. "The idea being that the way we perceive the world doesn't happen in any specific mode, or sense, but it really happens
at the junction of all senses." I thank Pat Fletcher and her friend Carol Bartlett for exposing me to thismarvelous topic. I thoroughly enjoyed my conversation with Pat and Carol, who
were very generous with their time on a Sunday afternoon.Before our talk ended, I asked Pat to indulge me in a silly exercise. Without saying why, I asked her to complete this sentence: "Using the Voice system is
..." Her response was firm and immediate: "the pathway to sight." For Pat Fletcher, that seems to say it all.Those interested in learning more can visit www.seeingwithsound.com. This site is the place to go to download the Voice software at no charge. It's also a rich
source of information. Following the "news" link, for example, leads to a well-maintained collection of annotated links to press coverage and events.Also, the Canadian Broadcasting Company produced a delightful 22-minute radio
segment on the Voice system. It features Pat and what she's hearing. It alsoincludes more about Dr. Alvaro Pascual-Leone, and the Harvard brain study. To
hear it for yourself, visit www.cbc.ca/quirks/archives/04-05/apr02.html. Andwww.metamodal.com is the site of the Pasadena startup, which includes a number
of videos of the system in use. Source URL: http://acb.org/downloads/bf012012.txt ---------- To send a message to the list, send any message to blindza@xxxxxxxxxxxxx ---------- To unsubscribe from this list, send a message to blindza-request@xxxxxxxxxxxxx with 'unsubscribe' in the subject line --- The 'homepage' for this list is at http://www.blindza.co.za
