[blindza] The Blind Climber Who "Sees" Through His Tongue

From: "Jacob Kruger" <jacobk@xxxxxxxxxxx>
To: "BlindZA" <blindza@xxxxxxxxxxxxx>
Date: Tue, 24 Jun 2008 11:58:21 +0200
---original message---
The Blind Climber Who "Sees" Through His Tongue.

Erik Weihenmayer's BrainPort translates images into electrical signals.

June 23, 2008, by Buddy Levy; photography by Beth Wald.

A wiry man in his late 30s hangs near the top of a 40-foot wall, the
toes of his climbing shoes barely gripping the sheer face. He clings
with his left hand and pauses, swinging his right arm to the side,
loosening the muscles. Then he raises his head and lunges, thrusting his
right hand above him and catching a rock hold with his fingertips. With
a few more moves he arrives at the top. Under normal circumstances this
extreme rock climber would be worth watching. But what makes his effort
even more remarkable is that he happens to be blind. Born with
retinoschisis, a rare disease akin to macular degeneration, Erik
Weihenmayer was sightless by age 13. Even so, he continued to pursue his
dream of mountaineering, and he succeeded: In 2001 he became the first -
and to date the only - blind climber to summit Mount Everest. Today he
is climbing with the aid of a tool that allows him to "see" in a new way
- with his tongue.

In normal vision, light hitting the retina provokes electrical impulses
that the brain translates into images. What the tool, called the
BrainPort, does is convert light into electrical impulses that stimulate
the tongue instead of the retina. With more tactile nerve endings than
any other part of the body except the lips, the tongue can discriminate
two points spaced less than a millimeter apart. That degree of
resolution is far beyond what the current BrainPort array, with only 611
electrodes, provides. But tests have shown the BrainPort delivers enough
information for users like Erik to navigate with.

"Climbing!" Erik calls out to his partner. He places the device in his
mouth, raises his head, and "surveys" the wall before him. Electrical
impulses from the BrainPort become to him "a tactile image that I'm
interpreting in space." Erik reaches for a handheld slider that controls
the zoom level and field of view of the BrainPort so that he can "see"
one or two rock climbing holds on the wall some 2 or 3 feet above him.
Then he slowly and deliberately raises his right hand, reaches up, grabs
hold, and begins to pull himself up the wall. His feet secure, he hangs
for a moment, adjusts the zoom again, then tentatively puts his hand out
and above him again, this time missing the hold by half a foot. Judging
the distance of an object in space is particularly difficult even with
the device, Erik says. To help him figure out how far away something is,
he sometimes waves his hand - an object of known size and distance - in
front of himself first to get a sense of scale via the BrainPort array.
The feedback helps him assess the distance of his next rock climbing hold.

Testing the BrainPort, Erik says, involves "learning to climb in a new,
different way. I'm learning another language in the same way someone
would be learning Braille or French for the first time. I'm figuring out
how to map it spatially." The challenges are significant. The BrainPort
provides information in two dimensions, like a line drawing on a piece
of paper, but the user's brain must learn to translate this information
into things like perspective, dimension, and location in space. When
Erik first used the BrainPort, the images in his brain appeared as
unidentifiable shapes and lines, but over time, through practice, his
brain adapted, eventually translating the tactile sensations into
recognizable patterns and symbols.

Erik has experimented with the tool at his home in Golden, Colorado. He
could find and grab a coffee cup placed on the center island in his
kitchen; play soccer with his daughter Emma in their large downstairs
playroom, kicking the ball back and forth over some 15 feet; play
rock-paper-scissors; and compete in a game of tic-tac-toe by identifying
x's and o's on paper mounted on an easel. Later, the two moved out to
the snowy front yard to play hide-and-seek. Eventually Erik could detect
Emma's movement as well as locate a snowman. Outdoor environments remain
particularly challenging, though.

"Outside, shadows create lines, which appear on the tongue as objects in
space," Erik says. "For sighted people, the eyes and the brain quickly
make adjustments, but the BrainPort doesn't always keep pace with the
shadows, and one has to learn to interpret these situations and contexts."

Paul Bach-y-Rita, the cocreator of the system Weihenmayer is testing,
challenged conventional wisdom when he proposed that "we see with the
brain, not with the eyes." He insisted that the brain was flexible
enough to learn to interpret data coming in on a different kind of
channel. Bach-y-Rita died in 2006, but Wicab - the company he cofounded
in Middleton, Wisconsin, in 1998 - continues to work on devices based on
the idea that one sense can substitute for another. By 2004, Wicab had
gained support for its research from the National Eye Institute, one of
the federal agencies of the National Institutes of Health.

Wicab intends to create a BrainPort vision device for consumers, and
over time the company has made it much less cumbersome. The prototype
Erik first worked with in 2003 was bulky, comprising a webcam with a
fixed field of view and a shoe box full of hardware that drove a basic
144-electrode array, all connected and run through a laptop computer.
The current BrainPort prototype includes hardware contained in a
belt-mounted 6-by-6-by-4-inch box, three sensitive cameras with a
90-degree field of view, and user-operated zoom and contrast control.

Still, despite significant improvements in size and portability, the
current prototype is not cosmetically appealing or portable enough for
ordinary use. Wicab plans eventually to integrate microcameras and
microprocessors into eyeglass frames; a dental appliance will
incorporate the tongue display, allowing the user to touch the display
intentionally with the tongue, but also to pull it away and talk.
Low-power radio frequency transceivers will link the cameras,
processors, and tongue arrays; later on, wireless technology will
connect these elements.

Other applications for the BrainPort include assisting military divers
in challenging low-light and no-light search-and-rescue operations, says
Rich Hogle, Wicab director of product development. "Imagine that you're
floating in a dense brown fog in daytime and visibility is such
that.there's no up, down, left, or right, and now your job is to swim
200 yards in a straight, level line, turn 180 degrees to the left, and
swim back in a line parallel to the first. If you're off by a few
degrees in either direction, or too shallow or too deep, you might miss
what you are looking for. Now imagine doing the same thing, in the same
brown fog, at night."

In research funded by the Defense Advanced Research Projects Agency,
Wicab researchers used BrainPort technology to help military divers swim
in a straight line in low-light and no-light conditions. "For this
project we used a digital compass to provide directional information
that was then displayed on the tongue as a tactile pattern," Hogle says.
"While swimming in the target direction, the diver's eyes and ears were
able to remain focused on the operational tasks at hand."

Erik Weihenmayer hopes that the device will eventually benefit other
members of the blind community, especially those who have never
experienced vision. "I wonder how great one's learning curve would be if
you used this device two hours a day, every day. All of these blobs and
circles and lines you would be able to learn and interpret. If you gave
it to a blind child, his spatial understanding of the world would be
through his tongue. It's a compelling thought."

The BrainPort provides a window into the vast complexity of our senses,
Weihenmayer says. "I'm reminded how amazing the brain really is, with
its incredible ability to differentiate all this information. It's a
trip to be able to 'look' at a hold and reach for it. For me, it's
beyond science. It's a kind of vision."

Source URL:
http://discovermagazine.com/2008/jul/23-the-blind-climber-who-sees-through-his-tongue


----------
To send a message to the list, send any message to blindza@xxxxxxxxxxxxx
----------
To unsubscribe from this list, send a message to blindza-request@xxxxxxxxxxxxx 
with 'unsubscribe' in the subject line
---
The 'homepage' for this list is at http://www.blindza.co.za
[blindza] The Blind Climber Who "Sees" Through His Tongue

Other related posts:
