----- Original Message -----
Wicab's wearable vision device nears U.S. market, thanks to Google.
By Jeff Engel, February 9th, 2015.
Paul Bach-y-Rita believed technology could help blind people to, in a way,
“see”
the world around them by substituting touch for sight. After 17 years and
nearly
$26 million in total funding, the late scientist’s company, Wicab, is
closer
than ever to turning his idea into reality.
Wicab has gotten some international attention in the past decade for its
“BrainPort” device that converts video signals to electronic pulses that
are
felt on the tongue. But what isn’t widely known is that the company had
been
on
the brink of failure—and that it took refocusing on the vision problem,
plus
some help from the U.S. military and tech giant Google, to right the ship.
The Middleton, WI, company is now seeking regulatory approval in the U.S.
In
pilot tests, Wicab’s device has helped blind people navigate sidewalks
without a
guide dog or cane, aided a blind rock climber to more confidently pursue
his
passion, and helped blind children in China learn to recognize Mandarin
characters and play games of darts.
The technology is based on decades of research by Bach-y-Rita, who
pioneered
the
field of “neuroplasticity,” the idea that the brain can reorganize itself
and
that senses can substitute for one another—in this case, the tongue’s
dense
group of receptors delivering information to the brain that would normally
arrive via the optic nerve. Bach-y-Rita and his team showed that the brain
can
be trained to interpret this sensory data and, although it wouldn’t
perfectly
replicate vision, it could help the blind to better perceive their
surroundings.
“Paul famously said we see with our brains and not with our eyes,” Wicab
CEO
Robert Beckman says. “The eyes are sensors. If the sensor is damaged or
not
working, you can provide an alternate sensor … to provide the information
to
the
person’s brain.”
The BrainPort device mounts a small video camera to sunglasses that are
connected via an electrical cord to a square-shaped, lollipop-like
mouthpiece
with a grid of 400 electrodes. The video feed is translated into digital
signals
expressed by the electrodes as light electronic pulses on the tongue. The
tongue
is an ideal choice for the contact point partly because it’s chock full of
nerve
endings. White pixels from the camera are translated into strong pulses,
gray
pixels feel slightly weaker, and black pixels result in no stimulation;
the
device can also reverse that so that darker images trigger the stimulation
and
lighter ones do not. The sensation, which feels similar to “Pop Rocks”
candy, is
meant to evoke in the mind a picture “painted on the tongue with tiny
bubbles,”
the company says—a much more sophisticated version of the children’s game
where
one interprets words traced by fingers on their back.
Bach-y-Rita, a former University of Wisconsin-Madison researcher, died in
2006
from lung cancer. But his company continued his work, and is now closer to
commercializing the BrainPort device in the U.S., after getting approval
to
sell
the product in Europe and Canada in 2013. Wicab is getting ready to
publish
positive results from a small clinical trial testing the technology, and
it
awaits U.S. Food & Drug Administration clearance to sell the medical
device
here, Beckman says.
Wicab’s story is an example of the twists and turns a medical startup can
take
as it tries to make the sometimes-perilous leap from the research lab to a
successful business. A decade ago, the company had a different focus.
Between
2005 and 2006, it had convinced investors to pump more than $10.5 million
into
commercializing its experimental technology primarily for the purposes of
helping people with balance problems. At the time, the company combined
the
electrode-equipped mouthpiece with an accelerometer, which can tell when
something tilts. The device would emit soft pulses of electric current
that
formed a pattern on the person’s tongue. If the person stayed upright, the
pattern would remain in the middle of the tongue, but it would shift if
the
person started to tip over. The technology was thought to help people with
chronic balance issues, perhaps through damaged inner ears or a stroke, to
train
themselves to maintain balance, Beckman says.
Wicab poured money into a clinical trial to test its theory. The device
indeed
showed it could help people improve their balance, but those in the
control
group who used a sham device also improved their balance via the series of
exercises completed as part of the study, Beckman says.
The clinical trial had failed, and Wicab was running low on capital and
forced
to lay off a chunk of its staff, which had been in the 20s. “To be quite
honest,
I thought we were dead in the water,” Beckman recalls.
But Wicab stayed afloat thanks to two things. First, it shifted its focus
toward
applying the technology to help the blind, Beckman says.
Second, it won funding in 2010 from two high-profile sources to pursue its
new
plan. One was the U.S. Department of Defense, which awarded Wicab a $3.2
million
grant to see if the technology could help soldiers blinded by improvised
explosive devices in Iraq and Afghanistan. The other source was Google,
which
gave Wicab $2.5 million to fund the 75-person study, but didn’t take an
equity
stake in the company, Beckman says.
The Google funding was serendipitous for Wicab. A U.S. Air Force general
and
a
Silicon Valley venture capitalist heard about the BrainPort device and
arranged
a demonstration at Google’s headquarters in California. The pair wanted to
help
Mike Malarsie, an Air Force senior airman who was recently blinded by an
IED
in
Afghanistan.
Malarsie tried out the BrainPort after a quick tutorial by neuroscientist
Aimee
Arnoldussen, who at the time was leading Wicab’s clinical research. Among
the
spectators were a few Google employees, including Eric Schmidt, Google’s
executive chairman and former CEO, Arnoldussen says. Google later decided
to
back further BrainPort research partly because “they witnessed somebody
benefiting from the technology and wanted to make a difference for people
who
are blind,” Beckman says.
Malarsie says he was confused at first by the concept of the device.
“You’re
going to put this thing on your tongue and it’s kind of going to draw what
you’re looking at. When they said it, it made absolutely no sense,” he
says.
But he quickly got the hang of the BrainPort, Arnoldussen says. In one of
the
tests, she held a ruler against a black sheet hung on the wall, and
Malarsie
had
to figure out whether the ruler was pointing horizontally or vertically,
based
on the pattern of the electrical pulses on his tongue as he moved his head
around. He felt “this weird tingly sensation,” but it was “not really
uncomfortable,” Malarsie says. He could immediately tell which direction
the
ruler was being held. “I could—I use the word ‘see,’ it’s not sight—I
could
feel
how she was moving this thing, can imagine what it would look like,” he
says.
Other exercises included discerning big letters put up on the wall and
looking
around the room to determine where the windows were, Malarsie says.
“He just really started to explore the room on his own,” Arnoldussen says.
“That
demo was purely Mike’s doing. I facilitated the trial, but what was the
most
exciting, was just how intuitive he was.”
For Malarsie, it was the closest thing to sight he’d experienced since an
IED
buried in a Kandahar road exploded in front of him several months earlier,
instantly killing two soldiers and blowing him off a bridge into the river
below. Shrapnel hit him in the face, destroying his left eye and tearing
his
right retina. “From the second it went off, it was pitch black,” Malarsie
says.
His memory of the immediate aftermath is “a bit murky.” But he knows that
a
fellow soldier and a medic pulled him out of the water, then the pair
advanced
into the nearby village, enemy gunfire raining down on them, to recover
the
body
of the soldier who stepped on the IED, Malarsie says.
In the hospital afterward, Malarsie’s father delivered the news: His two
rescuers didn’t make it back alive. All told, four men died and six were
wounded
that day, he says.
“It was right then that I knew I didn’t have any right to feel sorry for
myself
to let being blind hold me back,” Malarsie says. “I wouldn’t be alive if
it
wasn’t for what they did. I decided right then to live my life in a way
that
would make them happy, proud of me.”
Malarsie, 27, who later rose to the rank of staff sergeant, retired from
the
military in 2013. These days, he travels the country giving speeches about
his
experiences, writes a blog reviewing technology for blind people, and
consults
for a guide dog company. He has a wife and three children and sounds
upbeat
about the future.
The meeting at Google five years ago was one of the moments that made him
excited about potential technological advances, as he listened to Google
employees spit-balling ideas for improving the BrainPort device. “That was
kind
of the first time I thought, ‘You know what, with this kind of stuff
happening
right now, unless I die early, there’s no way I’m going to die without
being
able to see,’” Malarsie says. “At some point in my life, I’m going to look
back
and tell my grandkids, ‘Back when I was blind, I used to walk around like
this.’
Blindness will be a thing of the past.”
That’s still far from reality, but new technologies that hold promise for
the
visually impaired are starting to move from research labs into the hands
of
consumers. California-based Second Sight Medical Products (NASDAQ: EYES)
and
French company Pixium Vision developed retinal implant systems for people
blinded by retinitis pigmentosa. The technology takes images from a video
camera
attached to glasses and translates them into digital signals expressed as
electrical pulses by electrodes in the eye implants. The optic nerve then
delivers this information to the brain, which perceives patterns of
light—again,
not returning full sight, but still providing more stimuli to interpret
surroundings than without the device.
One of the challenges is these technologies have high price tags. Second
Sight’s
product has a base cost of more than $100,000, but the company has secured
reimbursement from some insurers, including Medicare in certain
situations.
BrainPort’s device costs $10,000, which partly explains why sales have
been
hard
to come by in Europe and Canada. The company intends to conduct additional
demonstration studies aimed at securing reimbursement from insurers in
Europe,
Canada, and the U.S., if the FDA clears the device for sale, Beckman says.
Wicab will try to raise at least $3 million more to fund the additional
studies,
Beckman says. The company will also try to break into the Chinese market,
aided
by Haiyin Capital, a Chinese venture capital firm that invested $3 million
in
Wicab last year.
Beckman doesn’t know when he’ll get an answer from the FDA—Wicab submitted
its
approval request in August 2013. The process has “taken a lot longer” than
he
imagined, which he partly blames on caution by the agency as it evaluates
the
new technology.
Beckman says the study of 75 subjects found that the electrical
stimulation
on
the tongue was safe. Eighteen people dropped out of the study for various
reasons, but the majority of those who completed the one-year assessment
successfully used BrainPort to identify objects, locate and identify signs
while
navigating a hallway, and read words on a computer screen. The company
intends
to market BrainPort as a device that, after some training, can assist
blind
people with “orientation, mobility, and object recognition,” but is not a
replacement for other aids like the white cane and guide dogs, Beckman
says.
If the FDA gives BrainPort the green light, the company would still have
its
work cut out to convince insurers to cover the device, and to continue
advancing
the technology and simplifying the design to make it more practical. But
FDA
clearance would still mark a significant step in Wicab’s journey to
market.
Malarsie’s experience with BrainPort during a six-month trial in 2011
gives
a
taste of how the device might help more users in the future. He trained to
the
point where he could use it to walk down the sidewalk without his guide
dog
or a
cane. “It was a liberating experience to walk somewhere outside without my
hands
outstretched in front of me, without a cane. It’s something I haven’t done
since
losing my sight. It was pretty awesome.”
In addition to navigation, BrainPort was useful for discerning where
people
were
located in a room. There would be too much stimuli for it to be useful in
a
crowd of people, Malarsie says, but he could use it to chase his children
around
the house and “see” where they were going, for example. “It kind of helps
with a
sense of inclusion,” he says. “So, just to have a sense of where people
are
sitting, how many people are around—that’s extremely helpful.”
Although Wicab has made the device less bulky over time, Malarsie says it
could
be improved if it used a smaller camera that wasn’t so noticeable, and
also
was
higher resolution; if it didn’t have any wires; and if it didn’t require
the
user to hold the lollipop device in the mouth and take it out with their
hand to
talk. He didn’t mind that strangers stared at him while wearing the
contraption
because it was useful to him, but he knows some blind people wouldn’t want
to
wear the device in its current form. “They already stand out; they don’t
want to
stand out more,” he says.
Beckman says Wicab is aware of these inconveniences and is working to
tweak
the
design and continue improving the technology.
Wicab intends to eliminate the handheld device that controls the intensity
of
the electronic pulses and the camera zoom, instead placing those controls
on
the
glasses. That would free up one of the user’s hands, which would be useful
because a cane or a guide dog leash might occupy the other hand, Beckman
says.
The company considered converting the lollipop device into a retainer that
would
sit on the roof of the mouth, and the person would lift the tongue and
touch
it
to feel the electrical stimulation. But focus groups raised concerns about
the
possibility of misplacing the retainer, so the company intends to keep the
lollipop device tethered to the glasses, Beckman says.
Beckman acknowledges that the current version of BrainPort looks “somewhat
strange,” and he recognizes that blind people “are still very much aware
of
aesthetics.” But people’s reactions can change after they get used to
seeing
new
gadgets. “The first time I saw somebody with a Bluetooth in their ear, I
thought
that was really odd,” Beckman says. “I believe that wearable technology,
including glasses, are going to continue to be developed. As that happens,
our
technology will fit right in.”
A next frontier for Wicab is partnering with software developers to
integrate
mobile apps with BrainPort, which would open up new possibilities for more
advanced and complementary features, Beckman says. “We need to couple the
capability we have—which is to interpret simple information, or the big
picture,
I would call it—with the Internet, which has the ability already to
decipher
and
interpret complex information.”
One of the early ideas is that a blind person could tell the mobile app
she
is
seeking, say, a bus stop. The app could look online to find the next bus’s
estimated time of arrival, while also helping steer the user to the bus
stop.
The app could have access to the BrainPort’s video feed and could
communicate to
the user—perhaps audibly, or through a signal on the tongue, or through
bone
conduction, a la Google Glass–that the bus stop is within view.
Beckman equates it to the technology that will enable driverless cars to
stay
within lanes and identify the signals of traffic lights.
“A lot has developed in computer vision, face recognition, contextual
understanding of surroundings, the idea of tapping into cloud resources,
that
didn’t exist” several years ago, says Arnoldussen, who left Wicab in 2012
but
still consults for the company. Once BrainPort can integrate those types
of
technologies, she adds, “I think the impact will be quite strong.”
Simplifying the logistics of operating the device and combining it with
mobile
apps are the key to making BrainPort a more practical technology right
“out
of
the box,” Beckman says. “I think the device, as it is, is useful and will
meet
with some success. But I think where we’re headed is in a direction that
will
greatly expand the number of people that want to purchase the technology.”
Source URL:
http://www.xconomy.com/wisconsin/2015/02/09/wicabs-wearable-vision-device-nears-u-s-market-thanks-to-google/?single_page=true
----------
To send a message to the list, send any message to blindza@xxxxxxxxxxxxx
----------
To unsubscribe from this list, send a message to
blindza-request@xxxxxxxxxxxxx with 'unsubscribe' in the subject line
---
The 'homepage' for this list is at http://www.blindza.co.za
