----- Original Message ----- Konekt blog - Emotional machines: and the blind could see.
Jeff Fraser.03/15/2011 – TVSS stands for Tactile Visual Sensory Substitution, an assistive technology pioneered by Paul Bach-y-Rita in the seventies to help blind persons
sense and react to their physical environment. Although at the time the technology wasn’t capable of resolution beyond a measly 20×20 cells, what ittaught us about the brain is that prosthetics are not only feasible, they’re a
lot closer to our reach than one might expect. With the processing power we now have at our disposal, we could be seeing anelegant, noninvasive visual prosthesis for the blind within a couple of decades.
TVSS is based on a similar principle as a walking cane or what’s known as the “facial vision” of the blind. Certain blind persons report a light touch on the
forehead or cheeks, as if being brushed with a veil, when encountering largeobjects within 30cm-80cm from the face, and can reliably identify the objects
causing it.When discovered in the late nineteenth century, facial vision was thought to be a kind of extrasensory perception; now, thanks to the work of Dallenbach et al. (1944), we know that facial vision isn’t tactile at all – it’s auditory. Facial
vision is based on the ability to detect the intensity and direction ofreflected sounds. Astonishingly, one researcher (Kohler 1967) went so far as to anesthetize subjects’ faces – and discovered that they still felt facial vision
occurring in their cheeks or forehead!The TVSS system works like this: a sensor placed somewhere on the person’s face,
for instance in a pair of glasses, maps the intensity of incident light. Thesensor’s electrical output is then translated into a set of regular instructions for a tactile stimulation device – a pad about 8″ square that can be worn on the
back, stomach, or forehead. The pad massages the skin in an analogue of the light hitting the sensor. The hope is that, just like auditory signals beingtransformed into feelings of touch in facial vision, here tactile signals will
be transformed into something like sight. It’s so simple, you wouldn’t think it could work. But it does.With practice, wearers become adept at using the system to locate, react to, and manipulate objects in space. Regardless of where the pad is placed on the body,
eventually the wearer comes to orient the sensation toward the space beingsensed, so that, rather than having to actively “interpret” the incoming tactile signals, they can respond to the TVSS similar to a person responding to sight.
In a telling anecdote given by Bach-y-Rita, a subject accidentally magnified the zoom on his camera, causing the object before him to suddenly “loom” bigger. The
subject jumped backwards in surprise – despite the fact that the tactile stimulator was on his back.Some wearers report a vaguely vision-like sense of where things are in the room.
(The “vagueness” here is usually explained as an effect of the system’s poorresolution – the original system was effectively colorblind, stereoblind, and incredibly myopic.) Even more interesting is the fact that once trained with the TVSS, a pad can be placed on any sensitive region of the body – even distributed in a metric-preserving way – without loss of aptitude. It’s as if the brain has
learned to treat the TVSS as an extension of the sensory nervous system. What’s the explanation? It’s all about brain plasticity.Typically, blind persons have not lost the cortical regions that enable them to
see, but only the retinal synapses that connect their eyes to their brain.Recent neuroimaging studies using PET and fMRI have shown that during tactile
visual substitution, blind subjects recruit extra-striate occipital areas thought to be involved with visual processing. A Trans-Cranial Magnetic Stimulation (TMS) study, which simulated lesions inthese areas, disrupted blind but not sighted blindfolded controls in performing
a depth-perception task. All of the studies found that activation of theoccipital areas was minimal upon introduction to the substitution system, but
increased with training. Cheers to the brain that changes itself.On the other hand, brain plasticity can’t explain everything. The brain doesn’t
need to be induced to transfer tactile stimulation to the visual processing areas – it does this automatically. How does the brain know that TVSS stimulation should be processed as sight, rather than some other perceptual modality? One possible explanation is that we use the same “mental imagery”apparatus to “picture” the unseen stimulus; however this fails to explain why
blindfolded sighted controls show less activation in the same areas.If you read my last post, you may have guessed where this is going: sensorimotor
contingencies. O’Regan and Noe (who are quickly becoming my heroes) have used TVSS as a foundational argument for their theory that sight is not just about where stimulus gets processed, but about the nature of the signal itself. TVSS is vision-like, in the sense that it responds in regular ways to the subject’s movements and associated changes in hearing and proprioception. The brain isable to tell that TVSS stimulation needs to be routed to visual centres because it bears distinctly “visual” relations to the rest of the sensorimotor system.
As a final note, it’s worth pointing out that the strangeness of TVSS isn’treally all that unusual, if you consider the fact that when someone pokes your arm, you feel it in your arm – despite the fact that the sensory information is being processed in your brain. The fact that you hear at your ears, see at your eyes, and feel at your skin doesn’t have anything to do with where your sensory nerves are placed; it has to do with the way your brain projects perception onto
your body.In just the same way, this kind of perceptual projection accounts for all sorts of strange phenomena – like sympathetic or remote tactile sensing, in which a subject feels an experimenter touch the arm of a mannequin as if it were her own arm. Another experiment showed that a subject can be “tricked” into feeling a touch on her index finger when the experimenter is actually poking her middle
finger.The practical upshot of all this is that we don’t need to find the specifically
“visual” neural submodule in the brain or nervous system in order to createsensory prostheses – we cam simply throw signals at the nervous system through
auditory or tactile channels, and let brain plasticity do all the real work. If this interpretation of the research stands up, then given ten years and enough research dollars, sensory handicaps could be a thing of the past.To leave off with a promising example, researcher Kevin Warwick, aka “Captain
Cyborg,” has shown that a form of sensory substitution can be interfaceddirectly with the human nervous system. In 2002, he had a chip implanted in his arm that allowed him to feel stimulation of a robotic hand and move it as if it was his own. This video presents some of his ideas on how cybernetic prostheses
could change our future. Source URL: http://www.konekt.ca/blog/emotional-machines-athe-blind-could-see ---------- To send a message to the list, send any message to blindza@xxxxxxxxxxxxx ---------- To unsubscribe from this list, send a message to blindza-request@xxxxxxxxxxxxx with 'unsubscribe' in the subject line --- The 'homepage' for this list is at http://www.blindza.co.za