While they seemed to use sighted experimentee's, the concept of conceptualisation is still relevant, etc. - see below.
Stay well Jacob Kruger Blind Biker Skype: BlindZA "Roger Wilco wants to welcome you...to the space janitor's closet..."----- Original Message ----- Brain Wave - Psych majors dive into the mind-bending world of sensory substitution.
By Chris Lydgate ’90. Orestis Papaioannou ’15 takes a cotton swab, pokes it through a hole in amesh-fabric cap and gently massages a dab of saline gel into the scalp of his
experimental subject, an English major named Dan.As he adjusts the electrodes in the cap, Orestis - who sports a neatly-trimmed beard, flip-flops, and a t-shirt proclaiming “Life of the Mind” - runs through the experimental protocol. “We’re going to show you some shapes and play some sounds at the same time,” he explains. “You’re going to learn how to translate
sounds back into shapes.”Dan takes a sip of black-tea lemonade, settles himself in a chair, and dons a pair of headphones while Orestis and Chris Graulty ’15 huddle around a computer monitor that displays the subject’s brainwaves, which roll across the screen in
a series of undulations, jumping in unison when the subject blinks.The students are investigating the brain’s ability to take information from one
perceptual realm, such as sound, and transfer it to another, such as vision. This phenomenon - known as sensory substitution - might seem like a mere scientific curiosity. But in fact, it holds enormous potential for helping people overcome sensory deficits and has profound implications for our ideas about what perception really is. === We’re sitting in Reed’s Sensation, Cognition, Attention, Language, and Perception Lab - known as the SCALP Lab - and this time I’m the subject. Mymouth is dry and my palms sweaty. Orestis does his best to help me relax. “Don’t worry,” he smiles. “It’s a very simple task.” Orestis is going to show me some shapes on the computer screen and play some sounds through the headphones. For
the first hour, each shape will be paired with a sound. Then he will play mesome new sounds, and my job will be to guess which shapes they go with. I enter
the soundproof booth, sit down at the keyboard, slip on the headphones, and click “start.” The first shapes look like the letters of an alien alphabet. Here’s a zerosqueezed between a pair of reverse guillemets. At the same time, panning left to right, I hear a peculiar sound, like R2-D2 being strangled by a length of barbed
wire. Next comes an elongated U with isosceles arms: I hear a mourning doveflying across Tokyo on NyQuil. Now a triplet of triangles howl like a swarm of
mutant crickets swimming underwater. To call the sounds perplexing would be a monumental understatement. They aresonic gibberish, as incomprehensible as Beethoven played backwards. But after an hour listening to the sounds and watching the shapes march past on the screen,
something peculiar starts to happen. The sounds start to take on a sort of character. The caw of a demented parrot as a dump truck slams on the brakes? Strange as it seems, there’s something, well, squiggly about that sound. Amarimba struck by a pair of beer bottles? I’m not sure why, but it’s squareish.
Now the experiment begins in earnest. I hear a sound; my job is to select which of five images fits it best. First comes a cuckoo clock tumbling downstairs - is that a square plus some squiggles or the symbol for mercury? The seconds tick away. I guess - wrong. On to the next sound: a buzz saw cutting through three sheets of galvanized tin. Was it the rocketship? Yes, weirdly, it was. And so it goes. After an hour, my forehead is slick with sweat and concentration. It feels like listening to my seven-year-old son read aloud - listening to him stumble over the same word again and again, except that now I’m the one who’s stumbling
blindly through this Euclidean cacophony. And yet (swarm of crickets, threetriangles) my guesses are slowly getting better. Something strange is happening
to my brain. The sounds are making sense. I am learning to hear shapes. ===Photo caption: Orestis Papaioannou ’15, Chris Graulty ’15, and Phoebe Bauer ’15
probe the brain’s ability to extract visual information from the auditory channel in Reed’s SCALP Lab. How it Works: Shapes to Sounds. The system used at Reed relies on the Meijer algorithm, developed by Dutch engineer Peter Meijer in 1992 to translate images into sounds. The verticaldimension of the image is coded into frequencies between 500 Hz-5000 Hz, where higher spatial position corresponds to higher pitch. The horizontal dimension is
coded into a 500-ms left-to-right panning effect. The resulting sound - intheory - includes all the information contained in the image, but is meaningless
to the untrained ear.Photo caption: Subjects learned to hear the shapes after as little as two hours
of training. Figure caption: AUDIO: The Reed team created 180 simple geometric shapes and transformed them into sounds. Click on each shape above and listen to the corresponding sound. The interlocking problems of sensation and perception have fascinatedphilosophers for thousands of years. Democritus argued that there was only one sense - touch - and that all the others were modified forms of it (vision, for
example, being caused by the physical impact of invisible particles upon theeye). Plato contended that there were many senses, including sight, smell, heat, cold, pleasure, pain, and fear. Aristotle, in De Anima, argued that there were
exactly five senses - a doctrine that has dominated Western thought on the subject ever since. In Chaucer’s Canterbury Tales, for example, the Parson speaks of the “fyve wittes” of “sighte, herynge, smellynge, savorynge, and touchynge.” Even today, a cursory internet search yields scores of websites designed to teach children about the five senses.The five-sense theory got a boost from psychological research that mapped the senses to specific regions of the brain. We now know, for example, that visual
processing takes place primarily in the occipital lobe, with particularsub-regions responsible for detecting motion, color, and shape. There’s even an
area that specializes in recognizing faces - if that part of the brain isinjured by a bullet, for example, the subject will lose the ability to recognize
a face, even his own.But the Aristotelian notion that the senses are distinct and independent, like TV channels, each with its own “audience” sitting on a couch somewhere in the
brain, is deeply flawed, according to Professor Enriqueta Canseco-Gonzalez [psychology 1992–]. For starters, it fails to account for the fact that our sense of taste islargely dependent on our sense of smell (try eating a cantaloupe with a stuffy nose). More important, it doesn’t explain why people with sensory deficits are
often able to compensate for their loss through sensory substitution -recruiting one sensory system as a stand-in for another. The Braille system, for example, relies on a person’s ability to use their fingers to “read” a series of
dots embossed on a page.In addition, psychologists and neuroscientists have identified several senses
that Aristotle didn’t count, such as the sense of balance (the vestibular system) and the sense of proprioception, which tells you where your arms and legs are without you having to look.In truth, says Prof. Canseco-Gonzalez, our senses are more like shortwave radio stations whose signals drift into each other, sometimes amplifying, sometimes interfering. They experience metaphorical slippage. This sensory crosstalk is reflected in expressions that use words from one modality to describe phenomena in another. We sing the blues, level icy stares, make salty comments, do fuzzy math, wear hot pink, and complain that the movie left a bad taste in the mouth. It also crops up in the intriguing neurological condition known as synesthesia,
when certain stimuli provoke sensations in more than one sensory channel.(Vladimir Nabokov, for example, associated each letter of the alphabet with a
different color.)Canseco-Gonzalez speaks in a warm voice with a Spanish accent. She grew up in
Mexico City, earned a BA in psychology and an MA in psychobiology from the National Autonomous University of Mexico, and a PhD from Brandeis for her dissertation on lexical access. Fluent in English, Spanish, Portuguese, andRussian, she has authored a score of papers on cognitive neuroscience and is now focusing on neuroplasticity - the brain’s ability to rewire itself in response
to a change in its occupant’s environment, behavior, or injury. She also knows something about psychological resilience from personal experience. She was working in Mexico City as a grad student in 1985 when a magnitude 8.1 earthquake occurred off the coast of Michoacán, crumbling herapartment building and killing more than 600 of her neighbors. At least 10,000
people died in the disaster. To demonstrate another example of sensory substitution, she gets up from her desk, stands behind me, and traces a pattern across my back with her finger.“Maybe you played this game as a child,” she says. “What letter am I writing?” Although I am reasonably well acquainted with the letters of the alphabet, it is surprisingly difficult to identify them as they are traced across my back. (I
guess “R” - it was actually an “A.”)Incredibly, this game formed the basis for a striking breakthrough in 1969, when
neuroscientist Paul Bach-y-Rita utilized sensory substitution to helpcongenitally blind subjects detect letters. He used a stationary TV camera to
transmit signals to a special chair with a matrix of four hundred vibratingplates that rested against the subject’s back. When the camera was pointed at an “X,” the system activated the plates in the shape of an “X.” Subjects learned to read the letter through the tactile sensation on their backs - essentially using
the sense of touch as a substitute for the sense of vision.Bach-y-Rita’s experiments - crude as they may seem today - paved the way for a
host of new approaches to helping people overcome sensory deficits. But this line of research also poses fundamental questions about perception. Do the subjects simply feel the letters or do they actually “see” the letters?This problem echoes a question posed by Irish philosopher William Molyneux to
John Locke in 1688. Suppose a person blind from birth has the ability todistinguish a cube from a sphere by sense of touch. Suddenly and miraculously, she regains her sight. Would she be able to tell which was the cube and which
the sphere simply by looking at them? The philosophical stakes were high and the question provoked fierce debate. Locke and other empiricist philosophers argued that the senses of vision andfeeling are independent, and that only through experience could a person learn
to associate the “round” feel of a sphere with its “round” appearance. Rationalists, on the other hand, contended that there was a logical correspondence between the sphere’s essence and its appearance - and that aperson who grasped its essence could identify it by reasoning and observation.
(Recent evidence suggests that Locke was right - about this, at least. In 2011, a study published in Nature Neuroscience reported that five congenitally blind individuals who had their sight surgically restored were not, in fact, able to match Lego shapes by sight immediately after their operation. After five days,
however, their performance improved dramatically.)On a practical level, sensory substitution raises fundamental questions about the way the brain works. Is it something that only people with sensory deficits
can learn? What about adults? What about people with brain damage? As faster computers made sensory-substitution technology increasingly accessible, Canseco-Gonzalez realized that she could explore these issues at Reed. “I thought to myself, these are good questions for Reed students,” shesays. “This is a great way to study how the brain functions. We are rethinking
how the brain is wired.”She found an intellectual partner in Prof. Michael Pitts [psychology 2011–], who obtained a BA in psych from University of New Hampshire, an MS and a PhD from Colorado State, and worked as a postdoc in the neuroscience department at UC San Diego before coming to Reed. Together with Orestis, Chris, and Phoebe Bauer ’15, they bounced around ideas to investigate. The Reed students decided to tackle a
deceptively simply question. Can ordinary people be taught how to do sensory substitution?To answer this, they constructed a three-part experiment. In the first phase,
they exposed subjects to the strange shapes and sounds and monitored theirbrainwaves. In the second phase, they taught subjects to associate particular shapes with particular sounds, and tested their subjects’ ability to match them up. They also tested subjects’ ability to match sounds they had never heard with shapes they had never seen. Finally, they repeated the first phase to see if the
training had any effect on the subjects’ brainwaves.“I was skeptical at first,” Prof. Pitts admits. “I didn’t really think it was
going to work.” The trials took place over the summers of 2013 and 2014, and the preliminaryresults are striking. After just two hours of training, subjects were able to
match new sounds correctly as much as 80% of the time. “That’s a phenomenal finding,” says Pitts. What’s more, the Reed team found that the two-hourtraining had far-reaching effects. Subjects who were given the test a year later
still retained some ability to match sounds to shapes.Just as significant, the subjects’ brain-waves demonstrated a different pattern after the training, suggesting that the shape-processing part of the brain was
being activated - even though the information they were processing came in through their headphones. “Here is the punch line,” says Canseco-Gonzalez.“There is a part of the brain that processes shapes. Usually this is in the form of vision or touch. But if you give it sound, it still can extract information.”
The results - which the Reed team hopes to publish next year - suggest that the
brain’s perceptual circuits are wired not so much by sense as by task. Some tasks, such as the ability to recognize shapes, are so important to human survival that the brain can actually rewire itself so as to accomplish them. Does this mean people can really hear shapes and see sounds, or does it meanthat the difference between seeing and hearing is fuzzier than we like to think?
Either way, the experiment suggests that the brain retains the astonishing property of neuroplasticity well into middle-age - and that we are only beginning to grasp its potential.Editor's Note: This research was made possible by a grant from the M.J. Murdock Charitable Trust Research Program for Life Sciences and the Reed College Science
Research Fellowship. Source URLs: http://www.reed.edu/reed_magazine/december2014/articles/features/brainwave/brainwave.html http://www.reed.edu/reed_magazine/december2014/articles/features/brainwave/brainwave1.html ---------- To send a message to the list, send any message to blindza@xxxxxxxxxxxxx ---------- To unsubscribe from this list, send a message to blindza-request@xxxxxxxxxxxxx with 'unsubscribe' in the subject line --- The 'homepage' for this list is at http://www.blindza.co.za
