[accessibleimage] Visual Ears Aural Eyes

Hi,
A related link to the article  Visual Ears Aural Eyes, from Sylvie about
the publisher of the book and a related internet radio station
"Dog Rose Sound is the broadcasting station of the [
http://www.dogrose-trust.org.uk/ ]Dog Rose Trust, the Charity which
develops audio and tactile interpretation for all. Universal and
multi-sensory design and universal access are all part of the Trust's aims.
Dog Rose Sound includes news, views, audio information such as guides,
interviews and events and we hope that they will include contributions
from a wide range of producers and groups.
the internet radio station for everyone!"
http://www.dogrosesound.org/home.htm
Best,
Lisa

accessibleimage@xxxxxxxxxxxxx skrev 3. september 2006 kl. 16:49 +0000:
>Here is a chapter from the very interesting book,
>Another Eyesight:
>Multi-Sensory Design in Context, which also substantiates the idea of the
>primary importance of touch. It is from  CHAPTER 2.26
>
>"Visual Ears, Aural Eyes: A Cross Sensory Approach to Design Education"
>by Eric Somers
>
>Educating the Visual and Aural Designer
>
>Educators in the visual and aural arts often misunderstand the education
>required for applied designers in contrast to that for fine artists. While
>fine artists often base their work on self expression, playful novelty, or
>conscious copying from an established tradition, the applied artist is
>usually in the position of designing a structure which communicates
>information to meet the needs of an external client. Thus training for the
>designer, whether a designer of sound or image, must include learning to
>think of visual and aural structures as solutions to design problems.
>Recently I have come to realize that an effective way to teach this kind
>of
>thinking is to ask students of sound design to analyze works of visual art
>then translate those structures and symbols into aural equivalents and,
>conversely, to ask visual design students to analyze sound compositions
>and
>use those structures as organizing principles for visual designs. Mladen
>Milicevic [5] has called this process "semiotic transformation."
>
>The Function of the Designer
>
>There is a great tendency to confuse the role of the designer with that of
>either the fine artist or the stylist. The term "designer" is often
>applied
>to the stylist. Most properly thought of, the information designer is a
>problem solver or information "architect" who takes ideas or information
>developed by a client and puts it into a form which will communicate
>effectively. A graphic designer takes text and images which communicate
>information about a business, for example, and creates a publication which
>presents the information clearly and interestingly to the reader. The
>stylist, though often confused with the designer, usually takes a design
>already produced by an original designer and re-shapes or decorates it to
>make it more novel or individualistic. Thus a creator of "designer" jeans
>does not really design a piece of clothing which functions more
>effectively
>than the original Levi Strauss product of a century ago, but creates a
>more
>novel variation of the original design. Such a "designer" is most properly
>called a "stylist." The fine artist may develop his or her work in a
>variety
>of ways. Some artists consciously copy the tradition of a certain genre of
>art. Others treat art as a form of free self-expression and create works
>meaningful to themselves, but which sometimes are not meaningful to others
>without explanation. Others may try to challenge the viewers' or
>listeners'
>expectations or perceptions with art forms that are perplexing or
>shocking.
>
>Visual and Aural Imagination
>
>Clearly the designer of a visual or aural structure must understand the
>information to be communicated then develop a concept for an effective
>presentation of that information. Ideally this should he done by
>understanding the nature of perception in the viewer/listener and
>developing
>an idea about how the information to he communicated could he designed for
>successful perception.
>
>Often art education and design practice fail in meeting this ideal. Too
>often design is copying of a look, a sound, indeed a "style," and applying
>it to a problem which may or may not he suited to the solution. One only
>has
>to look at the many copy-cat confusing web sites to see this phenomenon,
>or
>listen to much popular music. Yet real design progress requires original
>conception, not ornamentation or augmentation.
>
>The Integration of the Senses
>
>There is an inherent integration of the human senses as noted by Sergei
>Eisenstein [6]. Erich M. von Hornhostel [20], Oliver Sacks [18] and
>others.
>The experience of one sense can, at a minimum, remind one of a sensation
>of
>another sense. For example, when most people hear someone talking on the
>radio, they develop a mental picture of the appearance of that person.
>William Gaver [7] has noted that in everyday perception of sound in the
>environment people tend to visualize a sound by associating it with the
>source they believe produced it. In the most extreme form of sense
>integration, synesthesia, a person exposed to stimulation of one sense
>actually experiences the physical sensations of another. Richard Cytowic
>[4]
>has documented cases in which people actually see colours when hearing
>music
>and feel shapes when tasting food. But true synesthesia is a neurological
>condition. My pedagogy is an attempt to make use of the inherent
>"everyday"
>integration of the senses (in Gaver's sense) in order to allow sense
>experience of one form to promote creative thinking in another.
>
>Thinking Visually and Aurally
>
>The idea of an artist studying inure than one art form in order to derive
>first principles is not new. The painter Wassily Kandinsky wrote "One art
>must learn how another uses its method, so that its own means may than he
>used according to the same fundamental principles, but in its own medium.-
>He added that "The natural result of this work is a comparison of the
>elements of one art with those of another. Music is found to be the best
>teacher." [10] Though perception and creative thinking are certainly
>related, there is a vast difference between the value placed on visual
>experience and on aural perception.
>
>Scholars studying the anthropology of communication, especially Edmund
>Carpenter [3], Marshall McLuhan [14], Walter Ong [16] and David Berlo [2]
>have noted that literate cultures tend to emphasize sight over the other
>senses. Western societies, especially, have tended to privilege the sense
>of
>sight since the widespread adoption of literacy made possible by the
>printing press. In such cultures people equate seeing with thinking and
>use
>visual terms as a kind of code to describe understanding of thought: "I
>see
>your point," "Can you make yourself more clear'?" "Your logic is fuzzy,"
>"I
>get the picture;' "Seeing is believing. etc. Cultures which do not equate
>seeing with thinking would have difficulty decoding such statements in a
>meaningful way since they do not share the same code.
>
>Cultures which do not have a strong tradition of literacy develop an aural
>culture with its own particular characteristics and code. In these
>cultures
>terms related to thinking often get expressed in aural terms. For example,
>"I hear you" is often used to mean "I understand you:' Though people from
>most literate cultures would rather go deaf than blind, people from oral
>cultures feel deafness to be a greater hardship than blindness.
>Non-literate
>cultures tend to believe something heard over something seen. Even Plato
>expressed skepticism and distrust of the then-new medium of writing,
>believing it to be inferior to word-of-mouth communication. (17] Speech in
>non-literate cultures tends to make use of several aural and visual
>elements
>of expression: the meaning of the words chosen ("diction"), the vocal
>inflections of the person saying the words, and the body movements, facial
>positions and gestures of the speaker. Speakers in non-literate cultures,
>including children in all cultures who have not yet learned to read, tend
>to
>use many inflections and gestures. But as people become educated in
>literate
>cultures they are often taught to "modulate" their vocal inflections,
>stand
>still as they talk, and not use gestures. Thus speech becomes reduced to
>the
>single element which can be coded by writing or printing: the meaning of
>the
>words themselves. [3, 14, 16]
>
>David Berlo [2] and Tony Schwartz [19] remind us that meanings do not
>reside
>in words, images or sounds, but in the cultural experience of each
>reader-viewer-listener. Thus perceptions bring to mind meanings which may
>not be the same for each recipient of the same act of communication. The
>complexity of meaning is apt to be greater than the symbol used for
>communication. Schwartz writes "A listener or viewer brings far more
>information to the communication event than a communicator can put into
>his
>program, commercial or message." [19] Maribeth Back [1] has taken a
>similar
>approach to sound design, letting a well chosen sound evoke a deeper
>cultural meaning. Miroslav Malik, [12] a Czech-Canadian designer of
>multimedia exhibits for worlds fairs and other multicultural festivals,
>often used a "mosaic" design approach in which many different images are
>presented at once drawn from the cultural backgrounds of different
>cultures.
>Viewers from each culture are then able to decode at least part of the
>message. The ability for a sound, word or image to evoke a complex thought
>allows for very powerful communication with a minimum of means or a narrow
>channel. But such associations can also lead to "stereotyping" in which a
>symbol always brings to mind a single instance of what could otherwise be
>a
>multidimensional thought. These symbols become icons that are not analyzed
>for the information they contain but which simply redirect the perceiver
>to
>his/her own stored meaning. Indeed, some people try to find symbolic
>associations in certain kinds of messages where none are intended. A
>common
>example is the preoccupation naive art viewers often have trying to figure
>out what a non-representational work of art is "supposed to be."
>
>Context and Imagination
>
>Designer Steve McCallion relates an assignment a professor of his gave in
>architecture school: "to create an innovative office tower. He insisted
>that
>we not reference outside resources - no books, no photographs... The
>result?
>A depressingly familiar display of foam core and hot glue." 113] Given a
>lack of context, designers often resort to the stereotypical thinking
>mentioned above. They tend to recall a stored impression and make it an
>"original" design by changing small details. They become stylists. By
>greatly changing the context of a design problem it is much more likely
>that
>original designs, not "styles," will result. Thus if one asks a class
>simply
>to design a "car" most will create a stylistic variation of the typical
>car.
>If however, one stipulates that the car has to he able to cross water and
>also drive up sand dunes, then the discontinuity of the new context will
>more likely produce innovative thinking. The architect John M. Johansen
>has
>produced some highly original experimental designs-the kind McCallion may
>have been seeking from his students by setting for himself some
>imaginative
>recontextualizing problems: a structure which hangs between twin
>skyscraper
>towers and is supported by them; a theatre which uses magnetic levitation
>to
>move people and objects; buildings which "adapt" themselves to changing
>environmental conditions similar to adaptation in biological forms, etc.
>[9]
>
>Discontinuity as an Agent of Contextual Change
>
>Eric Hoffer [8], Peter Drucker [5] and others have discussed how
>discontinuity leads to innovation. Immigrants to the U.S. have a better
>than-average record of success as inventors, business entrepreneurs, and
>artists. The change of social context leads these immigrants to creative
>thinking more easily than someone brought up within the culture. Dealing
>with the discontinuity of change seems to produce creative thinking.
>Discontinuities can he caused by various forces. Many young people in the
>1960s became disillusioned with certain social structures and political
>policies in the U.S. and sought innovative changes. Writers and painters
>often travel to consciously experience discontinuities which might awaken
>them to new ideas and images. In my teaching of visual and aural design I
>have produced discontinuity as a stimulant to creative thinking by asking
>students to "re-frame" their visual perceptions in aural terms and to
>re-frame their aural perceptions in visual terms. This process of semiotic
>transformation involves the conversion of the symbols and structure of one
>sensory medium to those of another. It is an attempt to re-integrate the
>senses so that auditory experience will play a role in visualization and
>visual experience will play a role in aural thinking.
>
>Semiotic Transformation as a Stimulant to Design Thinking
>
>There is a tradition of using art of one form as "inspiration" for
>another.
>Some teachers of fine art play music in the classroom and ask students to
>"paint what they feel." A similar approach is used by some writing
>teachers.
>In music there is the tradition of "program music" which is intended to
>bring to mind a specific experience. Thus in Beethoven's Sixth Symphony we
>hear passages intended to remind us of a storm, of villagers playing, etc.
>Lest we miss the intent Beethoven thoughtfully annotated the score. The
>pedagogy being presented here is not related to any of those strategies.
>Its
>purpose is not to evoke shared cultural memory of certain images or
>experiences, nor is it intended to focus the student on his or her own
>"feelings." It is analytical and structural in nature. The student is not
>asked to "interpret" a sound or image, but to understand the nature of its
>elements and how they are related, then create an artwork in another
>medium
>which uses elements perceived to be of a similar nature arranged in a
>similar structure. What sounds are similar to certain pictorial elements,
>and what pictorial representations would be good "translations" of sound
>elements, are decisions the student designer must make. The result then is
>not self expression (the domain of the fine artist) but analytical problem
>solving (the domain of the true designer).
>
>Sound Composition Based on Sonification of Visual Experience
>
>In teaching a course in sound composition and musique concrete a few years
>ago I faced a problem similar to that of Steve McCallion's professor (see
>above). When I asked students to make sound compositions from "found
>sounds"
>recorded by them using a portable DAT (digital-audio tape) recorder, I
>tended to either get compositions that were more or less random
>collections
>of sounds or else imitations of non-concrete music. Without a structural
>framework the students floundered. Since most of these students were media
>studies majors with no background in music composition, they could not
>rely
>on their understanding of the design of music to design these sound
>compositions. But each student had taken a foundations course in two
>dimensional visual design. As an experiment I took a abstract painting
>from
>the wall in my office and asked the sound design students each to create a
>sound composition equivalent to the visual work of art. I explained that
>the
>students were to study the visual art. break it down into separate visual
>elements, then find sounds equivalent to each element. Finally, they were
>to
>study the arrangement, or composition, of the elements on the canvas and
>arrange their sound elements so they seemed equivalent in time and sonic
>space to the arrangement in visual space of the visual elements.
>
>This process of semiotic transformation greatly improved the structures of
>the student sound compositions. In the following semesters we repeated the
>assignment successfully using visual works by a number of artists
>including
>Ann Wilson, Elsworth Kelly, Mark Rothko, Cy Twombley, and Jackson Pollack.
>Most recently I expanded the technique by showing two works of visual art
>and asking students to create a composition which begins with the form of
>the first visual design and gradually develops into a piece having the
>form
>of the second visual composition. In all cases the visual designs
>presented
>are non-representational in order to keep the assignment focused on form
>rather than on outside references. The results can be summarized as
>follows:
>
>1. By limiting the composition to elements re-mapped from the visual
>model,
>students used fewer different sounds and made better use of repetition and
>variation.
>
>2. Greater structural unity resulted from their "seeing" in the visual
>model
>how to think of the sound composition as a structure having multiple
>overlapping elements, not just a string of linear events.
>
>3. In having to do the semiotic transformations, the students improved
>their
>analytical skills by learning how to break down visual and aural
>structures
>into individual elements.
>
>4. By re-framing their visual experience aurally, the process of
>composition
>became focused in the brain, with the tools of the sound studio more a
>means
>of execution and less the means of composition itself.
>
>Visual Design Based on Visualization of Sound Experience
>
>With the success of the sound composition project, I tested the
>possibility
>of reversing the process described above: to use sound composition as a
>pedagogical aid in teaching visual design. When presenting a student with
>a
>visual design problem one often finds that student solutions tend to copy
>other designs the student has seen. Indeed, some teachers actually show
>worked-out solutions as models. But this tends, in my view, to promote an
>emphasis on styling rather than design. I once joked that it would he
>better
>to teach visual design by radio (so students would have nothing visual to
>copy), but then realized that using sound might provide a way to train
>students to come up with original visualizations. Basing visual art on
>sound
>is not entirely a new concept in teaching art and design. In addition to
>the
>"listen and paint what you feel" exercises noted above, some major artists
>have experimented with the visualization of music. Paul Klee, in his
>Notebooks [11], describes exercises in basing visual art on musical rhythm
>and structure. He includes an analysis of a three part passage by Bach,
>but
>most of his analysis seems to have been made from the musical score rather
>than from the sound itself. For my teaching experiment, I designed
>electro-acoustic sound clips produced by computer sound synthesis. As with
>the visual art using in my sound composition exercises, the sounds were
>non-representational. Again, too, the task was analytical: to find
>elements
>in sound that can he represented by equivalent visual structures. Students
>were trained to make the transformations a step at a time. Small, short
>sounds were produced and students asked to produce visualizations of them.
>Then longer sound passages were presented and students were given time to
>create drawing which represented both individual sound elements and the
>relationship of these elements to each other. In the most advanced stage
>four minute compositions were played and students were asked to create
>color
>paper cut-out collages representing the sound piece.
>
>Several results were noted:
>
>1. There was sufficient formal consistency between both the elements and
>overall compositions to indicate that the semiotic transformation from the
>sound events was genuine. An example can he seen in the three drawings
>below, each a different student representation of two plucked string
>notes,
>one louder than the other:
>
>Figure 1: Student sketches by Bill Miller, Frank Corridori and Sophie Phan
>
>2. Though nothing was ever said to the students about how the sounds were
>produced, many of the visual images showed a striking understanding from
>the
>aural perception alone. The examples below are based on a sound produced
>by
>modulation. The visual reproductions are clear representations of
>structures
>where one element controls (or modulates) another:
>
>Figure 2: Student sketches by Jay Diesing, Ashley Benner and Frank
>Corridor'
>
>3. The students developed a remarkable ability to articulate their reasons
>for making visual choices based on their sound perceptions, thus
>supporting
>the analytical purpose of this pedagogy.
>
>4. Many of the student visual compositions were interesting and original
>visual designs completely pleasing on a purely visual level.
>
>5. Many of the students felt the experience helped them to understand the
>design process better.
>
>Conclusion
>
>With the specialization inherent in much of modern society, it is often
>easy
>to forget a past where an educated person made simultaneous contributions
>to
>several fields including visual art, theatre, music, the natural sciences
>and philosophy. In such a context the idea of using one's ears to aid
>visual
>thinking, or the reverse, would probably not have seemed unusual. Today,
>when someone asks "what do you do?" they sometimes seem unsatisfied by any
>answer that can't "peg" you to a certain narrow area of specialty.
>
>Sometimes specialization is necessary. One probably cannot be a noted
>concert violinist and an accomplished oil painter simultaneously because
>of
>the amount of time it takes to study and practice the technique of each
>art.
>But with the aid of modern technology, especially the computer, one can
>design sonic information structures and visual ones and can perform tasks
>that are both technical and artistic. With the visual-aural demands of a
>multimedia culture, it is not unreasonable for a young person to want to
>understand the underlying principles of both aural and visual design.
>
>The experiments I have done and described above show that semiotic
>transformation can be a useful tool of design education. Thus the
>multi-sensory pedagogy described in this paper seems to be an appropriate
>direction for design educators to consider in the 21st century.
>
>NOTE: This article first appeared, in slightly different form, in the
>proceedings of the 1998 International Conference on Auditory Display (ICAD
>'98) under the title of "A Pedagogy of Creative Thinking based on
>Sonification of Visual Structures and Visualization of Aural Structures."
>
>References
>
>1.  Back M. "Micro-Narratives in Sound Design: Context, Character, and
>Caricature in Waveform Manipulation." In: Proceedings of the International
>Conference on Auditory Display, 1996.
>
>2.  Berlo D. The Process of Communication: An Introduction to Theory and
>Practice. Holt, Rinehart, Winston, New York 1960.
>
>3.  Carpenter E. Oh, What a Blow That Phantom Gave Me!. Holt, Rinehart and
>Winston, New York, 1972.
>
>4.  Cytowic R. The Man Who Tasted Shapes. Tarcher/Putnam, New York, 1993.
>
>5.  Drucker P. The Age of Discontinuity: Guidelines to Our Changing
>Society.
>Harper and Row, New York, 1969.
>
>6.  Eisenstein S. Leyda J (cd and trans) Film Sense. Harcourt Brace
>Jovanovich, 1942.
>
>7.  Gaver W. "Using and Creating Auditory Icons." In: Kramer G. (ed)
>Auditory Display: Sonification, Audification, and Auditory Interfaces.
>Addison Wesley, Reading MA, 1994.
>
>8.  Hoffer E. The Ordeal of Change. Harper and Row, New York, 1963.
>
>9.  Johansen J. A Life in the Continuum of Modern Architecture. 13 Area
>Edizioni, Milan, 1995.
>
>10.  Kandinsky W. Concerning the Spiritual in Art. George Wittenborn, New
>York, 1947,
>
>11. Klee P., Spitler J. (ed), Maheim R. (trans) Paul Klee Notebooks Volume
>1: The Thinking Eye. The Overlook Press, Woodstock New York, 1992.
>
>12.  Malik, M. Private conversation with the author (though some
>references
>to his work can he found in various reviews of the Czech pavilion at Expo
>'67 held in Montreal).
>
>13.  McCallion S, Muoio A (ed.) from a profile in "They Have a Better Idea
>... Do You?" Fast Company, August-September 1997.
>
>14.  McLuhan M. The Gutenberg Galaxy.
>
>University of Toronto Press, Toronto, 1963.
>
>15.  Milicevic M. Private conversation with the author after hearing a
>presentation of preliminary findings from experiments with the pedagogy
>presented.
>
>16.  Ong W. Orality and Literacy, Methuan, London and New York. 1982.
>
>17.  Plato, Phaedrus. also discussed by Ong (see above).
>
>18.  Sacks O. Seeing Voices: A Journey Into the World of the Deaf.
>HarperCollins, New York 1990.
>
>19.  Schwartz T. The Responsive Chord. Anchor Press/ Doubleday, Garden
>City
>(NJ) and New York, 1973.
>
>20.  Von Hornhostel E. "The Unity of the Senses." In Psyche Vol. VII No.
>4,
>April 1927.
>
>Eric Somers is Professor of Design and Communication at Dutchess Community
>College of the State University of New York, where he served as Chair of
>the
>Department of Visual and Performing Arts for 15 years. He began his career
>as a fine arts television producer which triggered a life long interest in
>the relationship of sound to image. He currently maintains a production
>service, The Sandbook Studio, which specializes in visual and aural
>documentation of the fine arts and in presenting workshops related to high
>end sound and image capture and reproduction. Professor Somers has served
>as
>Chair of the New York section of the Audio Engineering Society (AES),
>President of the International Community for Auditory Display (ICAD), and
>President of the Museum for Preservation of Illustrative Art.


Lisa Yayla
Huseby Kompetansesenter 
Oslo Norway
lisa.yayla@xxxxxxxxxx

Other related posts: