[asvs] Re: the Concept
- From: Grigori Evreinov <grse@xxxxxxxxx>
- To: asvs@xxxxxxxxxxxxx
- Date: Sun, 10 Oct 2004 16:50:51 +0300
Hi,
Zorro wrote:
>Hi Grigori and all,
>
>Nice to see you here! I agree with
>
>"the software fit to the user, not the user fit to the software"!!!
>
>but at the same time I'd paraphrase and say
>
>"the user fit to the physics, not the physics fit to the user"!!!
>
>
>
If physics is suitable or the mapping (artificial mapping anyway) is
intuitive
The intuitive mapping means the user can control (!) and perceive the
imaging
by the natural way without the long training but after the strategy or
specific rules were explained once. In general, the technique guide
should be similar and not more difficult than typing.
It is not necessary take into account the colors at all, excluding the case
when it is used for a segmentation or other processing of the image.
But, up to now even sonification of the simple graphics is still carried
out
by non-intuitive unnatural way. Thus, black and white graphical images
could be enough but the mapping should be changed radically.
The main error, I talk about that all the time, is not the use of the
sonification
of the image, but the sonification of interaction (!) with the visual
objects.
Visual perception ab ovo is visual interaction and development of technique
and the strategy of visual interaction. That is, the motor cortex and
integration
of the position and its optical characteristics this is the subject for
the research
and development the mapping for auditory or other physical signals the use
for alternative visualization of visual images.
Certainly, ?one-point? method or the use of sonification technique
based on virtual sound source is a case study. Nevertheless, this method
has also shown that the motor activity of the eye is not less important
than
hearing activity of the head (eye tracker had head movements compensation).
http://www.cs.uta.fi/~grse/ConferenceWork/AES_22/ThreeMappsEval.pdf
The ?full image? sonification is not so intuitive and contradicts to
the visual perception basics. As this way excludes the active motor
component,
exploratory movements, and substitute them by intentional fixation.
Exploratory movements are the basis for demodulation or decoding the
optical
(visual) information about the ?grid? of the image presented.
However, the parameter ? ?the column? which was proposed by Peter
is very important for development Artificial SVS.
I have tried to explore the sonification of interaction with graphics
based on a ?stick?-metaphor that is a flexible analog of the Peter?s column.
Moreover, I proposed to change the length of the ?stick? (adaptive stick)
in dependence on the speed of the exploratory movements.
Sorry, I did not explain the stick-metaphor:
the idea of the stick interaction is not the tip of the stick
interaction like one-point style
but the crossing of the stick like the line in the same plane of the
graphics.
That is, if the stick (vertical line having the definite length) will
interact with
a slope line it will have one point of the crossing.
If the stick will interact with rectangle, it can have one, two, or all
the points
coincided with array of the stick. But the system should adapt
and minimize information for the user.
That is, if all the points coincide sound chord is difficult for
interpretation
and will be perceived as noise or strong sound distractor.
Therefore, the stick has a possibility for rotation.
The best implementation could be the physical stick which interact with
virtual graphics, while the stick position is captured by digital camera
like in the paper
http://www.dgp.toronto.edu/~ravin/papers/siggraph2004_visionwand.pdf
In dependence how we can organize the mapping (absolute or relative),
the point(s) will be sonified with different frequency or/and timbre,
volume or/and L-R position?
The absolute mapping means the mapping of sound parameters concerning
the virtual sound plane.
The relative mapping means the mapping of sound parameters concerning
the stick itself.
Grigori
Other related posts:
