[accessibleimage] Re: Haptic rendering of images

Hi Peter,

Well, this has not been exhaustively investigated, but the
impression that I and one of the blind beta-testers had was that somehow it seemed very difficult to mentally integrate the tactile feedback from a manual scan with a > single point of tactile contact. I suppose it was analogous to someone creating a cardboard raised drawing for you and then asking you to try and figure out its content by scraping it with a match or a needle instead of using multiple fingers at the
same time to feel overall shapes and textures and such.

That's a great description of what I consider to be one of the big problems Peter. At the moment, it's just a hypothesis, but I suspect that trying to feel the surface of a shape to eidentify an object overloads human memory. Sensory memory only lasts for between 200ms and around one second, and so it is only really useful for very small objects or small segments of objects. I suspect that this isn't enough capacity to identify most individual characteristics that lead to object identification when simply feeling the surface of a shape.

If you find a way to make the single-point-of-contact devices give more intuitive feedback, I'd be very interested. Like I wrote, I played with friction, vibration > and gradient based force feedback settings, but overall > I did not find the results encouraging.

The hypothesis I have regarding object identification, for all senses not just kinesthetics, is that there are two mechanisms used by humans. The first is a simple pattern matching task, where the stimuli contained in sensory memory is matched against previously stored patterns of stimuli in long term memory. This is the faster of the two approaches, which is why I think it's carried out first. The second method is to use deductive logic and heuristic rules to identify objects. This works by reducing an object into it's components and then matching those components against heuristic rules that describe shapes and objects. For example, someone could identify that a shape has four corners and all sides are equal in length. After identifying the attributes of this object, they would then match them against previously learnt heuristics describing shapes and come up with a square on the basis that a square has four corners and sides of equal length. This second method of object identification is useful in that it enables humans to learn to associate names with patterns of raw sensory stimuli that they haven't previously encountered, and it fits in with the hypothesis testing concepts found in learning theory.

It's this second method of object identification that I think is used in single contact point haptics. A user can scan a single characteristic of an object and likely hold that information in sensory memory, eventually matching it with the name of a shape characteristic. They can repeat this for a reasonable amount of identifying characteristics of the shape, and eventually get enough to match it against a heuristic describing a shape. Not only is it possible to use components of a shape, such as corners, but textures and changes in texture could also be used in the heuristic matching process.

One idea for applying this to 2D haptics would be through the use of snap constraint forces. Constraints are a mechanism similar to glue, and are used to stick the contact point to an object's surface whilst allowing the user to move around the surface of that object. Given that an object's outline shape is quite important in the heuristics regarding objects, one improvement might be to have a snap constraint force applied to the outline of the shape. This would enable a user to trace around the outline of the shape identifying the key characteristics of the outline, information that they then could use in the deductive logic identification process.

The disclaimer is, that at the moment it's all a hypothesis and I haven't proven any of it. However, the theoretical basis for the hypothesis is quite strong.

It was meant to provide supplementary information for > the soundscapes under direct user control (after all, you
cannot wiggle your ears in a functionally relevant way to > hear out any details), but so far I was a bit disappointed > with the multimodal experiment. The final show-stopper > was that most of the tactile gaming peripherals disappeared from the market.

I did notice tthat, and thought it was a bit odd at the time. I was thinking about buying a Logitech iFeel Mouseman, but couldn't find one in the UK when I looked a couple of years ago. Maybe it was down to Immersion, as they seem to manufacture most of the underlying haptic systems for these sort of devices.

I played with a Phantom stylus from Sensable once when I was at the NASA booth at SIGGRAPH in 1998 in Orlando, and that was quite cool and intuitive, > e.g. for 3D sculpting work, but at that time it had a $25,000 price tag if I remember correctly.

They've gone up in price since then, although it depends on which model you purchase. The Desktop model, which provides feedback along the x, y and z axis costs around £15000, or $30000 approximately, and the six degrees of freedom Premium model, which also provides torque feedback on three of the joints of the arm, costs around £50000, or $100000 approximately. There's even a model above that one, although we just have those two here at Bristol. So, I don't think it would be something an individual would purchase, but maybe institutions may.

Will


Other related posts: