New Now Know How

16th April 2010

Post with 1 note

Natural User Interfaces are Not

In my PhD thesis I said, quite early on,

The largest problem with interfaces that use a recognition paradigm is that they are error-prone in their interaction. This is not to say that it is easier for users of recognition systems to make mistakes with such systems but that the systems themselves make mistakes.

Compare with Steve Ballmer in the Huffington Post of January this year:

But I believe we will look back on 2010 as the year we expanded beyond the mouse and keyboard and started incorporating more natural forms of interaction such as touch, speech, gestures, handwriting, and vision — what computer scientists call the “NUI” or natural user interface.

(Don Norman wasn’t impressed.)

The list of “natural forms of interaction” is quite extensive. And in the “natural” world, they work pretty well. But computers and the ways in which they are made to understand the world are not natural.

A recent article for the BBC quoted Google research Mike Cohen on speech recognition:

The obstacles for this system are not only technological but also environmental, as Google research scientist Mike Cohen explained.

“There’s background noise, and people may be saying all sorts of things with pauses and hesitations like ‘um’ and ‘erm’ - that’s one thing that makes it hard,” he said.

“People can talk about almost anything. It’s enormous vocabs, unpredictable queries, and we need to be able to handle that,” he added.

It’s not that speech recognition, or gesture or whatever, aren’t useful. It’s just that they aren’t natural. The problems of recognition are technological, environmental and social. There’s no shared context with a machine that’s recognising a human activity. Even if you could overcome all the technical and environmental obstacles for accurate recognition of speech or gesture, there’d be the, insurmountable in my opinion, obstacle of the social aspects of communication to overcome. It’s a lack of shared social context that makes these recognition interfaces hard to use.