HCI: UI beyond the WIMP Paradigm

Question

With the popularity of the Apple iPhone, the potential of the Microsoft Surface, and the sheer fluidity and innovation of the interfaces pioneered by Jeff Han of Perceptive Pixel ...

What are good examples of Graphical User Interfaces which have evolved beyond the

Windows, Icons, ( Mouse / Menu ), and Pointer paradigm ?

Answer 1

Are you only interested in GUIs? A lot of research has been done and continues to be done on tangible interfaces for example, which fall outside of that category (although they can include computer graphics). The User Interface Wikipedia page might be a good place to start. You might also want to explore the ACM CHI Conference. I used to know some of the people who worked on zooming interfaces; the Human Computer Interaction Lab an the University of Maryland also has a bunch of links which you may find interesting.

Lastly I will point out that a lot of innovative user interface ideas work better in demos than they do in real use. I bring that up because your example, as a couple of commenters have pointed out, might, if applied inappropriately, be tiring to use for any extended period of time. Note that light pens were, for the most part, replaced by mice. Good design sometimes goes against naive intuition (mine anyway). There is a nice rant on this topic with regard to 3d graphics on useit.com.

Answer 2

People are used to WIMP, the other main issue is that most of the other "Cool" interfaces require specialized hardware.

Answer 3

I'm not in journalism; I write software for a living.

Answer 4

Technically, the interface you are looking for may be called Post-WIMP user interfaces, according to a paper of the same name by Andries van Dam. The reasons why we need other paradigms is that WIMP is not good enough, especially for some specific applications such as 3D model manipulation.

To those who think that UI research builds only cool-looking but non-practical demos, the first mouse was bulky and it took decades to be prevalent. Also Douglas Engelbart, the inventor, thought people would use both mouse and (a short form of) keyboard at the same time. This shows that even a pioneer of the field had a wrong vision about the future.

Since we are still in WIMP era, there are diverse comments on how the future will be (and most of them must be wrong.) Please search for these keywords in Google for more details.

• Programming by example/demonstration
  In short, in this paradigm, users show what they want to do and computer will learn new behaviors.

• 3D User Interfaces
  I guess everybody knows and has seen many examples of this interface before. Despite a lot of hot debates on its usefulness, a part of 3D interface ongoing research has been implemented into many leading operating systems. The state of the art could be BumpTop. See also: Zooming User Interfaces

• Pen-based/Sketch-based/Gesture-based Computing
  Though this interface may use the same hardware setup like WIMP but, instead of point-and-click, users command through strokes which are information-richer.

• Direct-touch User Interface
  This is ike Microsoft's Surface or Apple's iPhone, but it doesn't have to be on tabletop. The interactive surface can be vertical, say wall, or not flat.

• Tangible User Interface This has already been mentioned in another answer. This can work well with touch surface, a set of computer vision system, or augmented reality.

• Voice User Interface, Mobile computing, Wearable Computers, Ubiquitous/Pervasive Computing, Human-Robot Interaction, etc.

Further information: Noncommand User Interface by Jakob Nielsen (1993) is another seminal paper on the topic.

Answer 5

If you want some theoretical concepts on GUIs, consider looking at vis, by Tuomo Valkonen. Tuomo has been extremely critical of WIMP concept for a long, he has developed ion window manager), which is one of many tiling window managers around. Tiling WMs are actually a performance win for the user when used right.

Vis is the idea of an UI which actually adapts to the needs of the particular user or his environment, including vision impairment, tactile preferences (mouse or keyboard), preferred language (to better suit right-to-left languages), preferred visual presentation (button order, mac-style or windows-style), better use of available space, corporate identity etc. The UI definition is presentation-free, the only things allowed are input/output parameters and their relationships. The layout algorithms and ergonomical constraints of the GUI itself are defined exactly once, at system level and in user's preferences. Essentially, this allows for any kind of GUI as long as the data to be shown is clearly defined. A GUI for a mobile device is equally possible as is a text terminal UI and voice interface.

Answer 6

How about mouse gestures?

A somewhat unknown, relatively new and highly underestimated UI feature. They tend to have a somewhat steeper learning curve then icons because of the invisibility (if nobody tells you they exist, they stay invisible), but can be a real time saver for the more experienced user (I get real aggrevated when I have to browse without mouse gestures).

It's kind of like the hotkey for the mouse.

Answer 7

vim!

It's definitely outside the realm of WIMP, but whether it's beyond it or way behind it is up to judgment!

Answer 8

Sticking to GUIs puts limits on the physical properties of the hardware. Users have to be able to read a screen and respond in some way. The iPhone, for example: It's interface is the whole top surface, so physical size and the IxD are opposing factors.

Around Christmas I wrote a paper exploring the potential for a wearable BCI-controlled device. Now, I'm not suggesting we're ready to start building such devices, but the lessons learnt are valid. I found that most users liked the idea of using language as the primary interaction medium. Crucially though, all expressed concerns about ambiguity and confirmation.

The WIMP paradigm is one that relies on very precise, definite actions - usually button pressing. Additionally, as Nielsen reminds us, good feedback is essential. WIMP systems are usually pretty good at (or at least have the potential to) immediately announcing the receipt and outcome of a users actions.

To escape these paired requirements, it seems we really need to write software that users can trust. This might mean being context aware, or it might mean having some sort of structured query language based on a subset of English, or it might mean something entirely different. What it certainly means though, is that we'd be free of the desktop and finally be able to deploy a seamlessly integrated computing experience.

Answer 9

NUI Group people work primarily on multi-touch interfaces and you can see some nice examples of modern, more human-friendly designs (not counting the endless photo-organizing-app demos ;) ).

Answer 10

I would recommend the following paper:

Jacob, R. J., Girouard, A., Hirshfield, L. M., Horn, M. S., Shaer, O., Solovey, E. T., and Zigelbaum, J. 2008. Reality-based interaction: a framework for post-WIMP interfaces. In Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy, April 05 - 10, 2008). CHI '08. ACM, New York, NY, 201-210. DOI= http://doi.acm.org/10.1145/1357054.1357089