Designing Interaction 2.0: Microsoft Natal Reinvents “The Screen”, Google Wave Morphs “Context”

My friend, and Diamond Fellow, Dan Bricklin wrote a wonderful essay on July 7, 2009 called "New Modes of Interaction: Some implications of Microsoft Natal and Google Wave."  Given that Dan not only co-invented the spreadsheet, but also heavily influenced the design of PowerPoint (through his Demo Program), and worked on the creation of early word processors, he is, to me, the uneconomically motivated Henry Ford of the modern office because the majority of knowledge workers live inside one or more of his creations for the bulk of their workday.  (Of course, Dan, being Dan, would blush at such a suggestion.)  So, when he says there is a new mode of interaction on the horizon, I sit up and take notice.

To my eye, Dan's essay reports on the reinvention of "the screen", and "context" - my words, not his.  By the screen I mean that digital proscenium arch which we only pierce with our keyboard, mouse, or other device.  That "barrier" is falling.  By context, I mean the environment in which information is presented, linked and displayed in a time sequence.  I'll deal with each below. 

Dan notes that Microsoft's Natal, enables a computer to read the gestures of user, and thereby allows a completely controller/mouse/keyboard free interaction with the machine.  In this example Dan sees a general purpose technology that allows the developer to use any "mixture of gesture/motion/image recognition with video and still photo recording and voice recognition and sound recording."   This means that designers can create deeper or shallower levels of involvement with the material and/or experience as the user desires.  As Dan notes, it will start in gaming, but move beyond it.  I think Natal has the potential to dissolve the alienation between the "user" and the "digital world" "behind" the screen.  Involvement with information as shown in the movie Minority Report will be available and you won't even need Tom Cruise's fancy gloves. 


It even shatters the idea of a screen because as the machine "locates" you, your gestures and your expressions, it can respond accordingly.  There will be many applications, including advanced retailing in which the lines between "TV" and "retail location" will blur.  This is why I think we are moving beyond the paradigm of the screen. 

By context I mean the sequence, linkage, and related information that surrounds and interacts with any content.  The layout of the New York Times is a context, as is Fox News.  One of the biggest wastes of time in modern knowledge work is the reinvention and updating of context across different people performing complex, interdependent work.  Ask yourself how much time you waste finding the most current versions of the PowerPoint deck, reading old email streams, understanding disparate spreadsheets, all forced on us by the fact that the personal computer has as much context sensitivity as a White Rhino in heat.  Of course there are previous solutions that embody ideas about synchronization and "truth" maintenance - including Lotus Notes.  Google's solution is inspiring.  As Dan points out:

a "˜Wave' is sort of like a threaded conversation on a traditional online discussion forum.  Conceptually, a wave is stored as a single document on a single server.  There are usually multiple waves on a Wave Server, and there may be any number of servers with their own waves, or copies of waves.

There is one definitive home server, and the Wave contains many things including participants to the Wave, security access levels for each participants, and a list of all changes and previous versions.  This architecture allows many things including a "playback" function which allows people who join a wave in mid stream to watch a playback of all emails, media, documents, etc.  Now that's what I mean by reinventing context.  Anyone who joins the conversation can catch up, and fast.


Many of these ideas have been around a long time.  At a conference I remember sitting next to Alan Kay who was ranting about how retrograde HTML was because the idea that anything should be "only a reader and not a publisher too" was backward beyond measure (as well as ignorant of existing solutions).  Also, in my limited technical understanding, it certainly seems like Wave is implementing the principles of object oriented programming articulated over forty years ago.  That delay makes sense, because as Peter Drucker pointed out so well, it often takes fifty years for good ideas to join the commercial main stream. 

So what?  Well, I think, (as Dan points out too), that these two new paradigms together may reinvent how people collaborate, and participate in complex, urgent, high-value-added knowledge work.  Because Wave allows for efficient synchronization, and is open to letting others build functionality into the platform, there will be many, many new capabilities.  For example, custom digital catalogues would be easy to implement and could even have certain super-users designated to create and republish user generated content to the whole market.  In effect, you could launch an eco-system of experts around a topic in real time.  This is a new form of editorial, and context.  Forbes too has a recent article about why Google Wave is a big, big deal from a development platform standpoint. 

When will it all hit?  Who knows, because technology waves are idiosyncratic events.  No one predicted Twitter.  I imagine that some killer app - around gaming, investing, sports or some more base passion will show us the possibilities and we will then think our collaborative, post-keyboard, high resolution, continuous context interaction with the digital world was always such.  The greatest technologies go from amazing to invisible in the shortest amount of time. 

Is “The Gig” The Right Way to Design Creative Knowledge Work?

Designing Promotions in the Internet Age: Groupon’s Five Key Lessons for All Companies