Personal Digital Augmentation

Latest revision as of 03:28, 10 November 2009

Requirements

My goal is to design a maker-built device (a Personal Digital Augmentation (PDA) device) that would replace an iPhone — something that would induce geek envy in an iPhone user, and eliminate the need to own one. It would, of course, use open source hardware and software, and makers would customize the hardware and software to suit their needs. Like the RepRap project, there would be a public reference design, with detailed build instructions. There would be enough standardization of the interfaces so that we could have a community repository of software and hardware components that are compatible with the platform. The more people build one of these, the more benefits there are.

I want this to be an "elite" device, with capabilities that obviously go beyond mass market handheld devices like the iPod nano and iPhone. The cooler it is, the more motivation there is to build one. So that's a requirement — what design elements are needed to meet this requirement?

What are the weaknesses of an iPhone?

• Text entry is painful.
• You need to pull it out of your pocket, turn it on, and navigate to the right application before you can use it.
• The multitouch interface is beautiful and addictive, but it consumes all of your visual attention to use it, more so than any other interface I've used. These facts might be related. But I want a device that can quickly respond to my needs with very little distraction or cognitive load. I'd like to be able to perform other tasks while using it. A very simple example is that with an iPod nano, you can easily advance to the next song without distracting yourself from another task, like driving or exercising, but you can't do this with an iphone.

Here are some problems with simply cloning an iPhone:

• Lameness: a maker-built iPhone clone will be thicker and clunkier than a real iPhone, and will suffer by comparison.
• Fitness for purpose: see iphone weaknesses, above.

Here are some alternatives to the iphone:

• A Steve Mann style wearable computer, with an eyetap and a septambic keyer as i/o devices.
• A "sixth sense" device: http://www.ted.com/index.php/talks/pattie_maes_demos_the_sixth_sense.html
  This avoids the "gargoyle" effect, or the socially repellent miasma that is emanated by crudely built head-mounted displays.

Alas, there are inevitably tradeoffs to be made. A possible out is to design a PDA platform that supports a range of input and output devices.

The project needs a more unique, catchier project name. I've considered "OH/M" (One Handheld per Maker), after the OLPC, except I don't know yet if this device will ultimate be a "handheld".

I want to base it on existing work as much as possible. In particular, if someone has already designed an open source "PDA" that meets all my requirements, I'll just use this design.

Here are some of the functions that the platform should support:

• listening to music
• listening to (and maybe recording) FM radio
• still camera
  • geotag your photos using GPS
• video camera
• audio recording / voice notes / voice recognition
• remote control
• augmented reality
  • the eyetap and sixth sense videos have some nice examples
• games
• social networking (in the broadest possible sense)
  • eg, ad hoc mesh networking with other PDAs at a maker gathering
  • eg, augmented reality multiplayer gaming with other PDA users
• email, web surfing
• telephone
• note taking
• calendar, watch, stopwatch, alerts
• usb drive / flash drive / bulk data storage and retrieval

How does text entry work?

• Could use a multi-touch display, like iPhone. It's slow, error prone (tried typing a password?) and you can't touch-type.
• Blackberry style thumb keyboard.
• Palm style Graffiti input, using stylus
• Chord keyboard. Potentially the fastest input method (stenotype is faster than qwerty), and allows touch typing. Also very "elite" and hackish.
  • spiffchorder: http://chorder.cs.vassar.edu/spiffchorder/forside
  • wearcam septambic keyer: http://wearcam.org/septambi/
• Data glove: can you enter text with a data glove? Efficiently? But a data glove is potentially useful for augmented reality, and of course VR.
  • too expensive: http://www.cyberglovesystems.com/products/hardware/cyberglove.php
  • flexion sensor for building a dataglove: http://infusionsystems.com/catalog/product_info.php/products_id/140
• sixth sense style projected keyboard + video analysis of hand gestures.

Even if we don't have a fully realized chord keyboard, there needs to be a way to provide input to your PDA using buttons, which doesn't consume visual attention. (I guess voice recognition via the microphone could also work.)

How does the display work?

• handheld LCD display, like iPhone
• head mounted display
  • with integrated camera, like eyetap: http://en.wikipedia.org/wiki/Eyetap
    This is a brilliant design for hands-free operation and augmented reality applications. Also, very elite and hackish, you can't buy an eyetap commercially.
• sixth sense wearable projector

I guess I'm currently attracted to the eyetap / chord keyboard combination. (The eyetap has the best augmented reality integration, although I worry about the gargoyle effect.) But I have no experience using such a system, so I don't know in detail the reasons why such a system may suck. (I do know what I don't like about the iPhone.) Steve Mann teaches a course at U of T on how to build an eyetap + keyer.

The PDA must be position and orientation aware, in order to fully support augmented reality.

• need GPS
• need a compass and orientation sensors
  • http://infusionsystems.com/catalog/product_info.php/products_id/157

a Wearable PDA design

It has 3 parts.

1. The 'head' part consists of an eyetap (head mounted display and video camera combo), plus an earphone/microphone. plus a compass, so that the PDA knows the compass direction in which you are currently looking (for augmented reality). Plus a Zigbee or BlueTooth link to the 'base' part.

2. The 'hand' part consists of a septambic keyer, plus additional functionality that provides 'mouse' functionality, which could be a thumb trackball, or a 3D position sensor (relative to the base) so you move your hand around in space like a WiiMote to provide mouse movements. Plus radio link to the base. Plus maybe infrared for controlling a TV.

3. The 'base' part has an ARM computer, the clock, the mass storage (flash or a mini hard drive), the cell phone and wifi and gps and fm radios. Plus the PAN radio for connecting the head and hand components (bluetooth or zigbee).

I don't want the head and hand parts to be physically wired to the base. I want to be able to quickly remove them and shove them in a coat pocket, in social situations. Hence the PAN.

According to the web, a ZigBee web cam was observed to be painfully slow, due to the low bit rate of ZigBee (max 250 kbps). So I guess we'll need blue tooth for the head mounted video. I'll focus on a BlueTooth based PAN.

The benefits of using a PAN for connecting components is that the whole system is easily expandable with a range of i/o devices, to suit different use cases and personal tastes. And with BlueTooth, there's lots of stuff available off-the-shelf:

• bluetooth cameras from kodak and samsung
• foldable keyboards, mice
• head sets, of course