Mentioned before in this blog and elsewhere, but here is some footage of my mixed reality painting prototype. This project was created in order to facillitate physical therapy for stroke patients with recovering upper arm difficulties.

So far this demonstration has been part of ISMAR 2009 and Otronicon 2010. The focus of this project is on making therapy a less difficult and tedious experience through enjoyable virtual activities. It is designed for three different skill levels based on the mobility of the patient. This was a challenge because at the lowest level patients may have essentially a frozen limb that can barely move millimeters.

Some specific features of this system:

  • 3d stereo Using the same technology in current movie theaters, we project the virtual world onto the real space in 3D.
  • Head Tracking The stereo glasses are tracked so that the user’s head movements translate the view of the virtual world, allowing them to move and change brush color or look behind the canvas at the fruit bowl they are painting.
  • Many Activities For a variety in context we designed nine distinct games that may appeal to different people.
  • Same Gameplay The main purpose of this system is to ease the burden of physical therapy drills, so despite the contextual difference in gameplay between activities, the controls are almost the same.
  • Passive Haptics The feel of the real paintbrush against the clear acrylic pane gives the user the tactile sensation of painting.

Work on this project is on going as we wrap up the software and plan future user testing. This project was made possible by Emiko Charbonneau, Steven Braeger, Daniel Mapes, Eileen Smith, and Charles E. Hughes.

Advertisement