Mixed reality (MR) is an augmented reality (AR) approach where three-dimensional holograms are placed in the real physical environment. Viewed through a head-mounted transparent display, these ghost-like holographic elements blend with reality. You can reach out to them and interact with them using gestures. Because you can see the physical and virtual world in parallel, negative aspects of conventional virtual reality devices are overcome. The eerie sense of isolation and disorientation that enclosed headsets have caused disappears, and the viewer is no longer alone. The technology for this exists: teams can use the first available MR headset, Microsoft's HoloLens, to work together in holographic interface settings that blend with the tactile aspects of physical environments. This is a sea change for interaction designers: visual displays are no longer confined to small rectangular displays that are two-dimensional. The new visual interfaces in MR are spatial, borderless, anchored, and contextualized in real environments, and they can be combined with existing physical control elements.
This past year, students in the Interaction Design (IxD) Program have explored the new possibilities of holographic interactions with a number of projects:
Associate Professor Axel Roesler worked with a team of nine students on a $120K research grant with Honeywell Aerospace to explore the new interaction concepts for applications in the commercial aircraft context. In collaboration with scientists and pilots at Honeywell, Roesler's team developed new holographic aviation displays for the commercial flight deck, new AR support concepts for pilot training, and MR applications for airplane inspection and maintenance. This work was related to a previous design research project with Boeing. Roesler and his team presented a paper on new interaction design techniques for next generation commercial flight decks in June at the Designing Interactive Systems Conference (DIS2017) in Edinburgh, Scotland. Read paper.
Roesler also led a team of four Division of Design seniors for a two-quarter capstone project in collaboration with NASA’s Jet Propulsion Laboratory. Dillon Baker, Allison Chan, Janice Kim, and Emily Pasternack developed a HoloLens-based MR collaboration platform for geologists and rover controllers to coordinate rock sample collection for the 2020 Mars Rover Mission. Watch video. Dillon Baker, who was instrumental in developing MR prototypes for both projects, recently joined Amazon's AR design team.
MR was also the theme for the Microsoft Design Expo 2017. Every year, Microsoft invites the world's top interaction design programs to Redmond to showcase exemplary projects and process. Among the six participating programs this year were Carnegie Mellon, MIT Media Lab, Art Center College Pasadena, and The Royal College of Art. IxD seniors Nick Allred, Dillon Baker, Kylen Dent, Jun Lee, Abe Poultridge, and Dwight Stoddard presented Trace, a MR tool suite that assists crime scene investigators in documenting and sharing evidence. The project was developed in a quarter long course, Roesler’s DESIGN 483 (Advanced Projects in Interaction Design), which was sponsored by Microsoft. Other projects that were developed in the course include Muse, a concept that generates holographic narratives as inspiration for writers, and Totem, a MR experience that associates shared memories with objects. During a week of Design Expo events, design studio tours, and presentations in July, UW IxD students had opportunities to network with the Microsoft Design community.
Recent Master of Design recipient Tate Strickland explored in his thesis, Envisioning the Museum Voice, how holographic annotations and verbal interpretations by art historians can augment the museum experience. After graduation, Tate relocated to Zurich, Switzerland, to join Google's voice interaction team.
There will be more projects like these. The Division of Design’s strong network and interdepartmental connections with human computer interaction groups on campus and in industry place our faculty and students in a unique position to explore the latest technological advances with cutting edge methods. Design at UW is rapidly shaping into one of the most exciting places for investigating the intersections between design, cultural relevance, new technologies, and social impact.
Help shape the future of interaction design by sponsoring a research project or donating. Start a discussion about sponsorship by contacting Merith Bennett. Choose from among these three funds for your gift: