My Involvement – Lead UI Designer, UX Researcher, Lead Visual Designer
Date Completed – Late 2016
Below is a great example of the how of Engineers use Protospace as a tool to help examine spacecrafts before they are built
The Challenge: What can hierarchy look and feel like in ProtoSpace?
Use Case: A group of engineers
For this project, our team was given two challenges from the JPL client team:
– Design a way to navigate through CAD model file hierarchy in augmented reality
– Figure out how to communicate the design to the ProtoSpace team.
When tasked with finding a solution to this problem I worked with a team of six, which consisted of a budding VR/AR expert (Yun), a software engineer (Joe), a psychologist (Lexus), and two marketing/communications professionals (Rose and Najma). I was mainly involved in the UX research and was responsible for the visual design of the user interface.
Because of the short, six-week time frame of the project, our team quickly cycled through various ideas, prototypes, and conducted several rounds of user testing before moving forward with our final prototype. The final prototype was demoed to Marijke and Matt from the JPL team and received positive feedback along with constructive criticism.
Hardware: Microsoft Hololens
Interaction and Gesture Controls: Hololens
Double air-tap: Same position and movement as the air tap but twice. This is a quick motion like double-clicking. This is used in ProtoSpace to open and close the models
Air-tap and drag: Air-tap but instead of lifting your finger back up you keep it down and drag it. This gesture is used for things like scrubbing through animations or drawing.
Gaze ray: Your head movement in ProtoSpace is marked by a small white dot inside the program. This is called your gaze ray. Your gaze ray acts as a cursor that is controlled by the movement of your head. To select things in Hololens, move your gaze ray to the thing you want to select and use the air-tap gesture to select it.
Clicker: Hololens ships with a handheld device that can replace the air-tap and double air-tap gesture. Like the air-tap gesture this tool works in conjunction with the gaze ray. Future versions of the clicker may include the ability to click and drag.
Protospace Features: Toolbar
Select Tool: Users can drop a shared “Point of Interest” indicator on a part of the spacecraft and hide and show selected parts.
Translation Tool: Users can move the spacecraft.
Scale Tool: Users can change the size of the spacecraft.
Rotate Tool: Users can rotate the spacecraft
Cut Plane: Users can manipulate a transparent plane to view select portions of the spacecraft.
Animation Tool: Users can play, loop, and scrub animated CAD models.
Manipulation Tool: Users can select and manipulate individual pieces of the spacecraft with 6 degrees of freedom.
During this early stage in our process, our team felt like there were too many missing pieces for us to understand the workflow of mechanical engineers. To change this situation, the team proceeded to conduct interviews with engineers who are familiar with various 3D CAD software. So we conducted interviews with NYU robotics engineers who are a part of the Poly Lunabots. The robotic engineers showed us how they use various 3D CAD software (mainly Solidworks) by going through some basic workflows.
Through communicating with clients and conducting interviews, we collected the following insights:
1. Identified use cases: From our clients, we got that there were two typical use cases. The first was to use ProtoSpace to support a collaborative discussion about the model with colleagues. The second was to use ProtoSpace as a presentation tool.
2. Identified users’ habits with CAD software: Observing users’ workflow helped us a lot in terms of understanding how the Hololens and the Protospace application could help improve such experiences.
After the first round of user research, we narrowed down the design challenge into the following pain points that we needed to address:
How can we quickly display parent-child relationships in the file hierarchy?
How is lengthy text displayed in 3D space?
What movements or gestures do users use to navigate the interface?
Ideation: Exploring Design Concepts
All initial prototyping for our concepts was done with paper and cardboard in order to get a feel for how users would interact with it in 3d space. Building low-fi prototypes allowed us to save time by quickly cycling through multiple iterations before deciding on a concept we felt more confident in moving forward with.
Relatively appropriate distance and size
More in line with the use scenario of ProtoSpace
A visual representation of how an engineer would use the Totem feature. The storyboard below outlines a few of the basic functionalities of the Totem.
Through testing in HoloLens, we realized some other issues that came with AR UI design:
1. Interaction: The way to quickly jump between different hierarchy and the visibility of the system received positive feedback. Users could easily understand how to use the interface. However, users reported that using “click and drag” to scrub through objects could easily become a tiring gesture.
2. Visual: Users noticed that sometimes the UI would blend into the environment. This reminded us to be more careful of the typography, color, and transparency used in UI elements.
2. Sustainability/Scalability for extreme usage: We have to be more mindful of designing a more sustainable UI solution for more extreme users. i.e., when the totem expands to a point where the design can poentionally become severely compromised.
In our next step, we plan to explore deeper in the above areas to iterate a more efficient solution.
UX Research Rose Bender, Joe Besilin, Lexus Aboloh, Yun Li
Unity Developer Yun Li
UI/UX Designer Ariana Ijaduola
Visual Designer Ariana Ijaduola
Client Designer NASA (Marijke Jorritsma and Matthew Clausen)
– NYU Tandon School of Engineering
– Nature World News