6th May 2023
How might we...
...encourage customers to consent to providing facial data when they don’t necessarily trust a digital avatar?
[XR] Showcasing the Value of Realistic Avatars
This pattern is a co-created output a Design Jam which took place in Singapore on the subject of Data Transparency and Control in the Metaverse. During the Design Jam, six design teams created simple prototypes which helped deliver better transparency and control over data use in fictional services.
Singapore Airlines is Singapore’s national carrier and one of the world’s leading airlines. In the fourth season of the Singapore Accelerator, an ongoing research collaboration between the TTC Labs team at Meta and the Singapore Infocomm Media Development Authority (IMDA) and Personal Data Protection Commission (PDPC), Singapore Airlines explored ways that emerging XR technology can enhance the customer experience – particularly the value of sharing voice and facial data in customer service contexts.
Problem & Opportunity
Singapore Airlines explored the potential for assessing and responding to people’s physical characteristics, such as their voice and facial expressions, to create positive customer experiences. People are not always aware of the role their emotional state plays in service interactions, especially when it comes to unfamiliar XR technologies and data collection of their physical characteristics such as voice and facial expressions.
Before people are willing to use customer support services in XR, they must be confident of the value that data sharing can bring to their experience.
Data being used
- Facial Detection
- Facial Landmarks / Mapping
- Voice
Key Features of Prototype
For their prototype, Singapore Airlines considered the deployment of a digital concierge that could respond to customer needs in real-time. In their use case, a service disruption has resulted in long queues to speak to customer support. Travelers are encouraged to use the digital concierge to seek assistance.
Enhanced Digital Concierge
The enhanced concierge uses voice and facial characteristics to infer the emotions of the customer, providing it with the ability to respond in a more timely and empathetic manner. However, if a user does not wish to share their data, they can continue using the standard concierge, which only utilizes voice and camera data.
Concierge Demo
A key feature of Singapore Airlines’ prototype is the ability for people to preview the enhanced concierge in order to make a decision about whether or not they would like to share their body-based data. If someone decides to demo the enhanced concierge, they are shown a scenario highlighting the benefits of emotional recognition. They can then decide if they would like to opt in to the enhanced version or continue using the standard concierge.
Opt-In Only
To ensure that personal data will not be infringed, Singapore Airlines adopted a strict opt-in only approach where user data would only be captured and processed with people’s consent. This placed an important emphasis on demonstrating the value of sharing body-based data, such as voice and facial data, and building people’s trust.
Design Pattern: Preview Slider
Singapore Airlines used a variety of XR prototyping techniques combining data rationalization, sketching and storyboarding to bring their solution to life. The resulting prototype and those of the other participating companies were later synthesized to form a collection of privacy UX design patterns.
The Preview Slider design pattern was developed from Singapore Airlines’ prototype.
This UX pattern allows people to see the impact of different data types on their avatar’s appearance before they start an experience.
Similar to a Zoom or Microsoft Teams preview window, the pattern shows someone how they outwardly appear to others — moving from low fidelity (no data sharing) to high fidelity (more extensive data sharing). This fidelity reflects the information that is available to other users within an experience, as represented by the appearance of their avatar. It also reflects what the system can infer about them, such as interest and emotional state (derived from facial data and voice analysis). In doing so, the Preview Slider helps to demystify the use of body-based data, highlighting its value in a visual and interactive way.