PRODUCT: A mapping of behaviors, use scenarios, and a framework to ensure META's AR wearables and smart devices are welcomed as valued partners in human social interactions.
ROLE: Inspired by the issues with Audio AR wearables, I initiated and led the development of this framework, partnering with the Meta AI Research team, contractors, and hardware developers for feasibility assessment, prototyping, and definition of use cases.
IMPACT: By using the framework and feasibility assessment, the team was able to prioritize and plan the implementation of use cases based on social intelligence's current and future sophistication. Video scenarios helped leadership visualize the potential of different investments and make decisions around form factor, feature set, and roadmap. The scenarios still inform the development of the MR assistant.
Designing the relationship between AI and people
The new generation of devices we will be introducing in our homes and on our bodies need to dynamically know when to reach, what to share, and how to present content. Social Intelligence is the design of a relationship between AI and people with the purpose of connecting you with the people, communities, and the world around you in a deeper and more meaningful way.
Social Intelligence would work as the common foundation among products and new projects, unifying learnings and providing a cohesive and meaningful experience.
This relationship needs to be grounded on trust and clear communication. It is humans and computers working together to reach shared goals.
Problems & user scenarios
A person's mood, receptivity, cognitive capacity, and energy levels can vary greatly throughout the day, the week, and throughout their life, and these variations are greatly impacted by factors such as the presence of others, the current activity, and past events.
However, current technology often ignores these cues when deciding how to interact with users. Smart devices and AR wearables, in particular, can be disruptive if unaware of what is happening. To be accepted and integrated seamlessly into daily life, they must be designed to be unobtrusive and respectful of the user's current state and context.
To address this challenge, we conducted research based on user feedback and explored a range of scenarios to gain a deeper understanding of the need and utility of Social Intelligence in smart devices and AR wearables. We divided our findings into two sections: situations for smart devices placed in the home, and situations for wearables worn on-the-go or in public places.
Social Intelligence for smart devices at home
Examples of social situations at home that are currently challenging for Meta Portal:
Social Intelligence for wearables devices
We created several video scenarios for the brooch form factor of the Meta's AR Audio wearable. They mostly also apply to AR Glasses. The situations are grouped into Social Intelligence value propositions.
Understand which information is more meaningful to the recipient:
Be aware of when and how it is appropriate to reach someone:
Infer what kind of suggestion is expected at a given moment:
Define the appropriate level of information to share:
Identify which social hints would help the recipient better navigate the current situation:
Adapt messages based on how much complexity the recipient can handle at the current moment:
Frameworks & Diagrams
Context understanding has been an elusive goal of UX and AI for a long time. For this exploration, we created frameworks and system maps to help us visualize the different steps and modules of the process so that we can tackle specific areas in parallel.
“...in the literature, people talk about an orchestration needed — I like that metaphor of the orchestra — that the technology needs a conductor to help make the beautiful symphony of a social situation happen.”
Meta AI Researcher
Meta AI Researcher