Cortana Natural Language Tasks
Design Problem
How do we evolve current Natural Language tasks and create a framework that supports new tasks?
Role
Full design ownership for porting old experiences, creating a new visual style, and flows for new features. On some projects, full voice design ownership (flow, writing, GUI in coordination with voice models) as well.
Investigations were made into how existing tasks could be more personalized, and a new visual model was made in the short timeframe of our release date. Designs considered tap target affordability and multi-modal interactions that could switch from touch to voice and vice versa easily.
Results
This work has enabled various teams to work on Natural Language tasks for Cortana. The framework was designed to scale well across a large company, to afford the ability for Cortana to grow with relation to Natural Language tasks in various markets. Overall the framework I created scales to suit the markets and needs that our users have, as well as allowing for innovation as new aspects of the design are needed while Cortana’s capabilities continuously improve. Partner teams at Mirosoft are using this design work as guidance when designing for Cortana, enabling more efficient and productive collaborations.
Messaging Task
Natural Language Screen Types
Natural Language Confirmation UI
Cortana sends the user to another app
A summary of Natural Language task UI
Multi-Modal UI to support voice or touch disambiguation
A visual summary for partners to understand how touch UI leads to different pickers
After each task where Cortana takes an action on the user's behalf, a Rollup like this one is shown to confirm with the user what has been done.
A single turn task where Cortana does not require follow up questions. Here, the user said "Open Excel"
Task Alert Solutions:
​
Cross-platform and proactive / reactive interface allows users to set reminders and alarms and be notified with system -level alerts from Cortana.
​
OS-level consistency combines with Cortana branding to provide consistent experiences.
​
On the right, the Desktop that a Reminder is set on may differ from the device an alert fires on.
See how thousands of companies are
using this framework as Cortana Skills
As the designer for Cortana Skills, I met with various third parties including Capital One, Expedia, and OpenTable to deliver UI solutions that fit their individually complex scenarios. The result was a suite of Skills promoting the use of Cortana.