Resume upon request
Resume upon request
Live Captions
Ideated the intial experience and framework for Windows 10 before it became a key accessibility feature in Windows 11.
Role: Design Intern
Click on the thumbnail to watch the full video or go to
How can we design for people who are hard of hearing and extend to everyone that loves Windows? What kind of features would be useful for live captioning to include?
The goal was to find a solution that was effortless, adaptable, and customizable.
Painpoints: People who are hard of hearing have to set up multiple screens to stay productive while multitasking and following along in meetings through captions. Captioning offered by apps stay inside the app frame; this means they can be blocked off by other windows and apps obstructing captions.
Live Captions take audio input and turns it into text. It always stays on top of whatever you're doing, whichever app you are in, allowing you to follow along while multitasking on other things. I designed the foundation of what live captions looks today and hasn't deviated too far from what I had proposed.
I started by brainstorming all the different form factors it could appear in, maybe it can be snapped to the bottom, to the top, only show one line or be stretched to show paragraphs. I initally proposed four different orientations that live captions could appear in which I thought would have various uses. Through iteration and feedback, we clarified and simplified the idea further, so that it is still a powerful tool that allows for a lot of customization for how people want to use it, but uncomplicates how they get there. The two styles, docked and undocked are also familiar gestures for anyone who has used the handwriting panel and touch keyboard.
In the Press
Windows 11 Is Getting Live Captions for All Audio Content
Windows 11 Insider Preview Build 22557 is here, and it’s a huge one… One of the more exciting features is live captions, which lets you see captions for any spoken content played on your computer. The captions are generated on the local computer, which means you won’t have to send what you’re watching to Microsoft’s servers or anything like that. This lets you maintain your privacy while taking advantage of this helpful feature.
Microsoft is testing one of Android's best accessibility features in Windows 11 …Google introduced Live Caption functionality to Android back in 2019, allowing users to get locally generated captions for any video and audio content… Now, Microsoft has announced that a new Windows 11 Insider preview build (version 22557) comes with a so-called Live Captions feature as well… The Windows maker recently said that some features tested in preview builds wouldn’t actually come to stable releases. But we do hope that Live Captions in particular come to stable Windows 11, as it’s an invaluable tool for the hearing-impaired and can be pretty convenient for others too. |
Live Captions will soon be built into Windows 11, accessible from a Winkey + CTRL + L shortcut. Live Captions will greatly benefit people who are deaf or hard of hearing, but they’re also available for anyone to use and will be automatically generated on-device from any content that includes audio… This looks like a great accessibility improvement to Windows 11. |
Windows 11 to Gain Start Menu Folders, Live Captions for Videos …Of course, these new features aren’t as impressive as live captions, a new accessibility tool that uses voice-to-text technology to automatically draw captions over videos. Live captions are processed on your device and can snap to the bottom or top of your display (they can also float in a window). And interestingly, you can use the live caption keyboard shortcut (WIN + Ctrl + L) without ever enabling the feature in Accessibility. |
Microsoft brings Live Captions, Start Menu folders, Do not disturb, and more to Windows 11 Dev Channel
Microsoft is bringing support for Live Captions which will automatically generate captions from any applications on your device that’s playing audio including video players or web browsers. Captions are generated on-device without sending any data to the cloud… A similar feature has been available via the Google Chrome web browser since last year, but building the feature into the operating system means that it will be available for more apps.
Windows 11 can now automatically generate Live Captions for any audio
Among the huge number of new features, Windows 11 build 22557 brought the ability to create Live Captions, similar to how Chrome does. The new Live Captions feature supports any audio source with text, even user's dictation via the connected microphone… Windows 11 will generate subtitles automatically for any content with sound and speech. It works even when your device is not connected to the Internet.
Captioned content appears on virtually all streaming services, including Netflix and YouTube, among others. In Build 22557, Windows itself generates the captions, auto-generating them on any content with audio. That means that the captions can be auto-generated with just about any content, even videos stored in your Videos folder.
A big new feature coming to this build is Live Captions, similar to those from Google on Android and Chrome. Windows’ Live Captions provide captions for spoken content from any sound source on the PC, including microphones.
For people with disabilities, Microsoft is finally adding support for live captions, which should help users better understand audio. As the name suggests, the Live Captions feature would display captions from any content with audio. |