Apple Watch’s AI features for rewinding and recapping conversations begin beta testing

The sound recognition feature will help deaf and hard-of-hearing users identify important sounds like sirens, alarms, or doorbells from their watch, while the song recognition feature taps into Apple’s Shazam app to detect music playing in the background and identify the song name and artist.

However, the more potentially controversial features are those that listen to human conversations: Live Rewind and Siri Recaps.

Given the backlash to AI recording devices, like Meta’s AI glasses and other AI wearables, Apple has attempted to carve out a space where the features can be useful to Apple Watch owners, while still respecting the privacy of those around them.

Apple says that users will also be able to tap an “Ask Siri” option that appears below the text snippet if they have questions about something that was said — like a technical term they don’t understand, or perhaps some sauce option the waiter mentioned that they’re unfamiliar with, among other things.

When you double-press the digital crown, an audible chime appears and an animation displays on the watch. This is meant to signal to people through the combination of the gesture, sound, and the visual display that you’ve recorded something that was said.

These snippets will be deleted automatically after 30 seconds if you don’t save them.

The other option, Siri Recaps, takes this ambient listening technology even further by allowing Apple Watch owners to take high-level notes of their conversations.

Importantly, Apple says it’s not identifying speakers, saving the raw audio, or providing word-for-word transcriptions. Instead, the feature summarizes a conversation with a short sentence and some bullet points that identify key points. For instance, depending on the topic, the bullet points might include things like important dates or deadlines, pricing or payment plan information, and other factual information.