After brutal privacy critiques, Microsoft Recall will get major privacy and security changes

Last month, with great fanfare, Microsoft announced its new line of Copilot+ PCs, with a signature AI-powered feature called Microsoft Recall. That feature is intended to make life easier for hundreds of millions of customers worldwide. But the company forgot to factor in the privacy risks associated with that product’s data collection practices or the ingenuity of hackers and security professionals, ethical or otherwise.

The result was a flood of criticism about the feature and its risks. Security professionals who were able to enable the feature in advance of its release put together detailed critiques of its design, with Kevin Beaumont delivering probably the most brutal assessment:

I think it’s an interesting entirely, really optional feature with a niche initial user base that would require incredibly careful communication, cybersecurity, engineering and implementation. Copilot+ Recall doesn’t have these. The work hasn’t been done properly to package it together, clearly.

[…]

I think they are probably going to set fire to the entire Copilot brand due to how poorly this has been implemented and rolled out. It’s an act of self harm at Microsoft in the name of AI, and by proxy real customer harm.

Apparently that critique triggered a five-alarm fire drill in Microsoft, with the company announcing some significant changes to the feature. In a blog post blandly titled “Update on the Recall preview feature for Copilot+ PCs,” Microsoft Corporate VP Pvan Davuluri, who runs the Windows + Devices division, acknowledged the criticism: “[W]e have heard a clear signal that we can make it easier for people to choose to enable Recall on their Copilot+ PC and improve privacy and security safeguards. With that in mind we are announcing updates that will go into effect before Recall (preview) ships to customers on June 18.”

According to the blog post, the following changes will be implemented in the released version of the feature:

• The setup experience of Copilot+ PCs will offer “a clearer choice to opt-in to saving snapshots using Recall.” The feature will be off by default and will only be enabled if the user chooses to enable it. (In my earlier post, I had suggested this as a crucial change.)
• Enabling the feature will require Windows Hello enrollment, with secure proof of the user’s identity, typically via biometrics. In addition, Microsoft says, “proof of presence” will also be required to view the Recall timeline and search its contents.
• As for the database itself, it will get an extra layer of data protection, including “just in time” decryption, which Microsoft says will be protected by Windows Hello Enhanced Sign-in Security (ESS). The search index database will be encrypted as well.

That last change is probably the most interesting one and should prevent some of the more dire scenarios that security researchers have warned about. The “just in time” decryption feature means that Recall snapshots will be protected by a second layer of encryption, using Windows Hello Enhanced Sign-in Security (ESS). Even if an attacker is able to access the user’s database, they won’t be able to decrypt and access its contents unless they can provide secure authentication on a device that’s enabled for that user with Windows Hello.

This is a developing story. I’ll update when I have more details.