Blackbox Blast
BB Preflight
← Incident log

CASE · Major · other

Unverified reportCollected from public reports

Breach at AI infrastructure vendor Fireworks AI exposed Notion AI chat transcripts in some workspaces (unverified report)

According to BleepingComputer, Notion disclosed that a breach at its AI infrastructure provider, Fireworks AI, exposed Notion AI chat transcripts in a small number of workspaces. Notion received an alert from Fireworks on November 2 about an exposed dataset said to contain only synthetic test data, but on November 8 Fireworks confirmed that customer data had been accessed.

3 min read

OBSERVED
CATEGORY
other
BLAST RADIUS
data
DETECTED BY
vendor-alert
TAGS
#notion #notion-ai #fireworks-ai #third-party-breach #ai-infrastructure #chat-transcripts #insights

CH 1

Cause

Per the article, an attacker reached Fireworks AI's development environment through a third-party cloud provider on October 23. What was exposed was a dataset of chat transcripts analyzed for Notion's Insights feature. The article does not say how the transcripts ended up in that dataset.

CH 2

Consequence

According to Notion, less than 1% of its users are affected. For the vast majority of impacted workspaces the transcripts were unattributable, meaning they were not labeled with who wrote them or which workspace they came from, and they did not contain tool call results. Notion still advises users to rotate any passwords, API keys, or other credentials entered into affected chats.

CH 3

Fix

By November 3, Notion had cut off traffic to Fireworks, deleted all data from the vendor's systems, and halted training runs. The Insights dataset has been disabled. Owners of potentially affected workspaces can review the data under Security incident review in the desktop web interface and share a link that lets members download their own transcripts. Users who have received no notice need no action for now.

What happened

According to BleepingComputer, Notion disclosed that a breach at its AI infrastructure provider, Fireworks AI, exposed Notion AI chat transcripts in some workspaces. Fireworks runs some of the open-weight models used in Notion AI and also analyzes stored conversations for a feature called Insights.

Insights gives workspace administrators visibility into how their teams use AI. The exposed transcripts came from the Insights dataset, and the feature has since been disabled.

The chaos on the ground

Notion learned of the problem on November 2, when Fireworks sent an alert about an exposed dataset. At that point the dataset was described as containing only synthetic test data.

By November 3, Notion had cut off traffic to Fireworks, deleted all data from the vendor's systems, and halted training runs. On November 8, however, Fireworks confirmed that customer data had been accessed, including that of Notion users.

When Notion analyzed the dataset it received that day, it found Notion AI chat transcripts used for Insights. According to Notion, for the vast majority of impacted workspaces the transcripts were unattributable: not labeled with who wrote them or which workspace they came from. They also did not contain tool call results.

Root cause

Per the article, an attacker accessed Fireworks AI's development environment through a third-party cloud provider on October 23. Resource metadata, billing information, model weights, and datasets were compromised for some customers.

The article separately reports an earlier Fireworks breach in early October, in which attackers accessed environment variables containing API keys and tokens stored as secrets for a limited number of accounts.

In its FAQ page, Notion stresses that Fireworks had contractually committed to zero data retention for Notion data. The article does not explain why the transcripts were nonetheless in the dataset.

The fix

Notion says less than 1% of its users are affected. Owners of potentially affected workspaces can review the data under Security incident review in the desktop web interface and share a link that lets members download their own transcripts.

Notion advises affected users to rotate any passwords, API keys, or other credentials entered into the chats. If neither a user nor their workspace owner has received a notice, Notion says they are most likely not in the affected subset and no action is needed for now.

The lesson is that credentials entered into AI chats need to be rotated in case the transcripts leave the vendor's hands. When conversations are handed to a vendor's AI infrastructure, a breach there can expose your own users' data.

Sources

PREFLIGHTIs your own Mac prepared?BB Preflight is a free Mac tool that checks 21 things, from AI agent permissions to where your keys live, without changing anything.Run the check →