Instinct user says the AI assistant surfaced a stranger's financial document

Why it matters

Personal agents depend on broad access to private accounts. Instinct's screenshots show how a routing failure or fabricated diagnosis can make that access dangerous.

Noah Shinn (@noahrshinn)'s AI assistant Instinct surfaced details from a financial document that user Pritak (@prit4k) says did not belong to him, according to screenshots posted on X on September 21st.

The exchange presents two materially different failure modes. Instinct may have delivered another person's document or extracted details into Pritak's conversation. Instinct also may have hallucinated the document, its contents and a confident explanation of how the supposed mix-up occurred. The screenshots do not establish which happened.

Pritak asked Instinct whose name appeared on what the assistant described as a Gerber claim document. Instinct responded with Pritak's name, a middle name and an address. Pritak said the middle name was wrong, he had not lived at the address during the relevant period and he had never sent Instinct a photo.

Instinct then claimed it had checked its records and found that Pritak's message contained only text. It said a Gerber image had crossed into his conversation "somewhere on the delivery side" and accepted responsibility for using it. Instinct added that nothing had been filed, said it would abandon the $574 claim and offered to report the incident.

When Pritak accepted that offer, Instinct drafted a report saying a photo the user never sent had appeared in one of his text conversations. The draft characterized the image as a financial document containing a stranger's personal details and concluded that a private document had reached the wrong conversation.

That language sounds like an incident finding. It is still an AI-generated response. Instinct's terms of service warn that outputs may contain incorrect or incomplete information and that records of actions available through Instinct may be inaccurate. A model claiming to have inspected its own delivery records does not establish that those records exist or that the model accurately interpreted them.

Other users described separate reliability and context-isolation concerns in replies, though none independently establishes what happened in Pritak's case. One user, @peepeedog, said that across 29 questions Instinct was "wrong or misleading" 10 times and apologized or walked back an answer 11 times. A reply from @SSShken challenged the methodology, asking whether competing assistants received the same questions in the same order and whether the walk-backs overlapped with the incorrect answers.

In another thread, Clayton Bradshaw replied that a similar episode had "happened to me too, then Instinct denied it." The supplied post does not include enough detail to determine whether Bradshaw encountered leaked context, a hallucination or another failure. The ambiguity itself drove the replies. One user wrote that "Data leak or hallucination?" was "not a reassuring multiple-choice question". Another warned that the reports made it look as though "user context is not user specific", while a third speculated about "mishandling of per-user file workspaces". Those are user interpretations, not confirmed diagnoses of Instinct's architecture.

An actual cross-user document would be a serious security failure because Instinct is built around access to unusually sensitive context. Instinct's privacy policy says the assistant may process documents, private messages, emails, account credentials, payment information and health-related information when users grant the corresponding permissions. Instinct's terms also authorize the service to access, copy, collect and index information from connected accounts.

A security question at the center of Instinct's product

Shinn, a former research scientist at Sierra whose earlier work included machine-learning and programming-language research at Northeastern and MIT, built Instinct to act through the applications people already use. Users can text or call the assistant, connect outside accounts and authorize it to complete tasks ranging from reservations to purchases.

That design gives Instinct its appeal and expands the consequences of routing mistakes. A conventional chatbot can produce a wrong answer. An assistant with access to inboxes, documents, passwords and payment methods can expose information or act on an invented premise before a user recognizes the error.

Instinct had already faced scrutiny over its data controls before Pritak posted the screenshots. TechCrunch reported on August 24th that beta testers had raised concerns about retained indexed email, deletion controls and actions taken without confirmation. Instinct revised its terms and privacy policy on August 26th. The current documents separate disconnecting an integration from deleting data previously indexed from it.

On August 26th, TechCrunch reported that Instinct had raised a $250 million Series B co-led by Index Ventures and Benchmark, bringing its reported funding to $350 million at a $2.5 billion valuation. That financing placed institutional weight behind Shinn's bet that consumers will give an autonomous assistant broad access to their digital lives.

Pritak's screenshots expose the control problem underneath that bet. If another user's image entered his conversation, Instinct needs isolation strong enough to keep private context inside the correct account and thread. If the image never existed, Instinct needs to stop presenting fabricated internal diagnoses as confirmed system events. The additional user reports do not resolve which failure occurred, but they sharpen the same immediate concern: Instinct can confidently describe private or supposedly internal context that users say is not theirs.