What Is Dual Provenance? Verifiable AI Data Security for LLM Inputs and Outputs

Today's estimates put daily global data creation at roughly 400 million terabytes, and AI and LLM adoption keeps pushing that number higher. Data security itself is nothing new. It can be traced back to the first telegraph hack in 1834. But the current pace of data creation and exchange demands a new approach to protecting those bytes. AI security is a new field, born out of users who need frictionless access to their AI tools while keeping control, transparency, and security over the information LLMs take in. Dual provenance is built to do exactly that.
Why you can’t trust LLM outputs if you can’t verify the inputs
We've reached a point where real and fabricated content are hard to tell apart, for people and machines alike. That matters more every month as organizations now lean on language models for decisions that can move markets, affect patient outcomes, and bear on national security.
Most of the conversation about AI trust focuses on the model itself: how accurate it is, whether it hallucinates, and how it was aligned. But a model's answer depends just as much on what it's fed. That includes the data it was trained or fine-tuned on, the documents it retrieves when it answers, the memory it carries between sessions, and the instructions it receives. If any of those inputs are corrupted, the output will be too, and it will often look perfectly normal.
What is data poisoning in AI?
Data poisoning is the deliberate corruption of the data a model learns from or relies on. It shows up in a few places:
- Training and fine-tuning data. An attacker plants crafted examples that bias the model or create a hidden backdoor, where a specific trigger makes it behave the way the attacker wants. A 2025 study by Anthropic, the UK AI Security Institute, and the Alan Turing Institute found that about 250 malicious documents were enough to backdoor every model size they tested. A bigger model or a bigger dataset didn't raise the bar.
- Retrieved knowledge. Many enterprise AI systems use retrieval-augmented generation (RAG), which pulls documents from a knowledge base at answer time. Anyone who can slip content into that knowledge base, whether a shared drive, wiki, ticket system, or scraped web page, can steer answers without touching the model. The PoisonedRAG research (2024) showed that a handful of planted passages can reliably push a model toward attacker-chosen answers.
- Memory. Assistants that remember past conversations can be fed false "facts" that persist and quietly shape every session after.
What is prompt injection?
Prompt injection works differently. Instead of corrupting knowledge, the attacker hides instructions inside content the model reads, such as an email, a web page, or a document, and the model follows them as if they were legitimate commands. When those instructions arrive through retrieved or shared data (known as indirect prompt injection) the line between poisoning and injection disappears. The poisoned data is the attack.
The industry recognizes both threats. OWASP's 2025 Top 10 for LLM Applications ranks Prompt Injection first (LLM01) and lists Data and Model Poisoning as LLM04.
Why poisoned AI outputs are hard to detect
A poisoned system doesn't crash or throw an error. It gives confident, plausible, wrong answers, sometimes only when a specific trigger appears. By the time anyone notices, the output may already have shaped a trade, a diagnosis, or a briefing.
How sensitive data leaks out of LLMs
Poisoning is only half the risk. The same pipelines that let bad data in also carry sensitive data out. Every prompt, attached file, and retrieved document is data leaving your control and entering a system you may not fully govern. In regulated industries, that exposure can be as damaging as a poisoned answer.
That leaves organizations with questions they often can't answer today. What went into the model, and was it clean? What came out, and was it changed? And did anything sensitive end up somewhere it shouldn't be? Answering one without the others isn't enough.
What is dual provenance in AI?
Dual provenance is a verifiable record of both sides of every AI interaction: what data entered a large language model and what the model produced. Inputs are signed before they reach the LLM and outputs are signed after, and the signed receipts are chained together and stored on the organization’s own systems, so anyone with the right access can prove that sensitive inputs were protected and that outputs were not altered. Dual provenance is a capability of Karst_Fathom, Qanapi's AI and LLM security app.

AI use is spreading quickly across regulated public and private sectors, and with it the need for strong data security. Sensitive data flows in and out of LLMs every day, whether anyone intends it to or not. A clinician's assistant summarizes a chart containing protected health information (PHI). A defense contractor handles mission-critical data at several classification levels. A lawyer drafts briefs with help from AI. In each case the goal isn't just to reduce the chance of data spillage but to prevent it, because a single misrouted prompt can move data into a system that isn't cleared to hold it.
Making matters worse, most organizations have no easy way for an administrator to verify exactly what went into or came out of an LLM. They need a way to make sure the data entering a model is clean and verified, and that sensitive data stays protected even while the model works with it, so leaks, spillage, and accidents don't happen. And if an attacker does reach protected information or controlled unclassified information (CUI), all they should find is values they can't read.
That's what dual provenance delivers: end-to-end data lineage, full auditability, and protection against both compromised inputs and confidential data leaks. Karst_Fathom takes in the raw data, encrypts only the sensitive fields, and produces an LLM-optimized input file. The rest of the content stays readable, so the model can still do its job, and the original records are never altered. The encrypted values pass through the LLM untouched, and only authorized users can decrypt them in the output. Every step produces a tamper-evident receipt, giving you verifiable proof that your sensitive data was protected inside your own environment before it ever reached the LLM of your choice.
Why does AI need provenance?
Outside of technology, provenance is the paper trail behind an object: where it came from, who has handled it, and whether anyone tampered with it along the way. Applied to AI, that trail is the difference between an answer you can defend and one you hope is right. An output is only as trustworthy as the record of what produced it.
Dual provenance applies that paper trail to both ends of an AI system.
What is input provenance?
The first side is the inputs. Staying compliant often means tracking which prompts entered the model, who sent them, and how much sensitive text or how many sensitive documents they contained. That information should be captured at the door, down to each byte of data, along with the person and system that brought it there.
Karst_Fathom inspects every input before it reaches the model. It detects sensitive data, such as PHI or CUI, and dangerous content, such as material that could poison an answer or carry hidden instructions. It encrypts what it finds and records the event in a tamper-evident receipt. If something goes wrong later, that record lets your team trace exactly what entered the model, where it came from, and who sent it, and investigate from there.
What is output provenance?
The second side is the output. Protecting inputs only matters if you can also trust what comes back. Output provenance answers three questions: what did the model produce, when did it produce it, and which inputs was it based on?
- Linking outputs to inputs. Every response is tied back to the protected inputs that produced it. When a reviewer, auditor, or investigator asks why the model said something, there's a record that connects the answer to its source, instead of a gap between the prompt and the result.
- Proving nothing changed. A response that was accurate when it left the model is worth very little if it was edited on the way into a court filing, a patient record, or an intelligence briefing. Changes can be accidental, like a copy-paste error, or deliberate, like someone altering a finding after the fact. Because the output is recorded in a tamper-evident receipt, any later change to it can be detected. And because receipts can be verified offline, that check doesn't depend on reaching a central server, which matters in disconnected or classified environments.
- Controlling who can read it. Sensitive values stay encrypted as they pass through the LLM and come back out. Only authorized users can decrypt them, so an output that is forwarded, stored, or intercepted doesn't expose the protected data inside it. Every later event, such as sharing, re-protecting, or destroying that data, gets its own receipt, so the trail continues after the AI interaction ends.
Together, the two sides close the loop: protected inputs, untampered outputs, and a verifiable record connecting them.
How is dual provenance different from AI logging and data lineage?
Logging, data lineage, and dual provenance all try to answer the same question: what happened to this data? The difference is whether you have to trust the answer or can check it yourself.
Why AI logs are not proof
A log is an assertion. An administrator can edit it, a script can overwrite it, and an attacker can wipe it. Even an untouched log only tells you what the system wrote down, and you have to take its word for that.
Dual provenance replaces assertions with evidence. Each input and output is signed, and each receipt is chained to the one before it, so changing or removing any record breaks the chain at that exact point. A third party can check the signatures and confirm the chain hasn't been touched, without taking anyone's word for it.
How zero-knowledge proofs verify records without exposing data
Signatures prove a record wasn't changed, but checking them usually means someone gets to see the record. In regulated work, that's often the problem. The auditor confirming your AI handled a patient's chart correctly shouldn't need to read the chart.
A zero-knowledge proof (ZKP) is a cryptographic method that lets one party prove something is true without revealing the data behind it. Businesses already use it to verify, comply, and transact while handing over far less sensitive information:
- Identity checks. Proving a customer is over 21 or already verified, without sharing a birthdate, address, or ID documents.
- Finance and compliance. Proving an exchange's reserves or that a transaction passes sanctions and anti-money-laundering rules, without exposing balances or counterparties.
- Blockchain scaling. Zero-knowledge rollups such as zkSync, Starknet, and Polygon zkEVM bundle thousands of transactions and post one small proof that they're all valid.
- Healthcare and AI. Proving a patient qualifies for a trial, or that an AI model was trained or run correctly, without exposing the underlying records or model weights.
The appeal is the same in every case: less sensitive data changes hands, so there's less to breach, regulations like GDPR and HIPAA are easier to meet, and parties can verify each other without having to trust each other.
Applied to provenance, a zero-knowledge proof means a record can be proven intact, and properly protected, without exposing its contents. An auditor can confirm nothing was altered without ever reading the patient's chart or the client's memo. Verification and confidentiality stop being a trade-off.

How does sensitive data leak into LLMs?
AI data loss rarely looks like an attack. All it takes is an employee or client pasting in an internal document and asking for something as harmless as a summary. In a few prompts from an internal machine, sensitive data can slip past your security team without anyone noticing.
The data may never leave the building in the traditional sense, but it still lands somewhere it isn't cleared to be. Privileged material or classified content ends up in a model's context window, logs, or a third-party service. Once that happens, it's very hard to pull back. Nothing was stolen, yet the data is now out of bounds, and the organization is still obligated to handle it according to current regulations.
Provenance does more than block these moves. It shows exactly what moved, where it went, and whether it was contained.
Karst_Fathom encrypts sensitive values before a prompt ever reaches the model. Get early access.
How do you audit AI usage for compliance?
Enterprise LLM use adds a new layer to both security and efficiency audits. Traditional paper trails, and even traditional ways of measuring ROI, can't keep up with how fast organizations are adopting AI.
Dual provenance gives CISOs and CTOs full visibility from data ingested through prompts and outputs, recorded on their own systems and available at any time. Beyond cryptographic proof that records are intact, it provides the evidence trail security audits require. It also makes token and prompt efficiency easier to measure. Leadership can look past individual users to see which prompts, context, data analysis, and other AI uses are costing the most in compute and budget.
That visibility pays off in speed. Teams adopt AI faster when they can prove what happened. They spend less time double-checking outputs, because the lineage already shows what a response was built from, and they can hand finished work downstream without restarting the review.
What is verifiable lineage, and who can see it?
Data lineage traces where data came from and where it went. Verifiable lineage goes a step further: every step in that trail is backed by a signed, tamper-evident receipt that anyone with the right access can check independently. You don't have to trust the system that produced the record, because the record proves itself.
Inside the organization, this gives leadership an honest picture of how AI is actually being used. It changes the conversation about risk, because it starts from evidence instead of assumptions. Security teams can investigate incidents from a trail they know hasn't been altered, and compliance teams can answer audit questions with proof instead of policy documents.
Outside the organization, it changes what you can offer. A patient, client, regulator, or mission partner can be shown the provenance of an output instead of being promised a policy about it. With zero-knowledge proofs, they can confirm the record is intact without seeing the sensitive data inside it. Who sees what stays under your control. Authorized users can decrypt protected values, while everyone else can verify the trail without reading it.

Conclusion: AI trust you can prove
AI is moving faster than the safeguards built for it. Models now read medical charts, legal briefs, financial records, and mission data every day, and the old answers to "is this safe?" no longer hold. A policy document can't prove what entered a model. A log can be edited. A promise to a regulator or a patient is only as good as the evidence behind it.
The risks run in both directions. Poisoned or malicious inputs can quietly corrupt what a model says, and sensitive data can quietly leak into systems that were never cleared to hold it. Either way, the damage often stays invisible until a decision has already been made on top of it. Organizations that want to use AI in high-stakes work need more than good intentions. They need to protect data before it reaches the model, prove what happened while it was there, and show that nothing changed afterward.
That's the standard dual provenance sets, and it's what Karst_Fathom was built to meet:
- Protected inputs. Karst_Fathom inspects every input, detects sensitive and dangerous data, and encrypts it before it reaches the LLM, leaving the rest readable so the model can still do its job.
- Data that stays unreadable. Encrypted values pass through the model and come back out protected. Only authorized users can decrypt them, so anyone who reaches the data without authorization finds nothing they can use.
- Untampered outputs. Every output is linked to the inputs that produced it and recorded in a tamper-evident receipt, so any later change can be detected.
- Proof, not logs. Signed, chained receipts can be verified offline by a third party, and zero-knowledge proofs let auditors confirm a record is intact without reading what's inside it.
- Your systems, your record. Provenance is recorded on your own infrastructure, your original data is never altered, and you keep the freedom to use the LLM of your choice.
The organizations that lead with AI won't be the ones that adopt it fastest. They'll be the ones that can prove, at any moment, exactly what their AI saw, what it said, and that both were protected. Dual provenance turns AI from something you hope is safe into something you can show is safe.
Karst_Fathom is in early access now for macOS and Windows. Enter your access code or join the waitlist.



