Why epistemic infrastructure has to get built now, why AI is the reason we can finally build it, and why we're starting where we're starting.
AI is great. It's the most transformative technology we will ever build, and I am, on net, extremely glad it exists (maybe not entirely in the forms that it is today).
I want to put that up front because most of what follows is about a problem. It's easy, when writing about a problem, where the reader ends up thinking the solution is less of the thing that caused the problem. That's not my view. The AI we have now and the AI we'll have in three years will accelerate almost everything downstream of cognition: science, code, analysis, design, the rate at which humanity produces new ideas. I want all of that, hopefully faster. I subscribe to d/acc.
I worry that AI will help produce far more than our existing machinery for sorting truth from plausible-sounding nonsense can handle, and we'll end up making consequential decisions on information that isn't as true as we'd like.
I also think AI brings capabilities that let us rethink and fundamentally redesign the epistemic infra that was impossible to build at scale before.
So here is what I want to argue,
The institutions for figuring out what's true all share a design assumption: that outputs arrive slowly enough that a small number of humans or tools can check them.
Peer review works because humans can read a paper in a few days and editors can find reviewers in a few weeks. Policy debate works because the pace of legislation allows months of deliberation between introduction and vote. Code review works because the rate of commits has been bounded by how fast humans can write code.
None of those assumptions hold anymore. The rate at which AI systems can now produce artifacts has decoupled from the rate at which they can be checked. Software engineers are already building things faster than they can collaborate on. Some examples:
Software engineers are already building things faster than they can collaborate on.
Retractions and hallucinated citations: more than 10,000 papers were retracted in 2023, three times the rate of a decade earlier. Peer review mechanism cannot keep up with this. Tens of thousands of 2025 publications cite papers that don't exist. Every 2025 proceeding studied at four CS conferences contained at least one. Reviewers don't have time to verify citations.
Supply chain attacks: in March 2026, attackers published compromised versions of LiteLLM (usually get 95M downloads/month). The compromised package was live for 40 minutes, downloaded 40,000+ times, stole credentials. The attackers used AI agents to spam the disclosure issue with 88 bot comments in 102 seconds. First documented operational use of AI agents in a supply chain attack. It won't be the last.
Our epistemic tooling is no longer sufficient. We need to build better epistemic infra, AI-native, and we need to accelerate doing it. As a society we are only as good as our epistemic infra. If we cannot make sense of what is being produced, we cannot figure out what is true. If we cannot figure out what is true, we cannot decide things well: which drug is safe, which policy works, what to believe.
Everything will be mediated through AI. The way humans interface with reality will be through AI. We already see chat-based systems doing this, and the next layer is real-time personalized software: interfaces that is built for you on the fly. Google's generative UI in Gemini already does this, where each query produces a custom interactive interface generated for that question alone. A2UI is an open protocol for it.
What comes after that might be the end of software altogether: pixels generated directly from models, with no intermediate software layer at all. You can see early implementations of this in projects like Flipbook, my own website, and in what Elon is planning with the hundred gigawatts of distributed inference sitting inside future Teslas, Optimuses, and SpaceX hardware. The model isn't a thing the software calls anymore. The model is the software and everything humans need will be directly generated by AI on the go.
We've been calling this regime subjective interfaces, and if you stack these layers together, you get something close to full personalization across the whole experience. I think this could be good, but it might also be bad. If we have too much subjectivity in how each person experiences reality, we lose the shared-external-artifacts that collective truth-seeking depends on. Truth-seeking has always meant comparing one view of the world to another, and it gets harder when there's less of a same-world left to compare against.
Today, if I read a paper and you read the same paper, we're reading the same artifact. We might disagree about what it means, but we can point at the same text. That shared-artifact property, the fact that two people can look at the same object and disagree about it, is what makes collective truth-seeking possible at all. (This is ofcourse with nuance that the shared-artifact event though might be identical the meaning derived from it might be different - the meaning each of us constructs from that information is shaped by background knowledge, reading conventions, the interpretive community we're embedded in, even what we read just before - which collectively I call our memetic gatekeeping - but this outside the scope of this blog.)
The tools that enable truth-seeking are our epistemic infrastructure, and with AI-mediated reality and the speed of outputs that AI enables, this infrastructure is no longer good enough. One of the places this gets hit hardest is science. The whole value chain, from forming a research question to doing experiments, recording them, peer review, publishing, and science communication, is already failing to keep up. With AI in the loop, it gets much worse.
We shouldn't reject the subjective-interfaces regime. It's going to happen, and a lot of what it enables is genuinely useful. What we need to do alongside it is build the thing-outside-the-frame: an external, persistent, verifiable substrate that AI itself can query.
One way to build such a substrate is to construct a dataset that's true, like an oracle, and let the AI rely on it. This approach is popular, but I think it's boring. I don't think it actually helps society or humans have better epistemics past a certain threshold. Humans don't have the bandwidth to consume the volume of information we'd need to maintain it. The labor cost of keeping it current with humans in the loop is unsustainable. And the approach doesn't scale with AI in any interesting way: it just builds a better static reference rather than something that grows with the rest of the system.
What we need is a substrate that maintains itself, that gets richer as AI improves, and that gives every AI-mediated rendering something underneath it to point at.
Every medium of knowledge has been shaped by the substrate it had to run on. Scientific communication runs on papers because of the printing press - the unit of knowledge became a paper, or a set of papers wrapped into a form factor like book or journal, because that's what the press could produce. We kept the form long after the press itself had been replaced, which is why a PDF, even today, is still typeset like something out of the 1600s.
The early internet was mostly text because that's what the protocols and the bandwidth allowed, and as bandwidth and codecs improved, images and video took over the medium. Video conveys things text cannot, and images convey things words struggle with, and these new forms gave us new ways of absorbing information, partly because human eyes can take in much more data, much faster, than we can by reading. (as the saying goes, a picture is worth a thousand words, a video is worth more.)
AI is a bit different. The earlier shifts mostly expanded the kinds of content a medium could carry. AI expands what a medium can do. With AI, we can manipulate the information space - we can structure it however we want, all at digital speed. Work that used to take a week now takes a model a minute, and the cost of working with information at high resolution has collapsed by orders of magnitude.
To see why this matters, let’s take a step back. Humans invented language to communicate and coordinate, and language is essentially a translation of the intentions-in-our-minds into something we can share with another person. This translation always loses some information. It's why we have trouble communicating with other people, why we can't write bug-free code, why we discovered other forms of expression like music, painting, and memes to express. Each of these is a different attempt to translate intention into a form that can leave one’s mind and reach another, and it is a lossy-function.
The interesting question AI lets us ask is which of these lossy channels were lossy by design, and which of them we can now redesign.
Let me give an example of voting in democracy. Voting is an act of communicating our preferences to elect a governing body, and one-person-one-vote is a kind of summarization. Each voter has a lot of opinions in their head: the things they care about, how much they care, what they'd trade off, with what confidence. The voting mechanism takes that whole object and compresses it down to a single yes-or-no. This is information-destroying by its form. A more thoughtful voter doesn't help, because the bandwidth of the channel is just one bit, and whatever doesn't fit through that one bit is gone.
Quadratic voting changes the shape of that channel. Voters get a budget of things-I-care-about-credits and allocate them across issues, with the cost of each vote growing quadratically in how many votes they cast for the same issue. Because of how the quadratic cost works out, the aggregate result now carries something it couldn't carry before: the intensity of preference. By changing the channel, we can fit it more information.
The same pattern shows up in Community Notes, which uses bridging-based ranking to surface notes only when they get agreement across people who normally disagree, escaping the failure mode of plain upvote/downvote that collapses everything to majority preference. It shows up in Pol.is, which uses dimensionality reduction over agree-disagree votes to surface statements that achieve cross-cluster consensus, where ordinary polling would have averaged the same statements out of existence.
What these mechanisms do is redesign the transformation-function itself, such that the move from intention to something-they-communicate loses less information along the way. The study of this kind of redesign goes by various names: mechanism design, collective sense-making, protocol design etc.
AI makes this kind of redesign genuinely cheap, because the work the channel has to do in order to be richer (structuring, surfacing, aggregating, propagating) is the kind of work that used to be expensive and is now affordable.
Once you start looking for it, you can see this pattern at every step of every epistemic value chain. Take the value chain of research, which we can think of as something like research-question to transformation-function to know-something-that-is-one-step-closer-to-the-truth.
An experiment happens in a lab. The result is a high-dimensional thing: raw data, methodological choices, instrument readings, the full state of what was tried, the things that didn't work, the judgments the researchers made along the way. The researcher writes a paper, and the paper compresses that experiment into prose with a fixed structure. By the time the paper is written, most of the experiment has been compressed away. The failed pilots, the calibration runs, the analyses that didn't pan out, all the things that didn't fit the narrative the paper needed.
Peer reviewers read the paper and write a recommendation, which compresses their judgment further into accept-or-reject plus a few comments. Citations get added when other researchers reference the paper. A paper either appears in a reference list or it doesn't, with no room to mark how strongly it supports the citing claim or under what conditions the support holds. Systematic reviews come along and aggregate citations into an evidence base, compressing many studies into a plot or a recommendation. Clinicians and policymakers read the recommendation and make decisions. By the time a decision gets made, the original experiment has been compressed several times in succession, and what reaches the decision-maker is a long way from what the lab actually saw. That is why we’ve invented words like science communication.
Peer review was designed for a world where reviewers wrote letters. Citations were designed for a world where reference lists were typeset by hand. Systematic reviews were designed for a world where a small team of humans could keep up with a manageable literature. The form of each step has the constraints of its era. The information loss gets worse when we have many papers to make sense of together. In other words, our epistemic capability in science is already poor, and it will get a lot worse when AI scales up the number of papers we produce.
I sometimes wonder how things would be if we could preserve all of the information at every step. A long-shot future where this might be possible is one with brain-computer interfaces, where I can send my brain signals to you with no information loss, and we eventually invent a way to do that with every other human at once, and with every other species, and we become a collective-consciousness of the universe and achieve epistemic singularity. Okay, that's far away. But I think today we can do something a lot better than what we have.
AI has properties that let us redesign each step of these value chains so that the channel at each step carries more, and so that we can collectively make sense of all the papers together at the speed AI itself produces them.
One think we could do is take boring oracle-dataset approach I described in the previous section and replace it with something different. Instead of maintaining a static dataset, let the dataset maintain itself in higher-fidelity forms.
What I mean by that is something like this. We take research papers and turn each of them into an autonomous AI agent, a paper-agent, one that maintains its own epistemics, talks to other paper-agents, updates its confidence levels as evidence comes in, reproduces its own findings where possible, and contributes to a graph of self-maintaining-knowledge. This has been tried before, in projects like the Semantic Web. But those projects didn't have intelligences they could summon at will from thin air.
This is what we mean when we say AI gives us a reason to redesign. The redesign is at the level of the channels along the value chain, so that less information gets destroyed at each step, because AI can communicate in higher fidelity than the channels we have today, and because it can present what it carries to humans in much richer forms than prose or static figures. AI is the new substrate. The mechanisms that were shaped by the old substrate are the ones we should be redesigning now.
AI is going to accelerate everything. If we accelerate the things that produce information much faster than the things that help us make sense of it, we get a world where we know less, on net, even though we're producing more. If we accelerate the sense-making channels at the same rate or faster, we get a world where the production is actually useful. We need to accelerate our epistemic capabilities defensively and differentially (d/acc). Forethought has also recently written more about using AI for epistemics.
We're starting with two domains: science and cybersecurity.
Almost every consequential decision a society makes eventually rests on a scientific claim. Medicine, biosecurity, energy, climate, food, AI itself: each of these has its rate of progress capped by how fast we can collectively make sense of what's known. We are already doing badly at this. The number of papers being produced is going to grow dramatically as AI gets used more aggressively in research, and the existing infrastructure for sense-making across that volume of papers is not built to scale. If we don't redesign the channels that turn papers into a maintained body of knowledge, every downstream decision that depends on those channels gets worse.
One such redesign we are exploring is to make each paper become its own autonomous-paper-agent: that seeks new evidence, that updates its confidence and that talks to other paper-agents to propagate implications across the literature. Reproduces its own work, hire humans to check things, hire labs to re-run experiments etc.
Something like this would enable much lower loss across the chain, because of properties LLMs bring. What this would enable, in practice, is a way to address the replication crisis, give humans a way to interact with science at high fidelity, and maintain claim-level auditability in the system. This could be a better epistemic infrastructure, one that keeps humans in the loop even as scientific discovery itself becomes automated.
Cybersecurity is the second domain, and the urgency is different. The shape of the problem there is that AI is making attackers faster than it is making defenders faster, and the gap is widening. Coding agents, which are increasingly writing the software that everything else runs on, are pulling libraries and packages without any way to reason about whether those libraries have been compromised in the last hour, whether the maintainer's account is behaving anomalously, or whether a dependency in the tree has just been yanked. The defensive infrastructure that exists, like CVE databases and security scanners, was designed for a world where attacks evolved over weeks. It does not work at the tempo of attacks generated by AI.
We are applying the same architecture here, where each library and code block becomes an autonomous agent that holds coherence about its own epistemic state by running multiple scenarios, watches for anomalous behavior in its own commit history or its dependency tree, and talks to the agents of related libraries when supersession or compromise is detected. Coding agents querying that layer before pulling a package have a defense that runs at the same tempo as the attack.
If we can build the substrate for science, we can build it for libraries, and from there for legal precedent, regulatory state, news, and other domains where the same shape of problem applies. We're starting with science and cybersecurity because the urgency is highest and the architecture is most demonstrable.
The whole substrate-redesign with AI only works if the AI sitting inside the substrate can be embedded in an external epistemic discipline that makes truth-seeking better.
Sutton's bitter lesson is usually read as a claim about AI capabilities, that general methods that scale with computation eventually beat methods that depend on hand-coded human knowledge. The same lesson might generalize to epistemics: any verification mechanism whose throughput is bounded by human cognitive labor will be outrun by any claim-generation mechanism whose throughput is bounded by compute. If our oversight relies on humans reading and checking everything, the system loses.
In order to build this, we will have to address a few open research questions that connect to AI alignment and safety. These are questions we expect to engage with seriously while we build.
At the model-level, we want to address the behavior of the model itself, and how these models relate to truthfulness and faithfulness. We are exploring whether the model can keep three things separate: what the source claims, what the model believes about the world, and what the model chooses to report.
At the system-level, these are questions about the larger system around the model. The scalable-oversight problem is the obvious one and model pluralism - we think that when multiple models disagree, the disagreement is an interesting epistemic artifact to surface, and the open question is what the disagreement actually signals.
A third question, and the one we think is most underexplored, is what I'd call things-we-don’t-know. The science that is out there is only the ones that made it out of the societal-machinery, it is heavily distorted by publication bias, withdrawn studies, unpublished nulls, unsaid statements etc. An epistemic system that doesn't reason about what's missing is going to inherit the bias of its corpus and call it ground truth. We want to explore whether AI systems can calibrate confidence for the fact that the corpus is a sample.
At the agent-level, we want to explore how agent societies, or hybrid human/AI institutions, would work.
Another is meta-epistemics. These are more general questions about how truth-maintaining systems should be built and how resilient it could be. Under truth-maintainence, we will have to think about how they can be governed, attacked, audited, plural. Under resilience, we will have to think about how to preserve epistemics under monoculture, platform capture, politics, and other cyber attacks.