We’ve moved past cops doing stuff because a dog gave them permission to do it. Now, we’re at the post-truth era of policing, where cops will do stuff because a machine told them to do it.

We already know ALPRs (automated license plate readers) generate false positives. And we know that these false positives can generate the sort of guns-out “encounters” no law-abiding citizen is prepared to handle. When you add addled “certainty” (an erroneous ALPR ping) to citizen bewilderment, things can get deadly pretty quickly.

But even when the guns don’t come out, it’s not because actual justice is being done. Cops are increasingly relying on tech to make decisions for them. And while your regular human being might acknowledge a blown judgment call of their own, they seem incapable of recognizing a mistake when it’s made by a computer.

That’s how we end up with situations like these, which seem like outtakes from a dystopian farce, rather than the new reality of American policing.

Supporters of Flock cameras and other automatic license plate readers often say that you have nothing to worry about if you’ve done nothing wrong. Yet a student in Georgia now has to fight a citation for not having insurance, despite presenting a valid insurance card when she was pulled over. It’s all thanks to an ALPR alert on her new car, reports WSB-TV. 

There are things we’ve always been told are wise — if not imperative — to carry in any care we’re driving: driver’s license, registration, and proof of insurance. You do know why they call it proof of insurance, right?

Well, despite the fact that the officer could have verified all three of these things using the handy-dandy laptop that fired off the Flock alert, the officer apparently refused to do so and took Flock’s word for it instead.

“Upon showing proof of insurance in this instance, she was still issued a ticket citation,” Jones said in an interview with WSB-TV. She did everything she was supposed to do, including having insurance and presenting proof when requested. Apparently, even the proof that law enforcement requires isn’t good enough when the computer says otherwise. This wasn’t even a one-time fluke, either. The very next day, she got pulled over in nearby Cobb County for exactly the same reason.

The computer says you’re a criminal. The stuff law enforcement has always considered proof of legal vehicle use is no longer acceptable. The people cops tend to refer to as “subjects” in police reports are now being made aware that this isn’t a term of law enforcement art, but rather a reference to their position in comparison to those who supposedly enforce the law. Your word against a cop’s, as the “fighting a losing battle in court” saying goes. Now, it’s your word against a cop’s computer. And even if your word is the same word cops trusted prior to Flock’s arrival, it’s no longer worth the paper it’s printed on.

Remember the Disinformation Governance Board? As we were quick to point out, this organization was named in the dumbest possible way by some incompetent people within the Biden administration, which only fed into the [ahem] disinformation around it. The program, a part of DHS, was an advisory group with no enforcement capabilities, and was tasked with coordinating standard best practices for government agencies to respond to and counter foreign disinformation campaigns. But it was launched with a blatantly over-the-top Orwellian name and with only vague details of its actual role, leaving the field open to everyone to speculate… or to pollute.

And speculate they did. Almost immediately the media was awash in people screaming about how Biden was creating a “Ministry of Truth” who would “police your speech.”

Fox News absolutely lost their minds, apparently mentioning the Board and its named director, Nina Jankowicz approximately 300 times in the course of just eight months. According to a (somewhat misguided and unsuccessful) lawsuit that Jankowicz later filed, Fox News became obsessed about claiming she was the new censorship czar of America:

The next day, on April 29, 2022, 11 different speakers referenced Jankowicz and the Board on 11 separate segments. Hannity repeated the falsehood that the Board was “in charge of policing disinformation,” and that Jankowicz is now “the person that polices our thoughts.” Fox News tweeted: “What you’re seeing is a full-scale attack on free speech in a country that has been free for nearly 250 years.”

On Fox & Friends, Carly Shimkus said Jankowicz is “lead[ing] a government board in charge of deciding what we can and cannot say.” On Outnumbered, Emily Compagno said Jankowicz is “now the official referee of disinformation” and also a “self-obsessed social media neophyte [who the country has now set as] as the champion, as the arbiter of truth.” Greg Gutfeld followed suit on Gutfeld!, warning that the Board “will likely target you,” and stating unequivocally that the goal of the Board is “to police your opinions, to hide their bad news or magically turn it into good news.”

This went on and on and on and on, despite all of it being nonsense. The board had no enforcement authority at all. It wasn’t about policing anything, let alone free speech or “truth.”

Even today you’ll hear MAGA folks claim that Biden had planned to launch a “Ministry of Truth” to try to censor Americans for their political wrong think.

Now contrast that with the coverage of Trump from a month ago openly telling his FCC to get more aggressive about policing speech. We mentioned it briefly, and there were a few news reports talking about how Donald Trump was demanding his FCC punish speech about his negative polls.

And then he demanded that the FCC punish news anchor Kristen Welker for the thoughtcrime of noting that in recent elections Trump had a bit of a “mixed record” on the success of his endorsements.

He literally says: “she will be reported to the FCC for rebuke or punishment.”

That was a month ago. What have you heard about Trump “Ministry of Truth” since? There’s obviously been coverage of his banning of some White House reporters, but the story of him literally demanding the FCC start punishing reporters for not reporting things the way Trump wants them to report was literally him demanding the FCC act as a Ministry of Truth, with Brendan Carr gleefully at the helm.

And, unlike Jankowicz, who went out of her way to explain that she wasn’t policing anyone’s speech, and who resigned within weeks after the death threats and harassment made the job untenable, Carr has embraced the role of actual speech cop. Hell, a year and a half ago when The Hollywood Reporter called out Carr’s role as Trump’s speech police attack dog, Carr responded with a screenshot of the caricature of him as an attack dog and tweeted “woof woof.”

And, of course, when questioned about Trump’s latest ridiculous speech police demands, Carr immediately promised to dutifully get his Ministry of Truth working on it.

In an appearance on “The Sunday Briefing” on Fox News, the Trump-appointed Carr was asked about the president’s threat against Welker. Carr responded by suggesting that broadcast news channels could face consequences if the agency interprets their reporting to not serve the public, citing “fake polls” designed to “suppress” voters as a potential risk.

“President Trump is over the target. We’re looking at a lot of actions,” Carr said, responding to host Peter Doocy. “There’s a lot of interest right now in fake polls that are out there. The FCC may put guidance out soon to remind broadcasters about their obligations with effect to not airing fake polls, particularly if they’re done to suppress people heading into the fall.”

And yet, that story lasted all of what, two days? Does anyone even remember that it happened?

It’s incredible the double standard at work. Biden launches something that is very far from being “the Ministry of Truth,” and the MAGA media machine spends the next eight months — and, really, the next four years — blasting out the fake news every damn day that Biden had hired someone to police everyone’s speech.

Four years later, Donald Trump publicly demands that the FCC — a supposedly independent agency that, unlike the DGB, actually does have enforcement powers, in theory limited by the First Amendment — punish a network anchor. And the guy running it hops right to it, promising to go after polls showing Trump’s popularity is in the dumps.

And the story disappears.

In 2022, we got nothing but hand-wringing from the loudest “but muh free speech” voices. Bari Weiss’s “The Free Press” had an article “All Hail America’s New Truth Czar!” about the controversy:

And Weiss herself went on Fox News (naturally) to mock it as the Ministry of Truth.

Now that the FCC is actually acting as the Ministry of Truth with Weiss running CBS News (complete with Brendan Carr’s designated “bias monitor” to police its speech)… I can find zero coverage from CBS News — a newsroom with vastly more resources than The Free Press — of Trump demanding that his FCC police the speech of one of CBS’s main competitors.

Gosh. I wonder why.

The Disinformation Governance Board (still a terrible name and roll out plan) had no enforcement power and no authority over American speech. It never actually launched and was completely abandoned within weeks. Yet it received 300 Fox News segments over the rest of 2022, and to this day people will insist that Biden launched a “Ministry of Truth” to police domestic speech.

Meanwhile, the FCC actually licenses broadcasters, and its chair just took orders directly from the president to “punish” reporters for truthful speech, and said he’s “looking at a lot of actions.”

That news cycle lasted about two days and is barely remembered.

Woof woof!

This week, our first place winner on the insightful side is MrWilson with a comment about the data on improving mental health among teens:

But this contradicts my agenda! Can’t you just ignore the facts “for the sake of the children?!?” …and my political and business aspirations?

Notice that all the proposed “do something” solutions for mental health issues never involve making the world a better place for teens to grow up in, stopping greedy selfish assholes from running the country, or cracking down on all the adults who take out their issues on their kids. Conservatives making up fake issues and making targets out furries and trans kids and immigrants and women et cetera et alia ad nauseum ad infinitum nunc est bibendum… is making their lives actively worse.

In second place, it’s SpcAgentOrange pushing back on our post about AI, copyright, and “theft”:

This is a whole lot of words dedicated to deliberately misreading. The executive did not say that this was the “largest theft of writing,” or the “largest theft of literature,” or “largest theft of copyrighted material in human history.” He said that it’s the “largest theft of labor in human history.” Whether or not it was ‘copyright infringement,’ these AI companies took the written works from thousands and thousands of writers, mostly without their consent or even their knowledge, and used it to create their own commercial product. Calling it “the largest theft of labor” is not wrong or hyperbolic – it’s a simple statement about what happened. These AI companies could have simply paid workers to create new training material. But they didn’t. They chose to rely on the labor of others without their consent. They stole their labor.

(Yes, yes, I hear the whining – “They didn’t steal their labor, because the labor was already done before they took the work – the books and papers and posts and movies and shows and all the rest had already been made, so AI companies didn’t actually take anything.” And if these systems weren’t being created specifically to put the original creators out of a job, that argument might move me.)

For editor’s choice on the insightful side, we start out with a comment from Stephen T. Stone about the ideology of the Trump regime:

Throw a rock at the Trump regime and you’ve got a good chance of hitting a eugenicist. That includes Trump himself, whose steadfast belief in IQ⁠—itself an invention meant to quantify and therefore justify racist beliefs⁠—and obsession with genetics are hallmarks of eugenicist beliefs.

Next, it’s Flakbait with a comment about the ICE officer who dodged extradition and then got arrested:

How does it feel, ICE?

This is straight out of ICE’s playbook. They have arrested numerous people outside courthouses who were heading in for their legal immigration hearing. Now this ICEhole gets taken down while in town for a legal hearing. That’s just a tasty bit of comeuppance.

Over on the funny side, both our winners come in response to our post about Elon Musk’s extreme failure to fight CSAM. In first place, it’s Pixelation with a quick reaction:

Obviously…

Biden did it.

In second place, it’s Heart of Dawn with another response:

Musk has decided to fight in the war on CSAM on the side of CSAM.

Things were pretty slow on the funny side beyond that, so we’ll just keep it to one single editor’s choice. It’s Thad with a response to the reminder that Trump allegedly kept Hitler’s speeches as “bedside reading material”:

Well, kept them at bedside, anyway. I’m skeptical about the “reading” part.

That’s all for this week, folks!

This Week in 2016

• Indian Court Says ‘Copyright Is Not An Inevitable, Divine, Or Natural Right’ And Photocopying Textbooks Is Fair Use
• Former UMG Exec: Major Label Music Should Cost More And DMCA Safe Harbors Should Be Destroyed
• Will The Washington Post Give Back Its Pulitzer And Stand Trial With Snowden?
• Senate Comes To Its Senses: Does NOT Support Ted Cruz’s Plan To Block Internet Governance Transition
• Journalists Blaming Facebook For Decline Is Just As Tiresome As When They Blamed Craigslist & Google
• Arguments Over Internet Governance Transition Get Even More Stupid

This Week in 2011

• Pirate Party Takes 9% Of The Vote In Berlin Elections, Wins A Bunch Of Seats In Parliament
• Italy Proposes Law That Will Ban People From The Internet Based On Single Accusation Of Infringement From Anyone
• MPAA Shuffles The Deck Chairs
• Is It A First Amendment Violation To Get Pulled Over For Flashing Your Lights To Warn Others Of Cops?
• Craigslist Trying To Destroy The Life Of Someone Who Made Posting To Craigslist Easier
• Entertainment Industry Now Shaking Down People At $10 Per Infringement

This Week in 2006

• Target Targets Blogger Who Posted Anti-Theft Procedures
• Some Things Not To Say To Customers After Your Board Is Accused Of Spying…
• Napster, Still Struggling, Puts Itself Up For Sale
• Band Just Notices That Run DMC Sampled Their Song Twenty Years Ago; Files Lawsuit
• Facebook Ready To Seal The Deal — But For Only $1 Billion
• The Growing Backlash Against Automated Cheating Detectors

Pete Hegseth and Trump got a dubious win today: a 2-1 panel of the DC Circuit found that the designation of Anthropic as a supply chain risk was not unlawful. It dismissed Anthropic’s challenge despite (1) a different court having found the exact opposite not that long ago; (2) being predicated on a statutory interpretation of “supply chain risk” that would effectively deem most AI models, and potentially all software, a supply chain risk; (3) having made this designation selectively and punitively; and (4) it resulting in Anthropic being disqualified from selling its model to any government agency, including those without same concern the DC Circuit credited the military with having.

To some extent the difference in the two decisions can be explained by the two different statutes at issue. Hegseth claimed the authority to make “supply chain risk” designations under two separate ones, 10 U.S.C. § 3252, which was at issue in the California challenge, and 41 U.S.C. § 4713, which was at issue in this case. Congress is also partly to blame for this mess, because in writing the statutory definition for “supply chain risk” in the 4713 statute it created more space for dubious interpretations like this one (“Whatever paradigmatic examples individual members of Congress may have had in mind, the statutory definition is not limited to “adversar[ies],” 10 U.S.C. § 3252(d)(4), and instead covers “any person,” which cannot refer only to foreign entities, 41 U.S.C. § 4713(k)(6).”).  The statute also constrained how such designations could be challenged, sending them all directly to the DC Circuit, rather than a district court, which is why Anthropic’s challenge of the Hegseth action ended up in two separate cases.

But a bizarre situation has still resulted where one court has said that Anthropic’s First Amendment and due process rights had been violated, and another has now said they weren’t, even though the same action was involved with both. Anthropic argued that the California district court’s earlier decision should have been controlling, but the DC Circuit disagreed:

Anthropic contends that the Northern District’s decision is preclusive as well as persuasive. But because the Department’s designation authority is much broader under section 4713 than it is under section 3252, the issues flagged by Anthropic are not the same in both cases. So, for example, the Northern District’s determination that the section 3252 designation was arbitrary does not control our determination whether the section 4713 designation was arbitrary. Likewise, the Northern District’s determination of exigency under section 3252 does not control our determination of exigency under section 4713. In any event, Congress gave this Court exclusive jurisdiction to review procurement actions taken pursuant to section 4713 designations, see 41 U.S.C. § 1327(b)(1), and it specifically barred other courts from reviewing any other “action taken under” section 4713, see id. § 1327(a). That strict “allocation of jurisdiction” to this Court makes it inappropriate to constrain our review based on the Northern District’s judgment. Restatement (Second) of Judgments § 28 (1982); see Shaw v. State of Cal. Dep’t of Alcoholic Beverage Control, 788 F.2d 600, 607–09 (9th Cir. 1986); Lyons v. Westinghouse Elec. Corp., 222 F.2d 184, 188–89 (2d Cir. 1955) (L. Hand, J.).

Instead, because the DC Circuit read the statutory authority Hegseth drew from differently, apparently given its textual differences, it found Hegseth entitled to take the action that he did. But it is a dubious reading that would have broad implications the court did not address. In short, because Anthropic would still have control over its model, the court found that it could be considered to have the power to “manipulate” it, even after being deployed in government, and that made it a supply chain risk.

The Secretary reasonably concluded that removing Anthropic from the Department’s supply chain was necessary to protect national security by reducing supply chain risk to the Department’s information systems. Specifically, the Secretary credited a joint recommendation from two senior Department officials that Claude might be “subject to manipulation” by Anthropic “in such a manner as to inhibit the DoW’s use thereof.” App. 178. Likewise, he credited Under Secretary Michael’s conclusion that Anthropic might “manipulat[e]” the “design, integrity, and operation” of the Department’s Claude models, potentially causing “critical defense system[s] failing to engage” as intended by the Department. Id. at 182.

The record in this case amply supports the Secretary’s conclusion. To begin, it is undisputed that Anthropic can and does control how Claude responds—or fails to respond—to user prompts. Anthropic’s Chief Science Officer explained how the company “seek[s] to embed safety considerations directly into the model itself.” App. 8. Its CEO explained how such training gives the model an “identity, character, values, and personality” of its own, tethered to a “constitution” developed to impose “high-level principles and values” on Claude itself. Id. at 93–94. And the head of its public-sector business explained: “Model training is the primary mechanism through which Anthropic can influence the behavior of models used by the Department.” Id. at 276. Anthropic disclaims any ability to access or alter a model that has already been delivered to the Department, see id., despite the “technical measures” that it uses to police compliance with usage restrictions by private customers, id. at 8. Nonetheless, extant models reflect Claude’s “[c]onstitutional” training. Id. at 274–75. Moreover, Anthropic may encode additional restrictions each time it delivers any “new version of the model” to Department contractors. Id. at 276. Finally, it is undisputed that such model restrictions are vitally important to Anthropic, which describes them as lying “at the core of [its] mission.” Id. at 2.

The record also indicates that Anthropic’s model training is effective in enforcing usage restrictions and that, as a result, Claude has refused to answer legitimate queries from government users. Anthropic itself explained how early, commercially available versions of Claude frustrated Department and intelligence-community users by refusing prompts to evaluate classified materials. App. 255. Likewise, as Under Secretary Michael explained, the Department learned in 2025 that Claude had refused to process CDC prompts to support research to prevent the spread of infectious diseases. Id. at 212. Anthropic responds that these glitches reflected safety features appropriately built into models sold to private companies and were resolved after Anthropic engineers worked with the relevant government stakeholders. Id. at 255–56, 261–62. Perhaps so, but the point here is not that these model or usage restrictions were arbitrary; instead, it is that Anthropic’s model training does effectively enforce contractual usage restrictions.

Finally, the record reveals a recent, serious dispute about the scope of the contractual prohibitions on lethal autonomous warfare and mass domestic surveillance. Under Secretary Michael describes the incident in general but striking terms: [O]ne of Anthropic’s executives questioned the propriety of the potential use of their software for a sensitive military operation abroad despite that use being permitted under the existing Terms of Service. This led to alarm by the DoW and the prime contractor who provides Anthropic software, and raised material doubts as to whether they would cause their software to stop working or cause some other disastrous action that would put our warfighters[’] lives in danger. App. 181. Anthropic does not say much about this incident, except to suggest that it reflected a misunderstanding. Id. at 236–37. But regardless, Anthropic has made clear that it views the contractual prohibition on mass domestic surveillance as urgent to support “democratic values,” id. at 146, and the contractual prohibition on lethal autonomous warfare as urgent to avoid “put[ting] America’s warfighters and civilians at risk” of a catastrophic AI mistake, id. at 147. For its part, the Department has made clear that it views an “any lawful use” authorization to be critical to its “AI-first” strategic plan. Id. at 202, 206. With such diametrically opposed positions and with contractual limitations that are hardly self-defining, the prospect for disputes is apparent.

In sum, the Department reasonably feared that Anthropic might manipulate Claude’s design to prevent it from performing national-security functions that the Department deems contractually authorized and necessary.

The nightmare hypo that the court credited was what if the military had some sensitive plans that depended on Claude’s use, which Anthropic then changed on the fly, which jeopardized the mission. But there are multiple problems with the court’s acceptance of the government’s argument here.

For one, if the court’s statutory interpretation about the power to affect the operation of delivered software were correct, then pretty much any software product, at least those still subject to vendor-supplied updates, could be considered supply chain risks, given that any update could make substantive changes. In any case, it would seem to mean that any AI model would be too risky for the government to use, because there is nothing unusual about Anthropic’s model-control architecture—to the extent Anthropic could still control its model, so could any other AI vendor potentially control theirs. Whether they would or not would depend on the contract restraining them, and the only thing potentially different about Anthropic is that it did not want to be contractually obligated to allow certain functions that Hegseth really wanted—functions that were ethically dubious at best and monstrously dangerous at worst.

But because that contractual reluctance upset Trump and Hegseth, they singled Anthropic out, alone, for negative treatment, turning their pique that “we can’t agree with Anthropic on how the software would need to be designed for us to be able to buy it” into “and because we can’t agree then NO ONE ELSE IN THE GOVERNMENT CAN EVER USE IT.” Per the DC Circuit, such an overbroad measure—after all, not every agency had the same concerns about changeability that the military might, yet Hegseth was deciding for them, too, whether they could use Claude, even when its architecture created no particular risk to them—and clearly punitive measure was perfectly fine because it implicated the implicit “national security” exception to the First Amendment the Founders apparently wrote into it in invisible ink.

To succeed on such a First Amendment retaliation claim, the plaintiff or petitioner must prove that (1) it engaged in protected speech, (2) the government took materially adverse action against it, and (3) the speech caused the materially adverse action. See Houston Cmty. Coll. Sys. v. Wilson, 595 U.S. 468, 477–79 (2022); Aref v. Lynch, 833 F.3d 242, 258 (D.C. Cir. 2016). Anthropic has satisfied the first and second prongs of this test, but not the third. The First Amendment squarely protects Anthropic’s advocacy regarding the safe and appropriate use of AI products. Moreover, the Department’s exclusion of Claude from its supply chain plainly qualifies as a materially adverse action. However, we can discern no causal connection between the two. Instead, the record makes clear that the Department removed Anthropic from its supply chain not because of its advocacy, but because Anthropic refused to agree to a contract term the Department deemed essential to national security.

Because Anthropic wouldn’t do the deal Hegseth wanted to do, he was therefore entitled to declare it too risky for anyone in the government to use, without it being seen as punishing Anthropic for its disinclination.

Anthropic points to various pungent statements in the Secretary’s February 27 social media post. Among other things, the Secretary denounced Anthropic’s “sanctimonious rhetoric,” “virtue-signaling,” and “Silicon Valley ideology.” App. 77. Such rhetoric seldom provides a sound basis for judging the lawfulness of federal executive action. See, e.g., Mullin v. Doe, 146 S. Ct. 2121, 2139 (2026); Trump v. Hawaii, 585 U.S. at 700–02. In any event, for all its flourishes, the Secretary’s social media post squarely addresses Anthropic’s refusal to provide the “all lawful uses” contractual authorization. He described Anthropic’s behavior as a “textbook case of how not to do business” with the Pentagon. App. 77 (emphasis added). He reiterated the Department’s demand for “full, unrestricted access to Anthropic’s models for every lawful purpose in defense of the Republic.” Id. (cleaned up). And he characterized Anthropic’s refusal to provide that access as imposing an unacceptable “veto power over the operational decisions of the United States military.” Id. The nub of this dispute was contractual, and the First Amendment did not require the Department to continue a contractual relationship that it viewed as creating a national-security risk.

Per the DC Circuit panel, the First Amendment takes a backseat to the President and his Secretary’s determination that a technology can ever be used by the government, no matter what.

This case raises profoundly difficult questions about the appropriate military uses of an almost unimaginably powerful new technology. The Secretary raises the deeply sobering prospect of overly constrained AI models shutting down unexpectedly and thus causing important military operations to fail. Anthropic raises the deeply sobering prospect of unconstrained AI models hallucinating inappropriate targets for lethal military force. Both possibilities present obvious national-security concerns. But in our Republic, it is the President and the Secretary of War who must determine how best to balance the competing risks. In doing so here, the Secretary did not transgress any limits on his authority under the Supply Chain Security Act or the Constitution. Accordingly, we deny the petitions for review.

Which cannot possibly be right if the Bill of Rights is to have any meaning in limiting government power, and especially not on a record like this.  Yet here we are.

The Metric Is Not the Mission is a ten-part examination of how Big Tech moved from building and expanding the open internet to increasingly shaping it around its own metrics, incentives and assumptions. Across the series, the argument follows the evolution of the platform economy—from the optimism of the early internet to the growing tensions around power, prediction, geopolitics, accountability and the future of digital life.

The series will be published in two parts each week over five weeks, with each installment building on the one before it. At the end of the series, the complete essay will be brought together in a single PDF edition, providing the full argument in one place.

Part II — When They Still Understood Us

Part I looked at the slow transformation taking place beneath the daily controversies surrounding Big Tech. This second part goes back to the beginning, asking what these companies originally understood about people and how the metrics that once measured their success gradually became the definition of it.

It has become fashionable to tell the story of Big Tech as though it were always destined to end here. In retrospect, it is easy to portray the rise of the major platforms as the inescapable march of surveillance capitalism, monopolistic ambition, and unchecked technological power. That narrative is emotionally satisfying because it offers clear villains and a comforting sense of inevitability. It also happens to be incomplete.

Cory Doctorow has given this deterioration a memorable name: “enshittification.” His argument is that platforms initially serve users well, then, once users and business customers are locked in, progressively shift value away from both toward shareholders, degrading the service in the process. It is a powerful account of how platforms become extractive. But it is not quite the argument here. The deeper problem is not simply that Big Tech has learned to extract more from us but that it has become increasingly convinced that because it can measure and predict our behavior, it understands us and, by extension, the societies it has come to mediate. The failure is therefore not only economic; it is also epistemic. The metric has become a substitute for the mission.

One cannot understand why these companies now appear increasingly disconnected from the societies they helped shape without first acknowledging that, for a remarkably long time, they understood those societies exceptionally well.

Technology succeeds when it solves technical problems. It changes the world when it solves human ones.

That was the genius of the first generation of internet platforms. Their founders did not invent friendship, curiosity, creativity or community. They simply recognized that the internet had reached a stage where these deeply human instincts required new forms of expression. The web of the late 1990s was exhilarating, but it was also fragmented, uneven and, for many people, intimidating. Finding information often required patience. Discovering interesting websites depended on chance as much as design. Publishing demanded a degree of technical literacy that excluded far more people than it empowered. The internet was open, but openness alone does not necessarily produce accessibility.

The great platforms emerged not because they sought to replace the internet but because they made it intelligible. Google transformed an expanding wilderness of information into something navigable. Wikipedia demonstrated that knowledge could be organized through collaboration rather than hierarchy. YouTube lowered the barriers to publishing so dramatically that expertise escaped universities, broadcasters and production studios. Facebook addressed an even more fundamental challenge. It recognized that the internet was no longer simply about information; it had become about people. Until then, maintaining relationships online had been surprisingly cumbersome. Email was too formal, instant messaging too ephemeral, personal websites too static. Facebook reduced social interaction to something almost frictionless. Its success lay not in technological sophistication but in psychological intuition.

This is easy to forget because the platforms that dominate our lives today bear only a partial resemblance to the ones that first captured our imagination. Facebook did not begin as an endless stream of algorithmically selected content. It was, in essence, a digital address book enriched by photographs, conversations, and the ordinary rituals of everyday life. It became valuable because it mirrored existing relationships rather than attempting to manufacture new ones. There was comfort in discovering former classmates, following the lives of distant relatives, or organizing gatherings that would otherwise have required dozens of emails and phone calls. The platform expanded social life without yet attempting to redefine it.

YouTube offered a similarly modest promise. It was not originally designed to maximize engagement or optimize watch time. It functioned more like an immense public archive whose value derived from its unpredictability. One could arrive searching for a lecture on astronomy and leave having discovered a forgotten jazz performance, a documentary on Greek history, or a repair manual for a washing machine. Recommendation existed, but it remained subordinate to curiosity. Users still felt as though they were exploring rather than being guided.

Even Twitter (now X), before it became a battleground for politics, culture wars, and performative outrage, captured something important about the changing nature of public conversation. It collapsed distance between journalists, academics, politicians, and ordinary citizens in ways that would have seemed extraordinary only a few years earlier. For all its imperfections, it suggested that expertise and authority might become more accessible rather than less.

Looking back, what united these companies was not simply technological innovation but a particular philosophy of the internet. They assumed that openness generated value. The more people connected, the richer the network became. Every new participant increased the possibilities for everyone else. Economists describe this as a network effect, but the phrase barely captures its cultural significance. Participation itself became the source of optimism. The internet appeared to be validating one of the oldest liberal ideas: that societies flourish when individuals are free to exchange ideas, collaborate voluntarily, and build institutions from the bottom up.

It is difficult to overstate how persuasive this vision became. Most governments celebrated the digital economy as an engine of innovation. Investors poured unprecedented sums into technology because the opportunities seemed limitless. Civil society organizations embraced online platforms as tools for democratic participation and global advocacy. Even critics of globalization often regarded the internet as an exception, a domain where openness appeared to distribute power rather than concentrate it.

For a brief historical moment, these interests aligned. What was good for technology companies often appeared to be good for users, for markets, and, in many respects, for the internet itself. The incentives reinforced one another. Companies grew by making the network more useful. Users benefited from larger communities. Developers built new services on open standards. The web expanded because success depended on drawing people further into its richness rather than confining them within a single destination.

History, however, has an inconvenient habit of changing the problems that institutions are asked to solve. The sociologist Robert K. Merton once observed that organizations often become prisoners of their own success. Practices that were rational under one set of conditions gradually harden into routines, and routines into orthodoxies. Institutions continue refining the solutions that once made them indispensable even as the environment around them evolves. Success breeds confidence; confidence breeds certainty; certainty eventually makes adaptation more difficult than persistence.

There is no reason to believe technology companies are exempt from this pattern. If anything, their extraordinary success may have accelerated it.

The platforms that once competed to help users navigate an open internet eventually found themselves managing ecosystems of unprecedented scale. Their priorities changed almost imperceptibly. The models they developed during the internet’s age of expansion proved astonishingly effective at connecting people, organizing information, and lowering the costs of participation. The metrics through which they evaluated success, such as growth, engagement, scale, and network effects, were not arbitrary inventions of venture capital. They reflected a period during which connecting more people genuinely created more value for everyone involved. The problem is that the world changed while the metrics remained stable. A measure that once indicated success gradually became the definition of success itself.

There is an obvious parallel here with Goodhart’s Law: when a measure becomes a target, it ceases to be a good measure. The principle, first articulated by economist Charles Goodhart in the context of monetary policy, describes what happens when an indicator that works as a proxy for an underlying objective is turned into the objective itself. But the problem here is slightly different. The issue is not simply that platforms began gaming their metrics, or that users learned to optimize for them. It is that the metrics gradually became so deeply embedded in the companies’ understanding of success that the distinction between the measure and the mission was lost. The metric did not merely distort the objective; it quietly became the objective.

None of this happened because a group of executives gathered in a boardroom and decided to undermine the open internet. Institutional change is rarely so theatrical. More often, it emerges from countless rational decisions made in pursuit of perfectly reasonable objectives: improve the user experience, reduce friction, personalize recommendations, increase safety, remove inconvenience. Each adjustment appears modest in isolation but collectively they alter the character of the system itself.

This is where history becomes quietly ironic. The companies that had once understood the internet better than anyone else slowly began to forget what had made the internet exceptional in the first place. They continued to believe they were connecting the world, even as they increasingly replaced the world with carefully curated representations of it. They accumulated unprecedented quantities of information about human behavior while becoming progressively less attentive to the human condition.

The distinction is subtle, but it may prove to be the defining story of this technological era. To observe behavior is not the same as understanding experience. A platform can know how long we hesitate before clicking a link, which videos hold our attention for an extra seven seconds, or what sequence of images is most likely to keep us scrolling late into the night. It can infer preferences with astonishing accuracy. It can predict patterns that would have been unimaginable a generation ago. Yet prediction, however sophisticated, remains an impoverished form of understanding. It reveals what people do. It says far less about why they do it, what they fear, what they hope for, or what kind of society they are trying to build together.

That difference, almost invisible at first, is where the story begins to change.

Konstantinos Komaitis, PhD, is a veteran of developing and analysing Internet policy to ensure an open and global Internet.

Ctrl-Alt-Speech is a weekly podcast about the latest news in online speech, from Mike Masnick and Everything in Moderation‘s Ben Whitelaw.

Subscribe now on Apple Podcasts, Overcast, Spotify, Pocket Casts, YouTube, or your podcast app of choice — or go straight to the RSS feed. To get extended episodes with additional coverage, support us on Patreon.

In this week’s episode, Mike and Ben cover:

• Trump dismisses concerns about AI risks at UN, says he won’t rein it in (PBS News)
• A.I. Gone Rogue? Trump Proposes Not New Rules but an ‘A.I. Force.’ (New York Times) 
• 22 countries back call to keep AI ‘under human control’ (POLITICO)
• If AI is going to destroy humanity, Scott Bessent says hyperscalers, not government, must take the fall: ‘We cannot absolve you of responsibility’ (Fortune)
• Meet the algorithm you can actually control and understand (New_Public)
• YouTube’s Custom Feeds Give You More Control Over the Algorithm (Wired) 
• YouTube CEO dismisses the idea of joining Meta’s multistate child safety settlement (CNN)
• Harry Potter Survey Results & Final Decision (Reddit)

And in the extended episode for Patreon supporters, they cover:

• Safer for Teens, Same Discord for Adults (Discord)
• Age Assurance on Discord: Vendors, Methods and Your Data (Discord) 
• How Discord Estimates Age Without Collecting Your Identity (Discord) 
• Ofcom launches investigation into Pornhub’s age checks (Ofcom)
• Aylo Defends Apple Age Assurance in Response to Ofcom Investigation (Aylo) 
• Discord’s age verification era is upon us, despite community backlash (TechCrunch)
• Angry reaction to Discord’s age assurance (X/Twitter)

Our fun links this week include Jev, the super fast new AI classifier, and Persodex, the context layer for your Contacts (PS. hit Ben up if you’re in New York in mid-October).

Follow us on Instagram, YouTube, and Bluesky for video clips from this week’s episode!

If you’re already a Patreon supporter, you can get the extended episode on Patreon.