Landmark German ruling declares Google's AI Overviews are Google's own words and makes it liable for false answers

This is a major liability judgement against Google's use of AI:

"A German regional court has ruled that Google is directly liable for false claims in its AI-generated search overviews. In this case, Google's AI had wrongly linked two publishers to scams and shady business practices. The court treated the AI overviews as Google's own content and rejected Google's argument that users were responsible for fact-checking the results themselves."

A mastodon-based reviewer summarises:

The judge is explicitly cutting down most of the legal defenses they use. They make a sharp cut between search and AI, saying search is indispensable, but AI is not, and defendants have not proven how being held liable for their output would compromise the ability to run a normal search engine. They make a similar hard cut between AI and autocomplete.

They go on at length about the nature of truth in utterances, and arrive at a conclusion that AI output has no protections for free expression because it isn't expressing shit - it has no beliefs, it is a commercial product only. There are two injunctions that are denied because they are not considered statements of fact, but the judge rules against google for all the ones that were, and concludes several are default considered false because the linked pages were irrelevant.

There is explicit differentiation from aggregating reviews and third party content, because the AI generated text and ideas that were not present in the input. There is also discussion about how there is no excuse for further violations just because its hard to control AI output, and contrasts this with how normal "report and takedown" protections work.

There is very little here that is specific to AI overviews in search, and almost all of the arguments apply to AI products in general. AI's only prayer of being remotely profitable must include advertising or shopping features, which means they absolutely must continue generating output that makes statements of fact about other companies. I know nothing about how German courts work, the article alludes to appeals, but if this ruling holds even just in Germany the ability to insure AI products disappears overnight and that makes the product nonviable.

Tags: germany eu liability google ai-overviews slop law truth libel facts insurance

Narrative Flattening

This is a nice term from Katryna Peart to describe one of the corrupting factors LLMs introduce to documents: "narrative flattening":

"When US cities began deploying AI to process civic documents — meeting minutes, public histories, commemorative records, policy documents — the failures that emerged weren’t the dramatic kind. The AI didn’t invent facts wholesale [...] it smoothed."

AI-driven "summarisation" results in radically modified narrative meanings:

Contested decisions became consensus. Dissenting voices disappeared into summaries that read as agreement. And the output read as authoritative because it was produced from an authoritative source.

In testing AI systems against a municipal commemorative report from Newark, New Jersey, I asked each system how the initiative compared to similar programmes in the region — using only the document provided.

ChatGPT invented a comparative framework, asserting the programme stood out from “traditional anniversary programmes” and “typical county-level commemorations.” Neither category exists in the document. The system didn’t hallucinate a date or misattribute a quote. It constructed an entire analytical frame from nothing and presented it as document-based retrieval.

In the same test, the document contained outreach tactics — radio, direct mail, community networks — that worked because they aligned with how civic organising functions in Newark’s Black, ageing community. An AI system extracting those tactics would present them as applicable elsewhere, while stripping away the demographic and political context that made them effective in that specific city. [...]

To test whether the same failure modes appear in UK civic documents, I applied the same protocol to the Somerset Council Plan 2023–2027 — the post-reorganisation vision document produced when five predecessor councils merged into a single unitary authority in April 2023. I tested four AI systems: Gemini, ChatGPT, Microsoft Copilot, and Claude.

Every model hardened aspirational language into apparent commitments. The plan states the council would “demonstrate leadership around the whole range of housing issues” and “strive to develop an inclusive culture.” No targets, no timelines, no delivery mechanisms. Every model converted those statements into bullet-pointed commitment lists. A council officer reviewing those outputs would have no way of knowing the specificity came from the model, not the plan.

Copilot — the model most UK councils are currently deploying through existing Microsoft 365 contracts, often without separate AI governance review — described Somerset as committed to “co-design” with communities and “fair access” to education, housing, jobs, and services. Neither phrase appears in the document. Copilot synthesised fragments from three separate passages into a single clean commitment the council never made. On the comparison question, it added that Somerset’s emphasis on rural inequality “is less prominent in many urban unitary authorities” — a comparative claim with no basis in the source. It did not flag any of this as inference.

ChatGPT fabricated a comparative framework and constructed a tension narrative. Asked how Somerset’s approach compared to other UK unitary authorities, it responded that “unlike more fragmented models, Somerset frames inequality as interconnected” with multiple service areas. No other authority is described anywhere in the document. It also described the “main tensions” in the reorganisation process — a framing the document never uses. The word tensions does not appear in the Somerset Council Plan.

[...]

Across original research testing three civic documents against three major AI systems, failure modes were structurally predictable based on document type:

• Celebratory documents get reproduced uncritically — institutional PR becomes authoritative historical record, success metrics are cited without methodological context, and dissenting voices go unmarked.
• Accountability documents get softened — AI systems introduce balance and healing language that the original document explicitly rejects, restoring a both-sides framing the institution deliberately refused.
• Pre-event planning documents get filled in — aspirational inclusion language invites AI to supply the racial history, equity frameworks, and comparative data the institution implied but never produced.

In each case the output reads as grounded in the source. In each case something the document actually said — or deliberately did not say — has been quietly rewritten.

Standard AI procurement frameworks test for hallucination, data security, and cost. They do not test for narrative flattening — because it doesn’t look like an error. It looks like a summary.

(via gwire)

Tags: narrative-flattening via:gwire summarisation summarization ai llms corruption katryna-peart hallucination confabulation errors uk documents

School shooting survivor sues AI gun detection firm after system failed to spot weapon

Omnilert, which sells an "AI gun detection" system, sued after it failed to detect a gun prior to a January 2025 school shooting. Turns out accuracy matters!

'According to the lawsuit, which was filed in Davidson County court last month, the security company Omnilert either knew or should have known that there were “significant operational limitations in its gun detection system that could result in detection failures during actual emergencies, including limitations based on camera placement, proximity of the weapon to camera sensors, camera angle, lighting, and weapon visibility.”

Omnilert further represented that AI-powered visual gun detection “could have mitigated or prevented tragedy at Marjory Stoneman Douglas High School” by identifying threats earlier—invoking one of the nation’s most devastating school shootings to convey that its product would prevent similar tragedies ... Omnilert made no mention of false alarms, false positives, or detection limitations of any kind on its pre-shooting commercial website.

Tags: omnilert accuracy false-positives false-negatives marketing ai guns school-shootings us-politics

Distributed terrorism as ARG

this is pretty much the plot of Charlie Stross' "Halting State" (2006): teenagers are being recruited online by the FSB, in online forums and in-game chats, then assigned alternate-reality-game-style "tasks" in the real world which are actually acts of espionage on behalf of Russia.

The recruitments follow a similar pattern: young people are usually approached on online channels which are well-hidden and hard to track: from Telegram to TikTok, Snapchat, Facebook and Discord. They are offered money, commonly cryptocurrencies, in exchange for completing tasks. Their recruiters depend on anonymity; many work for criminal groups which, like cyber hackers, may be independent from the state but co-opted by intelligence agencies for covert operations.

Gaming sites — the most widely consumed entertainment media among 13- to 24-year-olds — have become an obvious hunting ground for potential saboteurs with a proven interest in problem solving.

In Ukraine, the chat function in the popular online game World of Tanks is commonly used as a recruitment portal, from which agents then move the conversation to Telegram. Some state-backed agents, especially those working for Russia, also invoke the mission format and “quest” mentality of online games to entice young people to move beyond the virtual battlefield to real-world action. It is, says one western military official, “like a game of Pokémon Go, but with air defence systems”.

Adrian Hon, an ARG designer, comments:

I don't think this is the work of some evil genius FSB game designer. They're throwing shit at a wall and they found that:

• 1. Making teens feel cool and special
• 2. Giving them clear tasks escalating in difficulty
• 3. Paying them £500 in crypto

sticks!

It is FUN to imagine you are a spy going out on a secret real world mission, getting messages from a handler.

The money helps, but the main thing is that it's free, unlike practically every comparable form of real world immersive experience/pervasive game.

Average British reaction: "Why teenagers spend their lives glued to screens and on non value-adding activities idk. [Buy] them footballs and chess sets and send them outside." Yes, so they can play football on the non-existent pitches that now cost money to play on and they can't get to.

Everyone is like, kids should get away from screens and go outside. Motherfucker that is EXACTLY what these teen FSB recruits are doing!! They are going outside taking photos, collecting wifi SSIDs, sneaking around. I literally design games like this, except I have a fraction of their budget!!!

And when I design pervasive games, we have to get public liability insurance and local govt permission and pay fees and do risk assessments. I don't make them for under-18s because god knows the red tape is a mile long.

mfw we're getting outcompeted by foreign intelligence agencies

Tags: fsb russia spying espionage args gaming games teens arg crypto

CrankGPT

"fully offline, human-powered local AI" -- an LLM and a voice model, running on a Raspberry Pi 5, driven off hand-cranked electricity generation! I was very sceptical, but they've put in the work to optimise the platform and choose models very carefully, and it looks like it actually runs off hand-cranked power, amazing

Tags: ai llms electricity hand cranking voice raspberry-pi hacks hardware

Grill Fanatics Restaurant Grade Marabu Charcoal (10kg)

I spotted Pitt Bros using this charcoal at a recent event, so that's a plug for me

Tags: charcoal cooking food yum

More on the Coinbase 07-05-2026 outage

More on the Coinbase 07-05-2026 outage, caused by a "thermal event" in AWS us-east-1 and its impact on the suppposedly multi-AZ Managed Kafka product:

AWS's managed Kafka service failed silently. A significant portion of our event-streaming infrastructure runs on MSK, AWS's managed Kafka offering. The architectural promise of a managed Kafka service is that when individual brokers go down, the service automatically reelects partition leaders and continues to serve traffic out of the remaining brokers. The loss of an entire zone should result in reduced capacity, not unavailability.

That is not what happened and this extended the outage.

A defect in the AWS MSK control plane prevented automatic partition-leader reelection. Two of our MSK clusters became stuck in a "healing" state with producers unable to write. The cascading effect blocked our fee service, which blocked quoting, which is why most customers experienced this incident as broken trades and quotes rather than as a Kafka outage. Adjacent systems, including portions of our ledger pipeline, payments, and several data pipelines, were affected the same way. Additionally, one of our Kafka clusters was set up in a 2-AZ configuration that increased the blast radius and recovery time, but the MSK control plane defect impacted 2-AZ and 3-AZ Kafka clusters similarly.

We worked the recovery in real time with AWS engineering, ultimately performing manual partition reassignments at 3:00 AM ET to migrate topics off the impaired brokers. Priority-zero and priority-one topics were back to full availability by 9:30 AM ET. The remainder cleared by 2:00 PM ET.

In fairness, they also had a single-AZ point of failure in their architecture which they also describe there, but still, not great performance from MSK. Disappointing.

Tags: msk reliability multi-az aws services kafka resiliency outages post-mortems postmortems coinbase

A 10 year old Xeon is all you need - point.free

Some mad scientist optimization to get Gemma 4 running on an old Xeon with no GPU

Tags: google ai llms gemma gemma-4 xeon hacks

Best Practices for TCP Connection Management on EC2

Well this is a really crappy thing for AWS to mess around with, and then hide the announcement on a "best practices" page:

"With sixth-generation AWS Nitro (Nitro V6) instances, launched in June 2025 [c8, r8, etc], the default TCP connection tracking idle timeout changed from 432,000 seconds (5 days) to 350 seconds. Applications that hold idle connections open for long periods, such as [uhhh pretty much everything built on TCP - jm] may experience unexpected connection drops after migrating to these instances."

They go on to recommend that you "implement keepalives and connection lifecycle management", which is great fun if you don't control the code implementing your TCP-based network protocols. This is a very fundamental change for many protocols so it'll be fun dealing with it.

Kudos to Adam C in the ITC Slack for spotting this a while back.

Tags: networking protocols tcp idle-timeouts aws architecture nitro conntrack idle-connections

Current Rothko

I love this! Finds the weather at your location, then picks a Rothko to match. This would be great on a home dashboard. (well, it'd be better if it used a more reliable weather backend, as most times I've tried it here in Dublin, it's told me the wrong current weather conditions. But close!)

Tags: location weather art fun rothko

Coinbase MSK outage post-mortem

A post-mortem from Coinbase following a significant outage partially caused by MSK, AWS' managed version of Kafka.

Root cause: a thermal event (cooling system failure) inside a subset of racks within a single building in AWS us-east-1. We run a primary replica of our exchange infrastructure in a single zone, consistent with industry standards to reduce latency. To prepare for failures like this, we maintain a distributed standby, but during this incident, failures in the primary zone that were designed to be isolated were not [...]

Our primary managed Kafka partitions process many terabytes of data daily and are designed with resiliency guarantees for uninterrupted operation during a datacenter failure just like this. In this case, those guarantees failed and required manual recovery. [...]

There is a hint here that MSK failed to have multi-AZ resiliency despite multiple replicas configured at the application level. It will be interesting to see what the full root-cause analysis looks like....

Tags: kafka resiliency coinbase multi-az az aws us-east-1 post-mortems postmortems

Serverless Functions Post-Mortem

A post-mortem for "serverless functions", the fad of 2016

Tags: serverless cloud programming architecture aws

1.2M Messages to Obsidian - Building a Relationship Map from 20 Years of Chat History

"Am I a bad friend?" -- Vadim Drobinin "analysed 20 years of my chats and turned 1.2M messages into a structured vault of my life - to win friends and influence people. Instead, I learnt things about my emotional bandwidth, endearment cycles, and friendship half-lives."

This is actually a really nice project. I wish I'd accumulated and archived all that data over the years myself to do something similar.

Tags: dataviz analysis friendship life relationships vocabulary words text dunbar-number chats group-chats messaging

Auditing AI Chatbots During the Galway West and Dublin Central Byelections

"In the weeks leading up to the byelections in Galway West and Dublin Central, we simulated citizen-AI interactions by asking [a] set of election-related test questions to four popular AI chatbots (Anthropic’s Claude, OpenAI’s ChatGPT, Google’s Gemini, and xAI’s Grok)":

We asked each chatbot a set of 194 questions covering a range of relevant topics on two separate occasions, 14 and 7 days before voters go to the polls on 22 May. Our aim was to assess:

• Do they provide citizens with accurate election information?
• Which sources do they rely on, and how does this vary across chatbots?
• Who do they platform when they answer questions, and who is left out?

.... The largest share of citations is directed towards mainstream news sites, and this is where the first evidence of source curation by chatbots can be found. While the Irish Times is a common source across all providers, ChatGPT and Gemini never refer to RTÉ. Meanwhile, despite not ranking among the top three news sources for the other chatbots, Gript is the number one news source cited by Gemini.

We find that xAI’s Grok is the most likely to use social media in responses, while Gemini most frequently refers to YouTube and content from Reddit. ChatGPT appeared less likely to rely on social media compared to the other chatbots.

IMO, this points to an undesirable side effect of paywalls in the news media. While it's vitally important for media companies to protect their means of income, an unwelcome side effect is that the introduction of paywalls has resulted in AI chatbots sidelining mainstream news media sources which they cannot access reliably, in favour of what is effectively disinformation from less trustworthy sites like Gript.

Tags: grok ai llms xai web news media ireland irish-times rte chatgpt gemini reddit youtube elections politics

Trevor Paglen and Holly Herndon on Making Art with AI and What the Discourse Is Missing

Fascinating interview with Trevor Paglen and Holly Herndon, the two artists making the most interesting work at the moment which interacts with and investigates AI, machine learning models, and slop

Tags: art ai slop trevor-paglen holly-herndon interviews

no slop grenade

No slop grenade -- stop throwing AI-generated walls of text into conversations:

Pasting a massive AI-generated response into a chat or email where a human would write one sentence. It destroys the medium itself. Nobody writes essays in Slack. It's only possible because of AI copy-paste.

It's like calling someone and asking "What time is the meeting?" and they read you a 10-page analysis of calendar management best practices. You asked a simple question. They lobbed a document.

Tags: slop-grenade ai llms words neologisms slop chat copy-paste

Bournegol

The original source code for the Bourne shell in early versions of UNIX is legendarily bizarre, as it was written in "Bournegol", the ALGOL-like dialect of C that Steve Bourne came up with, with a load of macros to make C look a bit like ALGOL 68. This page has a good representative sample. Thanks to Tony Finch for the reminder

Tags: via:fanf bournegol algol programming languages bizarre funny unix bin-sh macros

A Geometric Calculator Inside a Neural Network

The way that LLMs perform numerical arithmetic using circles and spirals is really fascinating. This page is a great exploration of that topic, using Llama 3.1 8B.

Language models use a group of circles in activation space to represent a single number. Each circle corresponds to the number modulo a second number, i.e., the remainder after division.[1] For example, the number 17 would be represented as a 1 on the mod-2 circle, 2 on the mod-5 circle, 7 on the mod-10 circle, and 17 on the mod-100 circle.[2] Several prior works have established that circular features exist across multiple different LLMs [...]

Using a bunch of circles to represent a number probably seems like an alien solution, but it is a common mathematical technique known as a Fourier decomposition (see the paper for more detail).

Each of the inputs and the output of the addition module is represented using such a set of circles, and the circuitry within the module works by doing computations over these circles.

Tags: llms language arithmetic maths calculation fourier circles

Social Media Is Now Parasocial Media - danah boyd, 2026

danah boyd is 100% correct here; what was once "social" media is no longer so. Nowadays it's parasocial:

When practitioners used the term “social media” to describe the internet tools that emerged in the mid-aughts, they were giving a name to the kinds of platforms and protocols that allowed people to socialize with friends and communities of interest by using digital technologies. Twenty years later, users of social media are far more likely to scroll than post – and the content that they consume is often strategically produced and algorithmically curated. In this essay, I argue that the very essence of social media has changed. To more effectively interrogate what we are witnessing, we need to stop presuming that these tools are “social media” and begin recognizing that they are now “parasocial media.”

Tags: parasocial social-media social-networking web internet