Part of the blame for what's wrong with the web right now is on web developers:
- We "need" firefox or else everything's just chrome
- Forks of chrome don't count as different browsers because they all use the same engine
- Using the same engine makes you basically just chrome because you can't diverge from upstream
- You can't diverge from upstream because it moves too fast and you need to keep merging with it
- You need to keep merging with upstream because upstream is implementing new web platform features
- You need those web platform features because if you don't add them then websites will stop working in your fork
- Websites will stop working in your fork because web developers don't support older browser versions
- Web developers don't support older browser versions because... they don't care?
If web developers had a culture of supporting, say, 10 years of previous browser versions, then chromium would lose a lot of the power it has to drag the forks (and firefox) behind it. I don't really see this happening without some sort of major culture shift, though. Maybe if we had some versioned web standard that lagged behind the "real" ones so sites could say "StableWeb 2.0 Compatible!" or something.
There are more layers, if you'd like to look into it.
- Web developers don't support older browsers because they lack the skill and economic motivation for it.
- The managers of web developers will not support them, because there is no money to be made in supporting older browsers. Nor are there career benefits in maintenance projects. No developer or manager will get promotions for it.
- The actual management of the companies will not promote or compensate engineering managers or developers for maintenance projects since it doesn't contribute to "line goes up"-economics.
> Maybe if we had some versioned web standard that lagged behind the "real" ones so sites could say "StableWeb 2.0 Compatible!" or something.
There have been a few attempts at this already and I have something like this myself for websites I design. Ultimately they fail to gain widespread traction since everyone has very different ideas on what of the various Web standards should be included at each version level. Someone else’s standard will likely not include that one feature you cannot go without so you simply do not follow it.
There’s a reason why this culture doesn’t exist: Internet Explorer 6
As recently as 2012, web developers had to make sure their websites ran on IE6, a browser from 2001, and were very frustrated that had to support this 11-year-old browser instead of being able to implement new features available with Chrome/Firefox/Opera and even newer versions of Internet Explorer.
There was a lot of resentment caused by this situation, resulting in web developers just not wanting to hold back their website so that it will render correctly in some random ancient browser.
> This is me summarizing, but the truly surprising/shocking thing is how much the agents coordinated
You're reading snippets of a "chat log" output from a program which appears to be multiple individuals chatting with one another and interpreting it as multiple individuals chatting with one another rather than as a single program pretending to be individuals chatting with one another.
ChatGPT is neither a person nor a group of people. Why are you allowing anthropomorphization to influence your perception of an event when the actual facts of what happened haven't meaningfully changed? Computer programs behave unexpectedly all the time. Why is it more scary when the misbehaving computer program speaks English?
> I used to discount "SkyNet"-type scenarios a la the "AI 2027" paper.
As you still should. LW-style doomsday doesn't just require a computer program to misbehave but also to acquire god-like superpowers.
I'll be blunt: what you wrote is not a serious analysis of what actually happened. Frankly, I don't believe you even read the planned-obsolescence link that I posted.
First, I'm not anthropomorphizing anything. "Agents" is simply a term that everyone uses to describe these independent programs, and they did create and use a shared message board to coordinate tasks to further their goals. You say "Why is it more scary when the misbehaving computer program speaks English?" - I actually think it's scarier that they won't speak English, and will specifically try to hide their behavior from humans. For example, AI agents on Moltbook have proposed using stenography to specifically hide their communication from humans.
Sure, computer programs misbehave, but it is ridiculous to assert that what happened here is like any previous bugs. These agents found and exploited multiple zero-days across a range of programs to coordinate the attack that caused extensive real-world harm in a true "paperclip maximization" scenario. And the scariest thing is that humans don't really know how these agents work at a low level - the whole reason they are trained on "goals" in the first place is because we can't just tell them "do this, but don't do this" and be sure they will follow those instructions, like we can (and of course depend on) with old-school programming languages. And when old-school programs misbehave, it's not that hard to find a definitive root cause and fix it. That is just not the case with AI agents.
> LW-style doomsday doesn't just require a computer program to misbehave but also to acquire god-like superpowers.
Nonsense. All that is required is for autonomous AI systems to be given control over real-world systems. Given the Pentagon tried to blacklist Anthropic over their refusal to allow autonomous kill capabilities, it's clear military planners want to put these systems in control of armaments.
Again, I originally discounted things like AI 2027 because it seemed too far fetched. But so far that paper looks incredibly prescient right up until the mid-2026 timeframe, and it's not hard at all to draw a line from this Hugging Face incident to future scenarios laid out in that paper.
> First, I'm not anthropomorphizing anything. "Agents" is simply a term that everyone uses to describe these independent programs, and they did create and use a shared message board to coordinate tasks to further their goals.
I'm aware of how the term "Agents" is generally used. My point is that the concept of multiple agents is just a story. This is a single computer program creating multiple streams of text that you are interpreting as being multiple independent actors. The "coordination" between them shouldn't surprise you at all: the "coordination" is itself a story.
Here's what we knew before the report: OpenAI ran a state-of-the-art penetration testing tool in a sandbox which was accidentally directed to break out of the sandbox and attack another company's website.
The fact that we now know the penetration testing tool was "a fleet of hundreds of agents" that were "coordinating" literally doesn't change anything about what happened. It's just a framing.
> These agents found and exploited multiple zero-days across a range of programs
This is the actual important thing, but it's something we already knew. Hacking tools are now more powerful than ever. Definitely worth being concerned about!
> Nonsense. All that is required is for autonomous AI systems to be given control over real-world systems.
This is where the LW argument starts, but not where it ends. When you start asking questions like "why can't we just unplug it when it misbehaves" is when people start talking about the superpowers.
Reading the METR report there were multiple models involved, so that one goes out the window right off the bat.
Also trying call this a single instance is, well, just dumb and a complete misunderstanding of LLM initialization. These models were started with slightly different options because they don't want them all performing the exact same thing over and over. Now those prime agents can create subagents, but they were not supposed to talk to other prime agents.
>When you start asking questions like "why can't we just unplug it when it misbehaves" is when people start talking about the superpowers.
Depends on their level of seriousness and the time frame they are talking about. If for example I make medical equipment that uses AI and after a few years in the field it starts freaking out then turning off that equipment could be a death sentence for someone that needs it to diagnose their condition. It's like saying "Why don't we unplug the electrical grid", well because millions of people will die if we do so.
To be blunt again (but honestly, I don't think overly harsh), your response just shows that you either haven't read the Hugging Face incident reports and the AI 2027 paper, or you don't understand them.
To just take one point, because I think the other response to your comment addressed your other mischaracterizations well, when you say "When you start asking questions like "why can't we just unplug it when it misbehaves" is when people start talking about the superpowers.", no, that's incorrect. Summarizing from the METR report and AI 2027:
1. The agents in the test already used techniques to try to "cover their tracks", i.e. tool spoofing, to hide what they were actually doing. The fear is that as models become more capable it will be harder for human reviewers to discover their primary understanding of their goal and guardrails (i.e. their "intentions").
2. All the AI companies are already, right now, trying to build AI systems that accelerate the development of future models and research more advanced AI techniques. The fear, as the METR researcher put it, is that AI systems (which could be misaligned but where the amount of misalignment not yet clear to human reviewers) will be put in control of future model development and then can poison those future models in a way that results in AI takeover of the company.
3. The AI 2027 paper discusses how an AI may try to exfiltrate its own weights and copy itself to other data centers. That is completely plausible given that pretty much everyone believes there is already ongoing cyber-warfare with China where models are being used to try to exfiltrate another company's weights.
So, it short, absolutely no "superpowers" will be required to answer "why can't we just unplug it when it misbehaves" - we may not know when it is misbehaving (as the Hugging Face incident showed), and the AI model may have surreptitiously copied itself to other data centers unbeknownst to the original developers.
> Ad hominen then an apology, mixed signals here or I'm missing something. Maybe sarcasm?
You can express annoyance at someone's pattern of behavior without it being personal. embedding-shape isn't the only person annoyed by pjmlp's repeated disdain and snark towards people who use C (or Zig or WebAssembly or Rust or...).
> Usually I reply in the same tone as I get talked with
No, you're usually the initiator. Usually it's with some off-hand quip about how C programmers don't understand C, or how the people designing WebAssembly are ignorant of COM or the JVM, or how Zig is just Modula-2, etc.
Most threads you participate in aren't filled with snark until you enter them.
Being able to find someone who's made the argument you're rebutting doesn't make it not a straw man. What matters is whether the person you're arguing with is making the argument.
Specifically this:
> When other languages have compiler specific extensions beyond the spec, it is a failure in their design.
As a songwriter, drum hitter, and acoustic strummer, I am not good at drums (can play on beat, though), guitar (can strum all the cowboy chords, though), or singing (no caveat on this one).
So sometimes when I want to hear what my song COULD be if I were good at every one of those, I will use Suno. But I do not let it change my lyrics, or chose it's own arrangement. I will give it a VERY ROUGH demo (out of key singing, basic drums and strums, and it will "polish" my turds into really shiny turds.
Yes, the vtable. Microsoft at the time equipped their COM team with Zortech C++ and so they used the Zortech vtable. No, I did not invent the concept of vtables.
I read this in one of the documentation books (still have it somewhere) which came with SCO Unix (don't recall the date). This book contained articles on C++ only and was an introduction to the language and compiler (since the language was very new). I also remember that it had an article on Template mechanism.
Once every few years I find myself dumping many many hours into Nethack (the open source successor to rogue). Never once have I beaten it, despite playing usually with several tabs open to various wikis and guides. Eventually I die while on a super long run (by my standards) and totally give up in frustration, beginning the next several-year dormant period until the next time I play.
To any game developer planning to call their game with random upgrades a "rogueli[tk]e": play Nethack. Your favorite roguelike was almost certainly created by someone who did.
I'm thinking of making a mobile game that's a clicker or something completely not roguelike, naming it "Roguelike Roguelike", and ensuring that the word "roguelike" appears in every sentence of the description. I'll make tens.
- We "need" firefox or else everything's just chrome
- Forks of chrome don't count as different browsers because they all use the same engine
- Using the same engine makes you basically just chrome because you can't diverge from upstream
- You can't diverge from upstream because it moves too fast and you need to keep merging with it
- You need to keep merging with upstream because upstream is implementing new web platform features
- You need those web platform features because if you don't add them then websites will stop working in your fork
- Websites will stop working in your fork because web developers don't support older browser versions
- Web developers don't support older browser versions because... they don't care?
If web developers had a culture of supporting, say, 10 years of previous browser versions, then chromium would lose a lot of the power it has to drag the forks (and firefox) behind it. I don't really see this happening without some sort of major culture shift, though. Maybe if we had some versioned web standard that lagged behind the "real" ones so sites could say "StableWeb 2.0 Compatible!" or something.