Boom XOR Bust

Even snarky Tiktokers have switched to deriding AI for being powerful

Last week we talked about how Gary Marcus, while still ridiculing AI day and night on his Substack and on Twitter, and while still being the poster child of the pooh-poohers, has steadily shifted to “AGI is at least a decade away”. It’s pretty shocking that that’s what being an AI bear has been reduced to. I’m barely more bullish than that myself. I think everything from 3 to 30 years is still on the table, with the extremes of that range both being quite unlikely. Of course the problem is that 3 years isn’t totally impossible and hoo-boy are we potentially screwed if it happens in anything like that kind of timeframe.

Here’s a different kind of ridicule I spotted on TwitterTok or wherever, posing eight rhetorical questions. I thought I’d give my answers to these (and I’m happy to debate them — hit reply if you got this by email or head to the Substack comments) and then we’ll get back to Gary Marcus and the emerging consensus about how freaked out to be. Then finally I’ll explain the “Boom XOR Bust” bit.

1. Is generative AI destroying the environment?

This may sound like a contradiction to non-economists but it’s simultaneously true that the aggregate energy use from AI is significant and that the marginal energy use for a given generative AI query is negligible. Even if you use AI all day every day, if you make a pie chart for your carbon footprint, the slice for AI will be paper thin. An article making this case calculates that you’ll use more energy reading the article, just from having your laptop’s display lit up for 10 minutes, than an LLM would’ve needed to write the whole article. But focusing on aggregate energy, it’s nothing to sneeze at.

I care about the environment and want clean energy and think data centers can and should be carbon neutral. But none of this amounts to a compelling reason to eschew personally using AI.

2. Do LLMs erode critical thinking skills according to an MIT study?

I think collaborating with LLMs enhances critical thinking skills, unless maybe you’re a student having it just do your homework for you. GPT-5 tells me the MIT study kinda sucks, which I believe without further investigation. (Haha, but also seriously, my prior on the study changing my mind is too low to be worth the time to read it and GPT-5’s assessment, untrustworthy as it is, is a legitimate Bayesian update here. But also, fine, now that I’m writing this whole parenthetical and since GPT-5 helpfully linked me to the study, I looked at it. It’s N=54 and I suspect the only thing we can conclude is the already obvious fact that writing an essay on your own, you remember it better.)

3. Does AI training violate copyright?

Matt Levine of “Money Stuff” fame asks, “Does an LLM mostly remix existing text, or does it mostly learn from existing text and generate new text? Or are those the same thing?” Hard questions! Levine also says, “I sympathize with the argument that if you read a book you can learn from it without paying royalties, but you undermine that case if you stole the book to read it.” So maybe AI companies (maybe especially Facebook?) are guilty of copyright infringement because they pirated books. But if they buy the books then probably they should be able to train on them? I.e., LLMs aren’t inherently violating copyright by consuming, digesting, and regurgitating copyrighted material. As long as they’re not reproducing text verbatim beyond the limits of Fair Use.

4. Do LLMs send people into psychosis?

Scott Alexander (an actual psychiatrist) convinced me with “In Search Of AI Psychosis” that this is rare enough to not worry about, at least if you’re confident you don’t have risk factors. But I’m in favor of an abundance of caution when it comes to mental illness that has even a tiny chance of making itself invisible to you. See footnote 2 of a Beeminder blog post I wrote that’s, I guess, adjacent to disordered eating. So I say precommit to believing your friends and family if they tell you your LLM use is unhealthy! Even if the AI psychosis thing turns out to be entirely bogus, there’s no downside to that.

5. Is AI taking people’s jobs?

The data seems conflicted here. In any case, I view job stealing as an almost unalloyed good, whether the job thieves are immigrants or machines. For reasons I talked about in another AGI Friday, I disbelieve the claims that programming jobs are in danger in the medium term, pre-AGI. I’m not sure yet if the empirical data, especially for junior developers, is flying in the face of my elegant theory. Expect a mea culpa in a future AGI Friday if so, but for now I’m sticking to my prediction: software jobs will boom the more AI booms (again, pre-AGI) and job destruction is (pre-AGI) mostly a good thing anyway.

6. Is AI a tool of environmental racism according to an NAACP lawsuit against xAI over Memphis air pollution?

I guess I’m happy to defer to the courts on this one. As I mention any time anything Elon-Musk-related comes up, the topic ends up so full of propaganda from every side that it’s very hard to research.

PS: This kind of thing always makes me pine for Pigouvian taxes.

7. Are deepfakes so realistic that it messes with our fabric of reality?

I don’t think so. My analogy is with screenshots of text messages. We all understand that those are fakeable and we have norms that make faking them a serious transgression. So if someone shows you screenshots of text and you distrust the person enough, you’ll discount them. Otherwise you’ll (rationally) believe they’re probably legit. In principle the same norms can evolve for photo evidence. Speaking of which, we’re currently at a 71% chance of instant deepfakes of anyone doing anything by the end of 2027.

8. Should you just embrace the new technology?

Questions 1-7, in the original TwitterTok thing, were rhetorical with an answer of yes. This is meant to be a no. I guess the irony here is that I think every one of the accusations in this clip has at best the kernel of a valid concern buried in it. In any case, the strategy of personally refusing to use AI does little good for any these. I don’t know how crazy I sound to people who do have this list of concerns. I’m basically like, “Pshaw, those are mostly non-issues, AI is so useful, you’re crazy not to use it. But also sometime in the next 3-30 years it’s liable to cross a threshold where we lose control of it and humans end up crowded out till we all starve to death.”

More generally, I’m happy to fall back on techno-optimist priors for all the downsides of AI that are short of AGI. Powerful tools get used for both good and evil, usually the good outweighs the evil, and we create new norms and laws to mitigate the downsides and we muddle through. But AGI is different and maybe it’s wise to ally with people who are worried about AI for other reasons. (Or the opposite — distance myself from them so when their bad arguments fail, I don’t get lumped in with them?)

Getting back to Gary Marcus, what’s striking to me about the above questions is how much the vibes have shifted. Twitter and TikTok-style derision used to be about how stupid and incompetent AI was and how naive the optimists were. Those people seem to have all done a 180 on the actual facts while staying perfectly consistent on tone and posture. Namely that the AI industry is awful, I guess.

But Gary Marcus is actually seeming (gradually) less guilty of that. In his latest article, reviewing If Anyone Build It, Everyone Dies, he leads by explicitly agreeing with eight core tenets of the book:

1. Rogue AI is a possibility that we should not ignore. We don’t know for sure what future AI will do and we cannot rule out the possibility that it will go rogue.

2. We currently have no solution to the “alignment problem” of making sure that machines behave in human-compatible ways.

3. Figuring out a solution to the alignment problem is really, really important.

4. Figuring out a solution to the alignment problem is really, really hard.

5. Superintelligence might come relatively soon, and we are not prepared for it.

6. The public should be more concerned than it is.

7. Governments should be more concerned than they are.

8. The short-term benefits of AI (eg in terms of economics and productivity) may not be worth the long-term risks.

Amen.

And to further begrudgingly converge, sort of, with Gary Marcus, his other hobbyhorse is that AI is a bubble about to burst. I think he’s ludicrously wrong to be certain about that (if his certainty isn’t just a bit he’s doing). But it’s absolutely a possibility. As I say almost every week (and it’s not just a bit I’m doing!) the one thing we’re sure of is that we have no idea how this is going to play out.

Except that seeing AI spending in the US shoot past 1% of GDP, I’m tempted to rule out or strongly discount some middle paths for how this plays out. Namely, either Marcus is right, this is totally unsustainable, the bubble bursts spectacularly, and we’re a couple years from the coldest AI winter in history… or it’s the investors who are right. I don’t know if that means literally AGI in a few years, but it would imply AI continuing its rocketship trajectory and becoming more and more transformative. And then whether that’s amazing or horrible is still a huge question mark of its own. Basically I’m saying we’ve narrowed it down to three possible paths:

1. The coldest AI winter ever. Life stays normal, just with new toys.

2. An infinitely hot summer. Curing death and the whole shebang.

3. A figurative nuclear winter — everyone dies.

Though if I had to actually bet, I do still think the middle paths are on the table as well.

Random Roundup

• ChatGPT in agent mode finally did something useful for me. It trawled the online race results for a skate marathon that various subsets of my extended family have done for the last 7 years and put together a table of finishing times for the 10 of us over those 7 years. Things like this probably need to get a bit more common before I’ll declare the “AI Agents To Get Useful in 2025” prediction to have come true.

  

• More, harder evidence that Waymos really are superhuman drivers. (And absolutely no interesting updates on the Tesla robotaxis, which is itself an update. I’m ever-so-gradually more sanguine about my at-the-time seemingly cope-addled prediction that Tesla is faking being at level 4 autonomy.)

• Someone on Twitter reinventing Gell-Mann amnesia: “It’s amazing how ChatGPT knows everything about subjects I know nothing about, but is wrong like 40% of the time about things I’m an expert on. Not going to think about this any further.”

• Epoch AI on what AI will look like in 2030. I suspect they’re right about their headline graph and, once again, I think we have no idea what the implications are of saturating all these benchmarks. AGI? Or just that the benchmarks were bad? (The benchmarks in this graph sure seem good though…)

Thanks to Theo Spears, Gabrielle Taylor, and Nathan Arthur for discussion that led to this edition of AGI Friday.