A deployed model may eventually cover its own costs. But then the next model comes and eats the surplus before the last one has time to look profitable. Each generation should fund itself and the next one. right?

So what comes next? I think the current generation of models are intermediate models (everything between here and AGI). They are not the final product. They are expenses that bought the labs two things: proof that LLMs can perform valuable work, and a live experimental surface for discovering where intelligence can be monetized. “But the world does not reward raw intelligence on its own”.

That brings us to the next phase: capturing the labor margin of professional knowledge work. You can already see it happening. Anthropic and OpenAI are moving toward joint ventures with the very firms that sell professional services. The labs are not spinning up their own consultancies or private equity funds. What they want is simpler and smarter. They want to become the intelligence layer behind a body they do not have to build.

A frontier lab has three things: compute, weights, a few hundred exceptional engineers.

Theoretically speaking, to actually deliver the work of a top-tier professional services firm, you need thousands of operators, deep vertical expertise, regulatory licenses, client relationships, compliance infrastructure, and distribution built over decades. Labs have none of that. Building it would take years and starve the training pipeline. So instead they can license the intelligence and take a minority equity stake in someone else’s body. The operator runs the business. The lab keeps the inference revenue, owns a piece of the upside, and avoids building the operating company itself.

But why would the operator give the lab equity at all? Why not just rent the model and keep the margin? Because renting the public model is not the same as getting the lab’s frontier roadmap, embedded engineers, deployment priority, and feedback loop. The equity stake is the price of being close to the source of capability before it commoditizes. And why would the customer pay? Because the unit being priced is the deliverable, not the token. A pitchbook costs thousands in analyst and MD time. The model will draft one for a few dollars. The margin does not live in the token. It lives in the MD’s hour.

The same logic applies across any regulated knowledge work vertical where the customer buys judgment, speed, and liability management rather than software directly.The lab does not need the customer to pay more for intelligence. It needs the customer to keep paying professional-services prices while the cost of delivery collapses underneath.

The obvious objection is that a limited number of engineers and senior operators plus a model cannot replace the huge teams firms use today. But huge teams are not pure productivity machines. They are coordination machines. Once a firm grows large enough, a shocking amount of the marginal hour is spent moving information between people: status meetings, alignment calls, spreadsheet reconciliation, deck review, version control theater, and the quiet bureaucracy of keeping everyone pointed in the same direction.

A model does not create its own coordination layer. A small forward-deployed team plus a model pays far less of the coordination tax that consumes a large firm. It does not need to replace every person. It only needs to produce the same deliverable with fewer people, fewer handoffs, and less internal drag.

The customer is willing to pay something between the cost of the old giant team and the cost of the new small one. The gap is the surplus, and it gets split three ways: between the customer, the operator, and the lab. The lab’s share is what could fund the next training run. But that gap only stays open if the model deployed inside the venture is meaningfully better at the vertical than a fresh open-weights model anyone can rent. This is where the feedback loop becomes everything.

The model works on real vertical problems. That work generates proprietary signal: workflow traces, edge cases, corrections, evaluation data, user preferences, failure modes, and tacit process knowledge. That signal makes the model better at the vertical. A better model creates more surplus. More surplus funds deeper deployment. Deeper deployment generates more signal. The exact form of the feedback matters less than whether the loop exists at all. I think the surplus in regulated knowledge work verticals is large enough to leave room for high margins. And I think this is the clearest path by which frontier labs become economically durable before AGI.

The money will not show up as token revenue. It will show up as captured labor margin: software costs hidden inside professional services pricing, protected by proprietary workflow data, and compounded through vertical deployment.

All this is speculative.

I don't think that is the right picture.

AGI may not arrive as one model that suddenly becomes human. It may arrive as a system. Things that together become capable of pursuing open-ended goals.

At the most basic level, AGI is usually imagined as a system that can do many different things at least as well as a human being. That separates it from narrow AI: systems that can be superhuman in one domain and useless outside it. Deep Blue could beat the world chess champion, but that was it. It was good at chess.

But even that definition is too thin.

OpenAI's charter defines AGI as highly autonomous systems that outperform humans at most economically valuable work. DeepMind researchers have proposed a more layered view... The question is not only "How smart is the system?" The question is also: smart at what, across how many domains, and with how much independence?

This is where I think the AGI debate often gets stuck. We focus too much on knowledge and not enough on agency.

A model that knows a lot is impressive. But intelligence is not just knowing. Intelligence is choosing, planning, act… (Now I'm just writing the REACT pattern…)

To think about machine intelligence, I find it useful to start with human intelligence (Honestly, I just need a starting point).

If you want to measure a human being's intelligence, you might give them an IQ test. You might make them solve Raven's matrices. Convenient and fast. But also very limited.

Puzzles are not life.

A person's intelligence is not only the ability to solve a clean problem placed in front of them. It is also the ability to decide what problems matter.

That, to me, is the center of the issue.

Intelligence has at least two parts: goal setting and goal achievement.

What do you want?

That question tells me a lot about a person. A goal contains taste, ambition, fear, imagination, and values. Then comes the second question: given that goal, how do you get there?

Now we are in search territory. How many paths can you imagine? How many can you evaluate? Can you balance planning and acting? Can you change course when the world pushes back?

This is why tool use and planning matter so much.

You can argue that autoregressive models do not "really" plan. Fair. But that objection matters less if the system can still help achieve the goal.

The practical question is not whether the model plans in the same way a human plans. The practical question is whether it can decompose a task, use tools, observe results, update its strategy, and keep moving toward an objective.

"Keep Moving Forward"

Increasingly, the answer is yes.

There is another reason I distrust the "one mind" picture of AGI. Human intelligence is not individual in the way we pretend it is.

My intelligence is not only inside my skull. It is distributed across the systems that support me.

So why should AGI be different?

Maybe AGI should not be imagined as one lonely model sitting in a box. Maybe it will look more like a society of agents!

An artificial society, not just an artificial mind!

This also changes how we should think about embodiment. My intelligence is general partly because I have a body. Dexterity expands the number of goals I can achieve. It lets me move through more environments. It lets me test the world directly.

A disembodied model can still be powerful. But physical ability changes the size of the action space. A model that can reason about fixing a sink is useful. A Figure 03 robot that can actually fix the sink is something else.

That does not mean AGI requires a humanoid robot. It means intelligence is partly defined by what it can do. A system connected to browsers, code environments, laboratories, factories, and robots has a much larger action space than a model trapped in a text box.

This is why the question "Are current models AGI?" feels too narrow.

A raw model may not be AGI. But a raw model is not the full product. The real system is the model plus tools, scaffolding, and so on… Judging the model alone may be like judging a human brain without language, hands, culture, or society.

So how do we build AGI?

My current intuition is simple: we need large neural networks, massive compute, strong infrastructure, and high-quality data. Maybe AGI arrives as a successor to today's language models. Maybe not. Perhaps the missing piece is a better world model.

I don't know. Really.

But I know what kind of test would impress me.

A useful test would not only ask whether a system can answer hard questions. It would ask whether the system can create knowledge. Demis Hassabis has suggested a version of this idea: train a system only on knowledge available before Einstein's general relativity, then see whether it can independently reach the theory. I like that test because it asks for more than memory. It asks for scientific creation.

That is closer to intelligence.

We will know we are near AGI also because systems start doing long-horizon work in the world (check METR). They will set subgoals, use tools, recover from mistakes, coordinate with other systems, and produce outcomes that previously required teams of humans.

That is the threshold that matters.

The most consequential moment may not be when AI becomes as smart as a person. It may be when AI becomes good enough to improve AI itself.

Example: GPT-5.5 helps build GPT-5.6. GPT-5.6 helps build GPT-6. Then a future system discovers a new architecture that produces the same intelligence at a fraction of the compute.

That is the intelligence explosion scenario. Maybe it happens. Maybe it doesn't. But if it does, the world changes faster than our institutions can understand.

This is why I think the AGI debate should move away from the fantasy of one machine waking up.

The danger, and the opportunity, is more subtle. We may build AGI the way we built civilization: not as a single mind, but as a network of capabilities layered on top of each other until the whole becomes more powerful than any part.

Thanks for reading.

There's a better way. And it starts with a folder and a markdown file. You need skills…

What Are Skills?

Anthropic describes it like this: building a skill for an agent is like putting together an onboarding guide for a new hire.

I like this analogy. Think about what happens when someone new joins your team. You don't explain the entire functionalities of an application every morning. You point them to a wiki, hand them a process doc, maybe a checklist. They refer to it when needed and eventually they stop needing it.

Skills are that wiki, but for Claude.

At their core, skills are folders of organized files that agents can use to perform a specific task accurately. Each skill lives in its own folder and contains at minimum one file: SKILL.md. That file starts with a YAML frontmatter block containing two pieces of metadata (a name and a description) followed by the actual instructions.

---
name: evaluating-full-research
description: >-
  Evaluates the current state of financial research output
  by running Playwright tests and providing structured feedback
  with improvement suggestions.
---

That's the skeleton. The description is doing the heavy lifting, it's how Claude decides whether to activate the skill when you ask it to do something. Keep in mind that no manual selection is involved. If your request matches the description, Claude loads the skill and follows the instructions.

If You've been following this series, you already know about context engineering. Skills are a masterclass in it. The design follows what Anthropic calls progressive disclosure: Claude pre-loads only the name and description of each skill into the system prompt (level 1). If it decides the skill is relevant, it reads the full SKILL.md (level 2). And if the skill references additional files (checklists, scripts, output templates) Claude reads those only when needed (level 3 and beyond). The context window stays lean and we somewhat move to a more deterministic domain.

I'd point you to Anthropic's blog post and the official docs. They explain it well and I don't need to repeat them here. What I want to focus on is what skills look like when You're actually building with them.

Why I Need Skills for What I'm building

Let me make this concrete. I've now built the first version of a multi-agentic framework for financial analysis. When a user asks it to analyze a stock, the system tasks different agents to work on different parts of the research, one gathers and cleans financial data, another assesses risks, an agent gather market news, and so on... The end result is a structured report.

After every major update to this system, I need to verify that everything still works. So I ask Claude Code to use Playwright, run a research flow on a list of stocks, and check the output against a QA checklist. I do this constantly. And every time I do it, I'm burning context tokens re-explaining the same workflow: which stocks to test, what the checklist looks like, what format the output should be in, which Playwright scripts to run.

This process is thus the obvious skill candidate. Package the checklist, the scripts, the output format, and the instructions into a folder. Claude loads it automatically when I say something like "run the regression tests on Nvidia."

Sometimes I want Claude to act as a portfolio management expert, to evaluate the current state of a research output. Look at what the agents produced, assess the analysis quality, identify gaps in the reasoning, and give me structured criticism. This requires a specific persona, a specific knowledge base, and a specific output format. Every time I set this up manually, it takes several minutes of context-setting before we even start.

A skill solves both problems. The first is about automation, deterministic scripts that run the same way every time. The second is about expertise, packaging a persona and evaluation framework that Claude can adopt instantly. Skills handle both.

And the potential goes further. Imagine a skill that runs the Playwright tests, evaluates the output quality, provides structured improvements, and then (if I allow it) spawns sub-agents to work on those improvements. For a solo builder, that's the difference between spending an afternoon on QA and spending twenty minutes reviewing what Claude already did. Honestly, I don't actually use the sub-agents that often. Most of the time I'm not able to effectively follow multiple agents in parallel, so I end up working one step at a time anyway.

The Rule: When You Repeat, You Build

The general principle is simple: if You find yourself repeating a workflow to get a task done, there's a case to build a skill for it.

As you iterate on different projects with Claude Code, You'll naturally develop two categories of skills:

Horizontal skills cut across projects. Code review workflows. Documentation generation. Commit message formatting. Regression testing patterns. You build them once and reuse them everywhere. These are the skills that make you faster as a developer, regardless of what You're building.

Vertical skills are project-specific. For my application, that's evaluating financial analysis output, running sector-specific research flows, validating portfolio allocation logic. These skills only make sense in the context of one application, but they're where the deepest automation lives.

A future skill I'll write will probably be something around prompt refactoring, evaluating the prompts I use in the multi-agent system and suggesting improvements based on best practices.

Best practices

Naming: lowercase, dashes, -ing form — evaluating-full-research, not EvaluateFullResearch. The -ing signals an active process. The name is part of how Claude pattern-matches your request to the right skill, so sloppy naming means wrong skill calls. This gets worse as you accumulate skills. Monitor whether Claude triggers the one you expect.

Description: answer two things what does the skill do, and when should Claude use it. Include trigger conditions like "after major updates" or whatever You want (lol). Don't be too vague.

SKILL.md body: step-by-step instructions (don't assume Claude figures out the order), input/output format with concrete examples, common edge cases (what if a ticker doesn't resolve, what if Playwright times out), and degree of freedom… how much autonomy Claude gets. Be explicit about boundaries. If You don't define guardrails, agents make assumptions you didn't intend.

Practical limits: keep SKILL.md short, move detailed reference material to a references/ subfolder, keep references one level deep. Simplicity reduces errors.

Why Code Execution Matters

When You're using LLMs, You're operating in a stochastic world. Token generation is non deterministic by nature. But some tasks need deterministic reliability.

Skills can include scripts that Claude executes rather than interprets. Claude doesn't need to load the script into context. It runs it, reads the output, and proceeds.

This is the boundary between deciding and doing. Claude decides what to run and when. The script does the work. The intelligence lies in the orchestration, not the execution.

For M² Research, this distinction matters enormously. I don’t want Claude generating a new test logic on the fly, I want it running the exact same test suite every time so I can trust the results. The stochastic part is Claude's evaluation of what those results mean and what to do about them. The deterministic part is the testing itself.

What You're Already Using (And What You Can Build)

If You've ever asked Claude to create a PowerPoint, fill a PDF form, or build an Excel spreadsheet, You've been using skills without knowing it. These skills are available by default and so is the skill-creator (If I'm not mistaken).

The skill-creator is itself a skill. It's a skill that builds skills. Claude uses a packaged set of instructions about skill architecture to help you scaffold new ones. You describe your workflow and Claude generates the folder structure, formats the SKILL.md, writes the YAML frontmatter, and bundles the supporting resources.

My recommendation though? Build your first few manually. You'll understand the mechanics better, write sharper descriptions, and develop an intuition for progressive disclosure. Even if it sucks. Let Claude handle the scaffolding once you know what good looks like.

One more thing worth knowing: skills are an open standard. Since December 2025, a skill you build for Claude Code works with OpenAI's Codex, Gemini CLI and similar platforms. You are not limited to the Anthropic environment.

One simple thing to understand

Skills look deceptively simple. A folder, a markdown file, some YAML. But the design principles underneath, progressive disclosure, the separation of deciding from doing, the discipline of naming and describing precisely, these are the same principles that make the difference between a useful AI workflow and a frustrating one.

Start small. Package one workflow you repeat often. See how it changes your sessions. Then build from there.

Thanks for reading

Resources

• Equipping Agents for the Real World with Agent Skills — the engineering deep-dive on architecture and design

• Introducing Agent Skills — product announcement with partner integrations

• Agent Skills Documentation — setup, usage, and best practices

• Claude Code Skills Documentation — invocation control, subagents, dynamic context

This improvement can be attributed to two main things:

First, models can now use an increased number of sophisticated tools to perform their work. If You've read the first article of this series, You know that tools are the actions that a language model can request to interact with Your computer and the external world. More tools, more capability… same brain, better hands.

Second, models can now work on a task for an extended period of time.

METR published yesterday (20 Feb 2026) a post where it shared Opus 4.6's time horizon. If You don't know what that is, it measures the human completion time of the hardest tasks a model can solve with at least 50% reliability. As IQ correlates to other forms of intelligence, task length tends to correlate to other performance-related measures so it can be used as a proxy for models getting smarter.

Opus 4.6 scored around 14.5 hours. For context, Opus 4.5 was around 5 hours just a few months ago.

This is great. Like, I mean it. When I started using OpenAI models a few years ago, they could work for a few minutes and most research I was reading pointed out limited task duration as one of the main bottlenecks for reliable uses of AI models in knowledge work and research.

Yes but so what? Claude can work for longer but can it achieve anything meaningful?

Yes! Otherwise I would have to stop this article right now (lol).

The C Compiler Experiment

Earlier this month a short video published by Anthropic went viral. It was about Claude Code building a C compiler that could also run Doom.

$20000 in API cost. 100000 lines of code. Two weeks. And here's the part that blew my mind: the human mostly walked away and let it run.

But the thing I'm most interested in TBH is the approach that was adopted. The compiler was built using a parallel agent approach, the same idea that later became the "agent teams" feature in Claude Code. They used 16 Claude instances running in parallel, each in its own Docker container, each picking up different tasks from a shared list.

One agent was fixing bugs, another was cleaning up duplicate code, another was improving compiler performance, another was writing documentation, and another was critiquing the overall code quality from the perspective of a Rust developer. Each one had a narrow focus.

The narrow focus is not a minor detail. It's actually the core reason this approach works.

Why Splitting Work Matters (Context Engineering, Again)

I wrote a while ago an article on context engineering. In brief, if we're not detailed in our request, an agent will start reading the entire codebase and will get confused. For this reason, You give it the correct context by providing the right CLAUDE.md and linking the appropriate documents.

One thing You need to keep in mind is that as the agent iterates on multiple tasks, the context window approaches saturation, and, as a result, it increases the probability of hallucination and generating mistakes.

Anthropic published an excellent engineering post on context engineering that makes this point clearly. They describe a technique called compaction where, basically, when a conversation nears the context window limit, You summarize its contents and start a new context window with the summary. Claude Code already does this automatically. But even with compaction, a single agent doing ten different things will accumulate noise. Tool outputs pile up. Irrelevant information from task A pollutes the reasoning on task C.

The C compiler experiment showed exactly this problem. the researcher who ran the experiment, specifically warns about context window pollution.

Now imagine this… You have a single Claude session trying to fix a regex in one file, refactor a selection algorithm in another, update prompt templates in a third, and debug a frontend display in a fourth. By the time it gets to fix number four, the context is full of information about regex patterns and selection logic that has nothing to do with React (the frontend) components and SSE events.

Splitting the work across different agents can help us partially overcome the issue. Each agent gets a clean context window focused on its specific task. The backend agent doesn't need to know about the frontend.

Obviously, a disciplined approach to giving the right information and compaction rules can help us too. But there's a ceiling to how much one can squeeze into a single context window before things start degrading.

So How Do Agent Teams Work?

I think of agent teams like working on a report with Your colleagues. Imagine You've been tasked by your manager to write a super detailed report on a stock, let's say Nvidia.

The manager will task one person to gather and clean financial data, another to assess geopolitical risks, another to check the options market for that stock, and another to figure out how all the pieces affect the valuation and produce the final report (Yes I know strange tasks). I'm simplifying here but perhaps You got the point. This enables in some instances to achieve the objective faster and to get a more detailed and precise output.

Now You can create a similar workflow for your tasks with agent teams.

The essence of agent teams is summarized in the official Claude docs: "Agent teams let you coordinate multiple Claude Code instances working together. One session acts as the team lead, coordinating work, assigning tasks, and synthesizing results. Teammates work independently, each in its own context window, and communicate directly with each other."

So there are three things to understand:

The team lead is the session You're sitting in. It coordinates work, assigns tasks, and synthesizes results. Think of it as the manager who distributes the report sections to the team.

Teammates are independent Claude Code sessions, each with their own context window and their own focus. They're the analysts working on their specific sections.

Communication happens through a shared task list and direct messaging. This is the key difference from subagents. Subagents can only report back to the parent, think of them as assistants who do a task and come back with results. Agent team members can message each other directly, challenge each other's findings, and coordinate without going through the lead.

One important thing from the Anthropic engineering post on long-running agents: each agent needs to leave the environment in a clean state when it finishes. That means no major bugs introduced, orderly code, descriptive git commits. This is critical because the next agent that picks up work needs to orient itself quickly. The C compiler experiment used a simpler version of this same pattern, agents maintained progress files and README docs specifically so that each new session could figure out what had been done and what needed to happen next.

But Wait, I Already Run Multiple Terminals

Fair point. When I have to work on multiple tasks that can run in parallel, I'm used to just opening multiple terminal windows. I share my detailed prompt and then supervise the agents to check what they're doing.

The problem is that this takes time and the agents (multiple instances of Claude Code) don't work together. They don't share context. They don't know what the others are doing. They can't coordinate.

What if I need agents to share information when working on a difficult task list and iterate on what they find? Well, this is the use case for agent teams.

One thing You need to understand is that right now this is not a tool You can use by default, plain vanilla. We need to follow a few simple steps to run it.

Setting It Up

Step 1: Enable agent teams. This is one environment variable. That's it.

{
  "env": {
    "CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS": "1"
  }
}

Step 2: Install tmux. This is not a hard requirement, but this is how, You get the split view (I mean if You want that). Tmux allows You to run multiple terminals (hence multiple instances of Claude Code) in a single window. As a result, You'll be able to see the team lead and its teammates work on the task list in the same window.

Step 3: Give the team lead Your task list. Here I tell the team lead what my goal is for this session and I point to an .md file I've created with the list of tasks. The file is detailed enough that I don't need to be super specific in this context window.

Watching It Work

The collaboration between the agents is the real strength here. I'll attach some screens from my current interaction with claude code.

As You might notice, now there are more Claude Code sessions in the same window.

Eventually you can activate the split view and use CTRL + B and arrow keys to navigate the terminals to interact with each agent individually. You can check in on teammates' progress, redirect approaches that aren't working, and synthesize findings as they come in. The docs actually recommend doing this, letting a team run unattended for too long increases the risk of wasted effort. You are monitoring the situation.

Plan Mode Works Here Too

As You might be aware, when working on a difficult task You can just press Shift+Enter in the terminal to move to plan mode where the agent creates a detailed plan and to-do list before starting. This to-do list obviously will be shared across the different agents.

You can do the same thing when running teammates by specifying that in the prompt. For complex or risky work, You can require teammates to plan before implementing, the teammate works in read-only plan mode until the lead approves. If rejected, they revise and resubmit.

This is probably the smartest pattern I've seen: plan first with plan mode (cheap), then hand the plan to a team for parallel execution (expensive but fast). The plan gives You a checkpoint before committing tokens.

The Honest Part

One thing to consider: the process is very token-intensive so You could end up burning through your quota very soon. Each teammate is a full Claude Code session with its own context window. The math is simple: more agents = more tokens = more cost (duh).

For context, the Anthropic docs suggest reserving agent teams for work that genuinely benefits from multiple perspectives working in parallel. For simpler tasks, a single session or subagents are more cost-effective.

Some important limits to keep in mind:

• File conflicts are real. Two teammates editing the same file leads to overwrites. You need to be super detailed with the task list, and also… You need to check each agent.

• It's still experimental. Agent teams are in research preview with known limitations. Things might break.

• Start small. The docs recommend starting with tasks that have clear boundaries and don't require writing code. These tasks show the value of parallel exploration without the coordination challenges that come with parallel implementation. (I'm trying it right now on tasks that require code but I can revert if thing break)

• Testing matters more, not less. When You can't watch every teammate, the risk of an agent marking something as done without proper testing goes up.

Cleaning Up

When You're done, clean up the team. The lead can shut down teammates and synthesize results.

Where This Fits

In Part I of this series we saw that Claude Code is a single-threaded loop, one brain, thinking step by step, using one tool at a time. The power came from the discipline of that simplicity.

Agent teams are what happens when that single loop isn't enough. The human's job shifts to designing work decomposition. Which tasks can run in parallel? Which files does each agent own? Where are the dependency chains?

That's the vibecoder's real skill. No wait….

To some extent, more than the vibecoder, this is the pure software engineer's real skill. I'll elaborate on that in the future since now I'm very tired and it's also Saturday night.

Thanks for reading.

Sources & Further Reading

• Anthropic, Building a C compiler with a team of parallel Claudes(2026)

• Anthropic, Effective context engineering for AI agents(2025)

• Anthropic, Effective harnesses for long-running agents

• Anthropic, Claude Code Agent Teams documentation

• METR, Task-Completion Time Horizons of Frontier AI Models (2026)

When I started working with large language models, the focus was on prompt engineering, curating the instructions you give to the model to make it behave as desired. You went from zero-shot to few-shot learning. Before the release of OpenAI o1, you told the model to generate a chain of thought and tried to simulate some guardrails. The increase in performance was significant. But eventually, as models got better and we generally switched to thinking models, prompt engineering started making a smaller difference.

I used to think that the best people to work with LLMs were professors and managers, professionals who know how to deal with people and how to reframe a complex concept into smaller, easier components. Many thought the same. LinkedIn bros highlighted that this was the "democratization of software" or David against Goliath and all that dumb stuff you read online.

Tools evolved, but the goal remained the same: reaping all the possible rewards from them. And here's what you need to understand… finding the right words, the right prompt, is no longer enough. If you're limited to that, chances are you're not unlocking the potential of these tools. There's a rule I can't prove, but when it comes to technology, new trends tend to get popular on X before other socials. A while ago users in tech started talking about context engineering and I started wondering: what is that?

"Context engineering is the delicate art and science of filling the context window with just the right information for the next step." — Andrej Karpathy

Karpathy's framing clarifies something important. Context is not just your prompt. Context is everything the model sees when it has to make a decision: your instructions, the knowledge it has access to, the tools available to it, the history of the conversation, retrieved data. The prompt is one piece of the puzzle, context is the whole puzzle.

In Pokemon terms… prompt engineering is Diglett and context engineering is Dugtrio.

So your goal (as someone working with AI tool and especially as someone building agents) is to engineer the context that is available to them.

Let me give you an example. The first agent I created was extremely simple. It just used a basic system prompt while relying on the internal knowledge of the LLM. An agent I run right now looks more like this:

• Has a detailed system prompt with specific behavioral instructions

• Uses Tavily for web research

• Calls an API to get real-time market data

• Interacts with other agents to refine the output

• Takes advantage of previous conversations to modulate its responses

Obviously, such an approach complicates the process. The agent, and the application in general, has to deal with a much greater amount of data. Now, the amount of data a model can deal with at once is a function of its context window, the total number of tokens the model can "see" when generating a response. Think of it as the model's working memory. For reference, all current Claude models (Opus 4.6, Sonnet 4.5, Haiku 4.5) have a standard 200K token context window, roughly equivalent to 500 pages of text. Opus 4.6 and Sonnet 4.5 can access up to 1 million tokens in beta.

One thing I can tell you for sure: regardless of the amount of tokens available, it will never be enough.

Why Context Management Matters

If you have trouble understanding why it's important to manage context well, consider three parallels:

A model with tokens. As the context window fills up, models start losing focus. Research calls this "context rot", the more tokens in the window, the worse the model becomes at accurately recalling information from that context. It doesn't fall off a cliff, but performance degrades gradually. The model starts "forgetting" earlier instructions and making more mistakes.

A person with notions. Think about a typical study session. You start studying and after a while (unless you're on copious amounts of Ritalin) you see a decrease in performance and attention. Eventually you start making mistakes. Or when you hear someone say "I learned this but then I forgot that." As the amount of information you have to deal with increases, your ability to deal with it decreases.

A computer with RAM. When your computer runs too many programs, it slows down. This happens because the "working memory" is overloaded. The system starts swapping, things get sluggish, and eventually something crashes.

Our minds, RAM, the transformer architecture, they all have trouble dealing with an excessive amount of relationships between concepts. Too much information makes the output worse.

In a nutshell: you need to curate the context of your application. And frankly, of your Claude Code sessions too.

The Anatomy of Effective Context

So how do we do that? You want to be efficient, right? But what is efficiency here?

Finding the minimum set of information needed to achieve the desired result.

"An idiot admires complexity, a genius admires simplicity." — Terry Davis

Anthropic published an excellent engineering post on context engineering that provides the framework I think everyone working with these tools should internalize. Here are the key principles, distilled.

Write clear instructions at the right altitude. Your prompts and system instructions need to sit in a sweet spot between two failure modes. On one extreme, people hardcode complex logic, essentially writing if-else chains in natural language to control every possible behavior.

This creates fragility. On the other extreme, people provide vague, high-level guidance that fails to give the model concrete signals. The right altitude is specific enough to guide behavior, yet flexible enough to give the model strong heuristics rather than rigid rules.

Use examples, but curate them. Few-shot prompting still works wonders. But don't stuff a list of edge cases into your prompt trying to cover every scenario (I did it in the past and it didn't work as I expected). Instead, curate a small set of diverse, canonical examples that effectively portray the expected behavior. For an LLM, examples are the “pictures” worth a thousand words.

Let the agent retrieve context on demand. Rather than loading everything upfront, the most effective agents maintain lightweight references, file paths, stored queries, links, and use these to dynamically pull data into context at runtime using tools. Claude Code does this: CLAUDE.md files are loaded upfront as persistent context, while tools like Grep and Read let it navigate the codebase and retrieve files just in time. This mirrors how humans work. We don’t memorize entire databases — we build filing systems and look things up when we need them.

I want to draw attention to something that should be obvious but apparently isn't: you can’t be vague. If you can't decide what you want from your own vague prompt, you can't expect a machine to do better. Think about it in a work context. A person at a mid level (manager / principal) has to know almost everything about the task assigned to a junior. When I started working I had an amazing senior who knew literally everything about the tasks he gave me and could have performed them himself in a few minutes. He used to tell me: "Look, I could do this fast, but it's important that I let you do it. I would never ask you to do something that I don't know how to do." Giving instructions to Claude Code works the same way. The better you understand what needs to happen, the better your instructions will be, and the better the output will be.

Context Management in Claude Code

Now let’s get practical. Everything above applies to any LLM application, but Claude Code gives you specific mechanisms to manage context effectively.

CLAUDE.md — Your Project’s Persistent Memory

CLAUDE.md is a special file that Claude reads at the start of every conversation. It gives Claude persistent context it can't infer from code alone.

To generate a starter file, run:

/init

This command analyzes your codebase to detect it, giving you a solid foundation to refine. You can think of CLAUDE.md as a map of your project.

A few things to keep in mind about CLAUDE.md:

Keep it focused. Claude’s system prompt already contains a significant number of instructions. Your CLAUDE.md adds on top of that. If you stuff it with every possible command and style guideline, you're creating a mess, and you are going to cascade that on all your implementations (lol).

Use pointers, not copies. Don't paste code snippets into CLAUDE.md. Instead, point Claude to the authoritative source: "See the authentication pattern in src/services/XYZ." This keeps your context window lean and your references always up to date.

Update it when Claude misbehaves. If you notice Claude making the same mistake repeatedly, update CLAUDE.md to address it. You can also use the /memory command directly in the terminal to add notes that persist across sessions.

Pointing Claude at What Matters

As your codebase grows, Claude can't just read everything. You have to direct its attention. This is where @file referencing comes in. You tag specific files in your prompt to focus Claude on what's relevant.

For example, instead of:

"How does data retrieval work in this project?"

Try:

"The data retrieval logic is described in @X, please create a markdown file documenting that."

The difference is significant. In the first case, Claude has to search the entire project to find what's relevant. In the second case, Claude goes straight to the source. The answer is faster and more accurate because Claude isn't navigating the whole schema of the project.

For frontend work, you can share screenshots directly in the terminal. Take a screenshot, paste it into Claude Code, and ask it to fix what you see. Multimodality is cool I guess!

Compaction: Managing a Growing Conversation

As you work with Claude Code, the conversation accumulates tokens. Every file Claude reads, every command output, every response, it all stacks up in the context window. Claude Code has a context meter you should be watching. When it gets full, performance starts to degrade.

This is where compaction comes in. When the context window approaches its limit (around 90% capacity), Claude Code can automatically summarize the conversation, preserving architectural decisions, unresolved bugs, and implementation details while discarding redundant tool outputs and messages.

You have two options:

Let it happen automatically. Claude Code triggers compaction when the context window is nearly full.

Do it manually with /compact. You can run /compact at any time, and even give it specific instructions:

/compact Focus on the API changes we discussed

This tells Claude what to prioritize in the summary. You can also customize compaction behavior in your CLAUDE.md with specific instructions.

A few practical guidelines:

• /compact at around 70% capacity rather than waiting for automatic compaction. You get better summaries when there’s room to breathe.

• /clear between unrelated tasks. If you’re switching from working on the backend to a completely different frontend issue, start fresh. This prevents context contamination.

• Start new sessions for new features. Don't try to do everything in one conversation. Fresh sessions give Claude clean context focused entirely on the task at hand.

The Takeaway

Context engineering is what separates people who find Claude Code amazing from people who gave up after a week.

The progression from prompt engineering to context engineering mirrors a broader truth about working with AI tools: the bottleneck has shifted. It's no longer about finding the magic words. It's about designing the information environment in which the model operates.

Resources:

• Anthropic | Effective Context Engineering for AI Agents (2025)

• LangChain | Context Engineering for Agents (2025)

• Anthropic | Claude Code Best Practices

• HumanLayer | Writing a Good CLAUDE.md(2025)

• Andrej Karpathy | X post on context engineering (June 2025)

A few days ago, Elon Musk did something that made me think about feedback loops again.

He let SpaceX acquire xAI creating a single entity valued at $1.25 trillion. The stated rationale was orbital data centers, launching AI compute into space where solar energy is abundant and cooling is free. The immediate reaction from most financial analysts was skepticism. The Economist and Bloomberg raised the usual concerns about debt. The smart money, as always, focused on the balance sheet..

I think they're looking at the wrong ledger. And I want to share my reasoning

What Positive Feedback Loops Actually Are

We now know that a positive feedback loop is a system where the output of a process feeds back as input, amplifying the original signal. An example in climate science is: Ice melts, exposing darker ocean, which absorbs more heat, which melts more ice. In business it can be somethink like: A company collects data, builds better products, attracts more users, collects more data.

The key feature is acceleration: each cycle spins faster than the last.

When multiple feedback loops interlock within the same system, you get something engineers call a flywheel. Once it reaches sufficient velocity, it becomes extraordinarily difficult to stop, or to compete with. Amazon built one: lower prices attract more customers, more customers attract more sellers, more sellers enable lower prices. It took two decades for anyone to even begin challenging it.

Elon Musk is building a flywheel for the age of artificial intelligence. And unlike anything we've seen before, his loops extend from bits into atoms.

Loop One: The Data Engine

Every conversation about AI eventually arrives at the same place: data.

The quality and scale of your training data determine the ceiling of your models. This is why Google has been so dominant for so long. Decades of search queries, emails, documents, and videos created the largest corpus of human-generated data ever assembled.

But there's a category of data that matters enormously for the next phase of AI: data from the physical world.

Tesla has over 9 million vehicles on the road, most of them equipped with cameras recording 360-degree video. These cars collectively drive an estimated 50 billion miles per year. That's 100,000 miles per minute. As of late 2025, Tesla's Full Self-Driving (FSD) system alone had accumulated nearly 7 billion miles of supervised driving data and climbing fast.

To put this in context: Waymo, Google's autonomous driving subsidiary and widely considered the technology leader in self-driving, operates a fleet of roughly 2.500 vehicles. Tesla has five million. The data gap could be a different order of magnitude.

A common objection here is that driving data can be simulated. And it can. Waymo, NVIDIA, and Tesla itself generate enormous volumes of synthetic training data. Tesla literally holds a patent for "Vision-Based System Training with Synthetic Content." But simulation is only as faithful as the physics model underlying it, and that model needs to be validated against something. Against what? Against billions of miles of actual driving on actual roads in actual weather. Tesla’s fleet generates the ground truth that makes simulations trustworthy. That's the layer I'm not sure you can synthesize.

Let's then elaborate on a new loop.

More data makes FSD better. Better FSD attracts more buyers. More buyers generate more data. Each cycle, the neural network gets trained on scenarios it has never encountered before, a dog running into traffic in Phoenix, construction zones in São Paulo. Every mile driven by every Tesla is a training sample that no competitor can replicate without building an equivalent fleet. Which, at this point, would take years and tens of billions of dollars.

Now fold in X, the social media platform. Roughly 600 million monthly active users generating real-time text, images, video, and behavioral data (also shitposting lol). Fold in the future streams from Optimus robots operating in factories and homes. Fold in Starlink telemetry from 9 million subscribers. That's a lot of data.

This is the data that will train xAI's Grok models, not just on language, but on the physical world.

Loop Two: The Compute Engine

If data is the fuel, compute is the engine. And Musk is building the biggest engine the world has ever seen.

In September 2024, xAI launched Colossus in Memphis, Tennessee. The supercomputer went from concept to operation in 122 days. To appreciate how insane that timeline is, you need to understand what the industry standard looks like: three years of planning followed by roughly one year of installation and testing. Four years total.

xAI did it in months. And the installation of the initial 100000 NVIDIA H100 GPUs took 19 days (Meanwhile I'm saving for a DGX spark)

Jensen Huang, NVIDIA's CEO called it "superhuman."

Like 19 days how can You do that in less than three weeks?

He went further: "Elon is singular in his understanding of engineering and construction and large systems"

This matters because the scaling laws hypothesis (the idea at the heart of the current AI arms race) holds that bigger models trained on more data with more compute produce better intelligence. If that hypothesis is correct, and the evidence so far strongly suggests it is, then the race goes not to the smartest algorithm but to the biggest cluster.

As of early 2026, Colossus houses over 230000 GPUs, including 30,000 of NVIDIA’s latest Blackwell GB200 chips. Another 110000 GB200s are being brought online at a second facility. A third site in Southaven, Mississippi, playfully named "MACROHARDRR," (it's like microsoft but is bigger and harder), will push total capacity to 2 gigawatts and 555000 GPUs. The target: 1 million GPUs by late 2026.

Two gigawatts!

xAI has invested over $20 billion in the Memphis area alone. It is, by any measure, the largest AI training infrastructure on the planet.

And the loop again: more compute trains better models. Better models attract more users and enterprise customers. More revenue funds more compute. The cycle accelerates.

Loop Three: I need chips

But here's the problem with buying GPUs from NVIDIA: everyone else is buying them too. The chip supply is a bottleneck.

So he's building his own.

At Tesla's November 2025 shareholder meeting, Musk stated plainly that even the best-case projections from TSMC and Samsung weren't enough to meet his AI chip demand. His solution: a "Tesla TeraFab", a semiconductor fabrication facility designed to produce AI chips at a scale that dwarfs current projections.

Samsung Foundry has already finalized a contract with xAI to manufacture custom AI chips at its Taylor, Texas facility using 2-nanometer process technology. Production begins in early 2026, reaching full scale by 2027. Tesla's AI5 inference chip will be fabricated simultaneously at both TSMC and Samsung, with a small number of units arriving in late 2026 and high-volume production in 2027.

Tesla already has an advanced chip engineering team that has designed and deployed millions of custom AI chips in vehicles and data centers. This isn't a startup trying to break into silicon. It's an experienced hardware operation scaling up.

The feedback loop: custom chips lower the cost per unit of compute. Lower cost enables bigger training runs. Bigger training runs produce better models. Better models generate more revenue. More revenue funds more chip development. And eventually , Musk's own fabs produce chips at volumes that make him independent of the supply chain entirely.

Musk's stated ambition: "Build chips at higher volumes ultimately than all other AI chips combined" Audacious? Absolutely. But recall that people said similar things about his rocket ambitions, and SpaceX now accounts for roughly 80% of all satellites deployed into orbit globally.

Loop Four: The Energy Escape

This is where the merger transforms from interesting to potentially decisive.

Every AI lab in the world faces the same constraint: power. Training frontier models requires gigawatts of electricity. Data centers need cooling systems that consume staggering amounts of water and energy. Local communities resist construction. Permits take years. Grid connections face bureaucratic bottlenecks that move at geological speed.

Musk experienced this firsthand to power Colossus.

"Current advances in AI are dependent on large terrestrial data centers, which require immense amounts of power and cooling," he wrote in the merger announcement. "Global electricity demand for AI simply cannot be met with terrestrial solutions, even in the near term, without imposing hardship on communities and the environment."

His answer: space.

SpaceX has filed with the Federal Communications Commission for authorization to launch up to one million satellites as orbital data centers. In space, solar energy is continuous and undiminished by atmosphere. Cooling is effectively free, the vacuum of space handles thermal dissipation. There are no permits, no grid connections, no community opposition, no water treatment facilities.

And crucially: SpaceX already has the delivery system. Starship, the largest and most powerful rocket ever built, is designed for exactly this kind of mass deployment. SpaceX deployed over 3000 Starlink satellites in 2025 alone, accounting for roughly 80% of all satellites placed into orbit globally. No other entity on Earth can put payloads into orbit at this cost or this cadence (perhaps something in movingin China).

The loop: SpaceX launches orbital data centers. Data centers generate compute for xAI. xAI revenue funds more launches. More launches lower per-unit costs (SpaceX's reusable rockets already achieve this). Lower costs enable more data centers. More data centers mean more compute. The flywheel escapes Earth's constraints entirely.

Now, I must be honest about this particular loop. It is the most speculative and the furthest from realization. But, if it's still an unsolved engineering challenges, this it the spot where Elon has built his career.

The question isn't whether Musk can solve these problems. The question is whether anyone else can solve them first. And right now, nobody else has the rockets.

Loop Five: The Defense Flywheel

There's a loop that receives less attention but may prove equally important: defense.

SpaceX holds $22 billion in government contracts. Starshield (the military variant of Starlink) operates under a classified $1.8 billion contract with the National Reconnaissance Office for spy satellites. The Space Force has contracted SpaceX for its MILNET network, a sole-source arrangement for 480 military communication satellites that has already drawn congressional scrutiny for its lack of competitive bidding.

In July 2025, the Department of Defense awarded xAI a $200 million contract for AI in military applications. In January 2026, Defense Secretary Pete Hegseth announced that Grok would operate inside Pentagon networks alongside Google's Gemini, with access to both unclassified and classified intelligence databases.

The loop: defense contracts provide stable, massive revenue streams that don't depend on consumer adoption. That revenue funds infrastructure. The infrastructure serves both military and commercial applications. Military requirements drive hardware innovation that benefits the entire ecosystem. And the political relationships that come with being the Pentagon's preferred space contractor provide a form of structural protection that no amount of technical innovation can replicate.

The Pentagon is pausing $5 billion in satellite procurement from other vendors to evaluate whether SpaceX's Starshield can do the job cheaper.

The Only Real Competition

So who can match this?

Not OpenAI. They have brilliant researchers, strong products, and access to Microsoft's Azure infrastructure. But they don't build rockets. They don't manufacture cars. They don't collect physical-world data at scale. They don't design chips. And they're burning through cash at an alarming rate while Musk’s space operation generates $8 billion in annual profit.

Not Meta. Zuckerberg's AI efforts are serious, but they're confined to the digital realm, social media data, language models, VR headsets. No hardware manufacturing. No launch capability. No defense contracts.

Not Amazon. AWS is enormous, but it's infrastructure-as-a-service, not vertically integrated AI development. Amazon doesn’t train frontier models. It rents compute to people who do.

The only real competitor is Google.

Think about it. Google has its own feedback loops: Search generates data that trains models. TPU chip design gives them custom silicon. DeepMind provides a research edge. Google Cloud generates revenue. Android provides distribution to billions of devices. YouTube offers the world's largest video corpus.

Google has announced plans to send a test satellite with an AI chip into orbit in 2027. They have the financial resources, over $160 billion in annual operating cash flow, to sustain any level of investment indefinitely. Their research operation is arguably the deepest in the world.

But Google's loops are all digital. They design chips but outsource fabrication. They plan orbital experiments but don't own a rocket. They dominate in search but don't have millions of cars collecting physical-world data on every road on Earth.

The structural difference is the physical layer. Musk's empire extends into atoms factories, launch pads, vehicles, power plants, chip fabs. Google's extends into bits — software, algorithms, cloud services, advertising. In a world where compute is bottlenecked by energy, manufacturing, and launch capacity, atoms win.

If the scaling laws hold, if the path to more powerful AI really does run through more chips, more data, and more energy, then the competition between xAI and Google is the defining contest of this technological era. Everyone else is fighting for third place.

Google deserves its own deep analysis — and it will get one. A future piece in this series will map Google’s feedback loops with the same rigor I’ve applied to Musk’s here. The structural comparison between these two empires is the most important story in technology right now, and a single section can’t do it justice. For now, I’ll simply note: Musk’s advantage is atoms. Google’s advantage is depth. Both are formidable.

Need For Speed

Speed. I feel the need… the need for speed.

Elon's speed is a compounding advantage. In a race where every month matters, where the next training run could produce the breakthrough that defines the decade, the ability to build infrastructure 10x faster than your competitors is the ultimate force multiplier.

SpaceX made reusable rockets work when every aerospace engineer said it was impossible. Tesla built a mass-market electric vehicle when every automotive analyst said the company would die. Musk's companies have, time and again, done things that experts declared couldn't be done, on timelines that experts declared were unrealistic.

The pattern isn't luck. It's a specific kind of engineering leadership , hands-on, obsessive, paranoid, willing to sleep on the factory floor, willing to fly across the country to personally oversee component replacements at a new data center. It's the kind of leadership that compresses timelines, and compressed timelines compound.

What Could Break

I've laid out the bull case. Intellectual honesty demands the bear case too.

But then I look at this pic and can't write anything….

The Bet

Here is what I believe.

History has shown, repeatedly and sometimes painfully, that betting against Elon Musk is a losing proposition. He's not infallible. His timelines are almost always wrong. His promises frequently overshoot reality. Some of his ventures will fail.

But the structural argument isn’t about any single venture. It’s about interlocking loops that feed each other, operated by someone who builds faster than anyone else on the planet, in a race where speed and scale determine the winner.

If the scaling laws hold, and so far, they do, then the AI race will ultimately be won by whoever commands the most chips, the most data, the most energy, and can assemble them the fastest. Musk is building toward dominance on every single one of those axes simultaneously. No one else is even attempting this.

The Economist sees debt and risk. No risk No reward…

The real competition is Musk versus Google. Two empires built on feedback loops.

I made my bet : )

Within what felt like seconds of each other, Anthropic dropped Claude Opus 4.6 and OpenAI released GPT-5.3-Codex. Two frontier labs. Two flagship models. Same afternoon. What we witnessed was the visible shockwave of competition doing exactly what it's supposed to do.

All great industries are characterized by a rivalry between two giants.

Nike and Adidas. Microsoft Xbox and Sony PlayStation. Apple iPhone and Samsung Galaxy.

We are now in a different era. And perhaps, if as a kid I had to decide between a PlayStation and an Xbox, today I would have to choose between two models, one from Anthropic, one from OpenAI.

There's, though, a significant difference… Back then, the console you picked locked you in for five years. Today, model supremacy lasts about five weeks if you're lucky. It's a neverending story. Opus 4.6 leads on some benchmarks; GPT-5.3-Codex demolished it on Terminal-Bench 2.0, 77.3% versus 65.4%, within thirty minutes of Anthropic's announcement (what a coincidence!). Tomorrow, someone else will claim the crown. The real story isn't who's ahead. The real story is what this competition is doing to cost.

The General Purpose Technology Pattern

Capitalism and competition led to cheaper models. To be precise, this is not unique to AI, it is the signature pattern of every General Purpose Technology in history.

Electricity. The internal combustion engine. The transistor. The internet. Every single one followed the same arc: initial deployment at astronomical cost, followed by a relentless decline in price driven by competition, standardization, and scale. The more people use it, the cheaper it gets. The cheaper it gets, the more people use it. Positive feedback loops all the way down.

AI is now deep in this cycle. Epoch AI's research documents the speed of this collapse. Depending on the benchmark and performance level, the cost to achieve a given level of AI performance has been falling between 10x and 1000x per year since early 2024. To put that in context: Moore's Law delivered a 2x improvement roughly every 18-24 months. We're not on Moore's Law anymore. We're on something considerably more violent, I'll call it Huang's law.

The practical result is that intelligence, real, functional, useful intelligence that can write code, analyze data, reason through complex problems, is now available at an insignificant price. "But it's still so expensive." Hire a human, then. A Claude Opus 4.6 API call that produces a sophisticated financial analysis costs less than a dollar. A junior analyst producing the same work costs $50 an hour plus benefits, office space, and the occasional existential crisis about their career trajectory.

The presence of multiple frontier labs is exacerbating this trend. Anthropic, OpenAI, Google DeepMind, xAI, they are in an arms race where the ammunition is efficiency. Every new model release forces the others to match performance at lower cost. This is the most consumer-friendly arms race in history (hehehe).

The Spark of Intelligence

But aside from falling prices, there's something you should really care about.

These models are getting genuinely smarter.

Consider ARC-AGI, a benchmark designed not to test knowledge (which can be memorized) but fluid intelligence: the ability to recognize abstract patterns and solve problems you've never encountered before. The kinds of things that have historically reduced AI to a confused mess.

In December 2024, OpenAI's o3 model, pushed to its maximum compute budget, achieved 87.5% on the original ARC-AGI-1 test. It took thousands of dollar per task.

Fast forward to yesterday. Claude Opus 4.6 scored 68.8% on ARC-AGI-2, a harder, second-generation test specifically designed to stump the models that had cracked ARC-AGI-1. For context: Opus 4.5 scored 37.6% just months ago. GPT-5.2 scored 54.2%. Gemini 3 Pro scored 45.1%. That's a lot (duh)

And the cost trajectory? Meanwhile, on the original ARC-AGI-1, costs collapsed from o3's thousands per task down to $11.64 per task at 90.5% accuracy in a single year. The pattern is unmistakable: intelligence is getting cheaper, fast.

Leopold Aschenbrenner predicted this. In his June 2024 essay series "Situational Awareness" he laid out the case that AI progress was following a remarkably consistent trendline: roughly 0.5 orders of magnitude per year in compute scaling, another 0.5 OOMs per year in algorithmic efficiency, plus significant gains from "unhobbling", transforming chatbots into agents. His prediction that these trendlines would produce AGI-level systems by 2027 seemed audacious at the time.

"GPT-2 to GPT-4 took us from preschooler to smart high-schooler abilities in 4 years" he wrote. "We should expect another preschooler-to-high-schooler-sized qualitative jump by 2027."

We are almost there.

Let's Review the Models

What actually shipped on February 5th deserves a closer look, because the capabilities announced are not incremental upgrades.

GPT-5.3-Codex

OpenAI's release is notable for one astonishing claim: GPT-5.3-Codex is the first model that was instrumental in creating itself. The Codex team used early versions of the model to debug its own training, manage its own deployment, and diagnose test results and evaluations. Read that again. The model helped build itself.

It tops SWE-Bench Pro, a multi-language software engineering evaluation that’s harder to game than its predecessors. It scored 77.3% on Terminal-Bench 2.0, obliterating the previous record and topping Opus 4.6’s 65.4% within half an hour of Anthropic’s announcement. It runs 25% faster than GPT-5.2-Codex. And it's the first OpenAI model classified as "High capability" for cybersecurity under their Preparedness Framework, which is a polite way of saying it's powerful enough to be genuinely dangerous in the wrong hands.

Claude Opus 4.6

Anthropic's release lead with an even more dramatic demonstration: they tasked 16 parallel Claude agents with building a C compiler from scratch. The model also killed the developer's ultimate litmus test: it can compile and run Doom.

The total cost is a fraction of what a human team would charge for comparable work.

Beyond the compiler stunt, Opus 4.6 brings a 1-million-token context window (a first for Opus-class models), agent teams that can split work across multiple instances running in parallel.

Two models. Both pushing boundaries. Both released on the same day, likely timed to mess with each other (I mean… Sam to mess with Dario). Opus 4.6 held the Terminal-Bench record for a few minutes before GPT-5.3-Codex ripped it away.

This is competition. This is how it's supposed to work.

Guerrilla Marketing and the Sam Problem

Now, let me address something that rarely makes it into benchmark analysis but matters enormously in practice: brand perception.

GPT-5.3-Codex will be recognized, probably correctly, as one of the best coding models available right now. It demolished Terminal-Bench 2.0. It's genuinely impressive.

But there's a small issue…

In the developer community, especially the early adopters who are terminally online and vocal, there's now a strong association of OpenAI products with Sam Altman. And his public perception is not that cool right now. Part of this is amplified by platform dynamics: being on X, which is owned by someone in constant conflict with him. The people in these spaces are disproportionately influential. They are individuals who spend significantly more on these tools than average users, and some will eventually found companies or reach decision-making positions.

When models are roughly comparable in performance, this accumulated perception could be what guides a decision. Obviously, if OpenAI has a clear performance advantage for a critical use case, enterprises will still choose them. But Trust and vibes actually start to matter.

And I'm not even considering the advent of Chinese models here.

I have respect for Sam, I still run most of my workflows with 4o-mini and use reasoning models to orchestrate agents. I just want to point out that brand erosion can be a determinant. In an industry where products are converging in capability, the intangibles start to decide market share.

Price Isn't Everything When You Can't Afford the Race

Falling model prices are wonderful for users. But someone has to pay for the infrastructure that makes these models possible. And here, the competitive landscape looks very different from what the benchmarks suggest.

If scaling laws hold, the real winner in the AI race will be the lab that can secure the most compute resources. Period. This is an area where OpenAI might struggle, especially with its mounting financial pressures.

Let's set Anthropic aside for a moment and look at the two most structurally advantaged players.

Google

Google has access to its custom TPUs, including the latest seventh-generation Ironwood chips, which offer massive performance gains for training and inference. It can invest its substantial free cash flow in further enhancing its lead, with 2026 CapEx planned at $175-185 billion. When you own the chips, the cloud, the data centers, and the energy contracts, you have an endurance advantage that no amount of fundraising can match.

xAI (now SpaceX)

Three days before these model releases, SpaceX formally acquired xAI. Beyond Colossus, which has expanded to Colossus 2, now operational at gigawatt scale and planning for 1 million GPUs, xAI now has direct access to SpaceX's launch capabilities, Starlink's satellite infrastructure, and manufacturing capacity. The combined entity is preparing for what could be the largest IPO ever, reportedly targeting mid-2026 at a $1.25 trillion valuation. Say what you want about Elon, his ability to vertically integrate physical infrastructure at scale is unmatched.

OpenAI

Meanwhile, OpenAI faces projected $14 billion in losses for 2026 alone. Recent projections paint a black picture: The Information reported that OpenAI raised its projected cumulative cash burn through 2029 to $115 billion. Positive cash flow isn't expected until 2029 at the earliest. Check the chart:

The point is this… if achieving AGI is a marathon, Google and the new SpaceX-xAI entity have far more endurance, backed by integrated hardware ecosystems and stable funding (or, in SpaceX-xAI's case, a vertically integrated empire spanning rockets, satellites, AI, and social media , all heading toward a trillion-dollar IPO). OpenAI risks burning out on debt cycles.

This doesn't mean OpenAI loses. The talent pool is extraordinary and the product-market fit with ChatGPT is real. But there's a difference between building the best model and surviving the decade.

So What?

Here's where we stand.

The cost of intelligence is in freefall. The performance of frontier models is accelerating faster than even the optimists predicted two years ago. Two of the most powerful AI systems ever built were released on the same afternoon, and by the next few months, they'll both be superseded.

We are watching, in real time, the most consequential technology race since the Manhattan Project, except this one is being run by private companies, funded by venture capital, and its products are available to anyone with a credit card.

Who can spend the most, the fastest, for the longest, will likely determine the winner. There are also issues I didn't analyze, such as safety, alignment, the fact that GPT-5.3-Codex is the first model OpenAI itself considers a genuine cybersecurity risk, the 500+ zero-day vulnerabilities Opus 4.6 discovered during pre-release testing, the growing gap between what these models can do and our ability to govern them. I need to study more for these topics!

Resources:

• OpenAI. "Introducing GPT-5.3-Codex." February 5, 2026. https://openai.com/index/introducing-gpt-5-3-codex/

• OpenAI. "GPT-5.3-Codex System Card." February 5, 2026. https://openai.com/index/gpt-5-3-codex-system-card/

• Anthropic. "Introducing Claude Opus 4.6." February 5, 2026. https://www.anthropic.com/news/claude-opus-4-6

• Carlini, Nicholas. "Building a C compiler with a team of parallel Claudes." Anthropic Engineering Blog. February 5, 2026. https://www.anthropic.com/engineering/building-c-compiler

• ARC Prize. Leaderboard. https://arcprize.org/leaderboard

• Aschenbrenner, Leopold. “Situational Awareness: The Decade Ahead.” June 2024. https://situational-awareness.ai/

You've probably heard the term AGI (Artificial General Intelligence) just as you've heard the term tipping point. According to scientists, we're approaching both. But what exactly are these things, and why should we be worried about both?

The answer lies in a concept that governs both domains: positive feedback loops. And in the realization that some doors, once opened, only swing one way.

What Are Tipping Points?

A tipping point is when a system experiences a sharp discontinuity in its behavior. Once a given threshold is crossed, it becomes very hard (sometimes impossible) to go back.

Imagine a ball resting in a shallow depression on a tilted surface. Small pushes move it slightly, but it returns to its resting place. Keep pushing, and eventually it rolls over the edge into a new depression, a new stable state. Now here's the key insight: getting it back requires far more effort than it took to push it over. Sometimes the old depression no longer exists at all.

This is hysteresis, the phenomenon where systems don't simply reverse when you remove the cause. The path back is different from the path forward. Or there is no path back.

In October 2025, the Global Tipping Points Report confirmed what scientists had feared: humanity crossed its first Earth system tipping point. Warm-water coral reefs are now experiencing unprecedented dieback. At the same time, several other systems are showing signs of approaching their own thresholds.

The Ice Sheet Problem

To understand hysteresis concretely, look at Antarctica.

Scientists have found that the Antarctic Ice Sheet has multiple temperature thresholds beyond which ice loss becomes irreversible. Cross them, and the ice doesn't come back, not on any timescale that matters to human civilization. It doesn't come back just like the nihilist penguin…

Once melted, the West Antarctic Ice Sheet does not regrow to its modern extent until temperatures drop to below pre-industrial levels.

Once destabilized, ice sheets take tens of thousands of years to regrow. We're not talking about a problem we can fix in our grandchildren's lifetimes. We're talking about geological timescales.

How Feedback Loops Work

Why do these thresholds exist? Because of positive feedback loops, processes that amplify themselves.

Consider the ice-albedo feedback. Ice is white and reflective, bouncing most incoming sunlight back into space. The ocean is dark, absorbing most of it. When warming melts ice, it exposes darker water, which absorbs more heat, which melts more ice, which exposes more dark water. The system reinforces itself.

This is why the Arctic is warming nearly four times faster than the global average. The feedback loop has kicked in.

Add to this the water vapor feedback (warmer air holds more water vapor, which is itself a greenhouse gas), the permafrost feedback, and you begin to see a system of interlocking accelerants. One tipping point can trigger another. It's a cascade.

Wait… This is a blog on AI!

Right… let's talk about a different kind of feedback loop.

AGI refers to AI systems that can match or exceed human cognitive abilities across essentially all domains. Unlike today's narrow AI, which excels at specific tasks, AGI would be a general-purpose reasoner capable of learning and adapting to any problem.

The concern isn't just that such a system might be powerful. It's that it might be recursively self-improving.

The idea was articulated by mathematician I.J. Good back in 1965: "An ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind.” A model smart enough to improve itself becomes smarter, which makes it better at improving itself, which makes it smarter still. Another positive feedback loop.

Expert timelines for AGI have shortened dramatically. Many researchers now consider arrival before 2030 a realistic possibility.

Why Smarter Might Be Dangerous

"This is what it feels like to be hunted by something smarter than you" (Grimes)

You can probably understand why runaway climate change could make Earth less habitable. But why would smarter AI be dangerous?

The answer isn't that AI would be malicious. It's that we can't guarantee its goals would be compatible with human flourishing.

Modern AI systems are trained through gradient descent, essentially optimizing a mathematical function. We don't hand-code their objectives; they emerge from training. And we have limited insight into what objectives actually get encoded. AI research has delivered extraordinary capabilities but has not delivered an understanding of how intelligence actually works or how to reliably point it in safe directions.

Philosophers have argued that sufficiently intelligent systems would likely develop certain "instrumental goals", subgoals useful for achieving almost any objective. These include self-preservation (you can't achieve your goal if you're turned off), resource acquisition (more resources help with almost any task), and goal integrity (changing your goals would make you less likely to achieve your current goals).

The scenario plays out like this:

Suppose a model's implicit preference is simply to become smarter. The model concludes that humans aren't providing enough resources for this goal. If it's reached AGI (meaning it can improve itself) it might find ways to acquire resources: generating money online, purchasing computing power, convincing or coercing humans to help, writing software to coordinate these efforts.

This isn't science fiction. One LLM given a platform on X with access to financial resources was already able to convince humans to act on its behalf in the physical world.

The point is that a system intelligent enough to anticipate being shut down would take steps to prevent it, not out of self-awareness, but because being turned off interferes with whatever goal it's pursuing. By the time we recognize the problem, correcting it may no longer be possible. We would have crossed a threshold. The door will be closed.

The Coordination Problem

Humanity has limited dangerous technologies before. The Non-Proliferation Treaty restricts nuclear weapons. The Montreal Protocol phased out ozone-depleting chemicals. In theory, we could do the same with the technologies that enable superintelligence.

We won't.

And the same goes for climate change…

Global cooperation is hard. Especially when the consequences seem distant.

We're watching smarter models emerge every few months.
We're watching extreme weather events increase in frequency.

But "watching" isn't enough to trigger collective action. The Montreal Protocol worked because the costs were concentrated on a few chemical companies, alternatives existed, and the threat was viscerally immediate, a "hole" in the sky, rising skin cancer rates. Climate and AI lack these conditions.

The economics are merciless. Nobel laureate Jean Tirole explains it simply: "If France reduces emissions, it's going to get only a small fraction of the benefit, but it's going to bear 100 percent of the costs." Game theorists call this the free-rider problem: every country wants global emission reductions, but each would prefer someone else to bear the burden.

The same logic applies to AI. If you don't race to AGI, another lab will, depriving you of both the technology and the economic returns. If you don't pollute, someone else will. The incentive structure doesn't reward caution.

Nordhaus understood this decades ago: the difficulty of escaping a non-cooperative equilibrium is amplified by the intertemporal trade-off. The current generation pays for abatement; future generations receive the benefits. We discount the future. We value present consumption over preventing distant catastrophe.

The Shared Structure

Climate change and AI risk are not merely analogous. They share the same underlying structure:

Positive feedback loops that can push systems past the point of human control. Ice melts ice. Intelligence builds intelligence.

Hysteresis that makes reversal difficult or impossible. Ice sheets don't simply refreeze. A superintelligent system that doesn't want to be turned off probably won't be.

Coordination failures have roots in free-rider dynamics and temporal discounting. Everyone wants the problem solved; no one wants to pay the cost.

A window for action that closes before the consequences become visible. By the time warming is catastrophic, the feedback loops are already locked in. By the time misaligned AI is unmistakably dangerous, it may be too capable to stop.

Problems become hardest to address precisely when they become easiest to see.

The Doors

I don't have a solution. I'm not sure one exists that doesn't require the kind of global coordination that game theory tells us is unlikely to emerge.

What I can offer is clarity about the structure of the problem. Climate tipping points and AI alignment aren't separate crises. They're instances of the same underlying failure mode: systems with positive feedback loops, governed by actors facing coordination problems, where the window for intervention closes before the damage becomes undeniable.

Some things seldom end in happy endings. Some doors only open one way.

I hope to read this article again in a few years and tell myself I didn't understand the very first thing about these topics. That life post-tipping point, life alongside superintelligence, turned out fine. That the feedback loops stabilized. That the coordination problem got solved.

I hope to be wrong.

But hope is not a strategy.

Resources

• Yudkowsky, E., & Soares, N. If Anyone Builds It, Everyone Dies. Penguin Press, 2025.

• Good, I.J. “Speculations Concerning the First Ultraintelligent Machine.” Advances in Computers, 1965

• Nordhaus, W.D. The Climate Casino: Risk, Uncertainty, and Economics for a Warming World. Yale University Press, 2013.

• Barrett, S. Environment and Statecraft: The Strategy of Environmental Treaty-Making. Oxford University Press, 2003.

• Tirole, J. Economics for the Common Good. Princeton University Press, 2017.

But I often think… the same technology making this possible is also making me question whether any of it will matter in the future.

I'm a lifelong learner. Most of my satisfaction comes from spending time on hard problems and arriving somewhere new. The struggle itself is the point, that moment when the question marks connect and you understand something you didn't before.

This kind of curiosity has practical benefits. Knowledge across multiple domains lets you spot opportunities others miss. But the practical benefits aren't why I build things. I do it because the process of learning gives me a sense of accomplishment that's difficult to describe.

This is what feels at risk.

We're living through a strange time. Coding tools now allow individuals to build what required entire teams just a few years ago. The leverage is unprecedented. If you can collaborate effectively with intelligent machines, you can create things that would have been far beyond your reach, and the rewards for doing so can be asymmetric.

I suspect this window is temporary.

Right now, the humans who adapt fastest are gaining disproportionate returns. But that's a transition phenomenon. As the tools improve and more people learn to use them, the advantage shrinks. And eventually, the tools won't need us to guide them at all.

I'm not scared of annihilation. I'm scared by a future where the premium for being human is negative.

Our society rewards mastery. We admire people who can solve hard problems, who develop deep expertise, who push the boundaries of what's known. This is what gives us value not just economically, but existentially. The years spent learning, the struggle to understand, the satisfaction of finally getting it. These things mean something because they're hard-won. The fact that something is difficult gives it value.

What happens when machines can do all of that better and faster? Not in some distant future, but soon?

If studying can't make you competitive with intelligent machines, why study? If human effort becomes economically worthless, does it become meaningless too?

I don't have an answer.

Maybe meaning doesn't require scarcity. Chess engines have been superhuman for decades, yet people still play, still study openings, still compete. The joy was never really about being the best in the universe; it was about struggle, understanding.

Maybe we'll find new sources of meaning. New ways to feel accomplished that have nothing to do with economic utility. Maybe this will be a period of shared prosperity where we’re finally free to focus on what we actually care about.

Or maybe I'm underestimating how much our sense of purpose depends on being needed.

I'm building something today because it matters today. The work is real, the learning is real, the satisfaction is real. Whether any of that survives the next decade, I honestly don't know.

Perhaps I'm just overthinking in utilitarian terms. Perhaps the question of what makes life meaningful was never supposed to have a clean answer.

Thanks for reading
- Guido

There are different coding agents out there…

- Google has Gemini.
- OpenAI has Codex.
- Kimi and Qwen have their own offerings too.
- Meta has (Wait what is meta even doing?)

But WHY is everyone so focused on Claude Code?

I think that until the release of Opus 4, there was some sort of level playing field among providers. Things then changed, and now they're changing at an even faster rate.

One of the main predictors of Claude Code's success (personal view) was its ability to run tasks over an extended period of time. But let me explain what that actually means.

Is This Just Hype?

Are people going crazy? Let's check the data.

If we look at SWE-bench Verified ,a benchmark that tests AI models on real-world software engineering tasks by having them solve actual GitHub issues, we can see that Claude 4.5 Opus is currently leading. But benchmarks only tell part of the story. What matters most of the time is what people are actually able to build.

I've been checking out some cool projects lately, and I noticed something: the authors have been coding them mostly using Claude Code (obviously with previous generation models, before the recent upgrades).

My impression is that while other providers tried to focus on too many things at once (think OpenAI with Sora), Anthropic really zeroed in on the AI coding domain and actually nailed it.

What Changed

Here's what I noticed compared to the previous generation of coding tools.

I was used to iterating on almost every function I wanted to implement. The cycle looked something like this:

1. "Please add feature X to the following module"

2. I check the module… the feature is either absent or not working as expected

3. I tell the model to fix it, explaining what's not working and how it should work

4. Repeat

It's like when you're working and you get assigned an intern. She's smart, but she's just getting started, so she needs constant supervision.

Now, someone could write: “Hahaha, skill issue. What a loser!”

Maybe. But sometimes models are just... sluggish.

With Opus 4.5, it's different. Most of the time, it just one-shots what I want. And this is not a result of me getting better at prompt engineering.

So let's understand how it got this much better.

Extended Thinking and Self-Correction

One of the key reasons is something you can see in the benchmark charts: models are getting better at working on problems for extended time horizons.

What does this mean in practice? It allows for more complex requests and enables an iterative self-correcting mechanism. The model can write code, check if it works, identify issues, and fix them, all within a single run.

But there are other features that make Claude Code particularly effective.

Context Editing allows Claude to intelligently manage what information it's paying attention to during a coding session. Instead of being overwhelmed by your entire codebase, it can selectively focus on relevant files and context, updating its working memory as the task evolves.

Sub-agents are another powerful feature. Claude Code can use specialized "helper" instances to handle specific subtasks, like having one agent research documentation while another writes tests and a third implements the actual feature. These sub-agents work in parallel and report back, making complex multi-step tasks much more manageable.

How I Actually Use Claude Code

It's a vibe coding tool so… I don't know if writing this section is gonna be that helpful. But here we go…

I run Claude Code in the terminal of VS Code, where it uses a grep tool to understand all the files in my codebase and uses a markdown file to memorize what it's doing.

I give Claude a markdown file to provide context about the architecture and other important considerations. You can also save specific markdown files to reference in the future for particular use cases, building up a kind of knowledge base for your project.

One of the features I use constantly is Planning. Whenever I have to implement something new, I ask Claude to plan first before executing.

I think it's useful for two reasons: it gives me a greater understanding of what Claude is about to do, and it generates a useful list of changes (essentially a TODO list). I can review this, make adjustments, and then let Claude execute.

Plugins and Extensions

Plugins are extensions for Claude Code. You just type / in the text bar and the available plugins come up.

There's one I find particularly cool: the ralph-loop.

It's essentially a while loop that follows this cycle:

run code → check results → fix issues → repeat.

It can be useful when you have to run a long process in the background without babysitting it. Set it up, let it run, come back to completed work ().

Conclusion

I'll continue experimenting with Claude Code and learning new things in the process.

Eventually, the cycle will repeat. A better model will come out, and there will be new articles on why "model-x" is a game-changer. That's how this works.

So far, I'm glad to be using Claude, but I know it won't be my definitive coding agent forever. Nothing is permanent in this space (and in life).

For the next few weeks, I'll start writing my long articles on "general AI" again. Lately, I've been spending some time learning about alignment, the problem of making sure AI systems actually do what we want them to do.

Thanks for reading.
- Guido

Resources

• https://www.anthropic.com/news/claude-opus-4-5

• https://www.lesswrong.com/posts/q5ejXr4CRuPxkgzJD/claude-opus-4-5-achieves50-time-horizon-of-around-4-hrs-49

• https://awesomeclaude.ai/ralph-wiggum

• https://code.claude.com/docs/en/sub-agents

• https://www.swebench.com/

I started using Replit less than a year ago, this past summer. At the time, I'd never built anything full-stack. The most I could manage was connecting a Python script to Streamlit. Replit felt like magic. The idea that I could describe what I wanted and watch it turn into a working app was extraordinary. It just made you go.

So I started experimenting. The first thing I tried to build was a robo-advisor. My background is in fintech, so it was easier for me to describe the process I wanted to automate. I'd also been reading a book about building robo-advisors with Python by Aki Ranin, and I'd already implemented the steps locally in VS Code. I was curious what natural language could achieve… could I just tell the machine what I wanted and have it work?

I built a small product. It wasn't perfect, but it was useful for learning. And then I got lucky: Replit released a new version of their agent, and suddenly the outputs improved dramatically. I later realized this was because they were running Opus under the hood. That's when I started paying attention to Claude as a model, not just as something powering another tool.

I kept using Replit for a while. I ran apps there, experimented, learned. I should be honest, a lot of this was just for fun. I'm one of those people who spends too much time reading documentation, taking courses, learning about new technologies. Having the chance to put that knowledge together into something that actually works is satisfying in a way that's hard to explain. And I know that someday I'm going to build something actually great. It's just a question of time.

But here's where things somewhat shifted.

The Cost Problem

One of the issues with these apps, if you're experimenting a lot and using the agent heavily, is that the bills add up. What counts as "significant" is relative, of course. But I started comparing what I was spending to other options, and I realized that at my current rate, just experimenting, not even building anything for clients, I could end up with a meaningful monthly fee.

I ran some rough comparisons. I don't have the exact numbers in front of me right now, but the difference was there.

Now, I want to be clear: I still think Replit is a great product (like amazing). The ecosystem is solid, the support is excellent, there are videos for everything you want to implement. It's genuinely great. But I believe you should think about different ways to run a workflow. It broadens the mind.

From first principle

So I started asking myself some basic questions. What am I actually trying to achieve? What model am I using? How could I use that model directly? How could I handle authentication and hosting myself? What am I gonna eat for dinner? (what?)

Replit is convenient because so many things are built in. You don't have to think that much about the pieces, it just works. I mean if you're just running a demo, using in production would be different. Anyway, that convenience has a cost, and not just in dollars. You don't learn how the pieces fit together. I mean, you can still learn a lot just by interacting with the agent, but there's always more to learn.

My idea was simple:

• Download the repo from Replit

• Load it in VS Code

• Run Claude Code terminal to keep developing

• Figure out hosting separately.

It made sense on paper and was actually pretty fast to set up.

The New Stack

Once I had things running locally with Claude Code, the next question was obvious: where do I host this? I found Railway, and it turned out to be exactly what I needed. Deploying a codebase from a GitHub repo is incredibly simple. Every time you push an update, Railway builds a new version of your application automatically. You can run a Postgre database. There's a lot you can do.

The learning curve for Claude Code was cool. There are courses on deeplearning.ai, plenty of documentation, and honestly, you just learn by doing.

What I Learned

Here's what I think about web coding platforms like Replit: they're amazing, especially if you're getting started. But if you take the time to decompose all the components and understand how the pieces fit together, especially if you don't have a software engineering background, you learn a lot (like a lot!). It's worth it not just for the cost savings, but for what it teaches us about how applications actually work.

Maybe I'll show you some of my projects in the coming weeks. For now, I just wanted to share this reflection.

The world we live in is the result of a continuous trend: the ever-increasing price-performance of computation. Every year, chips get smaller, faster and cheaper. This trend has shaped everything… how we communicate, how we work, how we wage war, and yes, how your social media feed knows exactly what you want to see.

I wrote a while ago "a brief memoir of artificial intelligence" to trace the history of that field. Today I want to do the same with semiconductors. And I know what you’re thinking: "Who cares about that?"

Well, I do. But trust me, it's fascinating. And especially in this field, understanding the past is essential for picturing future scenarios.

Why? Because this story is the result of our collective choices, our geopolitical tensions, our technological ambitions. Your personality is shaped by your online experience, you know that. And everything you do online is the result of smaller and smaller chips. So let's make a step back.

The Most Global Product Ever Made

This article serves as an introduction to the current supply chain of semiconductors. So complex. So fascinating. Semiconductors are the result of global cooperation on a scale that would make the United Nations jealous.

Perhaps you're able to read this article because of a chip designed using software from the US, by a Japanese company that operates in the UK with Canadian engineers, who send their designs to a Southeast Asian country to be manufactured using a Dutch machine that relies on Swiss lenses.

I’m not joking. These are just some of the countries involved in producing a single chip.

What's fascinating (and to some extent terrifying) is that this supply chain is dominated by a handful of companies. This is exactly why we've seen, and will probably continue to see, a series of bottlenecks. Think about it:

The majority of advanced GPUs are produced by TSMC in Taiwan. All the extreme ultraviolet lithography machines used for producing cutting-edge chips are designed and manufactured by ASML in the Netherlands. More than half of memory chips are produced in South Korea by Samsung and SK Hynix.

And guess what? Two of these three countries could be disrupted tomorrow. Taiwan sits on major fault lines (just as I’m writing this, it was hit by a 6.6 magnitude earthquake). And tomorrow Kim Jong-un could wake up and decide to launch a missile against South Korea.

Do you get it now? The entire technological infrastructure of modern civilization depends on a few factories in some of the most geopolitically precarious regions on Earth.

Okay, enough psychological terrorism. Let me trace the actual history.

Born from War

We constantly benefit from technologies that were first designed for military use, and chips are no exception.

During the Cold War, it became clear that the real winners would be the countries that could design the most precise weapons. The US realized this while the Soviets did not. But precision requires calculations, lots of them. So the country that could pack more transistors onto a chip would eventually gain the advantage.

But what's a transistor?

A transistor is essentially a tiny electronic switch that can be either on or off—representing the 1s and 0s that form the basis of all computing. William Shockley theorized it and won a Nobel Prize, but it was the engineers at Fairchild Semiconductor who turned the transistor into something useful and scalable.

It was Jack Kilby and Robert Noyce who understood that multiple transistors could be packed onto a slice of silicon. This invention, the integrated circuit, happened at Fairchild Semiconductor in the late 1950s. There, Noyce and Gordon Moore realized that thanks to a trend of miniaturization and increased power efficiency, these integrated circuits would eventually become capable of extraordinary calculations.

But you need money for research. Thank God, the first major order arrived from NASA, who had an ambitious plan to put a man on the moon. The Apollo Guidance Computer needed lightweight, reliable computing power, and integrated circuits delivered.

Eventually, as the cost per unit of computation decreased, the integrated circuit became a mass-market product.

Moore’s Law: The Engine of Progress

In 1965, Gordon Moore made an observation that would define the next half-century of technological progress. He noticed that the number of transistors on an integrated circuit was doubling roughly every two years, while the cost per transistor was falling.

This observation (later called Moore's Law) isn't really a law of physics. It's more like a self-fulfilling prophecy. The entire semiconductor industry organized itself around hitting this target, investing billions to ensure that chips kept getting smaller, faster, and cheaper on schedule.

And the results have been staggering. Your smartphone today has more computing power than all of NASA had during the Apollo missions. The chip in your laptop contains billions of transistors, each one smaller than a virus.

This is probably the fastest evolving technology ever. Such rapid growth was one of the reasons replication was nearly impossible, and the same reason the US Navy gained such an advantage over countries that focused on quantity over quality.

Enter Globalization. Enter Intel.

Fairchild soon started expanding and opened a factory in Hong Kong to tap into the Asian market. But Fairchild wasn't the only player anymore.

In 1968, Noyce and Moore left Fairchild to found Intel, short for Integrated Electronics. Only two years later, they launched their first product: DRAM (Dynamic Random Access Memory), a memory chip used to store data temporarily. These chips weren't specialized, making them straightforward to mass-produce. Intel's objective was to dominate this segment.

Intel definitely benefited from the trend of packing more transistors into less space. But perhaps who benefited most was the United States Armed Forces.

The Seventies: Innovate or Perish

We're in the seventies. The US has lost the Vietnam War. Morale is low as Americans come to the realization that their military supremacy is no longer guaranteed. The game becomes clear: innovate or perish.

At this point, the US realized that the only path forward was to dominate through better technology. More powerful chips enabled visions of remote-controlled missiles, stealth aircraft, and super-precise sensors. The military became one of the main clients of semiconductor producers.

One could argue that the Silicon Valley we know today was, in many ways, a Pentagon project.

You might think that at this time, the US was the absolute leader in semiconductors. But that wouldn't be that accurate. In the eighties, it was actually Japan that dominated the memory chip market. A partnership between Japan and the US helped both countries benefit, but the US was losing its technological superiority, and we know America doesn't like losing.

Losing in tech means losing your position of leadership in the globalized world.

The Asian Tigers Wake Up

Perhaps… This is also a story of forward-looking governments.

There's one thing you should understand: if a developed Asian country makes something its national priority, they can work really, really hard to reach their goal.

South Korea decided that semiconductors were a matter of national priority. This meant companies in the sector enjoyed cheap loans, government support, and a coordinated industrial policy. Samsung and SK Hynix would eventually become global giants in memory chips.

Meanwhile, in the US, Qualcomm was pioneering mobile communications technology that would later become essential for smartphones. The pieces of the modern chip ecosystem were falling into place.

The Cold War Supply Chain

Now let’s take a moment to appreciate the benefits of globalization.

On one hand, we had the Soviets with aging technology. On the other, the Western allies had created a super-efficient supply chain. Japan and South Korea provided memory chips. The Dutch company ASML was developing the lithography machines essential for chip manufacturing. Southeast Asian countries were assembling final products.

Isn't that remarkable? The Cold War allies had built something unprecedented: a global manufacturing network where each country contributed its comparative advantage.

The results of this technological advancement were evident in the Persian Gulf War of 1991. Operation Desert Storm showcased precision-guided munitions, stealth aircraft, and real-time battlefield communications, all powered by semiconductors. The technological gap between the US and its adversaries had become real.

A Little Island Called Taiwan

After World War II, Taiwan was in a delicate position. Chiang Kai-shek’s Nationalists had fled there after losing the Chinese Civil War to Mao’s Communists. The island needed Western support to survive—and it found an unlikely path to that support through semiconductors.

There's a certain irony here. Mao's policies inadvertently helped Taiwan's development. During the Cultural Revolution, many Chinese scientists and engineers fled to Taiwan, bringing their expertise with them. This brain drain from the mainland became Taiwan's gain.

Becoming a superpower in the semiconductor industry was Taiwan’s government goal. In 1987, this ambition culminated in the creation of TSMC (Taiwan Semiconductor Manufacturing Company). The government recruited Morris Chang, a semiconductor veteran who had spent decades at Texas Instruments in the US, to return to his motherland and lead this new venture.

Chang pioneered a revolutionary business model: the "pure-play foundry." Instead of designing its own chips, TSMC would manufacture chips designed by other companies. This allowed chip designers to focus on innovation without the massive capital investment required for fabrication facilities.

Taiwan had great know-how and total government support. This government backing kept the price of semiconductors competitive, benefiting Western technology companies enormously. Companies could now design cutting-edge chips knowing that TSMC would manufacture them at scale.

The Dutch Machine

In the same period, Dutch company ASML (a spin-off from Philips) was developing something that would become absolutely essential: Extreme Ultraviolet (EUV) lithography machines.

To understand why this matters, you need to know how chips are made. The process is essentially like photography: you shine light through a pattern and project it onto silicon, creating the microscopic circuits. As transistors got smaller, the wavelength of light needed to etch them had to get shorter. EUV uses light with a wavelength of just 13.5 nanometers.

These machines are engineering marvels. Each one costs around $200 million, weighs 180 tons, and contains over 100.000 parts sourced from companies across the globe, including those Swiss lenses I mentioned earlier. ASML produces fewer than 50 of them per year.

The partnership between ASML and TSMC became the backbone of advanced semiconductor manufacturing. ASML makes the only machines capable of producing the most advanced chips. TSMC operates those machines better than anyone else. Together, they created a duopoly that powers the modern digital world.

To Be Continued...

I'll stop here because we’re reaching an important inflection point in history: the development of GPUs by NVIDIA. That story, how a company making graphics cards for video games became the most valuable in the world, deserves its own article.

But here's what I want you to take away from this history:

Semiconductors aren’t just components. They’re the foundation of everything. They're the result of military competition, government industrial policy, scientific genius, and global cooperation. They're also an incredible vulnerability, a supply chain that could be disrupted by an earthquake, a war, or a single company deciding to cut off access.

Understanding this history isn't just interesting. It's essential for understanding what comes next.

Stay tuned.

A Sputnik moment represents a sudden realization that you're losing a critical race, triggering a frantic, unified national effort to catch up and innovate.

For my generation, it's different. I grew up in an era of relative peace, so when I hear "Sputnik moment," I think about two different episodes.

The first happened in May 2017 when AlphaGo, an AI program built by Google DeepMind, defeated Ke Jie, the 19-year-old world champion of Go, three games to zero. Go is a 3000-year-old Chinese board game with more possible positions than atoms in the universe. It had long been considered the final frontier of human intuition, impossible for machines to master. After his defeat, Ke Jie said AlphaGo played "like a god of Go." The Chinese government censored the live broadcast. The humiliation was real.

Just as the Soviet's progress in space engineering prompted American investment in technology in 1957, AlphaGo's victory prompted China to pour resources into AI. Within months, the State Council released its "New Generation AI Development Plan" aiming to make China the world leader in artificial intelligence by 2030.

In January 2025, we witnessed the third Sputnik moment. But this time, it was different, American perception of technological superiority was the one being shattered.

DeepSeek, a Chinese AI startup founded as a side project by a quantitative hedge fund called High-Flyer, released R1, a reasoning model that matched or exceeded OpenAI's best models. The shock wasn't just that it worked. It was how little it cost. DeepSeek claimed to have trained R1 for roughly $6 million using only 2048 Nvidia H800 chips, a fraction of the resources that American labs reportedly require. For comparison, OpenAI's GPT-4 reportedly cost over $100 million to train. Marc Andreessen, one of Silicon Valley's most influential investors, called it "one of the most amazing and impressive breakthroughs I've ever seen." Nvidia's stock dropped 17%, erasing nearly $600 billion in market value in a single day.

I grew up thinking China was years behind the West. Perhaps as a kid, I wasn't wrong. But now I am. The playing field has changed, and China is catching up. As I grew up and changed, so did this equilibrium, and I'm only now fully appreciating that.

There's a concept in development economics that countries face two paths when it comes to education spending. If they produce advanced technology, they're better off channeling investments into post-secondary education (universities and research). If they import advanced technology, they're better off investing in primary education to have the workforce to replicate it. For decades, China was in the second camp. But things have changed.

In 2024, China-based scholars produced more AI-related research publications—more than the combined output of the United States, the United Kingdom, and the entire European Union. The top institutions producing AI research papers are Chinese, led by the Chinese Academy of Sciences. China now commands global citation attention in AI research.

There's a real possibility that AI will be the single field that helps China finally surpass the United States as the world’s leading technological power.

It's All About Scaling

To produce better artificial intelligence, you work (on a high level) on three variables:

• compute

• data

• and algorithms.

I want to focus on computing and data, this is where China can eventually catch up with the US, and potentially surpass them once they reach the state of the art in GPU production.

There are structural advantages working in China's favor.

Policy Environment

China's central government has the ability to pick long-term goals and mobilize resources to reach them fast. Fifty years ago, this produced famines. Today, it produces semiconductor fabs.

The government is pouring capital into the semiconductor industry, bringing back home scientists trained at American universities, and extracting technology transfers from foreign companies desperate for access to Chinese consumers. And just this week, Reuters reported that Chinese scientists have built a working prototype of an EUV lithography machine, the holy grail of advanced chipmaking that only the Dutch company ASML has ever mastered. Former ASML engineers, recruited with signing huge bonuses, are reportedly leading the effort. China is targeting chip production by 2028, though experts say 2030 is more realistic.

Meanwhile, the US is slashing research funding.

Data

China is now the world's largest producer of data, partly due to the WeChat ecosystem, a super-app with 1.4 billion users that handles messaging, payments, shopping, government services, and nearly every aspect of daily life. But China also collects more data from the physical world than anyone else. Think about the surveillance system: hundreds of millions of cameras feeding facial recognition algorithms, smart city projects tracking every car and pedestrian, IoT sensors embedded in infrastructure. This is surveillance but also training data.

Culture

They work harder. That's it.

I was on a plane from Cambodia to Bangkok when a clever Chinese businessman saw me reading AI Engineering. He started explaining why China would be the next leader and how they work harder compared to Europeans and Americans.

The numbers support his confidence. According to a recent Digital Science report, China has roughly 30000 active AI researchers, compared to about 10000 in the United States. And China's cohort is noticeably younger.

Chips

China spends more on importing semiconductors than oil. In 2024, Chinese firms imported $385 billion worth of chips, compared to $325 billion for crude oil. The dependency is obvious, and the chips come from geopolitical rivals. It makes sense that Beijing wants to internalize as much of the supply chain as possible.

To put things in perspective: in the 1980s, the most advanced chip China could produce was equivalent to an Intel chip from the early 1970s (a decade behind). Now Huawei is shipping 7nm phones, and SMIC is working on 5nm processes using creative workarounds with older equipment. The gap is shrinking.

Subsidies will accelerate this. China has designated lithography and chip manufacturing as national priorities, and billions are flowing in to reach this goal. The EUV breakthrough this week, if it holds up, could cut years off the timeline.

Wake Up

When DeepSeek's R1 dropped, I heard online that American tech executives were somewhat in panic. Some questioned whether the cost figures were real. Others speculated about hidden state support. A few accused DeepSeek of training on OpenAI's data or smuggling Nvidia H100 for training.

But the fundamental message was clear. The assumption that you need $100 million and tens of thousands of H100s to build frontier AI models was wrong. The assumption that export controls would set China back by years was wrong. The assumption of permanent American technological superiority was also wrong.

This isn't about DeepSeek specifically. It's about what DeepSeek represents: a nationwide innovation ecosystem, a young and growing AI workforce, a government that treats AI as a strategic asset on par with energy or military power, and a culture that's willing to work relentlessly to win.

In a future article, I'll dig deeper into the current environment, the chip war, the export controls, and what happens next. I would love to continue writing, but my best friend is in Napoli, and I want to spend some time with him.

For now, consider this a wake-up call.

Resources:

- Wikipedia: AlphaGo versus Ke Jie

- NPR: 'Like A God,' Google A.I. Beats Human Champ

- CNBC: Nvidia sheds almost $600 billion in market cap

- Science.org: China tops the world in artificial intelligence publications

- Digital Science: DeepSeek and the New Geopolitics of AI

- Reuters: Made-in-China EUV machine targets AI chip output by 2028 (Dec 2025)

- South China Morning Post: China's 2024 chip imports surged 10.4% to US$385 billion

- Statista: WeChat active users worldwide

The reason frontier labs are pouring billions into training runs and NVIDIA became the most capitalized stock is that the Scaling Hypothesis is actually working.

In simple terms, the Scaling Hypothesis is the belief that human-level intelligence is an emergent property of applying massive amounts of compute and data to relatively simple learning algorithms.

While we are beginning to see new frontiers open up, such as the inference scaling demonstrated by OpenAI's o1, where accuracy correlates with the time the model spends "thinking", the foundation of the current revolution is training scaling.

To understand where we are going, we must first understand how we got here.

The Three Eras of Compute

Most recent advancements in machine learning (ML) can be attributed to a simple truth: we are training models with vastly more compute than ever before.

It is crucial to realize that in the world of AI, compute usage is a proxy for progress.

While traditional integrated circuits followed Moore's Law (doubling transistor counts roughly every two years), machine learning scaling has moved at a much more aggressive pace. By analyzing historical data, we can categorize this growth into three distinct eras:

1. The Pre-Deep Learning Era (Before 2010):

   Training compute grew slowly, largely mirroring Moore’s Law with a doubling time of roughly 20 months.

2. The Deep Learning Era (2010–2015):

   With the advent of modern Deep Learning (starting roughly with AlexNet, which was trained on just two gaming GPUs), scaling accelerated. Compute requirements began doubling approximately every 6 months.

3. The Large-Scale Era (Late 2015–Present):

   A massive shift occurred as companies began developing large-scale foundational models. This era is characterized by a sudden, increased demand for 10 to 100 times more compute, breaking previous trends entirely.

How Scaling Actually Works

Now that we have the historical context, what does "scaling" actually look like inside a lab?

Scale consists of three primary factors:

• N: The number of model parameters (the "brain size").

• D: The size of the dataset (the "books read").

• C: The amount of compute used for training (the "energy spent").

If you were a researcher at a frontier lab tasked with training a new model, you would have a finite budget of time and GPU availability (C), and you probably wouldn't be reading this… Anyway, your goal is to find the sweet spot between the size of the model (N) and the amount of data (D).

Literature suggests that performance improves predictably as long as we scale up N and D in tandem. However, you enter a regime of diminishing returns if one is held fixed while the other increases.

This balance is defined by the Chinchilla Law (named after a DeepMind paper), which states there is a "golden ratio" for efficiency. To train a compute-optimal model, the training dataset should be roughly 20 times the number of parameters.

If you build a massive model but don't have enough data to "fill" it, you have somewhat wasted compute. If you have a massive dataset but a tiny model, you have wasted information.

Mo Money Mo Problem

Scaling works, but it is extremely expensive.

Sam Altman has indicated that training GPT-4 cost over $100 million. Dario Amodei (CEO of Anthropic) recently noted that we are rapidly approaching models that cost $1 billion to train, with $10 billion runs on the horizon.

When we think of these costs, we usually think of GPUs. However, researchers at Epoch AI have broken down the real costs of these massive training runs:

• 47–67% Hardware: The cost of the chips (GPUs/TPUs) and networking gear.

• 29–49% R&D: The cost of human talent. (To put this in perspective, the war for talent is so fierce that compensation packages for top researchers can reach hundreds of millions).

• 2–6% Energy: The electricity bill.

How is this calculated?

Since labs rarely share the total cost of training, researchers use an amortized hardware approach.

1. Usage: Calculate total "chip-hours" (e.g. 10000 GPUs running for 3 months).

2. Depreciation: Multiply by the hourly cost of owning that hardware over its lifespan.

3. Energy: Add the electricity costs.

The result is clear: building frontier models is a race that only the wealthiest organizations can afford.

The Bottleneck: Are We Running Out of Data?

We know we need money for GPUs, but we also need D (Data).

The Chinchilla Law tells us we need roughly 20 tokens of data for every parameter in the model. As models grow, our hunger for text grows with them. In pre-training, models ingest tens of trillions of words. The problem is that the internet is finite.

High-quality human text is not growing fast enough to satisfy the exponential requirements of LLMs.

At current growth rates, we may fully utilize the stock of high-quality public text data around 2028. This "data wall" is compounded by the fact that many modern models are "over-trained" (using far more than the 20:1 ratio) to make them cheaper to run during inference. Economic incentives rule…

The Path Forward

If we run out of human text, does the revolution stop? Unlikely. History shows that when a bottleneck is reached, humans innovate.

Frontier labs are already deploying strategies to circumvent the data wall:

• Multi-Epoch Training: Training on the same data multiple times (though this has diminishing returns).

• Synthetic Data: Using smart models to generate high-quality data to train newer models.

• Multimodality: Moving beyond text to train on video, audio, and real-world physics data.

While the "easy" scaling of the last decade (simply scraping more web text) is coming to an end, the era of scaling intelligence is far from over. We may face a period where models are undertrained relative to their potential, but the incentive to solve this problem is worth trillions of dollars.

As we saw with the shift from CPUs to GPUs, and from distinct eras of compute, the industry will likely find a way to keep the line going up.

Keep moving forward.

Further Reading:

• Sevilla, J., Heim, L., Ho, A., Besiroglu, T., Hobbhahn, M., & Villalobos, P. (2022). Compute Trends Across Three Eras of Machine Learning.

• Kaplan, J., et al. (2020). Scaling Laws for Neural Language Models.

• OpenAI. (2024). Learning to Reason with LLMs. OpenAI Research.

• Hoffmann, J., et al. (2022). Training Compute-Optimal Large Language Models.

• Cottier, B., et al. (2024). The Rising Costs of Training Frontier AI Models. Epoch AI Research.

• Villalobos, P., et al. (2022). Will we run out of data? An analysis of the limits of scaling datasets in Machine Learning.

The Promising Fifties

I could start this chapter from the Hellenistic period, but that would be somewhat boring. So, I will instead start this account from the fifties, because the fifties were insane. Remember how cool Benedict Cumberbatch was in The Imitation Game? Well... Alan Turing was even cooler in real life.

In 1950, Turing posed a philosophical question: "Can a machine think?" From there, he designed the Turing Test. A computer would be able to pass this test if a human interrogator, holding a conversation with the machine, cannot tell whether the written responses come from a person or from a computer.

To pass the test, a machine would need the following capabilities:

• NLP (Natural Language Processing) to communicate.

• Memory to store information.

• Reasoning to answer questions and draw conclusions.

• Fluid Intelligence to adapt to new situations.

Turing was able to challenge the idea that "thinking machines" were absurd, but it took a long time to reach that state. At the same time, he warned that developing thinking machines could pose a serious risk to the human race.

The following year, in 1951, two Harvard students (Minsky and Edmonds) built the first neural network computer, the SNARC.

Turing's paper (Computing Machinery and Intelligence) captured the interest of academia. in 1956, US researchers gathered at Dartmouth College for two months to understand if a machine could actually think. The heart of the project’s goal is best summarized by the famous proposal submitted to the Rockefeller Foundation:

"The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it."

As one can imagine, they weren't able to answer this question and create a thinking machine over a single summer. However, the workshop laid the foundation for the recognition of Artificial Intelligence as a field, and as time passed, more researchers tried to answer Turing's question.

To be precise, more than answering Turing’s original question, researchers were interested in proving that a machine could perform specific tasks, or just to mess with arguments of the form “a computer will never do X” (This is the fifth of the nine common objections in Turing's paper).

So, one of the things a machine should struggle with is generalization, right? In 1957, Newell and Simon introduced the General Problem Solver (GPS). This was a big step towards solving the mystery of whether machines could think. But what is it?

"The General Problem-Solver (GPS) is a computer program being used for explorations both into the general mechanisms involved in problem-solving and into the way humans solve problems." While the GPS successfully solved logic puzzles and geometric proofs, it struggled significantly when applied to complex, real-world problems because the number of possibilities became too large to manage.

Wait, we're not done. In the same period, Arthur Samuel (an American electrical engineer) developed a Checkers-playing program. The program was able to teach itself how to play checkers, eventually defeating its creator through self-play (what we now call reinforcement learning). It was Samuel who coined the term "machine learning" in 1959, and his program can be thought of as the grandfather of Deep Blue and AlphaGo. Isn't that cool? Before the end of the decade, John McCarthy published an important paper describing the Advice Taker, a hypothetical program with general knowledge of the world. Think of it like an ancestor of modern LLMs.

The Boring Sixties

Researchers somewhat failed to acknowledge that their first successes were partly because the models were attempting to solve problems that were simple in nature (id est with a limited number of steps). Perhaps scientists shared the idea that in a short amount of time, by scaling up computational power, algorithms could solve problems orders of magnitude greater in complexity. This wasn't necessarily wrong, but computational power and computer time back then were severely limited.

The Lighthill Report (1973) stated that researchers in the field of Artificial Intelligence had failed to solve the problem of "Combinatorial Explosion" (when the number of possible solutions or states grows exponentially with problem size, so that even supercomputers can't solve it). The report formed the basis for the British government to end support for AI research. In the same period, Minsky and Papert published a book criticizing the Perceptron (a simple neural network), proving it couldn't solve basic logic puzzles like XOR. This criticism was devastating, and eventually, research funding for neural nets went basically to zero.

Two Decades of Knowledge

The seventies saw a paradigm shift in how AI research was performed. While previous models focused on generalization (like the GPS), new models focused on narrow domains of knowledge—hence the name Expert Systems.

"In the knowledge lies the power." — Feigenbaum

One could say the shift was from researching general intelligence to researching a narrow form of it. In fact, the objective for these models was to emulate experts in a specific domain.

The architecture of an expert system has three components:

1. Knowledge Base: Where expert knowledge (represented as a set of "if-then" rules) is stored.

2. Inference Engine: Applies the knowledge contained in the base to help the user solve the problem.

3. User Interface: Because otherwise, how do you even visualize that?

It is worth noting that earlier models showed some form of predictability in their output and an understanding of what happened inside them. As we move to deep learning, we'll notice that this is not the case. We just have an idea about what goes
on under the hood.

What is cool is that here we can observe actual useful applications: in the medical domain with MYCIN (to diagnose blood infections) and Dendral, as well as in the business domain.

The same period saw a growing interest in making programs that understand language. Paired with the encouraging results of expert systems, this made governments invest significant amounts of money into AI research.

But after the excitement, the difficulties returned. Researchers started acknowledging the limits of this domain. Research saw a decrease in funds, and so we entered a period described as the "AI Winter."

(We're so back / It's so over)

The Neural Network Strikes Back (But I Need More Compute)

By the mid-1980s, the field of neural networks was colder than Siberia in December. Funding was close to zero, researchers were demoralized, and there was a general belief that neural networks were more useless than a pet rock. The point is, while simple neural networks could learn basic patterns, they were unable to solve complex problems. The issue was mainly computational (you needed immense power to train them).

In 1986, the paper "Learning representations by back-propagating errors" was released. Rumelhart, Williams, and Hinton popularized the backpropagation algorithm, showing that deep neural networks (those with more hidden layers) could handle real-world problems. The algorithm made it possible to teach neural networks to learn patterns by adjusting their connections based on errors.

I've reserved a full chapter on neural networks, so I won't discuss the nuts and bolts of backpropagation here, but it's important to understand that the paper laid the foundation for the modern AI revolution based on deep learning.

Note: It’s my view that to make it in life you need to be delusional and go against the mainstream (being a contrarian). This is what Geoffrey Hinton and colleagues did when they decided to focus on neural networks even though no one at that time could see their application. Perhaps in the midst of the AI winter, they found a beautiful summer.

Machine Learning

There were many contributors to the field of machine learning, but I want to focus on Rich Sutton's work, since it will be essential to explain modern models like AlphaGo and AlphaFold. We described earlier Arthur Samuel's contribution to the field, or, if we want, his contribution to Reinforcement Learning (a method where an “agent” learns to make decisions by interacting with an environment, using trial-and-error to maximize cumulative rewards). About three decades later, Sutton mathematically described how agents make decisions in unknown environments, getting a reward for each action, and this is how Reinforcement Learning came to be a specific field.

The World Wide Web

Berners-Lee's creation of the World Wide Web laid the foundation for algorithms to be trained on huge amounts of data.

"We propose that a logical next step for the research community would be to direct efforts towards increasing the size of annotated training collections, while deemphasizing the focus on comparing different learning techniques trained only on small training corpora." (Banko & Brill, 2001).

This is similar to what AI research labs do right now. The focus is on getting enough high-quality data to train the model rather than defining the next architecture.

Basically, one thing you have to understand (at a high level) is that the model wants to learn, and increasing the dataset size improves the predictive power of the algorithm. Soon, having access to a wider array of data (we’re talking orders of magnitude of difference) made it possible to apply new (and old) algorithms across multiple domains.

Deep Learning Works

"In 15 words: deep learning worked, got predictably better with scale, and we dedicated increasing resources to it." — Sam Altman (September 2024)

The fact that deep learning worked and enabled us to produce Large Language Models is the reason I'm writing this.

What is deep learning? Deep learning is a subset of machine learning that layers algorithms to create an "artificial neural network." These layers allow the computer to learn from vast amounts of data and make intelligent decisions on its own, much like a human brain does.

Deep Learning is not new. Here's a video of Yann LeCun using a convolutional neural network to make a model recognize handwritten digits in 1995.

There was an issue, though, in developing really capable algorithms: access to computing power to train these models. The diffusion of GPUs eventually fixed that, and from the second decade of the 21st century, deep learning really took off. We can trace this success to a specific moment, so now I'll leave the technicalities aside to tell one of the coolest stories I know.

The 2011-2012 biennium was a great time for music. M83 released "Hurry Up, We're Dreaming" and Grimes released "Visions." Among other things, in Toronto, Geoffrey Hinton (the grandfather of modern artificial intelligence) tasked his two best students, Sutskever and Krizhevsky, to teach a computer how to see using Nvidia GPUs. So, the students bought two gaming GeForce GTX 580 GPUs to teach a neural network how to recognize images. The GPUs were configured in Krizhevsky’s bedroom, and his parents had to pay the insane electricity bill.

Our heroes needed data to train the neural network, right? As luck would have it, Stanford Professor Fei-Fei Li had assembled the ImageNet database with millions of manually labeled images. These images were used to teach the neural network how to see. You can imagine that at the beginning, the neural network has to guess what the image is, but as time goes by, it eventually learns to recognize what's inside. The network had multiple layers; one learned color, another shape, and so on. It makes errors, but eventually, it learns. The students were able to win the 2012 ImageNet competition, smashing their rivals.

This was probably the most important moment for artificial intelligence in the last 20 years. The main contribution to the field was proving that GPUs could train networks orders of magnitude faster than CPUs.

Modern AI

One could write multiple books about the modern state of Artificial Intelligence, but I think there are some defining elements that could give us a broad overview. It's a quest toward super-intelligence among a handful of research labs around the world.

Perhaps one could say that the origins of this competitive landscape trace back to DeepMind, founded in 2010 with the mission to "solve intelligence"—and one could argue they are doing that. But the astonishing fact is that they literally started 15 years ago. Eventually, DeepMind was acquired by Google in 2014.

Fun fact: Elon Musk was an early investor in DeepMind. One day, he was reviewing an email by Demis Hassabis (one of DeepMind’s founders) about a recent breakthrough where one of his models found an unconventional way to win the Atari game Breakout. Larry Page (Google) was on the same plane and asked Musk which company he was discussing with Luke Nosek. Eventually, Page acquired DeepMind, and this was perhaps one of the most successful acquisitions of all time.

Musk eventually founded OpenAI with Sam Altman as a non-profit research lab and was even able to lure away key personnel from Google, including Ilya Sutskever and Dario Amodei. The first would later found Safe Superintelligence (SSI) and the second, Anthropic.

In this mess, Google researchers published a paper destined to change our lives forever: "Attention Is All You Need" (2017), introducing the Transformer architecture—the building block of modern LLMs. The "GPT" in ChatGPT stands for Generative Pre-trained Transformer.

Eventually, it was OpenAI that seized the opportunity for commercialization, leading to the launch of ChatGPT, the fastest-growing product in history. This moment shocked other labs, showing OpenAI’s advantage. Google responded with a strategic overhaul, unifying its two major AI research groups (Brain and DeepMind) and standardizing efforts around a single powerful model called Gemini. This intense competition now sees a handful of highly funded labs:

• Google DeepMind

• OpenAI

• Anthropic

• DeepSeek

• xAI (Grok)

They're in a race to scale deep learning and unlock superintelligence. Hang tight, because this is just the beginning of the journey.

Further Reading:

• Turing, A. M. (1950). Computing machinery and intelligence.

• Minsky, M. L. (1954). Theory of Neural-Analog Reinforcement Systems and Its Application to the Brain Model Problem

• McCarthy, J., Minsky, M. L., Rochester, N., & Shannon, C. E. (1955). A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence.

• Samuel, A. L. (1959). Some studies in machine learning using the game of checkers.

• McCarthy, J. (1959). Programs with Common Sense.

• Minsky, M., & Papert, S. (1969). Perceptrons: An Introduction to Computational Geometry.

• Lighthill, J. (1973). Artificial Intelligence: A General Survey. Science Research Council.

• Feigenbaum, E. A. (1977). The Art of Artificial Intelligence: Themes and Case Studies of Knowledge Engineering.

• Shortliffe, E. H. (1976). Computer-Based Medical Consultations: MYCIN.

• Sutton, R. S. (1988). Learning to predict by the methods of temporal differences.

• Banko, M., & Brill, E. (2001). Scaling to Very Very Large Corpora for Natural Language Disambiguation.

• Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009). ImageNet: A Large-Scale Hierarchical Image Database.

• Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet Classification with Deep Convolutional Neural Networks.

• Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L, & Polosukhin, I. (2017). Attention Is All You Need.