Thoughts

Understanding AI

I’ve been building software for 7 years, but AI changed the way I think about software.

I’m not an ML researcher. I’m a developer who wanted to understand what is happening behind the AI tools I use every day.

AI is not magic

Traditional software follows rules written by developers.

AI works differently. You give a model a lot of data, it learns patterns, and then uses those patterns to make predictions.

At a simple level:

  1. Data
  2. Patterns
  3. Predictions

What does ChatGPT actually do?

A language model predicts what should come next based on the context.

You give it text, it processes that text as tokens, calculates probabilities, and generates the next token. It repeats this process until it produces an answer.

It’s more sophisticated than simply “predicting the next word,” but this is a useful mental model.

Tokens

Models don't read text the same way we do.

Text is broken into smaller pieces called tokens, converted into numbers, and processed by the model.

So roughly:

  1. Text
  2. Tokens
  3. Numbers
  4. Model
  5. Tokens
  6. Text

Understanding tokens also makes things like context windows and API pricing easier to understand.

Training

Before you can use a model, it has to be trained.

The model sees huge amounts of data, makes predictions, measures its mistakes, and adjusts itself. This happens over and over again using enormous amounts of computing power.

  1. Data
  2. Prediction
  3. Measure the mistake
  4. Adjust
  5. ↻ repeat

The goal is not simply to memorize answers, but to learn patterns.

Transformers

Modern language models are largely built around the transformer architecture.

One of its key ideas is attention. It allows the model to understand relationships between different parts of the context.

You don't need to understand all the mathematics to start building with AI. Understanding the basic idea is enough.

AI products still need good engineering

This is probably the part I care about most.

Adding an AI model to an app is relatively easy. Building a useful AI product is much harder.

You still need:

  • Good product decisions
  • Good UX
  • Reliable APIs
  • Good data
  • Error handling
  • Evaluation
  • Monitoring

The model is only one part of the system.

Evals

AI doesn't always produce the exact same output, so testing is different from traditional software.

Evals help answer a simple question:

Does it actually solve the problem reliably?

That matters more than how impressive the demo looks.

Coding agents

As a developer, the most interesting part of AI for me isn't just chat.

It's what happens when models can use tools.

A coding agent can understand a task, read the codebase, change files, run tests, check the result, and try again.

At that point, AI starts becoming more than autocomplete. It becomes a new way of interacting with software.

What I take away

I don't think every developer needs to become an ML researcher.

But I do think developers should understand the basic ideas behind the tools they use.

Models are tools. Data matters. Context matters. Evaluation matters. Product thinking matters.

AI is another layer of software engineering, not a replacement for it.