How AI Image Generation Works

IB Syllabus: none directly. This is a stretch page: nothing on it is examined. It sits behind A4.1.1 (generative AI as a real-world application of machine learning) and deepens A4.2.3, A4.3.2, A4.3.8 and A4.3.9 (all HL) plus the ethics points A4.4.1 and A4.4.2. SL and HL both welcome.

Why read this. Three genuinely different ideas can make a computer produce an image it has never seen, and a student who can say how the three differ understands generative AI better than one who can only name this year’s tools. The plain-language account comes first; the mathematics is layered underneath and labelled, so you can stop before it. In a hurry? Read the first paragraph of each of the three methods, then the ethics close. (Written July 2026 from sources dated 2022 to 2026; capability claims carry a date.)

Reading an Image and Generating One Are Different Problems
Method One: Two Networks in Competition (GANs)
Method Two: Compress, Then Invent (Autoencoders and VAEs)
Method Three: Learn to Undo Damage (Diffusion)
Steering the Output: How a Sentence Controls an Image
A Fourth Family, in Outline: Flow-Based Models
How the Methods Combine
Comparing the Three Methods
What About Video?
The Ethics Close: Who Made This Image?
What This Page Leaves Out
Where This Connects

Reading an Image and Generating One Are Different Problems

Stretch. Builds on A4.3.9, convolutional neural networks (HL). If you have met CNNs, you know how a computer reads an image; this page is the opposite direction, and it is not the same machinery run backwards.

A network that reads an image is handed the whole thing and asked a question about it: is this a cat, where is the pedestrian? A network that generates an image starts with nothing and must invent every pixel.

The second problem is harder in one specific way: there is no obvious right answer to compare against. Nearly everything in machine learning works by improving a measurable score, so the first job in generating anything is to invent a score that can be improved. The three methods below are three different answers to that one question, and keeping that question in mind is the best way to tell them apart.

If you want the “reading” half fresh in your mind first, it is on Training and Evaluating a Model under CNNs (A4.3.9, in the syllabus).

Method One: Two Networks in Competition (GANs)

Stretch. Builds on A4.3.2 (HL), because one of the two networks is an ordinary binary classifier, and on A4.3.8, since both are plain neural networks. There is no student-friendly video for this method (the source lecture is a long unedited recording), so the diagrams carry it. The source lecture’s running example is photographs of the moon; here we will use potted plants instead.

A generative adversarial network (GAN) is a competition between two networks with opposite jobs.

The generator produces images. At the start it is untrained, so it produces meaningless speckle.

The discriminator is shown a mix of real photographs (say, thousands of real potted plants) and the generator’s output, and must say which is which. That is classification in the completely ordinary sense, sorting inputs into two categories, and it is trained the ordinary way: we know which images really are real, so every wrong call gets corrected. At the start it, too, is untrained, so it guesses. That matters: neither network needs to be any good for training to begin.

The competition is the score that generation was missing:

If the generator’s image fools the discriminator, that counts as success for the generator, and it adjusts to produce more like it.
If the discriminator catches the fake, that is success for the discriminator and failure for the generator, which adjusts to avoid that kind of output.

Each network’s mistake is the other’s training signal. A generator barely better than random can still sometimes fool a discriminator barely better than random, and each small win pushes both forward: more real photographs make the discriminator harder to fool, which forces the generator to improve, which hands the discriminator harder cases. They ratchet each other upwards until the generator’s plants are ones neither the discriminator nor a person can reliably call fake.

The pair improves together, which is the part a single snapshot cannot show. One more thing worth naming: nobody ever tells the generator what a good image looks like. There are no labelled answers to copy. It learns entirely from whether it got away with it.

Try it: what is the laziest way to win? If the generator’s only goal is to fool the discriminator, what is the cheapest strategy it could stumble into? Think before revealing.

Show the failure this leads to

Find one image that reliably fools the judge, and produce it every time. This failure has a name, mode collapse: the generator’s outputs all land in one tiny corner of the possibilities instead of spreading across them. Reaching it by reasoning, as you just did, is better than being told, because it shows the failure is built into the incentive, not a random bug. Follow-up worth arguing about: what would you change in the training so that repetition stopped paying?

An honest note: no watchable video behind this page teaches mode collapse. The passage after this reveal completes the story of adversarial training going wrong.

When adversarial training goes wrong. Mode collapse is the failure you can reach by reasoning, and it teaches the deepest lesson on this page: nothing malfunctioned. The generator optimised exactly the goal it was given; the goal just was not what anyone wanted. That is the same lesson as a badly chosen reward in reinforcement learning, and connecting the two is worth a moment. Two more failures matter, because judging a generator on training stability is part of evaluating this method:

Failure to converge. The two networks are meant to improve together, but they can circle each other instead, each reacting to the other’s last move without either settling. Because the discriminator is a moving target, there is no single score that steadily falls the way a loss curve does in supervised training, so “is this working?” is genuinely harder to answer here than elsewhere.
One side overpowering the other. A discriminator that gets too good too quickly rejects everything with complete confidence. The generator’s feedback collapses to “wrong” with no hint of which direction would be less wrong, and it stops improving.

What is done about it, in broad terms: practitioners slow the stronger network down, stop the discriminator becoming too confident too fast, and change how the disagreement is measured so the losing side still receives useful direction. The detail is beyond this page. The point to keep is that adversarial training needs active balancing, which is a real reason the other methods stay attractive despite their softer output.

An honest note on sources: these failure paragraphs are written material with no video source, added so the method’s evaluation story is complete. If you are preparing the 2027 case study, the assessed treatment lives in the Case Study section.

Method Two: Compress, Then Invent (Autoencoders and VAEs)

Stretch. Builds directly on A4.2.3, dimensionality reduction (HL): an autoencoder is a working example of compressing data and rebuilding it, and this section’s exercise is the same idea that point teaches. No student-friendly video exists for this method either; the diagrams and the exercise below carry it.

Start with a puzzle, before any theory.

Try it first: compress three numbers into one. Here are four rows of three numbers:

Replace each row with a single number, chosen so that someone who sees only your number can rebuild all three originals. Find the rule before revealing.

Show the rule and why it matters

Each row is a number, its square, and its cube. So the rows compress to 2, 3, 4 and 5, and anyone holding “3” can rebuild “3, 9, 27”.

Four things just happened, and they are the whole lesson of this section:

Three values became one, and nothing was lost.
To find the compression you had to find the pattern. Compressing the data and understanding the data turned out to be the same act.
Nobody told you the rule. You worked it out from examples, which is exactly what the encoder below is trained to do.
The honest part: real data is messier than this, so a real system gets close to the original rather than exact. That is still useful.

Extension: what should happen to a fifth row that does not fit the rule, say 4, 10, 19? That question, what a code can and cannot represent, is a genuinely good one to sit with.

Now the machinery. An autoencoder is two networks joined in the middle. The encoder takes the input (perhaps thousands of pixel values) and squeezes it down to a short list called the code. The decoder takes only the code and tries to rebuild the original. The network is trained on exactly the game you just played: reconstruct what went in, through a bottleneck.

The shape is the explanation: wide, narrow, wide. The space the codes live in is called the latent space (latent just means hidden: it is the concealed middle representation, not anything visible in the image). The obvious question, “what is the point of getting out what you put in?”, has the answer you discovered above: to compress something and still rebuild it, the network has to find the pattern, so the middle ends up holding something meaningful.

A plain autoencoder is a compressor, not a generator: it can only rebuild things it is given. One change turns it into a generator. A variational autoencoder (VAE) additionally learns how the codes are spread through the latent space: where, in that middle space, real inputs tend to land. Once that is known, you can throw the encoder away. Pick a point at random from the region where real codes land, push it through the decoder, and out comes something that was never in the training data but plausibly could have been.

Sampling from the latent space is the single step that turns a compressor into a generator. If one idea from this section survives until next year, make it that one.

Method Three: Learn to Undo Damage (Diffusion)

Stretch. Anchored to A4.1.1: this is the method behind the image tools students actually use, which by 2026 made it the most popular of the three for new systems (a dated claim; the field moves). The source’s running example is a photograph of a bridge; picture a photo from your own camera roll instead.

Diffusion rests on an asymmetry that deserves its own sentence: destroying an image is easy, creating one is hard.

Destroying really is trivial. Take every pixel, nudge its red, green and blue values by a small random amount (that random speckling is what “noise” means here), and repeat. Do it enough times and any photograph becomes television static.

The trick is to train a network to run the easy direction backwards, one small step at a time:

Training. Take a real photograph. Add a little noise, and keep the before-and-after pair. Add a little more, keep that pair too. Repeat until the image is pure static. Every adjacent pair is a training example: “this is what one step of damage looks like; learn to undo it.”
Generating. Start from pure random static, which costs nothing to produce. Apply the trained undo-one-step network. Apply it again, and again, until an image emerges.

Why small steps rather than one big jump? Because a single leap from photograph to static leaves nothing to learn from: the connection between the two is gone. Small steps leave a trail.

The noise ladder. Each rightward step is trivial damage; the trained network learns only the small leftward step, and generation is that step repeated from pure static.

Two questions students reliably ask, with straight answers:

“So it recovers the original photo?” No, and this is worth labouring: adding noise genuinely destroyed information, so nothing can bring that exact photo back. The network produces a plausible image, not a recovered one. That distinction, plausible rather than retrieved, matters again in the ethics section.
“Is the noise just any randomness?” Random, but not arbitrary. The nudges follow a stated spread (small changes likely, large ones rare), and the undo step is trained against exactly that spread.

Try it: the two-minute version on paper. Draw a four-by-four grid, top two rows one colour, bottom two rows another. Shift each square’s colour slightly and randomly, and redraw. Could a friend reconstruct the original? Almost certainly yes: one step of damage is reversible by inspection. Now ask what happens after twenty repeats. That gap between “one step, easy” and “twenty steps, hopeless” is the entire insight, and it is why the network is trained on single steps only. (A bigger version of the same idea: take one of your own photos, export five increasingly noise-heavy copies from any photo editor, then view the sequence in reverse. The reverse film strip is what a diffusion model does.)

Finally, text enters. If every training photograph carries a written description, the undo-one-step network can be made to pay attention to those words as it works. Started from static, the same machinery can then be steered towards “a lighthouse at sunset” rather than any image at all. How the words get their grip is the next section, and it is the mathematical half of the page.

But how do AI images and videos actually work? (Welch Labs, from 8:15)

The strongest visual treatment of diffusion available, starting at the diffusion walkthrough. Long and mathematical (37 minutes in full); the segment from 8:15 is the accessible core, and the earlier part of the video covers the steering ideas in the next section. From 2022: the tools it names are that moment's snapshot; the mechanism is the durable part.

Steering the Output: How a Sentence Controls an Image

Stretch, and the advanced half of this page: the ideas are geometric and the sources are mathematical. Builds on A4.2.3 (a shared embedding space is a compact learned representation) and A4.3.2 (a classifier appears as a side effect). If this section is one layer too far, skip to the comparison: the rest of the page does not depend on it.

For text to steer image generation, text and images first have to be comparable at all. That is what CLIP provides (Contrastive Language-Image Pretraining). It is two encoders trained together: one turns an image into a vector (a list of numbers), the other turns a caption into a vector, and training pulls each matching image-and-caption pair close together while pushing mismatched pairs apart. That push-and-pull training is what “contrastive” means. Closeness is measured by cosine similarity, high when two vectors point the same way.

The result is one shared space holding both pictures and words, and it behaves like the word space on the language models page: directions carry meaning. The source demonstrates this with a hat; our version: take the vector for a portrait wearing sunglasses, subtract the vector for the same portrait without them, and the difference lands near the vector for the word “sunglasses”.

Pictures and captions in one space. A pleasant side effect links back to the syllabus: this alone gives you a classifier. To label an image, compare it against a set of candidate captions and take the one that scores highest, which is nearest-match classification (A4.3.2) running on meaning instead of pixels.

With the shared space in place, the steering machinery is three ideas stacked:

Conditioning: feed the caption’s vector into each denoising step, so every removal of noise also moves the image towards one matching the description.
Classifier-free guidance: run each step twice, once with the prompt and once without, take the difference between the two directions, and exaggerate it. The guidance strength setting controls the exaggeration: turned up, the output follows the prompt more forcefully (and, pushed too far, starts looking less natural).
Negative prompts: the same arithmetic in reverse. Describe what you do not want, and subtract that direction.

For the full mathematical treatment, the Welch Labs video above covers CLIP from 3:37, cosine similarity from 5:36, the shared space and its directions from 6:24, conditioning from 26:41, guidance from 29:53 and negative prompts from 33:37.

A Fourth Family, in Outline: Flow-Based Models

Stretch, kept deliberately brief. Its anchor is A4.2.3’s compression idea, because the contrast with information loss is what makes this family make sense, and knowing it exists keeps the three-method picture honest.

The three methods above all accept losing information somewhere. A GAN never reconstructs anything. An autoencoder’s decoder lands close to the original rather than exactly on it. Diffusion destroys information on purpose every time it adds noise.

Flow-based models refuse that trade. They build the journey from noise to image out of steps that are each exactly reversible, so the whole chain runs in both directions: noise fed forward becomes an image, and an image fed backward recovers precisely the noise it came from. What that buys is something none of the other three can do directly: the model can say how likely a given image is under its own picture of the data. What it costs is design freedom, because every step must be undoable, which rules out most of the operations a network would normally use, so these models tend to be larger and slower for comparable quality.

That is the whole of it at this level: the family exists, reversibility is its idea, and exact likelihood versus design freedom is its trade.

How the Methods Combine

Stretch, and the answer to a question a careful reader has already asked: two of these methods look combinable, and production systems do exactly that.

Diffusion as described above is expensive. Every denoising step touches every pixel of a full-size image, across many steps, on a picture that does not exist yet. An autoencoder compresses an image into a much smaller code that keeps what matters. Put the two together: run the denoising loop inside that small compressed space instead of on raw pixels, and only decode to a full image once a clean code exists. The expensive part now happens on something far smaller. That is the shape of most current text-to-image systems (a claim dated 2026), with a text encoder from the CLIP family doing the steering. Adversarial training turns up inside hybrids too: a discriminator attached to an autoencoder’s decoder pushes its output toward looking realistic rather than merely close to the original.

So the honest way to hold these families in your head is as components, not competitors. “Which one is best” is the wrong question. “What is each one good at, and which part of the pipeline should it do” is the right one. The assessed version of this idea is the Hybrid Models page in the Case Study section.

Comparing the Three Methods

Stretch, anchored to A4.3.10 (HL): choosing and comparing models is exactly the skill that statement assesses, and filling this table is practice for it. The comparison is deliberately a task rather than a verdict, because the sources genuinely do not crown a winner. The fourth family above is left out of the table on purpose; adding a flow-based row yourself is a worthwhile extension. The one dated fact available: by the 2026 sources’ vantage point, diffusion had proved the most popular family for new image systems in the preceding few years.

Copy this table and fill it from the page. The last column is intentionally the hardest.

	What score is being improved?	How many networks?	One pass or many?	What does failure look like?
GAN
VAE
Diffusion

Compare your answers

GAN: score = how often the generator fools the discriminator. Two networks. One pass to generate an image. Failure: mode collapse (everything looks alike), reached by reasoning earlier.
VAE: score = how closely the decoder’s rebuild matches the original (plus keeping the codes tidily spread). One network in two halves (encoder and decoder). One pass. Failure: honestly, this page’s sources do not name one, and that gap is worth noticing rather than papering over.
Diffusion: score = how well one small step of noise is undone. One network, applied many times. Many passes. Failure: again, not named by the sources here.

The empty failure cells are not you missing something; they are the page’s sources running out. A comparison that admits where its information stops is worth more than one that fills every box.

What About Video?

Stretch, and a deliberately narrow claim. If it earns a place in your notes, it is as A4.4.1 discussion material: moving images raise the same authorship and consent questions as stills, at higher stakes. The opening hook of the Welch Labs video shows pure noise becoming a moving scene (from 0:47), and the basic idea is the one you already have: diffusion, with the extra demand that the frames agree with each other over time. How that agreement (temporal consistency) is actually achieved is beyond every source this page is built on, so this page states the fact and stops. If a “how video generation works” section ever appears here, it will need new sources first. (Claim dated 2026.)

The Ethics Close: Who Made This Image?

In the syllabus: A4.4.1 and A4.4.2 examine exactly these questions with the Discuss command term. This section is the strongest discussion material in this page’s sources, and it is not optional extra reading in the way the mathematics was.

Everything above is machinery; these questions are about the world the machinery lands in.

The training data was mostly other people’s work. The photographs and artwork these systems learn from were largely scraped from the web, including work by living artists who never agreed. A model does not retrieve their images (the diffusion section showed the original genuinely cannot come back), but it learned its abilities from them. Is that inspiration, or copying at industrial scale? The same question hits text models and writers, which is why the language models page raises it too.
Plausible is not the same as real. A generator produces images that look like photographs of events that never happened. What happens to “seeing is believing” as evidence, in courtrooms, journalism and your own group chats?
Who is the author? If you wrote the prompt, the company trained the model, and ten million photographers supplied the training data, whose image is it, and who is accountable when it causes harm?

The closing section of the Welch Labs video (from 34:26) puts the first question sharply and is worth five minutes of anyone’s time. For the syllabus framing of accountability, consent and bias, continue at Ethics of Machine Learning.

What This Page Leaves Out

An honest list. None of these is covered by this page’s sources, so each is a limit to know about rather than a hidden assumption. (Three items that used to sit here, flow-based models, GAN training troubles, and how the methods combine, were written into the page in July 2026; the sources note below records that they have no video source.)

Editing rather than generating: filling in part of an image, extending one, upscaling, style transfer.
How generated images are judged: no measure of “better generator” appears in any source here. The assessed treatment is the Evaluating Models page in the Case Study section.

Where This Connects

In the syllabus: What Machine Learning Is (A4.1.1: generative AI as an application; A4.1.2: why this all runs on parallel hardware).
In the syllabus (HL): Training and Evaluating a Model for dimensionality reduction (A4.2.3, the autoencoder’s day job), CNNs (A4.3.9, the reading direction), and neural networks generally (A4.3.8); Types of Learning for classification (A4.3.2, the discriminator’s job and CLIP’s side effect).
In the syllabus: Ethics of Machine Learning (A4.4.1) and Computing in Daily Life (A4.4.2) for the questions in the ethics close.
Stretch: How Large Language Models Work shares the embedding-space idea (one space, meaning as position) and the copyright question; AI Beyond Machine Learning covers the half of AI where nothing is trained at all.
Assessed (Paper 1 Section B, 2027 cohort): the Case Study section covers image-generation techniques, GANs, diffusion, hybrid models, evaluation, and the ethics and law as examined material. This page is the conceptual background behind it.

Sources and dates for this page

Built July 2026 by rewording and reorganising ideas from: Welch Labs, “But how do AI images and videos actually work?” (embedded above, 2022 vantage point); and CS50’s “Fundamentals of AI” Lecture 5 “Generating” (2026), a long unedited live recording, linked for completeness rather than embedded: watch here. The reading-versus-generating contrast draws on the series’ Lecture 3 on computer vision. All worked examples and diagrams on this page are original; the compression exercise uses different numbers and framing from any source. The passages on adversarial training failures, flow-based models, and how the methods combine (added July 2026) are original written material with no video source, prepared against the 2027 case-study requirements; treat them as the first places to re-check if this page is ever reviewed for accuracy.