MiniGPT Lab — Build a tiny ChatGPT engine

Parts Tray

0/8

✂

Tokenizationinstall

Splits raw text into smaller chunks called tokens that the model can process.

Click to install →

Vocabulary / Token IDs

Looks up each token in a fixed vocabulary and replaces it with a unique numeric ID.

Install previous parts first

→

Embeddings

Maps each token ID to a dense vector of floating-point numbers that encodes semantic meaning.

Install previous parts first

Positional Encoding

Adds a position signal to each vector so the model knows the order of tokens in the sequence.

Install previous parts first

◎

Attention

Computes attention weights so the model can focus on the most relevant earlier tokens when predicting the next one.

Install previous parts first

∑

Logits / Probabilities

Produces raw logit scores for every vocabulary token, then applies softmax to produce a probability distribution.

Install previous parts first

⚡

Sampling

Applies temperature scaling to reshape the distribution, then samples one token according to the resulting probabilities.

Install previous parts first

∞

Autoregressive Output

Appends the sampled token to the sequence. In a real model this feeds back in for the next token.

Install previous parts first

Install parts left to right. Each unlocks the next stage.

Engine Chassis

⚙

The engine chassis is empty

Install modules from the Parts Tray on the left,
or click ▶ Built Demo to fill all slots instantly.

✂1. Tokenization

#2. Vocabulary / Token IDs

→3. Embeddings

014. Positional Encoding

◎5. Attention

∑6. Logits / Probabilities

⚡7. Sampling

∞8. Autoregressive Output

→

What just happened?

Run the engine to see each transformation step explained here. You'll see what the data looked like before each stage, what the stage did to it, and what came out — and why it matters.