In C programming, tokens are the smallest individual units (or “words”) of a C program that the compiler can recognize and use to build the program.

What is a token in C?

A token is the smallest meaningful piece of code—like a keyword, variable name, operator, constant, or punctuation mark.

For example, in the statement:

c

int age = 42;

the compiler breaks it into these tokens:

  • int → keyword
  • age → identifier
  • = → operator
  • 42 → constant
  • ; → punctuator

Main types of tokens in C

Modern C groups tokens into six primary categories :

Token type| What it is| Example(s)
---|---|---
Keywords| Reserved words with fixed meaning (cannot be used as names).| int, float, if, return 14
Identifiers| Names you choose for variables, functions, arrays, etc.| age, sum, myFunction 25
Constants| Fixed values that do not change during execution.| 42, 3.14, 'A' 35
Strings| Sequences of characters in double quotes.| "Hello, world!" 26
Operators| Symbols that perform operations (arith‑, rela‑, logical, etc.).| +, -, *, ==, && 36
Special symbols / punctuators| Punctuation marks used to structure code.| ;, {}, (), [] 18

Why are tokens important?

The C compiler first scans your source code and splits it into tokens during the lexical analysis phase.

Only after breaking the code into tokens does the compiler assemble them into expressions, statements, and functions, which is why understanding tokens helps you write cleaner, more precise C code.

In short:
👉 “C tokens” = the tiny building‑blocks of a C program: keywords, identifiers, constants, strings, operators, and special symbols.
Every valid C program is ultimately a sequence of such tokens stitched together by the rules of the language.

Information gathered from public forums or data available on the internet and portrayed here.