Tokens in C: Types, Uses, and Syntax

Tokens in C are the fundamental building blocks… Just like letters and words form a sentence in English, tokens form the instructions that the compiler can understand. They include keywords, identifiers, constants, operators, strings, and special symbols. Using tokens properly helps the compiler understand your code, ensures the program runs correctly, and makes it readable for humans. Without tokens, a program is meaningless to the compiler. In this guide, we will learn about tokens in C in detail with examples.

What are Tokens in C?

Tokens are the building blocks of the C programming language. The compiler uses these tokens to interpret what the program is intended to do. Tokens are important because the compiler cannot understand a program written in plain text, and tokens help the compiler to understand:

• Structure the code
• Define instructions clearly.
• Avoid confusion.

Tokens include keywords, identifiers, constants, operators, and special symbols.

Types of Tokens in C

1. Keywords

Keywords are the reserved words in C that have special meaning to the compiler. They are predefined words and cannot be used in place of variables or other identifiers. In C, keywords are usually written in lowercase, and you cannot use them for your own naming.

For example, int int = 5; is wrong as int is a keyword.

There are 32 standard keywords present in C. Some of them are as follows:

Example:

Output:

In the above C program, the two variables age and pi are initialized with the keywords int and float. The value of the variables is displayed, and the return keyword is used.

2. Identifiers

Identifiers are the names that are given to variables, functions, arrays, or any user-defined items in a C program. They are used to identify the values present in a code. Below are some rules for naming a variable in C.

• Must start with a letter (a–z, A–Z) or an underscore (_).
• Can contain letters, digits (0–9), or underscores after the first character.
• Cannot be a keyword in C.
• Case-sensitive: age and Age are different identifiers.
• No spaces or special characters allowed.

Example:

Output:

In the above C program, the age, pi_value, and _count are identifiers used to store and refer to values.

3. Constants

Constants are the fixed values in a C program that do not change during the execution of a program. They are used to store data that should remain constant throughout the program. In many C learning resources, constants are classified as Primary (like numeric or character) and Secondary (like those defined using const or #define).

1. Primary Constants: These are the constants like integer, float, or character constants.

2. Secondary Constants: These constants are created using #define or the const keyword. For example, #define PI 3.14 or const int MAX = 100;

Example:

Output:

In the above C program, PI and MAX are constants that cannot be changed during program execution. The variable age shows the difference between a regular variable and a constant.

4. Literals

Literals are the fixed values that are written directly in the code. They are used to represent constant data and are used to initialize variables or provide values in expressions.

Example:

Output:

In the above C program, 25, 3.14, ‘A’, and “Alice” are literals written in the code.

5. Operators

Operators in C are the special symbols that tell the compiler to perform specific operations on data or variables. Below are the types of operators in C:

• Arithmetic Operators
• Bitwise Operators
• Relational Operators
• Assignment Operators
• Increment or Decrement Operators

Example:

Output:

In the above C program, the operators + and * are used to perform the addition and multiplication operations on the variables a and b. After that, && is used to check the two conditions together.

6. Strings

Strings in C are a sequence of characters enclosed in double quotes ” “. They are used to store textual data such as names, messages, or any readable information.

Example:

Output:

In the above C program, name and greeting are strings (arrays of characters).

7. Punctuators

Punctuators are the special symbols in C that are used to structure and organize the program. They help the compiler understand where statements start and end, group code, or separate items. They are not part of the data but are used to organize code, and each punctuator serves a specific purpose in the program. Some of the common punctuators in C are:

Example:

Output:

In the above C program, different punctuators as below are used.

;
{ }
( )
[ ]
,

C Character Set

The C character set is the collection of all characters that are allowed to be used in a C program. Every C program is written using these characters. Some key points about the C character set are:

1. Characters are the smallest units of a program.
2. They include letters, digits, special symbols, and whitespace.
3. Using characters outside this set can cause compilation errors.

Components of the C Character Set

The C character set consists of the following components.

Example:

Output:

In the above C program, ‘A’ is a letter, 5 is a digit, ‘@’ is a special symbol, and ‘ ‘ is whitespace, and all are part of the C character set.

Uses of Tokens in C

Below are the common uses of tokens in C.

1. Structure the Program: Tokens make the code readable, and keywords, operators, identifiers, and punctuators tell the compiler what each part of the program will do. Without tokens, the compiler cannot interpret the code.

2. Clear Communication with Compiler: Each token in C has a specific meaning and purpose. For example, int tells the compiler that a variable is an integer, + tells that the addition operation has to be performed, and ; ends a statement.

3. Prevent Errors: Using the token in C ensures that the syntax rules are followed correctly. Misusing tokens can cause compilation errors and prevent the program from running correctly.

4. Improve Readability and Maintainability: Tokens allow programmers to write structured and organized code, which makes it easier for other people (or you in the future) to understand and debug the program.

5. Enable Code Reusability: Identifiers (a type of token) let programmers name variables and functions meaningfully, making code modular and reusable.

Conclusion

From the above article, we learned that tokens are essential for writing correct and efficient C programs because they structure your code so the compiler can interpret it, allow clear communication with the compiler to execute instructions as intended, prevent errors by enforcing C syntax rules, improve readability and maintainability to make the code easier to understand and debug, and help create modular and reusable code through meaningful identifiers.

Tokens in C – FAQs