Python Tokens and Identifiers: Keywords, Rules and Examples

 

Python Token

A token is the smallest individual unit of a Python program that has meaning to the Python interpreter.

For example:

This statement contains the following tokens:

• a → Identifier
• = → Operator
• 10 → Literal/Constant

Therefore:

A statement inside a Python program is a collection of tokens.

Types of Tokens in Python

The important types of tokens used in Python are:

1. Keywords
2. Identifiers
3. Literals / Constants
4. Operators
5. Separators / Delimiters

1) Keywords

Keywords are special words that have a predefined meaning in the Python language.

Keywords are also called reserved words because they have a specific meaning defined by Python and cannot normally be used as identifiers.

Each keyword is used for a specific purpose.

Example

import math

 

# Calculate square root

print("Square root of 9:", math.sqrt(9))

 

# Assign values to variables

a = 10

b = 5

 

# Check which number is greater

if a > b:

    print(a)

else:

    print(b)

In the above program:

• import is a keyword.
• if is a keyword.
• else is a keyword.

However:

• math, a, and b are identifiers.
• print and sqrt are names referring to built-in/function objects.

How to Find Keywords in Python?

Python provides the built-in keyword module to check the keywords.

>>> import keyword

>>> keyword.kwlist

output:

['False', 'None', 'True', 'and', 'as', 'assert', 'async', 'await',

 'break', 'case', 'class', 'continue', 'def', 'del', 'elif', 'else',

 'except', 'finally', 'for', 'from', 'global', 'if', 'import', 'in',

 'is', 'lambda', 'match', 'nonlocal', 'not', 'or', 'pass', 'raise',

 'return', 'try', 'while', 'with', 'yield']

The exact list can vary between Python versions.

In current Python 3 versions, Python has 35 hard keywords.

Soft Keywords

Python also has soft keywords. These words have a special meaning only in particular language contexts.

>>> keyword.softkwlist

output:

['_', 'case', 'match', 'type']

There are 4 soft keywords in current Python versions.

Therefore:

Python has 35 hard keywords and 4 soft keywords.

---

2) Identifiers

An identifier is a name created by the programmer to identify a programming element.

Identifiers are used for naming different programming elements, such as:

1. Variable names
2. Function names
3. Class names
4. Module names
5. Package names
6. Other user-defined objects

Example

rollno = 12

Here, rollno is an identifier because it is the name given to the variable.

Another example:

def calculate():

    print("Hello")

Here, calculate is an identifier used as a function name.

Characters Allowed in an Identifier

An identifier can be created using:

• English alphabets (A-Z, a-z)
• Digits (0-9)
• Underscore (_)

However, there are certain rules that must be followed while creating identifiers.

An identifier can contain letters, digits, and underscores, but it must start with a letter or an underscore.

For example:

student

student1

student_name

_student

are valid identifiers.

The underscore (_) is the only special character allowed in a normal Python identifier.

---

Rules for Writing Identifiers

Python follows some rules while creating identifiers.

Rule 1: Use letters, digits and underscore

An identifier can contain:

• Letters (A-Z, a-z)
• Digits (0-9)
• Underscore (_)

Example:

student
student1
student_name
marks_2026

All of these are valid identifiers.

---

Rule 2: An identifier cannot start with a digit

An identifier can contain digits, but it cannot start with a digit.

Valid:

student1
marks2026

Invalid:

1student
2026marks

The second examples will produce a SyntaxError.

---

Rule 3: Spaces are not allowed

Spaces cannot be used inside an identifier.

Invalid:

student name

Instead, use an underscore:

student_name

---

Rule 4: Special symbols are not allowed

Characters such as:

@  #  $  %  -  !

cannot normally be used in an identifier.

For example:

student-name

is not a valid identifier because - is interpreted as the minus operator.

Use:

student_name

instead.

---

Rule 5: Python identifiers are case-sensitive

Python treats uppercase and lowercase letters as different.

For example:

age = 20
Age = 25
AGE = 30

Here:

age, Age, and AGE are three different identifiers.

---

Rule 6: Keywords cannot be used as identifiers

Python has some reserved words, called keywords.

Examples:

if
else
for
while
class
def
return
import

These words have a special meaning in Python and cannot be used as ordinary identifiers.

For example:

class = 10

is invalid.

---

Simple Rule to Remember

An identifier can start with a letter or underscore, followed by letters, digits, or underscores. It cannot contain spaces or special symbols, and it cannot be a Python keyword.

Identifier and Variable in Python

In Python, Identifier and Variable are closely related concepts, but they are not exactly the same.

An identifier is a name given to a program element, whereas a variable is a name that is used to refer to a value (object) in a Python program.

For example:

age = 25

Here, age is an identifier used as a variable, and 25 is the value assigned to it.

A simple way to understand this is:

Identifier = Name
Variable = Name referring to a value

 Literal 

A literal is a value that we directly write in a Python program.

For example, 10, 25.5, "Hello", and True are literals.
Python uses these values directly in the program.