Working with Files in Python

aakash

May 29, 2025

Introduction
Coding often requires storing and accessing data, which is where file handling becomes an essential part of backend development. One of the most essential backend skills is mastering file handling in Python to manage data efficiently. It allows programs to interact with the real world through file data. In this blog, we’ll explore the file-handling capabilities in Python, everyday use cases, and best practices you should follow.

File handling in phython is reading, writing, and manipulating files on your computer like .txt, .csv, and .json using Python’s built-in functions like open(), read(), and write () and libraries like CSV, JSON, and Pandas. Python uses simple syntax so that developers can focus more on logic rather than on the process of opening and closing files.

Why File Handling Is Essential for Python Developers

File handling in Python is an essential process as it assists in the following:

Backend Systems and Applications generate log files for errors, user activities, and system events. These files are stored as .txt or .log files using functions like open () and write () for debugging later.
Automation Scripts need settings like file paths, API keys, user inputs, etc., and are stored in .txt, .ini, .json, or .yaml files. These scripts read these files for configuration and generate reports.
Data Processing Pipelines depend on a large volume of structured data. While importing the data from files like .csv, .json, or .xlsx, libraries like csv, json, and pandas manage these files for processing.
Web Development requires unstructured data from users, such as images, documents, or videos. File handling assists in properly saving such data temporarily.

Common Use Cases in Real-World Applications

Some practical examples of file handling in Python help in:

Reading Log Files: This means opening the file that stores records of an application, program, or system for analyzing errors or system usage.
Saving API Responses: This stores the API data in a .json file, allowing data retrieval without calling the API.
Storing Configuration: To make code cleaner, safer, and easy to update, .env or .ini files keep essential information outside the code like API keys, database details, or passwords, etc.,
Exporting Filtered Data: This makes it easy to share data with the team for reporting, analysis, and making data-driven decisions.

Understanding File Types in Python

Text Files: include formats like .txt, .csv, .json , contains human- readable characters, and can be opened in a text editor like notepad or VS code.
Binary Files: have extensions .jpg, .exe, .pdf) and contain non-human readable data stored in binary format.

File Handling Basics in Python

Let’s do some basic coding to learn file handling.

Opening and Closing Files

file = open(‘Notes.txt’, ‘r’)
content = file.read()
file.close()

File → is a variable,
Open → it the function returning file object representing the file
r → means read only mode
file.read() reads the entire content of a file and stores it in a variable content .
Close the file, to avoid corruption and memory leaks.

If the file is present in the current directory of the python script then you can simply write the file name, like Notes.txt (relative path). And, if the file is located in another directory then give an entire path(absolute path).

Basic file modes are:

‘r’ → Read (default)

‘w’ → Write (overwrites file)

‘a’→ Append

‘b’ → Binary mode

‘x’ → Create new file

Reading Files and Writing Files in Python

Reading Line by Line

with open(‘data.txt’, ‘r’) as file:
for line in file:
print(line.strip())

open data.txt in read mode(r).
‘with’ is a content manager and automatically closes the file.
The for loop goes through each line one by one.
Print the line and strip() removes spaces in end and beginning and new lines created by \n

Writing and Appending

# Writing
with open(‘coding.txt’, ‘w’) as file:
file.write(“I love coding”)

open coding.txt in write mode (w)
as file → is the variable that will store the file object. It allows you to write like a pen whereas a file is a box . It means first you open the box (file) , take out the pen and start writing.
If coding.txt not present, python create a new file and writes.
If it already exists, all the content is deleted and then, write I love coding .

# Appending
with open(‘coding.txt’, ‘a’) as file:
file.write(“\nAppended line.”)

It means open the file in append mode(a), that means to add content in the end and not to erase anything.

Working with Structured Data Files

Structured data files store, process and share data in an organized format, like a table e.g., .csv,.json or .xlsx.

Reading and Writing CSV Files

import csv
with open(‘data.csv’, ‘w’, newline=”) as file:
writer = csv.writer(file)
writer.writerow([‘Name’, ‘Age’])
writer.writerow([‘Alice’, 25])

csv is the built-in module for read/write CSV files,
open a data.csv file in write mode (w),
file is a variable that holds the file object,
writer is a functional object , to write data in csv format using commas,
This writes the first row with Name and Age as columns and the second row with values as Alice and 25 in columns

# Readingwith open(‘data.csv’, ‘r’) as file:
reader = csv.reader(file)
for row in reader:
print(row)

open a data.csv file in the read mode (r),
file is a variable that holds the file object (with which we read the file),
reader is created as an object , it knows how to read CSV files.
For loop reads each row and displays data in each row as a list
[‘Name’, ‘Age’]
[‘Alice’, ’25’]

Handling JSON Files

Writing JSON files

import json

data = {“name”: “Tom”, “age”: 21}
with open(‘profile.json’, ‘w’) as file:
json.dump(data, file)

JSON module is imported
This stores the data in a json file.
Json.dump() write to file

Reading

with open(‘profile.json’, ‘r’) as file:
user = json.load(file)
print(user)

It reads json file using json.load()

Managing Excel Files

import pandas as pd

# Reading
df = pd.read_excel(‘data.xlsx’)

print(df)

# Writing
df.to_excel(‘output.xlsx’, index=False)

Pandas is a python library it works on data like table and spreadsheet and here an alias name pd is provided to it
Df is a dataframe(it is a data structure with rows and columns like excel)
This line reads data.xlsx and loads it in dataframe df
This prints the table with rows and columns
This line saves the dataframe df to a new file output.xlsx
index=False tells pandas not to write row numbers into file.

Best Practices for Handling Large Files

Efficient File Processing

While working with large files or datasets, loading the entire file can cause a program to crash. Instead, use line-by-line or buffered reading.

Using Generators

def read_large_file(file_path):
with open(file_path, ‘r’) as file:
for line in file:
yield line.strip()

Defining a function read_large_file to read a large file.
Open the file in read mode .
For loop reads the file one line at a time.
strip() removes spaces and new lines and yields one line at a time ( generator),lowering down memory usage.

Error Handling in File Operations

Common Errors include:

File not found: FileNotFoundError
No permission: PermissionError

Try-Except Example

Exception handling is one of the best way to handle errors without crashing the program and in python try- except block is used:

try:
with open(‘unknown.txt’, ‘r’) as file
data = file.read()
except FileNotFoundError:
print(“File does not exist.”)
except Exception as e:
print(“An error occurred:”, e)

It means lets try these lines of codes open unknown.txt and read
If file doesn’t exist → FileNotFoundError → prints: “File does not exist.”
If any other error → prints: “An error occurred: .”, showing the error message.

Advanced File Handling Techniques

Context Managers

Use with to open files as it automatically opens and closes the file, even if an error happens.

Temporary Files

The temp file module allows you to create temporary files, which are useful for storing data temporarily.

Automating File Tasks

Automate tasks using loops, cron jobs and use modules like os and shutil to manage files and directories

Secure File Handling

Always use backups before overwriting files, validate file paths, and always close files or use ‘with’ block.

Conclusion and Next Steps

File handling in Python is a skill that can be mastered very easily and is applied in every domain from data science to web apps. This article has given you insights about Text vs binary files, Reading and writing structured formats, efficiency in handling files etc.
Once you master the basics, the next step is to delve into databases, APIs, and full-fledged data pipelines.