Chapter 9: Reading and Writing Files

by

Scene: The “Amnesia” Crisis

Chaitanya is sitting in the computer lab, his head resting in his hands. He looks completely defeated.

Aditi Ma’am walks in and spots his posture. “Let me guess. You spent three hours typing student registration data into your new PyInputPlus script, and then you closed the terminal.”

Chaitanya: “Yes! I typed in 50 students. Their names, ages, houses, everything. It was working perfectly! Then I closed the window to go to lunch. When I came back and ran the script again… it was completely empty. It asked me to start from Student #1. Where did all my data go?”

Aditi Ma’am: “It evaporated, Chaitanya. You are dealing with the fundamental difference between RAM (Random Access Memory) and the Hard Drive.”

Aditi Ma’am: “Variables—like your lists and dictionaries—live in RAM. RAM is like the school’s whiteboard. It is fast, you can erase things quickly, but the moment the power goes out (or the program closes), the whiteboard is wiped clean. If you want data to survive, you must write it to the Hard Drive. The Hard Drive is the school’s filing cabinet. It is slower, but it is permanent.”

Chaitanya: “So how do I open the filing cabinet using Python?”

Aditi Ma’am: “Before you can open a file, you have to know how to find it. You must understand File Paths.”

The Slash War (Windows vs. macOS/Linux)

Aditi Ma’am: “Here is the first hurdle of file management. The world of computers is divided into two camps when it comes to writing file paths.”

  • Windows uses the Backslash \ (e.g., C:\Users\Chaitanya\Notes.txt).
  • macOS and Linux use the Forward Slash / (e.g., /Users/Chaitanya/Notes.txt).

Chaitanya: “That’s annoying. If I write a script for my Windows laptop, it will crash on the school’s Linux servers?”

Aditi Ma’am: “In the old days of Python, yes. Programmers had to write messy if/else statements just to add the right slashes. But in modern Python, we have the pathlib module. It handles this automatically.”

Enter pathlib and the Path Object

Aditi Ma’am: “Let’s import Path from the pathlib module. Instead of writing file paths as raw strings, we pass folder names to the Path() function. It builds the correct path for whatever operating system the code is currently running on.”

Python

from pathlib import Path

# Let pathlib build the path for us
my_file = Path('SchoolData', 'Students', 'attendance.csv')
print(my_file)

Chaitanya: “Let me run that on my Windows machine.”

Output (on Windows): SchoolData\Students\attendance.csv

Aditi Ma’am: “If I run that exact same code on my Mac, the output will be SchoolData/Students/attendance.csv. The Path object is smart. It adapts.”

The / Operator (Path Joining)

Chaitanya: “What if I have a base folder, and I want to add file names to it inside a loop?”

Aditi Ma’am: “The Path object has a magic trick. It overloads the division operator /. You can use / to join Path objects together, or to join a Path object with a string.”

Python

from pathlib import Path

base_folder = Path('SchoolSystem')
houses = ['Red', 'Blue', 'Green']

for house in houses:
    # Joining Path with a string using /
    house_file = base_folder / house / 'roster.txt'
    print(house_file)

Output (on Windows):

SchoolSystem\Red\roster.txt
SchoolSystem\Blue\roster.txt
SchoolSystem\Green\roster.txt

Chaitanya: “That is so much cleaner than using string concatenation with +!”

Aditi Ma’am: “Yes. The only rule is that one of the first two items you are joining must be a Path object. You cannot do 'Folder' / 'File.txt', because those are just two strings, and Python will try to do math division and crash.”

The Current Working Directory (CWD)

Aditi Ma’am: “Every Python program runs inside a specific folder on your computer. This is called the Current Working Directory (CWD). If you tell Python to save a file named report.txt, it will drop it directly into the CWD.”

Chaitanya: “How do I know what my CWD is?”

Aditi Ma’am: “You ask Path.”

Python

import os
from pathlib import Path

print(Path.cwd())

Output: C:\Users\Chaitanya\PythonProjects

Chaitanya: “Can I change it? Say I want all my files to save to my Desktop instead?”

Aditi Ma’am: “Yes, using the os module’s chdir() function (Change Directory).”

Python

import os
from pathlib import Path

os.chdir('C:\\Users\\Chaitanya\\Desktop')
print(Path.cwd())

Output: C:\Users\Chaitanya\Desktop

Aditi Ma’am: “Notice I used \\ in the string? Because \ is an escape character, I had to escape the backslash itself. This is why using Path objects is safer than using strings.”

Absolute vs. Relative Paths

Aditi Ma’am: “There are two ways to describe where a file lives: Absolute and Relative.”

1. Absolute Paths:

  • They always begin with the root folder (like C:\ on Windows, or / on Mac).
  • School Metaphor: “Chaitanya lives at 123 Syntax Avenue, New Delhi, India 110001.” No matter where you are in the world, that address gets you to your house.

2. Relative Paths:

  • They are relative to the Current Working Directory.
  • School Metaphor: “Go out the door, turn left, and it’s the third classroom.” This instruction only works if you are already standing in the correct hallway!

Chaitanya: “So if my CWD is C:\School, and I just type Path('Data\students.txt'), Python assumes I mean C:\School\Data\students.txt?”

Aditi Ma’am: “Exactly. Data\students.txt is a Relative path. It relies on the CWD.”

The Dot . and Dot-Dot .. Folders

Aditi Ma’am: “Relative paths use two special folder names.”

  • . (Single Dot): Means “This directory.”
  • .. (Double Dot): Means “The parent directory” (go up one level).

Chaitanya: “Like climbing up a tree?”

Aditi Ma’am: “Yes. If your CWD is C:\School\Exams\Math, and you want to save something in the Science folder, you don’t need an absolute path. You can use .. to step backward into Exams, and then go into Science.”

Python

from pathlib import Path
import os

os.chdir('C:\\School\\Exams\\Math')

# Go up one level (..), then into Science
science_test = Path('..', 'Science', 'test1.txt')
print(science_test)

Aditi Ma’am: “This makes your code portable. If you move the entire School folder to a USB drive, your relative paths will still work, but your absolute paths (C:\...) will break immediately!”

Creating New Folders (mkdir)

Chaitanya: “Ma’am, my script generates a separate report for every student. If I dump 500 text files into my main folder, it’s going to be a mess. Can Python create folders?”

Aditi Ma’am: “Yes. You can use the mkdir() method (Make Directory) on a Path object.”

Python

from pathlib import Path

# Create a Path object for the new folder
new_folder = Path(Path.cwd() / 'StudentReports')

# Tell the OS to actually create it
new_folder.mkdir()

Chaitanya: (Runs code) “I just checked my file explorer. The folder appeared out of nowhere!”

Aditi Ma’am: “What happens if you run the script again?”

Chaitanya: (Runs code again)

Python

FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'C:\\Users\\Chaitanya\\PythonProjects\\StudentReports'

Chaitanya: “It crashed. It says the folder already exists.”

Aditi Ma’am: “To prevent that, we pass exist_ok=True to the mkdir() method. This tells Python: ‘Make this folder, but if it is already there, just ignore this command and don’t crash.'”

Python

new_folder.mkdir(exist_ok=True)

Chaitanya: “Perfect. Now I can organize everything safely.”

Aditi Ma’am: “That is the foundation of the filesystem, Chaitanya. You now know how to build paths, navigate directories, and create folders without worrying about Windows vs. Mac compatibility.

Chaitanya: “But I still haven’t saved any actual data! How do I put text inside those files?”

Aditi Ma’am: “Patience. In Part 2, I will teach you the three-step File I/O process: Open, Write, and Close. We will also cover how to check file sizes and list all the files currently inside a folder.”

PART 2

Scene: Looking Inside the Filing Cabinet

Aditi Ma’am: “Before we write new files, Chaitanya, a good System Architect always checks what is already in the filing cabinet. If you blindly create files, you might overwrite the Principal’s important documents.”

Chaitanya: “How do I check if a file already exists without leaving Python?”

Aditi Ma’am: “The Path object has three built-in methods that act like your eyes. They return True or False.”

  1. p.exists(): Does this path exist on the hard drive?
  2. p.is_file(): Is this path pointing to a regular file (like .txt or .csv)?
  3. p.is_dir(): Is this path pointing to a folder (directory)?

Python

from pathlib import Path

# Let's check the school's main data folder
data_folder = Path('C:/SchoolSystem/Data')

print('Does it exist? ' + str(data_folder.exists()))
print('Is it a folder? ' + str(data_folder.is_dir()))
print('Is it a file? ' + str(data_folder.is_file()))

Chaitanya: “That is incredibly useful for my if statements. I can write: if not data_folder.exists(): data_folder.mkdir().”

Aditi Ma’am: “Exactly. Defensive programming. Always check before you act.”

Finding File Sizes and Folder Contents

Chaitanya: “Ma’am, the server has a storage limit. How can I check how big a file is? Like the attendance.csv file?”

Aditi Ma’am: “For that, we dip back into the older os.path module. Specifically, os.path.getsize(path). It returns the size of the file in bytes.”

Python

import os
size = os.path.getsize('C:/SchoolSystem/Data/attendance.csv')
print(f'File size: {size} bytes')

Chaitanya: “And what if I want to see every file inside the Data folder? Like opening a drawer and looking at all the folders inside?”

Aditi Ma’am: “You can use os.listdir(path). It returns a list of filename strings.”

Python

import os
print(os.listdir('C:/SchoolSystem/Data'))

Output: ['attendance.csv', 'grades.txt', 'StudentReports']

Aditi Ma’am: “Alternatively, if you are using pathlib, you can use the glob() method to search for specific types of files. For example, p.glob('*.txt') will list only the text files.”

The pathlib Shortcut: Reading and Writing

Chaitanya: “Okay, I know how to find files. Now, how do I save the student data I collected yesterday so it doesn’t disappear?”

Aditi Ma’am: “In Python 3, the pathlib module gave us a beautiful shortcut for simple text files. You can do it in one line.”

  • write_text(): Creates the file (if it doesn’t exist) and writes a string to it.
  • read_text(): Opens the file, reads all the text as a giant string, and closes it.

Chaitanya: “Show me.”

Python

from pathlib import Path

# 1. Create a Path object for the new file
hello_file = Path('hello_school.txt')

# 2. Write data to it
hello_file.write_text('Welcome to St. Python High School!')

# 3. Read data from it
content = hello_file.read_text()
print(content)

Chaitanya: (Runs the code) “Whoa! I just looked in my folder, and hello_school.txt is actually there! And when I opened it in Notepad, the text was inside! It’s permanent!”

Aditi Ma’am: “Congratulations. You have successfully written your first piece of permanent data.”

The Traditional File I/O Process

Chaitanya: “If write_text() is that easy, why do I need to learn anything else?”

Aditi Ma’am: “Because write_text() is a sledgehammer. It opens the file, dumps the text, and closes it. But what if the file already has data? write_text() will completely overwrite it. It will erase everything.”

Chaitanya: “Oh no. That would delete the whole attendance log.”

Aditi Ma’am: “Exactly. For precise control, you need to learn the traditional File Input/Output (I/O) process. It is a three-step dance:”

  1. Call open() to get a File object.
  2. Call read() or write() on that File object.
  3. Call close() on the File object.

Chaitanya: “Like checking out a book from the library. You open it, read it, and return (close) it.”

Step 1: Opening Files with open()

Aditi Ma’am: “To open a file, pass its path to the open() function. It returns a File object.”

Python

hello_file = open('C:/SchoolSystem/hello_school.txt')

Aditi Ma’am: “By default, open() opens the file in Read Mode ('r'). You can look at the data, but you cannot change it.”

Step 2: Reading Files (read vs readlines)

Aditi Ma’am: “If you want to read the entire file as one giant string, use the read() method.”

Python

content = hello_file.read()
print(content)

Chaitanya: “But what if it’s a list of 50 student names, one on each line? I want them in a Python list, not a massive string.”

Aditi Ma’am: “Then use the readlines() method. It returns a Python list of strings, where each string is a line of text.”

Python

# Assuming 'roster.txt' has three names
roster_file = open('roster.txt')
lines = roster_file.readlines()
print(lines)

Output: ['Alice\n', 'Bob\n', 'Chaitanya\n']

Chaitanya: “Why is there a \n at the end of every name?”

Aditi Ma’am: “Because that is the invisible ‘Newline’ character that tells Notepad to hit Enter. You can easily clean it off using the strip() method we learned in Chapter 6.”

Step 3: Writing vs. Appending Modes

Chaitanya: “Okay, how do I add a new student to roster.txt without deleting Alice and Bob?”

Aditi Ma’am: “You must open the file in the correct mode. open() takes a second argument.”

  • 'r' (Read Mode): Default. View only.
  • 'w' (Write Mode): Overwrites the existing file completely. Starts fresh.
  • 'a' (Append Mode): Adds text to the end of the file. Preserves existing data.

Aditi Ma’am: “If you pass a filename that doesn’t exist to 'w' or 'a', Python will create the file for you.”

Python

# Let's add 'David' to the end of the file
roster_file = open('roster.txt', 'a')
roster_file.write('David\n')

Chaitanya: “Notice you put the \n in there manually?”

Aditi Ma’am: “Yes. The write() method does NOT add a newline automatically like print() does. If you forget \n, the next name will be glued to David, like DavidEve.”

Step 4: Closing the File

Aditi Ma’am: “Finally, you must close the file.”

Python

roster_file.close()

Chaitanya: “What happens if I forget to close it?”

Aditi Ma’am: “Python is usually smart enough to close it when your program finishes. But while your program is running, the file is ‘locked’. If another program tries to read it, or if you try to delete it, the operating system will say ‘File is in use’. Always clean up after yourself.”

Summary of the I/O Dance

Aditi Ma’am: “Here is the complete, safe way to add a new student to our system.”

Python

# 1. Open in Append Mode
attendance_file = open('attendance.txt', 'a')

# 2. Write the new data
attendance_file.write('Rahul - Present\n')

# 3. Close the file to save and unlock it
attendance_file.close()

Chaitanya: “This changes everything. I can build a real database now.”

Aditi Ma’am: “Not quite. Text files are great for simple strings. But what if you want to save a complex Dictionary of Lists? Like our nested student_grades dictionary? You can’t just write a dictionary to a .txt file easily.”

Chaitanya: “Why not?”

Aditi Ma’am: “Because write() only accepts strings. If you convert a dictionary to a string, you lose all its Python powers. You won’t be able to access student['Math'] anymore. It just becomes dumb text.”

Aditi Ma’am: “In Part 3, I will show you how to save Python lists and dictionaries directly to the hard drive using the shelve module. We will also build a project that automatically generates 35 randomized geography quizzes and their answer keys.”

PART 3

Scene: The “Dumb Text” Problem

Chaitanya: “Ma’am, I have a problem. I created a dictionary of student grades: grades = {'Rahul': 85, 'Simran': 92}. But when I try to save it using file.write(grades), Python crashes!”

Python

TypeError: write() argument must be str, not dict

Chaitanya: “If I convert it to a string using str(grades), it saves. But when I read it back tomorrow, it’s just a dumb string of text. I can’t do file_content['Rahul'] anymore. It lost its dictionary powers!”

Aditi Ma’am: “Exactly. Text files (.txt) only understand characters. They don’t understand Python dictionaries, lists, or integers. To save Python objects permanently without losing their structure, we use the shelve module.”

Saving Variables with the shelve Module

Aditi Ma’am: “Think of shelve as a magic dictionary that lives directly on your hard drive instead of in RAM.”

Aditi Ma’am: “You open a ‘shelf’ file, put your lists or dictionaries into it using keys, and then close it. When you open it later, all your data is perfectly preserved.”

Python

import shelve

# 1. Open the shelf file (it creates 'school_data.dat' on your drive)
shelfFile = shelve.open('school_data')

# 2. Save a list to it, just like adding a key to a dictionary
students = ['Rahul', 'Simran', 'Chaitanya']
shelfFile['names'] = students

# 3. Close the shelf
shelfFile.close()

Chaitanya: “And how do I get it back?”

Aditi Ma’am: “Just open the shelf and ask for the key.”

Python

shelfFile = shelve.open('school_data')
print(shelfFile['names']) # Returns the actual Python list!
shelfFile.close()

Output: ['Rahul', 'Simran', 'Chaitanya']

Chaitanya: “That is amazing! It’s still a list! I can loop through it, append to it… it’s a real database.”

Aditi Ma’am: “Yes. And just like dictionaries, shelves have keys() and values() methods. It is the easiest way to add a save feature to your programs.”

Saving Variables with pprint.pformat()

Aditi Ma’am: “There is one other trick. Sometimes you want your data to be readable as text, but you still want Python to understand it. You can use the pprint (Pretty Print) module to write your dictionary directly into a new .py file.”

Chaitanya: “Wait, Python can write Python code?”

Aditi Ma’am: “Yes. The pformat() function converts your dictionary into a perfectly formatted string of Python code.”

Python

import pprint

grades = {'Rahul': 85, 'Simran': 92}
grades_string = pprint.pformat(grades)

fileObj = open('my_grades.py', 'w')
fileObj.write('grades = ' + grades_string + '\n')
fileObj.close()

Aditi Ma’am: “Now you have a brand new file called my_grades.py. Tomorrow, you can just import my_grades in your script, and instantly access my_grades.grades. It’s brilliant for saving configuration settings.”

The Grand Project: Generating Random Quiz Files

Aditi Ma’am: “Now, let’s put all your file skills to the ultimate test. The Geography teacher is giving a test on Indian States and Capitals to 35 students.”

Chaitanya: “And he wants me to print 35 copies?”

Aditi Ma’am: “No. If he prints 35 identical copies, the students will just copy off each other’s papers. He wants 35 unique quizzes. The questions must be in a different random order for every single student. And the multiple-choice answers must also be randomized. Oh, and he needs 35 corresponding Answer Keys to grade them.”

Chaitanya: “Making 35 unique quizzes by hand in Microsoft Word would take me three days!”

Aditi Ma’am: “With Python, it will take you less than 3 seconds. Let’s build the Random Quiz Generator.”

Step 1: The Data Aditi Ma’am: “First, we store our geography data in a dictionary.”

Python

import random
from pathlib import Path

# The quiz data. Keys are states and values are their capitals.
capitals = {'Andhra Pradesh': 'Amaravati', 'Assam': 'Dispur', 'Bihar': 'Patna', 
            'Gujarat': 'Gandhinagar', 'Karnataka': 'Bengaluru', 'Kerala': 'Thiruvananthapuram', 
            'Maharashtra': 'Mumbai', 'Rajasthan': 'Jaipur', 'Tamil Nadu': 'Chennai', 
            'Uttar Pradesh': 'Lucknow', 'West Bengal': 'Kolkata'} 
            # (Imagine all 28 states here)

Step 2: The Main Loop Aditi Ma’am: “We need 35 quizzes, so we loop 35 times. Inside the loop, we create the file paths and open the text files.”

Python

# Generate 35 quiz files
for quizNum in range(35):
    # Create the quiz and answer key files
    quizFile = open(f'capitalsquiz_{quizNum + 1}.txt', 'w')
    answerKeyFile = open(f'capitalsquiz_answers_{quizNum + 1}.txt', 'w')

    # Write out the header for the quiz
    quizFile.write('Name:\n\nDate:\n\nClass:\n\n')
    quizFile.write((' ' * 20) + f'State Capitals Quiz (Form {quizNum + 1})\n\n')

    # Shuffle the order of the states
    states = list(capitals.keys())
    random.shuffle(states)

Chaitanya: “Okay, so states is now a randomized list of the 28 states. We have the files open and the headers written.”

Step 3: Generating the Questions and Answers Aditi Ma’am: “Now, we loop through all 28 states to generate the 28 multiple-choice questions for this specific quiz.”

Python

    for questionNum in range(len(states)):
        # Get right and wrong answers
        correctAnswer = capitals[states[questionNum]]
        wrongAnswers = list(capitals.values())
        wrongAnswers.remove(correctAnswer) # Remove the correct answer from the wrong list
        wrongAnswers = random.sample(wrongAnswers, 3) # Pick 3 random wrong answers
        
        # Combine and shuffle the answer options
        answerOptions = wrongAnswers + [correctAnswer]
        random.shuffle(answerOptions)

Chaitanya: “That is so clever! You grabbed all the capitals, removed the right one, picked 3 random ones to act as decoys, and then shuffled the 4 options so the right answer isn’t always ‘D’.”

Step 4: Writing to the Files Aditi Ma’am: “Exactly. Now we just write the question to the quizFile, and the correct letter (A, B, C, or D) to the answerKeyFile.”

Python

        # Write the question and the answer options to the quiz file
        quizFile.write(f'{questionNum + 1}. What is the capital of {states[questionNum]}?\n')
        for i in range(4):
            quizFile.write(f"    {'ABCD'[i]}. {answerOptions[i]}\n")
        quizFile.write('\n')

        # Write the answer key to a file
        answerKeyFile.write(f"{questionNum + 1}. {'ABCD'[answerOptions.index(correctAnswer)]}\n")

    # Close both files after the 28 questions are done
    quizFile.close()
    answerKeyFile.close()

Chaitanya: “Let me run it.”

(Chaitanya presses run. Instantly, 70 files populate his folder: 35 quizzes, 35 answer keys).

Chaitanya: (Opens capitalsquiz_1.txt)

Plaintext

Name:

Date:

Class:

                    State Capitals Quiz (Form 1)

1. What is the capital of Gujarat?
    A. Jaipur
    B. Gandhinagar
    C. Patna
    D. Dispur

2. What is the capital of Tamil Nadu?
    A. Chennai
    B. Mumbai
    ...

Chaitanya: “It’s perfect. The Geography teacher is going to think I’m a wizard.”

Aditi Ma’am: “You aren’t a wizard, Chaitanya. You just know how to automate the boring stuff.”

Summary Box (Chapter 9)

  • File Paths: Use Path('folder', 'file.txt') from the pathlib module to handle Windows/Mac slash differences.
  • Navigation: Path.cwd() gets the current directory. The / operator joins paths cleanly.
  • Creating Folders: Path.mkdir(exist_ok=True).
  • File Checking: p.exists(), p.is_file(), p.is_dir().
  • Simple I/O: p.write_text('data') and p.read_text().
  • Advanced I/O: open(path, 'w') (Write mode), open(path, 'a') (Append mode), followed by write() and close().
  • Saving Variables: Use the shelve module to save Python lists and dictionaries directly to the hard drive as a binary database.

Aditi Ma’am: “You have mastered creating and filling files. But look at your folder now. You have 70 text files cluttering up your desktop.”

Chaitanya: “It is a bit of a mess.”

Aditi Ma’am: “Tomorrow, in Chapter 10: Organizing Files, I will teach you how to write scripts that can automatically copy, move, rename, and delete thousands of files in the blink of an eye. We will even learn how to compress them into ZIP files.”

Leave a Comment

💬 Join Telegram