• Home
  • Get Started
  • Updates
  • Support
  • Shop
  • Pricing
  • AI News
Get Started
  • Login
  • Register
Neuraldemy
Cart / 0.00$

No products in the cart.

No Result
View All Result
Get Started
Neuraldemy
Get started
Home Free

Python OS Module: A Comprehensive Guide for Beginners

Amritesh Kumar by Amritesh Kumar
August 29, 2024 - Updated on September 3, 2024
in Free, Python
Reading Time: 42 mins read
A A
Python Os Module

The python os module is a powerful tool that allows you to interact with your operating system directly from your Python scripts. Whether you’re building a file management system, automating system tasks, or simply need to navigate directories, the OS module has got you covered.

In this tutorial, we’ll dive deep into the python os module, exploring its key functions and demonstrating how to use them effectively. By the end, you’ll have a solid understanding of how to leverage this module in your Python projects.

Free
Intermediate
5 Days to Learn
OS Module
Prerequisites
Basic Python Knowledge
Understanding of File and Directory Operations
What You Will Learn
  • How to use the OS module in Python
  • Working with files and directories
  • And a lot more
Tutorial Outcomes
By the end of this tutorial on the OS module, you’ll be proficient in handling file and directory operations in Python. You’ll also learn how to manage environment variables and execute system commands, enhancing your ability to interact with the operating system from within your Python programs.
This Tutorial Includes
Concepts & Explanation
Code Examples
Real World Projects

Table Of Contents

  1. What is the Python OS Module?
    1. Python Os Module Methods Quick Reference
    2. Navigating the File System Using Python Os Module
      1. Changing the Current Working Directory
      2. Listing Files and Directories
    3. File and Directory Manipulation Using Python Os Module
      1. Creating a Directory
      2. Renaming Files and Directories
      3. Deleting Files and Directories
    4. Working with Environment Variables
      1. Getting an Environment Variable
      2. Setting an Environment Variable
      3. Deleting an Environment Variable
    5. Executing System Commands
      1. Running a System Command
    6. File and Directory Path Utilities
      1. Checking if a Path Exists
      2. Splitting a Path
      3. Getting the File Extension
    7. Cross-Platform Compatibility
      1. Joining Paths
    8. Other Important Methods:
  2. Practical Projects
    1. Project 1: Directory Size Calculator
    2. Project 2: Directory Tree Printer
    3. Project 3: Disk Usage Monitor
    4. Project 4: Automated Data Preprocessing Pipeline

What is the Python OS Module?

The OS module in Python provides a way to interact with the underlying operating system in a way that is both powerful and platform-independent. Whether you’re on Windows, macOS, or Linux, the OS module has you covered, allowing your Python scripts to perform tasks that would otherwise require manual intervention.

Now, you might be wondering, “Why should I bother learning about this Os module?” Here’s why:

  1. Automation: Automate repetitive tasks like file management, directory navigation, and more.
  2. Cross-platform compatibility: Write code that works across different operating systems.
  3. System interaction: Interact with your computer’s file system and environment variables easily.
  4. Power and flexibility: Perform complex operations with just a few lines of code.

Trust me, once you start using the Os module, you’ll find yourself reaching for it all the time. Before we begin using this module, we need to import it into our script. Here’s how:

import osCode language: JavaScript (javascript)

This single line gives you access to a wide array of functions for interacting with your operating system. The OS module provides several functions for working with directories.

Python Os Module Methods Quick Reference

MethodArgumentsDescription
os.getcwd()NoneReturns the current working directory.
os.chdir()pathChanges the current working directory to path.
os.listdir()[path]Returns a list of entries in the directory at path.
os.mkdir()pathCreates a new directory at path.
os.makedirs()pathCreates a directory at path and any necessary parent dirs.
os.rmdir()pathRemoves the directory at path.
os.removedirs()pathRemoves directories at path recursively.
os.remove()pathRemoves the file at path.
os.rename()src, dstRenames src to dst.
os.path.join()*pathsJoins multiple paths into one.
os.path.basename()pathReturns the base name of path.
os.path.dirname()pathReturns the directory name of path.
os.getenv()key, default=NoneGets an environment variable or returns default.
os.environ.get()key, default=NoneGets an environment variable from os.environ.
os.path.abspath()pathReturns the absolute path of path.
os.path.exists()pathReturns True if path exists.
os.path.isabs()pathReturns True if path is an absolute path.
os.path.isfile()pathReturns True if path is a file.
os.path.isdir()pathReturns True if path is a directory.
os.path.split()pathSplits path into (head, tail) tuple.
os.system()commandRuns a shell command.
os.popen()command, mode='r', buffering=-1Opens a pipe to or from command.
os.open()path, flags, mode=0o777Opens a file descriptor.
os.close()fdCloses the file descriptor fd.
os.read()fd, nReads n bytes from the file descriptor fd.
os.write()fd, strWrites the string str to the file descriptor fd.
os.fstat()fdReturns status info about the file descriptor fd.
os.link()src, dstCreates a hard link pointing from dst to src.
os.symlink()src, dstCreates a symbolic link pointing from dst to src.
os.readlink()pathReturns the target of the symbolic link path.
os.chmod()path, modeChanges the permissions of the file at path.
os.chown()path, uid, gidChanges the owner and group of the file at path.
os.umask()maskSets the current numeric umask.
os.getuid()NoneReturns the current process’s user ID.
os.getgid()NoneReturns the current process’s group ID.
os.geteuid()NoneReturns the current process’s effective user ID.
os.getegid()NoneReturns the current process’s effective group ID.
os.getlogin()NoneReturns the current logged-in user’s name.
This table provides a quick reference for the most commonly used methods in the Python Os module. For more detailed information on each method, refer to the official Python documentation.

We are not going to discuss all of the above methods but the ones that are mostly used.

Navigating the File System Using Python Os Module

The current working directory (CWD) is the folder where your Python script is running. You can find out where you are in the file system with:

cwd = os.getcwd()
print(f"Current Working Directory: {cwd}")Code language: PHP (php)

This is useful when you want to know the starting point of your script or confirm that you’re in the right place.

Changing the Current Working Directory

You can change the current working directory to a different one using os.chdir(). This can be particularly useful if your script needs to work with files in a specific directory.

os.chdir('/path/to/directory')    # make sure to change the name otherwise you will get error
print(f"Changed Directory: {os.getcwd()}")Code language: PHP (php)

Listing Files and Directories

To see what’s inside a directory, the os.listdir() function will list all files and directories within a given path:

files = os.listdir('/path/to/directory')
print(f"Files and Directories: {files}")Code language: PHP (php)

File and Directory Manipulation Using Python Os Module

The os module also allows you to create, delete, and rename files and directories.

Creating a Directory

You can create a new directory with os.mkdir():

os.mkdir('new_directory')
print("Directory 'new_directory' created")Code language: PHP (php)

If you need to create a directory along with any necessary parent directories, use os.makedirs():

os.makedirs('parent_directory/new_directory')
print("Directory 'new_directory' within 'parent_directory' created")Code language: PHP (php)

Renaming Files and Directories

Renaming is straightforward with os.rename():

os.rename('old_name.txt', 'new_name.txt')
print("File renamed from 'old_name.txt' to 'new_name.txt'")Code language: PHP (php)

This works for both files and directories.

Deleting Files and Directories

To delete a file, use os.remove():

os.remove('file_to_delete.txt')
print("File 'file_to_delete.txt' deleted")Code language: PHP (php)

For directories, there are two options:

  • os.rmdir() deletes an empty directory.
  • os.removedirs() removes a directory along with its parent directories if they are empty.
os.removedirs('parent_directory/empty_directory')
print("Directory 'empty_directory' and 'parent_directory' deleted")Code language: PHP (php)

Working with Environment Variables

Environment variables are crucial for storing configuration settings and secrets like API keys. The os module lets you access and modify these variables easily.

Getting an Environment Variable

You can fetch the value of an environment variable using os.getenv():

home_dir = os.getenv('HOME')
print(f"Home Directory: {home_dir}")Code language: PHP (php)

os.environ.get('KEY') another way to access environment variables. If the variable KEY exists, it returns its value. If it doesn’t exist, it returns None (you can also specify a default value).

Setting an Environment Variable

To set an environment variable, you can use os.environ:

os.environ['MY_VARIABLE'] = 'some_value'
print(f"MY_VARIABLE set to: {os.environ['MY_VARIABLE']}")Code language: PHP (php)

Deleting an Environment Variable

If you need to remove an environment variable, you can do so with os.environ.pop():

os.environ.pop('MY_VARIABLE', None)
print("MY_VARIABLE deleted")Code language: PHP (php)

Let’s say you have a script that connects to a database. You don’t want to write the database password directly in the script because that’s sensitive information. Instead, you store the password in an environment variable and use it in your script.

On Windows: Open Command Prompt and run: set DB_PASSWORD=mysecretpassword

On Linux or macOS: Open Terminal and run: export DB_PASSWORD=mysecretpassword

Now, write a Python script that uses the DB_PASSWORD environment variable to connect to the database:

import os

# Access the database password from environment variable
db_password = os.environ.get('DB_PASSWORD')

# Use the password to connect to the databas 
if db_password:
    print(f"Connecting to the database with password: {db_password}")
else:
    print("No database password found!")Code language: Python (python)

By using os.environ.get('DB_PASSWORD'), your script gets the database password from the environment. If the environment variable is not set, it avoids trying to connect, which can be a security measure.

Executing System Commands

Sometimes, you need to execute system-level commands from within your Python script. The os.system() function allows you to do just that.

Running a System Command

Here’s an example of running a simple command like ls or dir:

os.system('ls')Code language: JavaScript (javascript)

This command lists the contents of the current directory on Unix-like systems. On Windows, you might use os.system('dir') instead.

File and Directory Path Utilities

The os.path submodule provides utilities to work with file paths.

Checking if a Path Exists

Before performing operations on a file or directory, it’s good practice to check if it exists:

if os.path.exists('some_file.txt'):
    print("File exists")
else:
    print("File does not exist")Code language: PHP (php)

Splitting a Path

You can split a path into the directory and the file name using os.path.split():

directory, file_name = os.path.split('/path/to/some_file.txt')
print(f"Directory: {directory}, File Name: {file_name}")Code language: PHP (php)

Getting the File Extension

If you need to know the file extension, use os.path.splitext() :

file_name, file_extension = os.path.splitext('some_file.txt')
print(f"File Name: {file_name}, File Extension: {file_extension}")Code language: PHP (php)

Cross-Platform Compatibility

One of the strengths of Python’s os module is its ability to handle tasks in a way that works across different operating systems. This means your script can run smoothly on Windows, macOS, or Linux without modification.

Joining Paths

When working with file paths in Python, it’s crucial to ensure that the paths are constructed correctly, no matter the operating system. Different operating systems use different path separators (e.g., / on Unix-like systems such as Linux and macOS, and \ on Windows). The os.path.join() function helps you build paths that are automatically adjusted for the correct separator.

full_path = os.path.join('folder', 'subfolder', 'file.txt')
print(f"Full Path: {full_path}")Code language: PHP (php)

On a Unix-like system (Linux or macOS), the output will be: Full Path: folder/subfolder/file.txt

On a Windows system, the output will look like this: Full Path: folder\subfolder\file.txt

Now that you have the full path, you can easily open and read the file:

if os.path.exists(full_path):
    with open(full_path, 'r') as file:
        profile_data = file.read()
        print("Profile Data:", profile_data)
else:
    print(f"The file {full_path} does not exist.")
Code language: Python (python)

The os.path.basename() function extracts the base name of the file or directory from a given path. The base name is the last part of the path, which is typically the file name. It takes the full path as an input and returns just the file name (file.txt) or the last part of the path.

path = '/home/user/documents/file.txt'
base_name = os.path.basename(path)
print(f"Base name: {base_name}") 

# output 
Base name: file.txt Code language: PHP (php)

The os.path.dirname() function returns the directory path, excluding the base name (file name or last directory). It strips the base name and returns the directory part of the path.

path = '/home/user/documents/file.txt'
dir_name = os.path.dirname(path)
print(f"Directory name: {dir_name}") # output will be /home/user/documentsCode language: PHP (php)

The os.path.split() function is a combination of os.path.basename() and os.path.dirname(). It splits the path into two parts: the directory path and the base name.

Other Important Methods:

os.walk() is a powerful function in Python’s os module that allows you to traverse a directory tree. It generates file names in a directory tree by walking either top-down or bottom-up. This method returns an iterator that provides a tuple for each iteration, containing three elements:

os.walk(top, topdown=True, onerror=None, followlinks=False)
__
top: The directory where the traversal begins.
topdown: If True (default), the tree is traversed from top to bottom. If False, it's traversed bottom to top.
onerror: A function to handle errors.
followlinks: If True, symbolic links are followed.Code language: PHP (php)
import os

for root, dirs, files in os.walk("/path/to/directory"):
    print(f"Current directory: {root}")
    print(f"Subdirectories: {dirs}")
    print(f"Files: {files}")
    print("---")
Code language: Python (python)

Important Related Methods

  1. os.path.abspath(path) Returns the absolute path of a file or directory. Example: os.path.abspath('file.txt')
  2. os.path.isfile(path) Checks if the path is a regular file. Example: os.path.isfile('document.txt')
  3. os.path.isdir(path)Checks if the path is a directory. Example: os.path.isdir('/path/to/directory')

Practical Projects

Let’s put our knowledge of Python Os Module into practice with a couple of small projects.

Project 1: Directory Size Calculator

Create a script that calculates the total size of a directory, including all its subdirectories and files.

import os 

# get size in bytes
def get_size(directory):
    total_size = 0
    
    for dirpath, dirname, filenames in os.walk(directory):
        for filename in filenames:
            file_path = os.path.join(dirpath, filename)
            total_size += os.path.getsize(file_path)
    
    return total_size

# run this function to get size
Code language: Python (python)

Project 2: Directory Tree Printer

This script prints a tree-like structure of a given directory:

import os

def print_directory_tree(directory, level=0):
    items = os.listdir(directory)
    for item in items:
        print('|   ' * level + '+-- ' + item)
        item_path = os.path.join(directory, item)
        if os.path.isdir(item_path):
            print_directory_tree(item_path, level + 1)

print_directory_tree('/path/to/directory')
Code language: Python (python)

Project 3: Disk Usage Monitor

Create a script that monitors disk usage and sends an alert if the usage exceeds a certain threshold

import os
import shutil

def check_disk_usage(directory, threshold_percentage):
    total, used, free = shutil.disk_usage(directory)   # this will be discussed in another tutorial
    used_percentage = (used / total) * 100

    print(f"Disk usage: {used_percentage:.2f}%")

    if used_percentage > threshold_percentage:
        print(f"Warning: Disk usage exceeds {threshold_percentage}%")

check_disk_usage('/', 80)  # Check disk usage for root directory and alert if usage exceeds 80%
Code language: Python (python)

Project 4: Automated Data Preprocessing Pipeline

Create a script that automates the preprocessing of data files, including reading, cleaning, and saving processed data to a specific directory.

import os
import pandas as pd

def preprocess_data(input_dir, output_dir):
    if not os.path.exists(output_dir):
        os.makedirs(output_dir)

    for file_name in os.listdir(input_dir):
        if file_name.endswith('.csv'):
            file_path = os.path.join(input_dir, file_name)
            data = pd.read_csv(file_path)
            
            # Drop rows with missing values
            cleaned_data = data.dropna()

            # Save cleaned data to output directory
            output_file_path = os.path.join(output_dir, file_name)
            cleaned_data.to_csv(output_file_path, index=False)
            print(f"Processed {file_name} and saved to {output_file_path}")

preprocess_data('/path/to/raw_data', '/path/to/processed_data')
Code language: Python (python)

In this tutorial, we’ve explored the versatile Python os module, a powerful tool for interacting with the operating system. We’ve covered essential methods and functionalities that enable you to manage files, directories, and system processes effectively. You don’t need to remmeber all the methods as you can always search online.

Feel free to experiment with the examples provided, and adapt them to fit your specific needs. The more you practice, the more proficient you’ll become in leveraging the os module to its fullest potential. Happy coding!

Tags: Python ProjectsPython Tutorial
Previous Post

Mathematics For Machine Learning: Mathematical Intuition Basics

Next Post

Pickle Module In Python: A Comprehensive Guide for Beginners

Amritesh Kumar

Amritesh Kumar

I believe you are not dumb or unintelligent; you just never had someone who could simplify the concepts you struggled to understand. My goal here is to simplify AI for all. Please help me improve this platform by contributing your knowledge on machine learning and data science, or help me improve current tutorials. I want to keep all the resources free except for support and certifications. Email me @amriteshkr18@gmail.com.

Related Posts

pickle module in python

Pickle Module In Python: A Comprehensive Guide for Beginners

Install Miniconda on Windows (The EASIEST Way! )

Install Miniconda on Windows (The EASIEST Way! )

TensorFlow Simplified Guide For Beginners

Create A To-Do-List App In Python: Complete Tutorial

Notes App In Python: Complete Tutorial

E-commerce App In Python – User Facing

Next Post
pickle module in python

Pickle Module In Python: A Comprehensive Guide for Beginners

MIT Researchers Develop AI Model for Human-Like Vocal Imitations

Understanding Memoization in JavaScript: Optimizing Function Calls with Caching

  • Customer Support
  • Get Started
  • Ask Your ML Queries
  • Contact
  • Privacy Policy
  • Terms Of Use
Neuraldemy

© 2024 - A learning platform by Odist Magazine

Welcome Back!

Login to your account below

Forgotten Password? Sign Up

Create New Account!

Fill the forms below to register

*By registering into our website, you agree to the Terms & Conditions and Privacy Policy.
All fields are required. Log In

Retrieve your password

Please enter your username or email address to reset your password.

Log In
No Result
View All Result
  • Home
  • Get Started
  • Updates
  • Support
  • Shop
  • Pricing
  • AI News
  • Login
  • Sign Up
  • Cart
Order Details

© 2024 - A learning platform by Odist Magazine

This website uses cookies. By continuing to use this website you are giving consent to cookies being used.
Are you sure want to unlock this post?
Unlock left : 0
Are you sure want to cancel subscription?
0