Python for Unix / Shell Users – Automation and Scripting Training Course

Day 1
Module 1: Introduction to Python for Unix Users

Topics
Why Python is ideal for scripting and automation
• Key advantages of Python for scripting
• Application of Python in system administration and DevOps

Comparison between Bash and Python
• Variables
• Control flow (if statements, loops)
• File handling
• Execution of system commands

Anatomy of a Python script
• Executing Python scripts
• Utilising the interpreter
• Command-line arguments
• Overview of argparse

Exercises
Exercise 1 – Translating a basic Bash script to Python
A Bash script that:
• Lists files in a directory
• Filters by extension
• Displays statistics
Participants will rewrite this script in Python.

Exercise 2 – Python script with CLI arguments
Developing a Python script that:
• Accepts parameters from the command line
• Processes files
• Outputs structured results

Module 2: File and Directory Manipulation

Topics
Interacting with the filesystem in Python
• Reading and writing files
• Navigating directories
• Managing paths

Utilising the pathlib module
• Path management
• File operations

Substituting Unix utilities
• grep
• find
• sed
• awk
with Python equivalents

Exercises
Exercise 1 – Filesystem scanning
A Python script that:
• Locates large files
• Identifies aged files
• Generates a report

Exercise 2 – Processing configuration files
A Python script that:
• Reads text files
• Extracts pertinent information
• Generates statistics

Day 2
Module 3: Log Processing and Text Manipulation

Topics
Handling text in Python
• String operations
• Data filtering and transformation

Regular expressions (regex)
• Pattern recognition
• Log parsing

Replacing shell pipelines
Example:
grep | awk | sed | sort
implemented via Python

Exercises
Exercise 1 – Log file analysis
A Python script that:
• Reads system logs
• Identifies errors
• Calculates error frequency

Exercise 2 – Converting a shell pipeline to Python
Transforming a complex Bash pipeline into a Python script

Module 4: Automating System Commands

Topics
Executing shell commands from Python
• subprocess module
• Capturing output
• Error management

Automating administrative tasks
• Executing system commands
• Integration with cron jobs
• Scripting for repetitive tasks

Process management
• Monitoring processes
• Basic parallel execution

Exercises
Exercise 1 – Python script for executing system commands
A script that:
• Runs Unix commands
• Captures output
• Generates reports

Exercise 2 – Service monitoring
A Python script that:
• Checks service status
• Collects information
• Displays results

Day 3
Module 5: Advanced Automation and Interaction with External Services

Topics
Working with APIs in Python
• Introduction to HTTP requests
• Authentication and tokens
• Processing JSON responses

Python for DevOps automation
• Gathering data from external services
• Integration with existing tools • Infrastructure scripting

Exercises
Exercise 1 – Python script consuming an API
A script that:
• Retrieves data from an API
• Processes the information
• Generates a report

Exercise 2 – Automating data collection
A Python script that:
• Collects system information
• Generates a JSON or CSV report

Module 6: Building a CLI Tool for Administration

Topics
Structuring a Python project
• Code organisation
• Function reuse
• Modules and packages

Creating a CLI tool
• Command-line arguments
• Structured output
• Error handling

Final exercise
Participants will develop a CLI automation tool in Python that:
• Analyses log files
• Identifies errors
• Generates statistics
• Can be executed from the command line

Course Outcomes
By the end of the course, participants will be able to:
• Convert shell scripts into Python scripts
• Utilise Python for system task automation
• Process logs and complex data
• Develop CLI tools for Unix system administration