File Manipulation and Text Processing

Core Concepts

Files serve as virtual storage units provided by operating systems to persist information. Text files include formats like .txt, .md, .py, .xml, and .ini that store character data, while multimedia files handle audio and video content.

Basic File Operations in Python

Locating Files

In development environments, right-click on any file and select "Copy Path" to quickly obtain its absolute path.

Opening Files

target_path = '/tmp/sample_document.txt'
file_descriptor = open(target_path)  # Loads file into memory without GUI
content = file_descriptor.read()
print(content)
file_descriptor.close()

Access Modes

Three primary modes control file access:

document_path = '/tmp/sample_document.txt'

# Read-only mode
file_handle = open(document_path, 'r')
print('Is readable:', file_handle.readable())
print('Is writable:', file_handle.writable())
# Attempting to write will raise an error
# file_handle.write('test')  # This would fail
data = file_handle.read()
print(data)
file_handle.close()

Mode Characteristics:

• r: Read-only, fails if file doesn't exist
• w: Write-only, creates or completely overwrites existing content
• a: Append-only, writes to end without truncating

Critical Warning: Write mode irreversibly clears existing content. Use version control or backup systems to recover lost data.

Encoding Specifications

Different regions use distinct character encoding schemes. While ASCII serves English content, Chinese systems require GBK or UTF-8. Always specify encoding explicitly:

file_handle = open(document_path, 'r', encoding='utf-8')

Advanced File Handling

Binary vs Text Modes

Binary Mode (b) Multimedia files must use binary mode since encodings like UTF-8 only apply to text.

# Correct: Reading video in binary mode
video_stream = open('/media/sample_video.mp4', 'rb')
binary_data = video_stream.read()
video_stream.close()

# Incorrect: This would fail
# video_stream = open('/media/sample_video.mp4', 'r', encoding='utf8')

Binary mode never uses encoding parameters and typically pairs with read operations for data duplication or scraping tasks.

Text Mode (t) Default when using r, w, or a without explicit specification. Always combined with text file operations.

Escaping Special Characters

Convert escape sequences to literal characters using either:

1. Raw string prefix: r'\n' prints as \n
2. Double backslash: '\\n' prints as \n

Simultaneous Read-Write Access

While modes like r+, w+, and a+ enable both operations, they create unpredictable behavior due to internal buffer management and cursor positioning. This approach is strongly discouraged.

Resource Management

Unclosed file handlers consume system resources, gradually increasing memory usage from kilobytes to gigabytes when processing large files.

Manual Closing:

file_handle.close()  # Releases OS resources

Automatic Closing with Context Managers:

with open('/tmp/document.txt', 'r', encoding='utf-8') as file_handle:
    processed_data = file_handle.read()
    print(processed_data)

# File automatically closes when exiting the indentation block
print("File is now closed")

Word Cloud Generation Pipeline

1. Content Extraction: Read source text
2. Tokenization: Apply segmentation tools like Jieba for Chinese text
3. Shape Masking: Provide silhouette images to the word cloud generator's mask parameter
4. Visualization: Generate and render the word cloud
5. Font Configuration: Specify system font paths to prevent character display issues

Web Scraping Architecture

Core Components

Requests Library: Handles HTTP client operations Selenium Framework: Automates browser interactions

Fundamental Concepts

Web scraping automates data extraction from online sources. Any visible internet content—product listings, media files, articles—constitutes extractable data.

Operational Flow

1. Transmit Request: Send HTTP request to target URL
2. Receive Response: Obtain raw data (HTML, JSON, binary)
3. Parse Content: Extract valuable information using selectors or patterns
4. Persist Results: Save structured data to files or databases

Practical Implementation

Downloading Media Files

import requests

# Image acquisition
image_response = requests.get('https://example-media.com/photos/sample-image.jpg')
binary_content = image_response.content

with open('local_image.jpg', 'wb') as image_file:
    image_file.write(binary_content)
    print('Image saved successfully')

# Video acquisition
video_response = requests.get('https://example-media.com/videos/demo-clip.mp4')

with open('local_video.mp4', 'wb') as video_file:
    video_file.write(video_response.content)
    print('Video saved successfully')

Response Handling:

• response.status_code returns HTTP status (200 for success)
• response.text provides decoded string content
• response.content delivers raw binary data for non-text assets

Text Data Extraction

Regular Expression Module

Python's built-in re module enables pattern-based text extraction without external dependencies.

Pattern Syntax

Greedy vs Non-Greedy Matching:

• .*? performs non-greedy matching, capturing minimal content between delimiters
• (.*?) creates capture groups for data extraction

Global Search Flag:

• re.S treats input as single-line, enabling cross-line pattern matching

HTML Parsing Example

import re

sample_html = '''
<div class="movie">
    <h1>Inception</h1>
    <span class="rating">9.3</span>
</div>
'''

# Extract title and rating
title_pattern = re.findall(r'<h1>(.*?)</h1>', sample_html, re.S)
rating_pattern = re.findall(r'<span class="rating">(.*?)</span>', sample_html, re.S)

print('Movie:', title_pattern[0])
print('Score:', rating_pattern[0])

View Optimization: When patterns exceed screen width, enable soft wrapping in your editor rather than inserting manual line breaks, which would alter the regex pattern.