Automated Word Extraction with VS Code and Python

Introduction

This article demonstrates how to implement an automated word extraction system using Visual Studio Code and Python. This solution is particularly useful for vocabulary building when reading academic papers or other texts where you need to collect specific words for later study.

Prerequisites

To implement this solution, you'll need the following tools:

• Visual Studio Code
• Python develpoment environment

The Python environment is required as we'll be using Python to process the text files. This implementation leverages Python's powerful text processing capabilities through regular expressions.

Configuration Process

The configuration is performed within the VS Code environment. This setup allows you to execute the word extraction process with a simple keyboard shortcut.

Setting Up Tasks

The first step is to configure a task in VS Code that will execute our Python script. This is done by creating a tasks.json file:

1. Press Ctrl + Shift + P to open the command palette
2. Type "Tasks: Configure Default Build Task" and select it
3. Choose "Create tasks.json file from template"
4. Select "Others"

Add the following configuration to your tasks.json file:

{
    "version": "2.0.0",
    "tasks": [
        {
            "label": "extract vocabulary",
            "type": "shell",
            "command": "python",
            "args": [
                "C:\\scripts\\word_extractor.py", "${file}"
            ],
            "problemMatcher": [],
            "group": {
                "kind": "build",
                "isDefault": true
            }
        }
    ]
}

In this configuration:

• "label" is the name of our task
• "command" specifies Python as the execution environment
• "args" contains the script path and the currrent file path (using ${file} variable)

Creating the Extraction Script

The core functionality is implemented in a Python script named word_extractor.py. This script processes text files to extract and organize vocabulary words:

# -*- coding: utf-8 -*-

import re
import os
import sys

def extract_vocabulary(input_path):
    """Extract and organize vocabulary words from a text file."""
    
    # Read the input file
    with open(input_path, 'r', encoding='utf-8') as file:
        content = file.read()
    
    # Process quoted words (removes Zotero references)
    processed_content = re.sub(r'"(\w+)".*\n', r'\1\n', content)
    
    # Split content into lines
    lines = processed_content.split("\n")
    
    # Filter out lines containing Chinese characters
    chinese_pattern = r'[\u4e00-\u9fff]+'
    filtered_lines = [
        line for line in lines 
        if not re.search(chinese_pattern, line)
    ]
    
    # Remove empty lines
    non_empty_lines = [line for line in filtered_lines if line.strip()]
    
    # Sort the remaining words alphabetically
    sorted_words = sorted(non_empty_lines)
    
    # Join the sorted words with newlines
    result_content = "\n".join(sorted_words)
    
    # Write the result back to the file
    with open(input_path, 'w', encoding='utf-8') as file:
        file.write(result_content)
    
    return result_content

if __name__ == "__main__":
    # Get the file path from command line arguments
    source_file = sys.argv[1]
    
    # Process the file
    processed_content = extract_vocabulary(source_file)
    
    print(f"Vocabulary extraction completed for: {source_file}")
    print(f"Extracted {len(processed_content.split())} unique words")

Script Logic Explanation

The Python script performs the following operations:

1. Reads the content of the specified text file
2. Removes Zotero-style references from quoted words
3. Filters out any lines containing Chinese characters
4. Removes empty lines
5. Sorts the remaining words alphabetically
6. Writes the processed content back to the file

Usage Instructions

Once configured, you can use the automated word extraction system as follows:

1. After reading an academic paper, copy the vocabulary words to a text file
2. Open the text file in VS Code
3. Press Ctrl + Shift + P to open the command palette
4. Type "Tasks: Run Task" and select it
5. Choose "extract vocabulary" from the list
6. The script will process the file, extracting and sorting the words

The processed file will now contain only the extracted vocabulary words, sorted alphabetical and ready for import into your vocabulary learning application.