Managing logs across multiple PHP application instances often leads to fragmentation. Relying on SSH to manually inspect server-side files during incidents is inefficient and reactive. To centralize eror tracking and enable immediate visibility, I developed a lightweight remote log aggregation service using Go.

Core Architecture

The service operates as a standalone HTTP server that centralizes incoming error reports, persists them into daily rotated files, and provides a protected interface for administration.

Log Submission (POST /ingest)

Applications submit diagnostic data as JSON payloads. The service ensures these are appended to daily storage files.

// Example log entry submission
curl -X POST http://logger.internal:9000/ingest -H "Content-Type: application/json" -d '{
    "timestamp": "2025-03-15T10:00:00Z",
    "severity": "CRITICAL",
    "payload": {
        "service": "order-processor",
        "error": "Connection timeout",
        "trace": "stacktrace_data_here"
    }
}'

Log Retrieval and Management (GET /stream & DELETE /purge)

To prevent unauthorized access, the dashboard is protected via Basic Authentication. The administrative interface allows users to view current logs and prune specific entries.

// Delete a specific line from a daily log file
curl -X DELETE "http://logger.internal:9000/purge?day=2025-03-15&index=5" 
     -u admin:securepassword

Implementation Details

• Concurrency Control: Since multiple web nodes may push logs simultaneously, the service utilizes sync.RWMutex to handle file I/O safety, preventing data corruption during concurrent write operations.
• Retention Policy: The system automatical manages disk space by pruning logs older than seven days, ensuring the storage volume remains manageable.
• Data Integrity: All logs are persisted as line-delimited JSON, simplifying downstream processing and backup tasks.

By shifting from local flat-file storage to this centralized collector, debguging transitions from "hunting through directories" to "monitoring a live dashboard." This setup provides a unified view of system health, enabling faster response times when application-level exceptions occur.