Home Original page

GitHub - parseablehq/n8n-observability: This repository contains all the information on n8n observability and monitoring using Parseable.

This repository provides a complete solution for monitoring n8n workflow automation with comprehensive observability using Parseable - an open-source, unified observability platform.

🎯 What This Provides

  • 📊 Traces: HTTP requests, database operations, and workflow execution spans
  • 📈 Metrics: Resource utilization, execution counts, and performance metrics
  • 📝 Logs: Structured workflow logs, node execution logs, and error tracking
  • 🔧 Easy Setup: Docker Compose based deployment with minimal configuration

🏗️ Architecture

┌─────────────┐    ┌──────────────┐    ┌─────────────┐
│   n8n       │───▶│ OpenTelemetry│───▶│ Parseable   │
│             │    │ Instrumentation  │ │             │
│ - Workflows │    │ - Winston Logs   │ │ - Traces    │
│ - Nodes     │    │ - HTTP Tracing   │ │ - Metrics   │
│ - Triggers  │    │ - DB Tracing     │ │ - Logs      │
└─────────────┘    └──────────────┘    └─────────────┘

🚀 Quick Start

Prerequisites

  • Docker and Docker Compose
  • Git

1. Clone and Setup

git clone https://github.com/parseablehq/n8n-observability.git
cd n8n-observability

2. Start the Stack

# Start all services
docker-compose up -d

# View logs
docker-compose logs -f n8n

3. Access Services

4. Verify Observability

  1. Create a workflow in n8n
  2. Execute the workflow
  3. Check Parseable for traces, metrics, and logs:
    • otel-traces stream: Workflow execution traces
    • otel-metrics stream: Performance metrics
    • n8n-logs stream: Structured logs

📋 What's Included

Core Files

  • docker-compose.yml - Complete stack orchestration
  • Dockerfile - Custom n8n image with observability
  • n8n-winston-logger.js - Winston transport for structured logging
  • simple-winston-tracing.js - OpenTelemetry setup
  • docker-entrypoint.sh - Container initialization script

Services

  1. n8n - Workflow automation platform with observability
  2. Parseable - Log analytics and observability backend
  3. PostgreSQL - n8n database backend

🔧 Configuration

Environment Variables

Key variables in docker-compose.yml:

# n8n Configuration
N8N_HOST: "localhost"
N8N_PORT: 5678
N8N_PROTOCOL: "http"
N8N_WINSTON_LOGGING: "true"

# Database
DB_TYPE: "postgresdb"
DB_POSTGRESDB_HOST: "postgres"
DB_POSTGRESDB_DATABASE: "n8n"

# Parseable Integration
PARSEABLE_URL: "http://parseable:8000"
PARSEABLE_USERNAME: "admin" 
PARSEABLE_PASSWORD: "admin"

# OpenTelemetry
OTEL_SERVICE_NAME: "n8n-comprehensive"
OTEL_EXPORTER_OTLP_ENDPOINT: "http://parseable:8000"

Parseable Streams

The setup creates three streams automatically:

  • n8n-logs: Structured application logs
  • otel-traces: Execution traces and spans
  • otel-metrics: Performance and resource metrics

📊 Observability Features

Logs

Workflow Events:

logger.workflow.started(workflowId, workflowName, executionId, metadata);
logger.workflow.completed(workflowId, workflowName, executionId, duration, metadata);
logger.workflow.failed(workflowId, workflowName, executionId, error, duration, metadata);

Node Events:

logger.node.started(workflowId, executionId, nodeType, nodeName, metadata);
logger.node.completed(workflowId, executionId, nodeType, nodeName, duration, metadata);
logger.node.failed(workflowId, executionId, nodeType, nodeName, error, duration, metadata);

HTTP Events:

logger.http.request(method, url, headers, metadata);
logger.http.response(method, url, statusCode, duration, metadata);

Traces

Automatic instrumentation for:

  • HTTP requests (incoming/outgoing)
  • Database operations
  • Workflow execution spans
  • Node execution spans

Metrics

Collected metrics include:

  • Request counts and durations
  • Database connection pools
  • Memory and CPU usage
  • Workflow execution statistics

🔍 Querying Data

Sample Parseable Queries

Recent Workflow Executions:

SELECT body, severity_text, time_unix_nano 
FROM "n8n-logs" 
WHERE body LIKE '%workflow%' 
ORDER BY time_unix_nano DESC 
LIMIT 10

Error Analysis:

SELECT body, severity_text, time_unix_nano 
FROM "n8n-logs" 
WHERE severity_text = 'ERROR' 
ORDER BY time_unix_nano DESC

Performance Traces:

SELECT span_name, duration_nanos, status_code 
FROM "otel-traces" 
ORDER BY start_time_unix_nano DESC

🎛️ Customization

Adding Custom Logging

Extend the Winston logger in n8n-winston-logger.js:

// Add custom log methods
logger.custom = {
  event: (eventType, data) => {
    logger.info('Custom event', {
      event_type: eventType,
      ...data
    });
  }
};

Adding Custom Instrumentation

Extend OpenTelemetry setup in simple-winston-tracing.js:

// Add custom instrumentations
const { YourCustomInstrumentation } = require('your-instrumentation');

registerInstrumentations({
  instrumentations: [
    // ... existing instrumentations
    new YourCustomInstrumentation()
  ]
});

🐛 Troubleshooting

Common Issues

1. Logs not appearing in Parseable

  • Check container connectivity: docker network ls
  • Verify Parseable is running: docker-compose logs parseable
  • Check n8n logs: docker-compose logs n8n

2. Authentication errors

  • Verify credentials in docker-compose.yml
  • Check Parseable access: curl -u admin:admin http://localhost:8000/api/v1/logstream

3. High memory usage

  • Adjust batch sizes in n8n-winston-logger.js
  • Configure log retention in Parseable

Debug Commands

# Check service health
docker-compose ps

# View real-time logs
docker-compose logs -f n8n
docker-compose logs -f parseable

# Test Parseable API
curl -u admin:admin http://localhost:8000/api/v1/logstream

# Check n8n database
docker exec -it n8n-postgres psql -U postgres -d n8n

🔐 Security Considerations

Production Deployment

  1. Change Default Credentials:

    PARSEABLE_USERNAME: "your-username"
PARSEABLE_PASSWORD: "your-secure-password"

  2. Use Environment Variables:

    export PARSEABLE_PASSWORD=$(openssl rand -base64 32)

  3. Network Security:

    • Use Docker networks for isolation
    • Configure firewall rules
    • Enable TLS/SSL for production

  4. Data Retention:

    • Configure log rotation
    • Set retention policies
    • Monitor disk usage

🤝 Contributing

  1. Fork the repository
  2. Create a feature branch: git checkout -b feature-name
  3. Commit changes: git commit -am 'Add feature'
  4. Push to branch: git push origin feature-name
  5. Submit pull request

📝 License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

🙏 Acknowledgments

📞 Support

⭐ Star this repo if you find it useful!