Monitoring

Monitor the health and performance of your Bloqd installation.

Health Endpoints

Server Health Check

# Basic health check
curl https://bloqd.example.com/api/v1/health

# Response
{
  "status": "healthy",
  "version": "1.0.0",
  "uptime": 86400,
  "database": "connected",
  "timestamp": "2024-01-15T10:30:00Z"
}

Detailed Health Check

curl https://bloqd.example.com/api/v1/health/detailed \
  -H "Authorization: Bearer f2b_your_api_key"

# Response
{
  "status": "healthy",
  "version": "1.0.0",
  "components": {
    "database": {
      "status": "healthy",
      "latency_ms": 2
    },
    "websocket": {
      "status": "healthy",
      "connections": 5
    },
    "queue": {
      "status": "healthy",
      "pending": 0
    }
  },
  "metrics": {
    "servers_total": 10,
    "servers_online": 9,
    "bans_today": 1523,
    "memory_mb": 256,
    "cpu_percent": 5.2
  }
}

Docker Health Check

The Docker image includes a built-in health check:

# docker-compose.yml
services:
  bloqd:
    image: clusterzx/bloqd:latest
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/api/v1/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

Check health status:

# View health status
docker inspect bloqd --format='{{.State.Health.Status}}'

# View health logs
docker inspect bloqd --format='{{range .State.Health.Log}}{{.Output}}{{end}}'

Logging

Log Levels

Configure in .env:

LOG_LEVEL=info

Level	Description
error	Errors only
warn	Warnings and errors
info	General information (default)
debug	Detailed debugging
trace	Very verbose

View Logs

# Docker logs
docker compose logs bloqd

# Follow logs
docker compose logs -f bloqd

# Last 100 lines
docker compose logs --tail 100 bloqd

# Filter by pattern
docker compose logs bloqd 2>&1 | grep -i error

Log Format

Logs are JSON formatted for easy parsing:

{
  "level": "info",
  "message": "Ban reported",
  "timestamp": "2024-01-15T10:30:00.000Z",
  "server": "web-server-01",
  "ip": "192.168.1.100",
  "jail": "sshd"
}

Log Rotation

Configure Docker log rotation in /etc/docker/daemon.json:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

Or in docker-compose.yml:

services:
  bloqd:
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"

Metrics

Prometheus Metrics

Bloqd exposes Prometheus-compatible metrics:

curl https://bloqd.example.com/api/v1/metrics \
  -H "Authorization: Bearer f2b_your_api_key"

Example metrics:

# HELP bloqd_servers_total Total number of servers
# TYPE bloqd_servers_total gauge
bloqd_servers_total 10

# HELP bloqd_servers_online Number of online servers
# TYPE bloqd_servers_online gauge
bloqd_servers_online 9

# HELP bloqd_bans_total Total bans processed
# TYPE bloqd_bans_total counter
bloqd_bans_total{jail="sshd"} 5000
bloqd_bans_total{jail="nginx-http-auth"} 1200

# HELP bloqd_api_requests_total API requests
# TYPE bloqd_api_requests_total counter
bloqd_api_requests_total{method="GET",status="200"} 10000

# HELP bloqd_websocket_connections Active WebSocket connections
# TYPE bloqd_websocket_connections gauge
bloqd_websocket_connections 5

Prometheus Configuration

# prometheus.yml
scrape_configs:
  - job_name: 'bloqd'
    static_configs:
      - targets: ['bloqd.example.com:443']
    scheme: https
    bearer_token: 'f2b_your_api_key'
    metrics_path: '/api/v1/metrics'

Grafana Dashboard

Import Dashboard

1. Open Grafana
2. Go to Dashboards → Import
3. Paste the dashboard JSON or ID
4. Select your Prometheus data source

Key Panels

Panel	Query
Servers Online	bloqd_servers_online
Bans Today	increase(bloqd_bans_total[24h])
Ban Rate	rate(bloqd_bans_total[5m])
API Latency	histogram_quantile(0.99, bloqd_api_latency_bucket)

Alerting

Alert Rules

Example Prometheus alert rules:

# alerts.yml
groups:
  - name: bloqd
    rules:
      - alert: BloqdServerOffline
        expr: bloqd_servers_online < bloqd_servers_total
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "Bloqd server offline"
          description: "{{ $value }} servers are offline"

      - alert: BloqdHighBanRate
        expr: rate(bloqd_bans_total[5m]) > 10
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: "High ban rate detected"
          description: "Ban rate is {{ $value }} per second"

      - alert: BloqdDatabaseError
        expr: bloqd_health_database != 1
        for: 1m
        labels:
          severity: critical
        annotations:
          summary: "Bloqd database error"

Discord Alerts

Configure in Bloqd settings for real-time alerts:

• Server offline notifications
• High ban rate alerts
• Error notifications

Email Alerts

Configure SMTP for email notifications:

• Daily summary reports
• Critical error alerts

Uptime Monitoring

External Monitoring

Configure external uptime monitors to check:

https://bloqd.example.com/api/v1/health

Expected response:

• Status code: 200
• Response time: < 1000ms
• Body contains: "status":"healthy"

Uptime Services

Compatible with:

• UptimeRobot
• Pingdom
• Better Uptime
• Uptime Kuma (self-hosted)

Example Uptime Kuma configuration:

Type: HTTP(s) - Keyword
URL: https://bloqd.example.com/api/v1/health
Keyword: "healthy"
Interval: 60 seconds

Agent Monitoring

Check Agent Status

From the dashboard API:

curl https://bloqd.example.com/api/v1/servers \
  -H "Authorization: Bearer f2b_your_api_key" | \
  jq '.servers[] | {hostname, status, last_heartbeat}'

Agent Health Metrics

Agents report:

Metric	Description
cpu_percent	CPU usage
memory_percent	Memory usage
disk_percent	Disk usage
fail2ban_status	Fail2ban running
last_heartbeat	Last check-in time

Alert on Offline Agents

Servers are marked offline after missing heartbeats:

# In .env
SERVER_OFFLINE_THRESHOLD=300  # 5 minutes

Performance Baseline

Expected Performance

Metric	Normal Range
API Response Time	< 100ms
Ban Processing	< 50ms
Memory Usage	200-500 MB
CPU Usage	1-10%
Database Size	Depends on retention

Performance Testing

# Simple load test
ab -n 1000 -c 10 -H "Authorization: Bearer f2b_key" \
  https://bloqd.example.com/api/v1/health

# Using wrk
wrk -t4 -c100 -d30s \
  -H "Authorization: Bearer f2b_key" \
  https://bloqd.example.com/api/v1/health

Troubleshooting

High Memory Usage

# Check container memory
docker stats bloqd

# Restart to clear memory
docker compose restart bloqd

High CPU Usage

# Check what's using CPU
docker compose exec bloqd top

# Check for heavy queries
docker compose logs bloqd | grep -i "slow query"

Connection Issues

# Check WebSocket connections
curl https://bloqd.example.com/api/v1/health/detailed | \
  jq '.components.websocket'

# Check network
docker compose exec bloqd netstat -an | grep ESTABLISHED