Skip to content
LogchefLogchef

Docker Logs

Vector’s docker_logs source connects to the Docker daemon socket and streams stdout/stderr from every running container, no log driver configuration needed. This is the simplest path from plain Docker (or Docker Compose) hosts into ClickHouse or VictoriaLogs for querying with Logchef.

Prerequisites

Section titled “Prerequisites”
  • ClickHouse or VictoriaLogs running and accessible
  • Docker Engine with the daemon socket available (/var/run/docker.sock)
  • A Logchef instance (for querying)

Create the ClickHouse Table (ClickHouse only)

Section titled “Create the ClickHouse Table (ClickHouse only)”

Skip this section when VictoriaLogs is your storage backend; it discovers fields from ingested events and does not need a table definition.

CREATE TABLE IF NOT EXISTS default.logs
(
    timestamp DateTime64(3) CODEC(DoubleDelta, LZ4),
    severity_text LowCardinality(String) CODEC(ZSTD(1)),
    severity_number Int32 CODEC(ZSTD(1)),
    service_name LowCardinality(String) CODEC(ZSTD(1)),
    namespace LowCardinality(String) CODEC(ZSTD(1)),
    body String CODEC(ZSTD(1)),
    log_attributes Map(LowCardinality(String), String) CODEC(ZSTD(1)),


    INDEX idx_severity_text severity_text TYPE set(100) GRANULARITY 4,
    INDEX idx_log_attributes_keys mapKeys(log_attributes) TYPE bloom_filter(0.01) GRANULARITY 1,
    INDEX idx_log_attributes_values mapValues(log_attributes) TYPE bloom_filter(0.01) GRANULARITY 1,
    INDEX idx_body body TYPE tokenbf_v1(32768, 3, 0) GRANULARITY 1
)
ENGINE = MergeTree()
PARTITION BY toDate(timestamp)
ORDER BY (namespace, service_name, timestamp)
TTL toDateTime(timestamp) + INTERVAL 30 DAY
SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1;

This is Logchef’s OpenTelemetry schema — see that page for field and codec details.

Configure Vector

Section titled “Configure Vector”
[api]
enabled = true


# Source: stream logs from every container
[sources.docker_logs]
type = "docker_logs"
include_containers = ["*"]


# Transform: map to the OTEL schema
[transforms.remap_docker_logs]
inputs = ["docker_logs"]
type = "remap"
source = '''
  .timestamp = .timestamp
  .service_name = .container_name
  .namespace = "docker"
  .body = .message


  .severity_text = if match(.message, r"(?i)error|exception|fail|critical") {
    "ERROR"
  } else if match(.message, r"(?i)warn") {
    "WARN"
  } else if match(.message, r"(?i)debug") {
    "DEBUG"
  } else {
    "INFO"
  }


  .severity_number = if .severity_text == "ERROR" {
    17
  } else if .severity_text == "WARN" {
    13
  } else if .severity_text == "DEBUG" {
    5
  } else {
    9
  }


  .log_attributes = {
    "container.id": .container_id,
    "container.image": .image,
    "docker.stream": .stream
  }


  del(.message)
  del(.source_type)
'''


# Sink: ship to ClickHouse
[sinks.clickhouse]
type = "clickhouse"
inputs = ["remap_docker_logs"]
endpoint = "http://clickhouse:8123"
database = "default"
table = "logs"
compression = "gzip"
healthcheck.enabled = false
skip_unknown_fields = true

VictoriaLogs sink

Section titled “VictoriaLogs sink”

To store the same normalized container events in VictoriaLogs, add this sink and either keep the ClickHouse sink for dual-write or remove it:

[sinks.victorialogs]
type = "elasticsearch"
inputs = ["remap_docker_logs"]
endpoints = ["http://victorialogs:9428/insert/elasticsearch/"]
api_version = "v8"
mode = "bulk"
compression = "gzip"


[sinks.victorialogs.healthcheck]
enabled = false


[sinks.victorialogs.request.headers]
VL-Stream-Fields = "namespace,service_name"
VL-Time-Field = "timestamp"
VL-Msg-Field = "body"

This maps timestamp to VictoriaLogs’ _time and body to _msg, while keeping the other event fields queryable. Keep the stream-field list stable and low-cardinality; do not add container IDs, request IDs, or trace IDs.

Vector’s docker_logs source emits container_id, container_name, image, stream (stdout/stderr), label (container labels as an object), and a timestamp already parsed from the Docker log entry. Use include_containers / exclude_containers (prefix match) or include_images / include_labels to scope collection to specific services instead of ["*"].

Run with Docker Compose

Section titled “Run with Docker Compose”

Mount the Docker socket read-only so Vector can talk to the daemon. This example starts both storage backends; keep the one your Vector config targets, or use both sinks to compare them in one Logchef dashboard:

services:
  vector:
    image: timberio/vector:latest-alpine
    volumes:
      - ./vector.toml:/etc/vector/vector.toml:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
    command: ["--config", "/etc/vector/vector.toml"]
    restart: unless-stopped


  clickhouse:
    image: clickhouse/clickhouse-server:latest
    ports:
      - "8123:8123"
      - "9000:9000"
    volumes:
      - clickhouse-data:/var/lib/clickhouse


  victorialogs:
    image: victoriametrics/victoria-logs:latest
    command:
      - -storageDataPath=/victoria-logs-data
      - -retentionPeriod=30d
    ports:
      - "9428:9428"
    volumes:
      - victorialogs-data:/victoria-logs-data


volumes:
  clickhouse-data:
  victorialogs-data:

Or run Vector standalone against an existing Docker host:

Terminal window
docker run -d \
  --name vector \
  -v $(pwd)/vector.toml:/etc/vector/vector.toml:ro \
  -v /var/run/docker.sock:/var/run/docker.sock:ro \
  timberio/vector:latest-alpine \
  --config /etc/vector/vector.toml

Adding Container Labels

Section titled “Adding Container Labels”

Docker labels (e.g. com.docker.compose.service, or your own env=production) are available on the event as .label. Promote useful ones into log_attributes in the transform:

[transforms.remap_docker_logs]
inputs = ["docker_logs"]
type = "remap"
source = '''
  # ... existing mapping from above ...


  .log_attributes."compose.service" = .label."com.docker.compose.service" ?? "unknown"
  .log_attributes.environment = .label.environment ?? "unknown"
'''

Connect Logchef

Section titled “Connect Logchef”

Once logs are flowing:

  1. Log in to Logchef
  2. Go to Sources > Add Source
  3. Add the ClickHouse table, the VictoriaLogs base URL, or one source for each
  4. Create or select a team and assign the source

For VictoriaLogs, use timestamp field _time and severity field severity_text. If Logchef runs in the same compose project, use http://victorialogs:9428 as the base URL rather than localhost.

Start querying:

namespace="docker" and severity_text="ERROR"
service_name="checkout-api" and log_attributes.docker.stream="stderr"
log_attributes."compose.service"="api" and body~"connection refused"

The same LogchefQL filters work on the VictoriaLogs source. You can also switch that source to native LogsQL:

namespace:="docker" AND severity_text:="ERROR" | fields _time, service_name, _msg

Next steps

Section titled “Next steps”