Metrics
Laminar collects metrics from multiple sources and stores them in GrepTimeDB.
Collection Architecture
Desktop Mode
Vector scrapes Prometheus metrics directly from Laminar components:
Laminar Backend āāāŗ Vector āāāŗ GrepTimeDB
- Controller (8004)
- Workers (6901-6920)
Kubernetes Mode
Vector runs as a DaemonSet collecting metrics from multiple sources:
āāāāāāāāāāāāāāāāāāāāāāā
ā Laminar Engine:8004 āāāāŗ Laminar-Metrics datasource
āāāāāāāāāāāāāāāāāāāāāāā
āāāāāāāāāāāāāāāāāāāāāāā
ā Vector host_metrics āāāā
āāāāāāāāāāāāāāāāāāāāāā⤠ā
ā kubelet cAdvisor āāāā¼āāāŗ GrepTimeDB-Metrics datasource
āāāāāāāāāāāāāāāāāāāāāā⤠ā
ā kube-state-metrics āāāā
āāāāāāāāāāāāāāāāāāāāāāā
Metric Sources
| Source | Collector | Interval | Datasource |
|---|
| Laminar engine | Prometheus scrape | 15s | Laminar-Metrics |
| Node metrics | Vector host_metrics | 15s | GrepTimeDB-Metrics |
| Container metrics | kubelet cAdvisor | 15s | GrepTimeDB-Metrics |
| K8s objects | kube-state-metrics | 30s | GrepTimeDB-Metrics |
Laminar Engine Metrics
Scraped directly from laminar-engine:8004/metrics.
Controller Metrics
| Metric | Type | Description |
|---|
laminar_controller_active_pipelines | Gauge | Number of running pipelines |
laminar_controller_registered_nodes | Gauge | Number of registered worker nodes |
laminar_controller_registered_slots | Gauge | Total slot count across all workers |
laminar_controller_free_slots | Gauge | Available (unused) slot count |
laminar_controller_compaction_tuples_in | Counter | Tuples received for compaction |
laminar_controller_compaction_tuples_out | Counter | Tuples emitted after compaction |
Worker Metrics
| Metric | Type | Labels | Description |
|---|
laminar_worker_messages_recv | Counter | node_id, subtask_idx, operator_name | Messages received by operator |
laminar_worker_messages_sent | Counter | node_id, subtask_idx, operator_name | Messages sent by operator |
laminar_worker_bytes_recv | Counter | node_id, subtask_idx, operator_name | Bytes received by operator |
laminar_worker_bytes_sent | Counter | node_id, subtask_idx, operator_name | Bytes sent by operator |
laminar_worker_batches_recv | Counter | node_id, subtask_idx, operator_name | Batches received by operator |
laminar_worker_batches_sent | Counter | node_id, subtask_idx, operator_name | Batches sent by operator |
laminar_worker_deserialization_errors | Counter | node_id, subtask_idx, operator_name | Deserialization error count |
GrepTimeDB Metric Views
Pre-defined views in laminar_metric_views database for easier querying:
| View | Description |
|---|
active_pipelines_metrics | Pipeline count over time |
slot_utilization_metrics | Slot usage percentage (free/registered) |
registered_nodes_metrics | Node count over time |
registered_slots_metrics | Total slots over time |
free_slots_metrics | Available slots over time |
worker_messages_metrics | Messages recv/sent per operator |
worker_bytes_metrics | Bytes recv/sent per operator |
worker_batches_metrics | Batches recv/sent per operator |
worker_errors_metrics | Deserialization errors per operator |
compaction_tuples_metrics | Compaction in/out counts |
Node Metrics
Collected by Vector's host_metrics source.
CPU
| Metric | Description |
|---|
node_cpu_seconds_total | CPU time in seconds |
node_logical_cpus | Number of logical CPUs |
node_physical_cpus | Number of physical CPUs |
node_load1 | 1-minute load average |
node_load5 | 5-minute load average |
node_load15 | 15-minute load average |
Memory
| Metric | Description |
|---|
node_memory_total_bytes | Total memory |
node_memory_free_bytes | Free memory |
node_memory_available_bytes | Available memory |
node_memory_used_bytes | Used memory |
node_memory_cached_bytes | Cached memory |
node_memory_buffers_bytes | Buffer memory |
node_memory_swap_total_bytes | Total swap |
node_memory_swap_free_bytes | Free swap |
Disk
| Metric | Description |
|---|
node_disk_read_bytes_total | Bytes read |
node_disk_written_bytes_total | Bytes written |
node_disk_reads_completed_total | Read operations |
node_disk_writes_completed_total | Write operations |
Filesystem
| Metric | Description |
|---|
node_filesystem_total_bytes | Filesystem size |
node_filesystem_free_bytes | Free space |
node_filesystem_used_bytes | Used space |
node_filesystem_used_ratio | Usage percentage |
Network
| Metric | Description |
|---|
node_network_receive_bytes_total | Bytes received |
node_network_transmit_bytes_total | Bytes transmitted |
node_network_receive_packets_total | Packets received |
node_network_transmit_packets_total | Packets transmitted |
node_network_receive_errs_total | Receive errors |
node_network_transmit_errs_total | Transmit errors |
Container Metrics (cAdvisor)
Collected from kubelet's cAdvisor endpoint.
CPU
| Metric | Description |
|---|
container_cpu_usage_seconds_total | CPU usage in seconds |
container_cpu_cfs_throttled_seconds_total | Throttled time |
container_cpu_cfs_periods_total | CFS periods |
Memory
| Metric | Description |
|---|
container_memory_usage_bytes | Current memory usage |
container_memory_working_set_bytes | Working set size |
container_memory_rss | RSS memory |
container_memory_cache | Cache memory |
container_spec_memory_limit_bytes | Memory limit |
Network
| Metric | Description |
|---|
container_network_receive_bytes_total | Bytes received |
container_network_transmit_bytes_total | Bytes transmitted |
container_network_receive_errors_total | Receive errors |
container_network_transmit_errors_total | Transmit errors |
Kubernetes Object Metrics
Collected from kube-state-metrics.
Pods
| Metric | Description |
|---|
kube_pod_info | Pod information |
kube_pod_status_phase | Pod phase (Running, Pending, etc.) |
kube_pod_status_ready | Pod ready status |
kube_pod_container_status_restarts_total | Container restart count |
kube_pod_container_resource_requests | Resource requests |
kube_pod_container_resource_limits | Resource limits |
Deployments
| Metric | Description |
|---|
kube_deployment_status_replicas | Current replicas |
kube_deployment_status_replicas_available | Available replicas |
kube_deployment_status_replicas_ready | Ready replicas |
kube_deployment_spec_replicas | Desired replicas |
Nodes
| Metric | Description |
|---|
kube_node_info | Node information |
kube_node_status_condition | Node conditions |
kube_node_status_capacity | Node capacity |
kube_node_status_allocatable | Allocatable resources |
Querying Metrics
Grafana (Prometheus datasource)
# Slot utilization percentage
(laminar_controller_registered_slots - laminar_controller_free_slots)
/ laminar_controller_registered_slots * 100
# Worker message throughput
rate(laminar_worker_messages_recv[5m])
# Container CPU usage
rate(container_cpu_usage_seconds_total{namespace="tenant-e6data"}[5m])
GrepTimeDB SQL
-- Recent slot utilization
SELECT * FROM laminar_metric_views.slot_utilization_metrics
WHERE ts > NOW() - INTERVAL '1 hour'
ORDER BY ts DESC
LIMIT 100;
-- Worker throughput by operator
SELECT operator_name, SUM(messages_recv) as total_recv
FROM laminar_metric_views.worker_messages_metrics
WHERE ts > NOW() - INTERVAL '1 hour'
GROUP BY operator_name;