Skip to content

SWIP-15: add BanyanDB queue batch/message metrics (apache/skywalking-banyandb#1169)#13904

Merged
wu-sheng merged 1 commit into
masterfrom
swip-15-banyandb-queue-batch-metrics
Jun 11, 2026
Merged

SWIP-15: add BanyanDB queue batch/message metrics (apache/skywalking-banyandb#1169)#13904
wu-sheng merged 1 commit into
masterfrom
swip-15-banyandb-queue-batch-metrics

Conversation

@wu-sheng

@wu-sheng wu-sheng commented Jun 11, 2026

Copy link
Copy Markdown
Member

Surface BanyanDB queue batch / message granularity metrics (apache/skywalking-banyandb#1169)

Follow-up to the SWIP-15 BanyanDB self-observability redesign. BanyanDB PR #1169 added per-batch and per-message catalogs to the batch-write queue metrics; this PR surfaces the subset that genuinely populates, modeled per the cluster / container / group scopes.

  • If this is non-trivial feature, paste the links/URLs to the design doc.
  • Update the documentation to include this new feature.
    • docs/en/banyandb/dashboards-banyandb.md — added the new metric rows under the Liaison / Data / Endpoint tables.
  • Tests (including UT, IT, E2E) are added to verify the new feature.
    • E2E assertions added to the no-FODC so11y case (test/e2e-v2/cases/banyandb/banyandb-cases.yaml); MAL fixture coverage regenerated. Verified locally: e2e 20/20 passed on a clean OAP image against a BanyanDB 0.11 (apache/skywalking-banyandb#1169) cluster (liaison + hot data); MALExpressionExecutionTest 0 banyandb failures.

New metrics (5 — all e2e-verified to return data)

Metric Scope Source family
publish_batch_throughput instance (liaison) banyandb_queue_pub_total_batch_finished
publish_batch_latency_p99 instance (liaison) banyandb_queue_pub_total_batch_latency
queue_sub_message_throughput instance (data) banyandb_queue_sub_total_message_finished
queue_batch_throughput endpoint (per group) banyandb_queue_sub_total_batch_finished
queue_message_throughput endpoint (per group) banyandb_queue_sub_total_message_finished

Intentionally not modeled (verified empty-by-design)

Confirmed by probing a live BanyanDB 0.11 cluster and reading the BanyanDB source:

  • queue_sub_batch_throughput / queue_sub_batch_latency_p99 at instance/data scope — a data node ingests writes via the per-message dispatch path, which ticks only the message counter; the sub-side batch counters fire on the liaison's OAP-facing handleEOF hop, so they are always empty on a data node.
  • queue_batch_latency_p99 at endpoint scope — banyandb_queue_pub_total_batch_latency is emitted with group="", so it cannot roll up per Endpoint (it is meaningful only at instance scope, where publish_batch_latency_p99 covers it).

Other changes

@wu-sheng wu-sheng added backend OAP backend related. enhancement Enhancement on performance or codes labels Jun 11, 2026
@wu-sheng wu-sheng added this to the 11.0.0 milestone Jun 11, 2026
wankai123
wankai123 previously approved these changes Jun 11, 2026
@wu-sheng wu-sheng force-pushed the swip-15-banyandb-queue-batch-metrics branch from a2a85cc to 541e83f Compare June 11, 2026 07:40
@wu-sheng wu-sheng changed the title SWIP-15: add BanyanDB queue batch/message metrics (BanyanDB #1169) SWIP-15: add BanyanDB queue batch/message metrics (apache/skywalking-banyandb#1169) Jun 11, 2026
@wu-sheng wu-sheng merged commit 75d9433 into master Jun 11, 2026
863 of 873 checks passed
@wu-sheng wu-sheng deleted the swip-15-banyandb-queue-batch-metrics branch June 11, 2026 13:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend OAP backend related. enhancement Enhancement on performance or codes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants