LLM observability: tools for developers

Monitoring, traces, evaluation and quality tracking for LLM applications. An entry can appear under several use cases when its topics or description provide several explicit signals.

Use case

Activity

Sort by

75 entries in this view.

Tool ranking

Ranked by measured growth, normalized across sources. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.

  1. 1

    Other
    Active

    A high-performance observability data pipeline.

    RustLicense: MPL-2.0
    • agent
    • cloud-native
    • data-transformation
    • datadog
    • etl

    githubmeasured growthOpen source ↗

    22 509GitHub stars+40 (+0.18 %)
  2. 2

    Other
    Active

    SigNoz is an open-source, OpenTelemetry-native observability platform for your team and their AI agents. Get logs, metrics, and traces in one tool with features like APM, distributed tracing, log management, infra monitoring, etc. Combined…

    TypeScript
    • apm
    • application-monitoring
    • distributed-tracing
    • go
    • good-first-issue

    githubmeasured growthOpen source ↗

    32 003GitHub stars+50 (+0.16 %)
  3. 3

    Other
    Active

    Open-source observability tool that uses AI agents to self-heal your software

    TypeScriptLicense: Apache-2.0
    • ai
    • llm
    • memory
    • observability
    • opentelemetry

    githubmeasured growthOpen source ↗

    1 404GitHub stars+2 (+0.14 %)
  4. 4

    Other
    Active

    AI SRE AgenticOps for Kubernetes and cloud infrastructure.

    Python
    • agenticops
    • ai
    • aiops
    • aisre
    • cloud

    githubmeasured growthOpen source ↗

    782GitHub stars+1 (+0.13 %)
  5. 5

    Other
    Active

    eBPF-based Networking, Security, and Observability

    GoLicense: Apache-2.0
    • bpf
    • cncf
    • cni
    • containers
    • ebpf

    githubmeasured growthOpen source ↗

    25 047GitHub stars+29 (+0.12 %)
  6. 6

    Other
    Active

    The fastest path to AI-powered full stack observability, even for lean teams.

    GoLicense: GPL-3.0
    • ai
    • alerting
    • cncf
    • data-visualization
    • database

    githubmeasured growthOpen source ↗

    80 412GitHub stars+85 (+0.11 %)
  7. 7

    Agent
    Active

    eBPF-powered network observability for Kubernetes. Indexes L4/L7 traffic with full K8s context, decrypts TLS without keys. Queryable by AI agents via MCP and humans via dashboard.

    GoLicense: Apache-2.0
    • cloud-native
    • devops
    • docker
    • ebpf
    • golang

    githubmeasured growthOpen source ↗

    12 068GitHub stars+8 (+0.07 %)
  8. 8

    MCP
    Active

    A Model Context Protocol (MCP) server for Langfuse, enabling AI agents to query Langfuse trace data for enhanced debugging and observability

    PythonLicense: MIT
    • agent-skills
    • ai-agents
    • claude-code
    • codex
    • developer-tools

    githubmeasured growthOpen source ↗

    105GitHub starsstable
  9. 9

    Other
    Active

    AgentX python SDK. Build multi-agent AI workforce. Run evaluation. Trace your agent. Full Observability.

    PythonLicense: Apache-2.0
    • a2a
    • agent
    • agent-monitoring
    • evaluation
    • mcp

    githubmeasured growthOpen source ↗

    68GitHub starsstable
  10. 10

    Skill
    Active

    🟪 Open-source runtime that ships any LangGraph or Google ADK agent as a production-ready FastAPI service. Bundled , AG-UI copilotkit API, chat UI, 15+ guardrails, MCP, OpenTelemetry, OIDC. One pip install. Self-hosted, no vendor lock-in.

    PythonLicense: GPL-3.0
    • a2a-protocol
    • ag-ui-protocol
    • agent-platform
    • agentic-ai
    • agents

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/Idun-Group/idun-agent-platform ~/.claude/skills/idun-agent-platform

    198GitHub stars-1 (-0.50 %)
  11. 11

    MCP
    Active

    AURA is a production-tested SRE agent platform you can deploy in minutes. AURA handles the guardrails, APIs, state management, streaming, and failure handling required to put AI to work safely on production infrastructure.

    RustLicense: Apache-2.0
    • a2a
    • ai-agent-framework
    • ai-agents
    • ai-sre
    • aiops

    githubmeasured growthOpen source ↗

    254GitHub stars-3 (-1.2 %)
  12. 12

    MCP
    Active

    74 open-source Agent Skills for Claude Code and Codex: AI SEO, AEO and GEO, code review with an A-F ship grade, CI gates, AI evals, design systems, conversion copy, Instagram growth, iOS and Android app shipping, creator rights, and…

    JavaScriptLicense: MIT
    • agent-orchestration
    • agent-skill
    • agent-skills
    • ai-agents
    • ai-coding

    githubmeasured growthOpen source ↗

    129GitHub stars-4 (-3.0 %)
  13. 13

    Skill
    Active

    Open benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic scoring, pre-registered rubric.

    JavaScriptLicense: MIT
    • aeo
    • agent-skills
    • ai
    • ai-seo
    • benchmark

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/aleclindz/seo-skill-bench ~/.claude/skills/seo-skill-bench

    51GitHub stars-5 (-8.9 %)
  14. 14

    MCP
    Active

    Production reliability, benchmarking, and evaluation layer for MCP agent runtimes — protocol negotiation, A2A interop, circuit breakers, connection pooling, adaptive streaming, and autoscaling, with measured throughput/latency and a…

    PythonLicense: Apache-2.0
    • a2a-protocol
    • agent-to-agent
    • ai-agents
    • async-python
    • autoscaling

    githubmeasured growthOpen source ↗

    3GitHub starsstable
  15. 15

    MCP
    Active

    Architecture-first Python scaffold for an auditable multi-agent corporate credit desk using A2A, MCP, deterministic credit policies, model routing, and OpenTelemetry.

    Python
    • a2a-protocol
    • agentic-ai
    • auditability
    • clean-architecture
    • credit-risk

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  16. 16

    Agent
    Active

    Generic DeepSeek Harness (dsh) plugins: A2A protocol server, session storage mirror, and Langfuse observability.

    TypeScriptLicense: MIT
    • agent2agent
    • ai-agents
    • cordis
    • deepseek-harness
    • dsh

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  17. 17

    MCP
    Active

    A typed, replayable, budget-aware programming language for LLM agents. Compiles to Python & TypeScript.

    Shell
    • a2a
    • agent-framework
    • agentic-ai
    • ai
    • ai-agents

    githubmeasured growthOpen source ↗

    2GitHub starsstable
  18. 18

    Skill
    Active

    Production patterns for agent orchestrators, harnesses, evals, MCP/A2A interoperability, memory, provider routing, and LLM usage metering.

    PythonLicense: MIT
    • a2a
    • agent-orchestration
    • agent-skills
    • ai-agents
    • developer-tools

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/ssheleg/agent-stack ~/.claude/skills/agent-stack

    2GitHub stars+1 (+100.0 %)
  19. 19

    Skill
    Active

    Multi-agent quality gate skill for Claude Code that researches, reviews, tests and challenges AI-generated work before the final answer.

    • agentic-coding
    • agentic-workflow
    • ai-agent-skills
    • ai-code-review
    • ai-workflows

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/ma-nucho-pro/supervisor-skill-claude ~/.claude/skills/supervisor-skill-claude

    2GitHub starsstable
  20. 20

    Agent
    Active

    Deterministic crash tests for agent control planes: paired worlds, hard stops, budgets, scopes, approvals, and hash-linked receipts.

    TypeScriptLicense: Apache-2.0
    • a2a
    • agent-evaluation
    • ai-agents
    • benchmark
    • deterministic-replay

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  21. 21

    Other
    Active

    Local-first Agentic AI Infrastructure Platform with LLM Gateway, Agent DAG Runtime, MCP Tool Hub, A2A Agent Mesh, Local RAG, Tool Sandbox and Observability.

    Python
    • a2a
    • agent-runtime
    • ai-infra
    • deepseek
    • fastapi

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  22. 22

    Agent
    Active

    面向长程研发任务的 Java Agent Harness，支持 A2A 跨语言协作、可恢复执行、上下文工程与 Eval 驱动开发。

    JavaLicense: Apache-2.0
    • a2a
    • agentic-ai
    • ai-agents
    • developer-tools
    • evals

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  23. 23

    Agent
    Active

    Local-first control plane for durable, governed agent teams: DAG orchestration, approvals, memory, signed skills, trace contracts, and realtime.

    PythonLicense: Apache-2.0
    • a2a-protocol
    • agentic-ai
    • ai-agents
    • fastapi
    • llmops

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  24. 24

    MCP
    Active

    Open-source, self-hosted control plane for OpenClaw and Hermes AI-agent fleets on Docker/Kubernetes — REST, CLI, and MCP.

    TypeScriptLicense: Apache-2.0
    • agent-management
    • agent-ops
    • agent-orchestration
    • ai-agent-platform
    • ai-agents

    githubmeasured growthOpen source ↗

    50GitHub stars+2 (+4.2 %)

Learning resources

Ranked by measured growth, normalized across sources. These resources remain available separately and do not take part in the main tool ranking.

  1. 1

    awesome-agentic-devops

    MCP
    ActiveOpen source ↗

    Curated + scored map of official MCP servers and agents for DevOps, Cloud, SRE, and Platform Engineering — every entry rated on production access, approval gates, and audit evidence.

    PythonLicense: MIT
    • agent-skills
    • agentic-ai
    • agentic-devops
    • ai-agents
    • awesome
    githubResourcemeasured growth
    77GitHub starsstable