LLM observability: tools for developers

Monitoring, traces, evaluation and quality tracking for LLM applications. An entry can appear under several use cases when its topics or description provide several explicit signals.

Use case

Activity

Sort by

117 entries in this view.

Tool ranking

Ranked by measured growth, normalized across sources. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.

  1. 1

    Skill
    Active

    OpenJudge: A Unified Framework for Holistic Evaluation and Quality Rewards

    PythonLicense: Apache-2.0
    • agent
    • agent-skills
    • ai-agent
    • alignment
    • evaluation

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/agentscope-ai/OpenJudge ~/.claude/skills/OpenJudge

    812GitHub stars+7 (+0.87 %)
  2. 2

    Skill
    Active

    Skills, prompts, and instructions for building AI agents on top of Dynatrace production context

    JavaScriptLicense: Apache-2.0
    • agent-skills
    • ai-agents
    • claude-code
    • devops
    • dql

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/Dynatrace/dynatrace-for-ai ~/.claude/skills/dynatrace-for-ai

    135GitHub stars+4 (+3.1 %)
  3. 3

    Skill
    Active

    Agent Skill evaluation harness for paired variants, trace artifacts, and runner adapters

    PythonLicense: MIT
    • agent-skills
    • evals
    • python
    • skills

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/adewale/skill-eval-harness ~/.claude/skills/skill-eval-harness

    73GitHub stars+4 (+5.8 %)
  4. 4

    Skill
    Active

    Mini Program Engineering Skill Suite is an Agent Skill suite for evidence‑first mini‑program development. It helps agents bring WeChat or other mini‑program projects from vague intent to reliable engineering work: project intake, product…

    PythonLicense: MIT
    • agent-skills
    • ai-agents
    • alipay
    • ci-cd
    • claude-code

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/NocodeMrLi/mini-program-engineering-skill-suite ~/.claude/skills/mini-program-engineering-skill-suite

    43GitHub stars+4 (+10.3 %)
  5. 5

    Skill
    Active

    🪢 Langfuse documentation -- Langfuse is the open source LLM Engineering Platform. Observability, evals, prompt management, playground and metrics to debug and improve LLM apps

    MDXLicense: MIT
    • agent-skills
    • agents
    • genai
    • hacktoberfest
    • langfuse

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/langfuse/langfuse-docs ~/.claude/skills/langfuse-docs

    239GitHub stars+3 (+1.3 %)
  6. 6

    Skill
    Active

    The container platform tailored for Kubernetes multi-cloud, datacenter, and edge management ⎈ 🖥 ☁️

    Go
    • agent-skills
    • ai
    • cloud-native
    • cncf
    • devops

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/kubesphere/kubesphere ~/.claude/skills/kubesphere

    17 035GitHub stars+2 (+0.01 %)
  7. 7

    Agent
    Active

    Amazon Bedrock AgentCore enterprise platform accelerator with AWS CDK, Terraform organization guardrails, MCP/A2A agents, security, memory, and observability.

    PythonLicense: Apache-2.0
    • a2a-protocol
    • agentcore
    • agentcore-agent
    • agentcore-gateway
    • agentcore-identity

    githubmeasured growthOpen source ↗

    6GitHub stars+2 (+50.0 %)
  8. 8

    Skill
    Active

    Dashboard for monitoring claude code sessions.

    PythonLicense: Apache-2.0
    • agents
    • agentskills
    • claude-code
    • claude-code-plugin
    • claude-code-skills

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add JayantDevkar/claude-code-karma

    323GitHub stars+2 (+0.62 %)
  9. 9

    MCP
    Active

    MCP server for Langfuse LLM observability — trace and observation analysis.

    TypeScriptLicense: MIT

      mcpmeasured growthOpen source ↗

      Install claude mcp add langfuse -- npx langfuse-observability-mcp-server

      80GitHub stars+2 (+2.6 %)
    • 10

      MCP
      Active

      Open-source, self-hosted control plane for OpenClaw and Hermes AI-agent fleets on Docker/Kubernetes — REST, CLI, and MCP.

      TypeScriptLicense: Apache-2.0
      • agent-management
      • agent-ops
      • agent-orchestration
      • ai-agent-platform
      • ai-agents

      githubmeasured growthOpen source ↗

      50GitHub stars+2 (+4.2 %)
    • 11

      MCP
      Active

      Deterministic, local-only audit reports for Claude Code AI agent sessions. Rust + SQLite. Zero network calls. MCP server for agents.

      HTMLLicense: MIT
      • agent-monitoring
      • agentic-ai
      • ai-agents
      • ai-engineering
      • audit

      githubmeasured growthOpen source ↗

      22GitHub stars+2 (+10.0 %)
    • 12

      Other
      Active

      AI SRE AgenticOps for Kubernetes and cloud infrastructure.

      Python
      • agenticops
      • ai
      • aiops
      • aisre
      • cloud

      githubmeasured growthOpen source ↗

      782GitHub stars+1 (+0.13 %)
    • 13

      Other
      Active

      A command-line tool to scaffold and manage enterprise-ready AI Agents powered by the A2A (Agent-to-Agent) protocol

      GoLicense: Apache-2.0
      • a2a
      • agents
      • ai
      • containerized
      • docker

      githubmeasured growthOpen source ↗

      14GitHub stars+1 (+7.7 %)
    • 14

      Skill
      Active

      Production-grade AI coding rules for Cursor and Claude Code. 15 rules + 9 doc templates + skills + agents + MCP setup. Drop into any project.

      PythonLicense: MIT
      • ai-agents
      • ai-coding
      • ai-development
      • ai-engineer
      • automation

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/aiagentwithdhruv/ai-dev-stack ~/.claude/skills/ai-dev-stack

      10GitHub stars+1 (+11.1 %)
    • 15

      Skill
      Active

      Audit which Claude Code skills you actually use — surface dead installs and hallucinated invocations from your session logs.

      TypeScriptLicense: MIT
      • anthropic
      • audit
      • claude
      • claude-code
      • claude-code-plugin

      githubmeasured growthOpen source ↗

      Install /plugin marketplace add sfrangulov/skill-graveyard

      10GitHub stars+1 (+11.1 %)
    • 16

      Skill
      Active

      🐙 ADLC Team Skills — Agentic SDLC for Engineering Teams

      ShellLicense: MIT
      • adr
      • agent-skills
      • agentic-sdlc
      • ai-coding
      • developer-tools

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/tikalk/adlc-team-skills ~/.claude/skills/adlc-team-skills

      132GitHub stars+1 (+0.76 %)
    • 17

      MCP
      Active

      Self improving agents through iterations

      PythonLicense: Apache-2.0
      • agent-learning
      • agent-memory
      • agentic-workflows
      • ai-agents
      • claude-code

      githubmeasured growthOpen source ↗

      105GitHub stars+1 (+0.96 %)
    • 18

      Skill
      Active

      Vendor-neutral OpenTelemetry skills for AI coding agents, grounded in upstream sources

      ShellLicense: Apache-2.0
      • agent-skills
      • ai
      • observability
      • opentelemetry

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/ollygarden/opentelemetry-agent-skills ~/.claude/skills/opentelemetry-agent-skills

      97GitHub stars+1 (+1.0 %)
    • 19

      Skill
      Active

      local-first analytics for AI agent skills

      TypeScript
      • agent-skills
      • analytics
      • bun
      • claude-code
      • cli

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/crafter-station/skill-kit ~/.claude/skills/skill-kit

      77GitHub stars+1 (+1.3 %)
    • 20

      Skill
      Active

      Make Claude Opus 4.8 behave like Claude Fable 5 — doctrine output style, drift-catching hooks, and an eval loop against golden Fable transcripts. Claude Code plugin.

      ShellLicense: MIT
      • anthropic
      • claude
      • claude-code
      • claude-code-plugin
      • evals

      githubmeasured growthOpen source ↗

      Install /plugin marketplace add rennf93/opus-fable-playbook

      34GitHub stars+1 (+3.0 %)
    • 21

      MCP
      Active

      Log Claude Code sessions to Opik, the open-source LLM observability and evaluation platform, built by Comet. Tracing, evaluation, and skills for observable AI applications.

      GoLicense: Apache-2.0
      • ai-agent
      • claude-ai
      • claude-code-plugin
      • llm-observation
      • mcp-server

      githubmeasured growthOpen source ↗

      22GitHub stars+1 (+4.8 %)
    • 22

      Skill
      Active

      Production patterns for agent orchestrators, harnesses, evals, MCP/A2A interoperability, memory, provider routing, and LLM usage metering.

      PythonLicense: MIT
      • a2a
      • agent-orchestration
      • agent-skills
      • ai-agents
      • developer-tools

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/ssheleg/agent-stack ~/.claude/skills/agent-stack

      2GitHub stars+1 (+100.0 %)
    • 23

      Other
      Active

      Open-source observability tool that uses AI agents to self-heal your software

      TypeScriptLicense: Apache-2.0
      • ai
      • llm
      • memory
      • observability
      • opentelemetry

      githubmeasured growthOpen source ↗

      1 404GitHub stars+1 (+0.07 %)

    Learning resources

    Ranked by measured growth, normalized across sources. These resources remain available separately and do not take part in the main tool ranking.

    1. 1

      awesome-llm-observability

      Skill
      ActiveOpen source ↗

      50+ curated LLM observability tools PLUS 26 Agent Skills (several with runnable, unit-tested scripts) to build, evaluate, debug, secure & monitor reliable LLM apps. Tracing, evals, guardrails, LLMOps.

      Python
      • ai-agents
      • ai-engineering
      • ai-observability
      • awesome
      • awesome-list
      github

      Install git clone https://github.com/ContextJet-ai/awesome-llm-observability ~/.claude/skills/awesome-llm-observability

      Resourcemeasured growth
      33GitHub stars+3 (+10.0 %)
    2. 2

      trigger_tree

      Skill
      ActiveOpen source ↗

      Documentation-discovery telemetry for Claude Code — heat/cold maps, health grade, evidence-backed router fixes. 100% local, zero tokens. /tt

      PythonLicense: MIT
      • ai-agents
      • claude-code
      • claude-code-plugin
      • docs-as-code
      • documentation
      github

      Install /plugin marketplace add Hedde/trigger_tree

      Resourcemeasured growth
      14GitHub stars+1 (+7.7 %)