LLM observability: tools for developers

Monitoring, traces, evaluation and quality tracking for LLM applications. An entry can appear under several use cases when its topics or description provide several explicit signals.

Use case

Activity

Sort by

75 entries in this view.

Tool ranking

Mixed ranking: measured growth takes priority; entries without two snapshots are estimated. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.

  1. 1

    MCP
    Active

    Architecture-first Python scaffold for an auditable multi-agent corporate credit desk using A2A, MCP, deterministic credit policies, model routing, and OpenTelemetry.

    Python
    • a2a-protocol
    • agentic-ai
    • auditability
    • clean-architecture
    • credit-risk

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  2. 2

    Skill
    Active

    Marketing Mix Model expert agent plugin for Claude Code - pymc-marketing v0.18.2+

    Python
    • claude-code-agent
    • claude-code-plugin
    • claude-code-skill
    • data-analysis
    • data-analytics

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add Yakoub-ai/agent-mmm

    4GitHub starsstable
  3. 3

    MCP
    Active

    Governed local-LLM observability: model policy, prompt scanner, capture proxy, 21 tools.

    PythonLicense: MIT

      mcpmeasured growthOpen source ↗

      Install claude mcp add ai-guardian -- uvx ai-guardian-aiops

      0GitHub starsstable
    • 4

      Agent
      Active

      Local-first control plane for durable, governed agent teams: DAG orchestration, approvals, memory, signed skills, trace contracts, and realtime.

      PythonLicense: Apache-2.0
      • a2a-protocol
      • agentic-ai
      • ai-agents
      • fastapi
      • llmops

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 5

      Agent
      Active

      Deterministic crash tests for agent control planes: paired worlds, hard stops, budgets, scopes, approvals, and hash-linked receipts.

      TypeScriptLicense: Apache-2.0
      • a2a
      • agent-evaluation
      • ai-agents
      • benchmark
      • deterministic-replay

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 6

      Skill
      Active

      Glanceable Claude Code and Codex state in your terminal tabs: white=idle, blue=working, orange=waiting. Multi-terminal (iTerm2, WezTerm, AI Power Term), a live status bar with Anthropic usage limits, /sfl and /nil window save-and-restore,…

      ShellLicense: MIT
      • ai-workflow-observability
      • claude-code
      • claude-code-hooks
      • claude-code-skill
      • claude-code-skills

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/wasulajr/headsup ~/.claude/skills/headsup

      1GitHub starsstable
    • 7

      Agent
      Active

      Generic DeepSeek Harness (dsh) plugins: A2A protocol server, session storage mirror, and Langfuse observability.

      TypeScriptLicense: MIT
      • agent2agent
      • ai-agents
      • cordis
      • deepseek-harness
      • dsh

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 8

      MCP
      Active

      Vendor-neutral OpenTelemetry tracing for A2A agents and MCP services, with W3C context propagation and privacy-safe telemetry.

      PythonLicense: MIT
      • a2a
      • agent2agent
      • agentic-ai
      • ai-agents
      • clean-architecture

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 9

      MCP
      Active

      Open-source, self-hosted control plane for OpenClaw and Hermes AI-agent fleets on Docker/Kubernetes — REST, CLI, and MCP.

      TypeScriptLicense: Apache-2.0
      • agent-management
      • agent-ops
      • agent-orchestration
      • ai-agent-platform
      • ai-agents

      githubmeasured growthOpen source ↗

      50GitHub stars+2 (+4.2 %)
    • 10

      Skill
      Active

      Real-time execution trace and cost intelligence for Claude Code

      TypeScriptLicense: MIT
      • agentic-workflow
      • ai-productivity
      • anthropic
      • anthropic-claude
      • claude

      githubmeasured growthOpen source ↗

      Install /plugin marketplace add DeibyGS/claudestat

      34GitHub starsstable
    • 11

      Agent
      Active

      The agent framework where the model never holds the trigger — every consequential action clears your policy first, waits for a human when it matters, and lands on a record you can verify. Build on it, or put it around the agent you already…

      PythonLicense: Apache-2.0
      • a2a
      • agent-framework
      • ai-agents
      • ai-governance
      • ai-safety

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 12

      Other
      Active

      Local-first Agentic AI Infrastructure Platform with LLM Gateway, Agent DAG Runtime, MCP Tool Hub, A2A Agent Mesh, Local RAG, Tool Sandbox and Observability.

      Python
      • a2a
      • agent-runtime
      • ai-infra
      • deepseek
      • fastapi

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 13

      MCP
      Active

      AgentGateway — independent third-party profile of a public API surface, by API Evangelist. AgentGateway is an open-source, AI-native proxy and gateway for routing, observing, and governing traffic to and from AI agents, LLM providers, and…

      • agent-to-agent
      • ai-gateway
      • api-gateway
      • apis-json
      • cncf

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 14

      MCP
      Active

      Deterministic, local-only audit reports for Claude Code AI agent sessions. Rust + SQLite. Zero network calls. MCP server for agents.

      HTMLLicense: MIT
      • agent-monitoring
      • agentic-ai
      • ai-agents
      • ai-engineering
      • audit

      githubmeasured growthOpen source ↗

      22GitHub stars+2 (+10.0 %)
    • 15

      MCP
      Active

      A typed, replayable, budget-aware programming language for LLM agents. Compiles to Python & TypeScript.

      Shell
      • a2a
      • agent-framework
      • agentic-ai
      • ai
      • ai-agents

      githubmeasured growthOpen source ↗

      2GitHub starsstable
    • 16

      Other
      Active

      AI-detector auditing toolkit: measures how often a detector flags genuine human writing, how stable its verdict is across seeds, and whether that verdict survives meaning-preserving edits. Detector-in-the-loop measurement harness. Claude…

      PythonLicense: MIT
      • ai-detection
      • ai-detector
      • benchmark
      • claude-code
      • claude-code-skill

      githubmeasured growthOpen source ↗

      18GitHub starsstable
    • 17

      Skill
      Active

      Multi-agent quality gate skill for Claude Code that researches, reviews, tests and challenges AI-generated work before the final answer.

      • agentic-coding
      • agentic-workflow
      • ai-agent-skills
      • ai-code-review
      • ai-workflows

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/ma-nucho-pro/supervisor-skill-claude ~/.claude/skills/supervisor-skill-claude

      2GitHub starsstable
    • 18

      Skill
      Active

      Measure prompt and skill improvements with blind A/B comparison.

      PythonLicense: MIT
      • agent-skills
      • ai-tools
      • claude-code
      • claude-code-plugin
      • claude-code-skill

      githubmeasured growthOpen source ↗

      Install /plugin marketplace add shinpr/rashomon

      18GitHub starsstable
    • 19

      Skill
      Active

      Production patterns for agent orchestrators, harnesses, evals, MCP/A2A interoperability, memory, provider routing, and LLM usage metering.

      PythonLicense: MIT
      • a2a
      • agent-orchestration
      • agent-skills
      • ai-agents
      • developer-tools

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/ssheleg/agent-stack ~/.claude/skills/agent-stack

      2GitHub stars+1 (+100.0 %)
    • 20

      Skill
      Active

      Deletion-first Agent Skill for removing test bloat, verification theater, and speculative fallbacks while preserving behavior.

      PythonLicense: MIT
      • agent-skills
      • claude
      • claude-ai
      • claude-code
      • claude-skills

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/MrZoyo/deslop-GPT ~/.claude/skills/deslop-GPT

      76GitHub stars+48 (+171.4 %)
    • 21

      Skill
      Active

      Mini Program Engineering Skill Suite is an Agent Skill suite for evidence‑first mini‑program development. It helps agents bring WeChat or other mini‑program projects from vague intent to reliable engineering work: project intake, product…

      PythonLicense: MIT
      • agent-skills
      • ai-agents
      • alipay
      • ci-cd
      • claude-code

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/NocodeMrLi/mini-program-engineering-skill-suite ~/.claude/skills/mini-program-engineering-skill-suite

      43GitHub stars+4 (+10.3 %)
    • 22

      Skill
      Active

      Build tested agent skills and govern their lifecycle through a user-defined marketplace: evidence, discovery, updates, rollback, quarantine, and 17-platform distribution.

      PythonLicense: MIT
      • agent-skills
      • ai-agents
      • claude-code
      • codex
      • cursor

      githubestimated momentumOpen source ↗

      Install git clone https://github.com/FrancyJGLisboa/agent-skills-platform ~/.claude/skills/agent-skills-platform

      2 376GitHub stars—

    Learning resources

    Ranked by measured growth, normalized across sources. These resources remain available separately and do not take part in the main tool ranking.

    1. 1

      trigger_tree

      Skill
      ActiveOpen source ↗

      Documentation-discovery telemetry for Claude Code — heat/cold maps, health grade, evidence-backed router fixes. 100% local, zero tokens. /tt

      PythonLicense: MIT
      • ai-agents
      • claude-code
      • claude-code-plugin
      • docs-as-code
      • documentation
      github

      Install /plugin marketplace add Hedde/trigger_tree

      Resourcemeasured growth
      14GitHub stars+1 (+7.7 %)
    2. 2

      awesome-llm-observability

      Skill
      ActiveOpen source ↗

      50+ curated LLM observability tools PLUS 26 Agent Skills (several with runnable, unit-tested scripts) to build, evaluate, debug, secure & monitor reliable LLM apps. Tracing, evals, guardrails, LLMOps.

      Python
      • ai-agents
      • ai-engineering
      • ai-observability
      • awesome
      • awesome-list
      github

      Install git clone https://github.com/ContextJet-ai/awesome-llm-observability ~/.claude/skills/awesome-llm-observability

      Resourcemeasured growth
      33GitHub stars+3 (+10.0 %)
    3. 3

      Agentic_AI_Engineer

      Agent
      ActiveOpen source ↗

      My complete journey to becoming an Agentic AI Engineer through structured learning, projects, experiments, and production-ready implementations of modern AI systems.

      Jupyter NotebookLicense: MIT
      • a2a-protocol
      • agent-orchestration
      • ai-guardrails
      • fastapi
      • kubernetes
      githubResourcemeasured growth
      18GitHub starsstable