LLM observability: tools for developers

Monitoring, traces, evaluation and quality tracking for LLM applications. An entry can appear under several use cases when its topics or description provide several explicit signals.

Use case

Activity

Sort by

75 entries in this view.

Tool ranking

Mixed ranking: measured growth takes priority; entries without two snapshots are estimated. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.

  1. 1

    Skill
    Active

    Real-time execution trace and cost intelligence for Claude Code

    TypeScriptLicense: MIT
    • agentic-workflow
    • ai-productivity
    • anthropic
    • anthropic-claude
    • claude

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add DeibyGS/claudestat

    34GitHub stars+1 (+3.0 %)
  2. 2

    MCP
    Active

    AgentGateway — independent third-party profile of a public API surface, by API Evangelist. AgentGateway is an open-source, AI-native proxy and gateway for routing, observing, and governing traffic to and from AI agents, LLM providers, and…

    • agent-to-agent
    • ai-gateway
    • api-gateway
    • apis-json
    • cncf

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  3. 3

    MCP
    Active

    Deterministic, local-only audit reports for Claude Code AI agent sessions. Rust + SQLite. Zero network calls. MCP server for agents.

    HTMLLicense: MIT
    • agent-monitoring
    • agentic-ai
    • ai-agents
    • ai-engineering
    • audit

    githubmeasured growthOpen source ↗

    22GitHub stars+2 (+10.0 %)
  4. 4

    MCP
    Active

    Governed local-LLM observability: model policy, prompt scanner, capture proxy, 21 tools.

    PythonLicense: MIT

      mcpmeasured growthOpen source ↗

      Install claude mcp add ai-guardian -- uvx ai-guardian-aiops

      0GitHub starsstable
    • 5

      Other
      Active

      AI-detector auditing toolkit: measures how often a detector flags genuine human writing, how stable its verdict is across seeds, and whether that verdict survives meaning-preserving edits. Detector-in-the-loop measurement harness. Claude…

      PythonLicense: MIT
      • ai-detection
      • ai-detector
      • benchmark
      • claude-code
      • claude-code-skill

      githubmeasured growthOpen source ↗

      18GitHub starsstable
    • 6

      Skill
      Active

      Measure prompt and skill improvements with blind A/B comparison.

      PythonLicense: MIT
      • agent-skills
      • ai-tools
      • claude-code
      • claude-code-plugin
      • claude-code-skill

      githubmeasured growthOpen source ↗

      Install /plugin marketplace add shinpr/rashomon

      18GitHub starsstable
    • 7

      Skill
      Active

      Make Claude write clearly, for everyone.

      PythonLicense: 0BSD
      • claude-code
      • claude-code-plugin
      • controlled-language
      • llm-evaluation
      • output-style

      githubmeasured growthOpen source ↗

      Install /plugin marketplace add stefanobaghino/simple-output-styles

      16GitHub starsstable
    • 8

      Other
      Active

      A command-line tool to scaffold and manage enterprise-ready AI Agents powered by the A2A (Agent-to-Agent) protocol

      GoLicense: Apache-2.0
      • a2a
      • agents
      • ai
      • containerized
      • docker

      githubmeasured growthOpen source ↗

      14GitHub stars+1 (+7.7 %)
    • 9

      Skill
      Active

      Glanceable Claude Code and Codex state in your terminal tabs: white=idle, blue=working, orange=waiting. Multi-terminal (iTerm2, WezTerm, AI Power Term), a live status bar with Anthropic usage limits, /sfl and /nil window save-and-restore,…

      ShellLicense: MIT
      • ai-workflow-observability
      • claude-code
      • claude-code-hooks
      • claude-code-skill
      • claude-code-skills

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/wasulajr/headsup ~/.claude/skills/headsup

      1GitHub starsstable
    • 10

      MCP
      Active

      Query Spanlens LLM observability from Cursor, Claude Desktop, or Continue via MCP.

      ↓ 32 downloads/weekTypeScriptLicense: MIT

        mcpmeasured growthOpen source ↗

        Install claude mcp add mcp-server -- npx @spanlens/mcp-server

        12GitHub starsstable
      • 11

        Agent
        Active

        Provide clear documentation for AgentStack’s MCP protocol, plugins, and ecosystem API with usage examples and tool references.

        • a2a
        • anthropic
        • claude-code
        • crewai
        • deep-learning

        githubmeasured growthOpen source ↗

        0GitHub starsstable
      • 12

        Other
        Active

        A Go-native framework for LLM agents, with OpenTelemetry observability built in.

        GoLicense: Apache-2.0
        • a2a
        • agents
        • ai
        • anthropic
        • bedrock

        githubmeasured growthOpen source ↗

        10GitHub starsstable
      • 13

        Skill
        Active

        Anchor — the production-grade AGENTS.md template for AI coding agents. 51 sections of battle-tested rules. Keep your agents grounded.

        License: MIT
        • agent-instructions
        • agents-md
        • ai-agent-instructions
        • ai-coding
        • ai-coding-agent

        githubmeasured growthOpen source ↗

        Install git clone https://github.com/peva3/anchor ~/.claude/skills/anchor

        8GitHub starsstable
      • 14

        Agent
        Active

        The agent framework where the model never holds the trigger — every consequential action clears your policy first, waits for a human when it matters, and lands on a record you can verify. Build on it, or put it around the agent you already…

        PythonLicense: Apache-2.0
        • a2a
        • agent-framework
        • ai-agents
        • ai-governance
        • ai-safety

        githubmeasured growthOpen source ↗

        1GitHub starsstable
      • 15

        Agent
        Active

        MemroOS / memroos: memory OS and governance layer for AI agents, agent workflows, dispatch, proof, and context continuity.

        TypeScript
        • a2a
        • agent-governance
        • agent-memory
        • agent-orchestration
        • agent-workflows

        githubmeasured growthOpen source ↗

        7GitHub starsstable
      • 16

        Agent
        Active

        Amazon Bedrock AgentCore enterprise platform accelerator with AWS CDK, Terraform organization guardrails, MCP/A2A agents, security, memory, and observability.

        PythonLicense: Apache-2.0
        • a2a-protocol
        • agentcore
        • agentcore-agent
        • agentcore-gateway
        • agentcore-identity

        githubmeasured growthOpen source ↗

        5GitHub stars+1 (+25.0 %)
      • 17

        MCP
        Active

        Vendor-neutral OpenTelemetry tracing for A2A agents and MCP services, with W3C context propagation and privacy-safe telemetry.

        PythonLicense: MIT
        • a2a
        • agent2agent
        • agentic-ai
        • ai-agents
        • clean-architecture

        githubmeasured growthOpen source ↗

        1GitHub starsstable
      • 18

        Skill
        Active

        Axiom is a curated marketplace of shared plugins for Claude Code and Codex.

        PythonLicense: MIT
        • agent-skills
        • anthropic
        • claude
        • claude-code
        • claude-code-marketplace

        githubmeasured growthOpen source ↗

        Install git clone https://github.com/netopsengineer/axiom ~/.claude/skills/axiom

        5GitHub starsstable
      • 19

        Skill
        Active

        Deletion-first Agent Skill for removing test bloat, verification theater, and speculative fallbacks while preserving behavior.

        PythonLicense: MIT
        • agent-skills
        • claude
        • claude-ai
        • claude-code
        • claude-skills

        githubmeasured growthOpen source ↗

        Install git clone https://github.com/MrZoyo/deslop-GPT ~/.claude/skills/deslop-GPT

        55GitHub stars+27 (+96.4 %)
      • 20

        Skill
        Active

        Mini Program Engineering Skill Suite is an Agent Skill suite for evidence‑first mini‑program development. It helps agents bring WeChat or other mini‑program projects from vague intent to reliable engineering work: project intake, product…

        PythonLicense: MIT
        • agent-skills
        • ai-agents
        • alipay
        • ci-cd
        • claude-code

        githubestimated momentumOpen source ↗

        Install git clone https://github.com/NocodeMrLi/mini-program-engineering-skill-suite ~/.claude/skills/mini-program-engineering-skill-suite

        39GitHub stars—
      • 21

        Skill
        Active

        Agent skills for Arize — datasets, experiments, and traces via the ax CLI

        PythonLicense: MIT
        • agent-skills
        • ai-agents
        • ai-observability
        • arize
        • claude-code

        githubmeasured growthOpen source ↗

        Install git clone https://github.com/Arize-ai/arize-skills ~/.claude/skills/arize-skills

        47GitHub starsstable
      • 22

        MCP
        Active

        Optimize any AI agent’s skills, tools/MCP, and prompts against your own evals.

        PythonLicense: Apache-2.0
        • agent-optimization
        • agent-skills
        • agentic-ai
        • ai-agents
        • claude

        githubmeasured growthOpen source ↗

        47GitHub stars+1 (+2.2 %)

      Learning resources

      Ranked by measured growth, normalized across sources. These resources remain available separately and do not take part in the main tool ranking.

      1. 1

        awesome-llm-observability

        Skill
        ActiveOpen source ↗

        50+ curated LLM observability tools PLUS 26 Agent Skills (several with runnable, unit-tested scripts) to build, evaluate, debug, secure & monitor reliable LLM apps. Tracing, evals, guardrails, LLMOps.

        Python
        • ai-agents
        • ai-engineering
        • ai-observability
        • awesome
        • awesome-list
        github

        Install git clone https://github.com/ContextJet-ai/awesome-llm-observability ~/.claude/skills/awesome-llm-observability

        Resourcemeasured growth
        32GitHub stars+2 (+6.7 %)
      2. 2

        Agentic_AI_Engineer

        Agent
        ActiveOpen source ↗

        My complete journey to becoming an Agentic AI Engineer through structured learning, projects, experiments, and production-ready implementations of modern AI systems.

        Jupyter NotebookLicense: MIT
        • a2a-protocol
        • agent-orchestration
        • ai-guardrails
        • fastapi
        • kubernetes
        githubResourcemeasured growth
        18GitHub starsstable
      3. 3

        trigger_tree

        Skill
        ActiveOpen source ↗

        Documentation-discovery telemetry for Claude Code — heat/cold maps, health grade, evidence-backed router fixes. 100% local, zero tokens. /tt

        PythonLicense: MIT
        • ai-agents
        • claude-code
        • claude-code-plugin
        • docs-as-code
        • documentation
        github

        Install /plugin marketplace add Hedde/trigger_tree

        Resourcemeasured growth
        14GitHub stars+1 (+7.7 %)