LLM observability: tools for developers

Monitoring, traces, evaluation and quality tracking for LLM applications. An entry can appear under several use cases when its topics or description provide several explicit signals.

Use case

Activity

Sort by

74 entries in this view.

Tool ranking

Ranked by measured growth, normalized across sources. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.

  1. 1

    Other
    Active

    AI-detector auditing toolkit: measures how often a detector flags genuine human writing, how stable its verdict is across seeds, and whether that verdict survives meaning-preserving edits. Detector-in-the-loop measurement harness. Claude…

    PythonLicense: MIT
    • ai-detection
    • ai-detector
    • benchmark
    • claude-code
    • claude-code-skill

    githubmeasured growthOpen source ↗

    18GitHub starsstable
  2. 2

    Skill
    Active

    Measure prompt and skill improvements with blind A/B comparison.

    PythonLicense: MIT
    • agent-skills
    • ai-tools
    • claude-code
    • claude-code-plugin
    • claude-code-skill

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add shinpr/rashomon

    18GitHub starsstable
  3. 3

    Skill
    Active

    Make Claude write clearly, for everyone.

    PythonLicense: 0BSD
    • claude-code
    • claude-code-plugin
    • controlled-language
    • llm-evaluation
    • output-style

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add stefanobaghino/simple-output-styles

    16GitHub starsstable
  4. 4

    MCP
    Active

    Query Spanlens LLM observability from Cursor, Claude Desktop, or Continue via MCP.

    ↓ 32 downloads/weekTypeScriptLicense: MIT

      mcpmeasured growthOpen source ↗

      Install claude mcp add mcp-server -- npx @spanlens/mcp-server

      12GitHub starsstable
    • 5

      Other
      Active

      A Go-native framework for LLM agents, with OpenTelemetry observability built in.

      GoLicense: Apache-2.0
      • a2a
      • agents
      • ai
      • anthropic
      • bedrock

      githubmeasured growthOpen source ↗

      10GitHub starsstable
    • 6

      Skill
      Active

      Anchor — the production-grade AGENTS.md template for AI coding agents. 51 sections of battle-tested rules. Keep your agents grounded.

      License: MIT
      • agent-instructions
      • agents-md
      • ai-agent-instructions
      • ai-coding
      • ai-coding-agent

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/peva3/anchor ~/.claude/skills/anchor

      8GitHub starsstable
    • 7

      Agent
      Active

      MemroOS / memroos: memory OS and governance layer for AI agents, agent workflows, dispatch, proof, and context continuity.

      TypeScript
      • a2a
      • agent-governance
      • agent-memory
      • agent-orchestration
      • agent-workflows

      githubmeasured growthOpen source ↗

      7GitHub starsstable
    • 8

      Skill
      Active

      Axiom is a curated marketplace of shared plugins for Claude Code and Codex.

      PythonLicense: MIT
      • agent-skills
      • anthropic
      • claude
      • claude-code
      • claude-code-marketplace

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/netopsengineer/axiom ~/.claude/skills/axiom

      5GitHub starsstable
    • 9

      MCP
      Active

      Production reliability, benchmarking, and evaluation layer for MCP agent runtimes — protocol negotiation, A2A interop, circuit breakers, connection pooling, adaptive streaming, and autoscaling, with measured throughput/latency and a…

      PythonLicense: Apache-2.0
      • a2a-protocol
      • agent-to-agent
      • ai-agents
      • async-python
      • autoscaling

      githubmeasured growthOpen source ↗

      3GitHub starsstable
    • 10

      MCP
      Active

      A typed, replayable, budget-aware programming language for LLM agents. Compiles to Python & TypeScript.

      Shell
      • a2a
      • agent-framework
      • agentic-ai
      • ai
      • ai-agents

      githubmeasured growthOpen source ↗

      2GitHub starsstable
    • 11

      Skill
      Active

      Multi-agent quality gate skill for Claude Code that researches, reviews, tests and challenges AI-generated work before the final answer.

      • agentic-coding
      • agentic-workflow
      • ai-agent-skills
      • ai-code-review
      • ai-workflows

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/ma-nucho-pro/supervisor-skill-claude ~/.claude/skills/supervisor-skill-claude

      2GitHub starsstable
    • 12

      Agent
      Active

      Generic DeepSeek Harness (dsh) plugins: A2A protocol server, session storage mirror, and Langfuse observability.

      TypeScriptLicense: MIT
      • agent2agent
      • ai-agents
      • cordis
      • deepseek-harness
      • dsh

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 13

      Agent
      Active

      Deterministic crash tests for agent control planes: paired worlds, hard stops, budgets, scopes, approvals, and hash-linked receipts.

      TypeScriptLicense: Apache-2.0
      • a2a
      • agent-evaluation
      • ai-agents
      • benchmark
      • deterministic-replay

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 14

      Other
      Active

      Local-first Agentic AI Infrastructure Platform with LLM Gateway, Agent DAG Runtime, MCP Tool Hub, A2A Agent Mesh, Local RAG, Tool Sandbox and Observability.

      Python
      • a2a
      • agent-runtime
      • ai-infra
      • deepseek
      • fastapi

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 15

      Skill
      Active

      Glanceable Claude Code and Codex state in your terminal tabs: white=idle, blue=working, orange=waiting. Multi-terminal (iTerm2, WezTerm, AI Power Term), a live status bar with Anthropic usage limits, /sfl and /nil window save-and-restore,…

      ShellLicense: MIT
      • ai-workflow-observability
      • claude-code
      • claude-code-hooks
      • claude-code-skill
      • claude-code-skills

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/wasulajr/headsup ~/.claude/skills/headsup

      1GitHub starsstable
    • 16

      Agent
      Active

      The agent framework where the model never holds the trigger — every consequential action clears your policy first, waits for a human when it matters, and lands on a record you can verify. Build on it, or put it around the agent you already…

      PythonLicense: Apache-2.0
      • a2a
      • agent-framework
      • ai-agents
      • ai-governance
      • ai-safety

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 17

      MCP
      Active

      Vendor-neutral OpenTelemetry tracing for A2A agents and MCP services, with W3C context propagation and privacy-safe telemetry.

      PythonLicense: MIT
      • a2a
      • agent2agent
      • agentic-ai
      • ai-agents
      • clean-architecture

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 18

      MCP
      Active

      Architecture-first Python scaffold for an auditable multi-agent corporate credit desk using A2A, MCP, deterministic credit policies, model routing, and OpenTelemetry.

      Python
      • a2a-protocol
      • agentic-ai
      • auditability
      • clean-architecture
      • credit-risk

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 19

      Agent
      Active

      面向长程研发任务的 Java Agent Harness，支持 A2A 跨语言协作、可恢复执行、上下文工程与 Eval 驱动开发。

      JavaLicense: Apache-2.0
      • a2a
      • agentic-ai
      • ai-agents
      • developer-tools
      • evals

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 20

      Agent
      Active

      Local-first control plane for durable, governed agent teams: DAG orchestration, approvals, memory, signed skills, trace contracts, and realtime.

      PythonLicense: Apache-2.0
      • a2a-protocol
      • agentic-ai
      • ai-agents
      • fastapi
      • llmops

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 21

      MCP
      Active

      AgentGateway — independent third-party profile of a public API surface, by API Evangelist. AgentGateway is an open-source, AI-native proxy and gateway for routing, observing, and governing traffic to and from AI agents, LLM providers, and…

      • agent-to-agent
      • ai-gateway
      • api-gateway
      • apis-json
      • cncf

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 22

      MCP
      Active

      Governed local-LLM observability: model policy, prompt scanner, capture proxy, 21 tools.

      PythonLicense: MIT

        mcpmeasured growthOpen source ↗

        Install claude mcp add ai-guardian -- uvx ai-guardian-aiops

        0GitHub starsstable
      • 23

        Agent
        Active

        Provide clear documentation for AgentStack’s MCP protocol, plugins, and ecosystem API with usage examples and tool references.

        • a2a
        • anthropic
        • claude-code
        • crewai
        • deep-learning

        githubmeasured growthOpen source ↗

        0GitHub starsstable

      Learning resources

      Ranked by measured growth, normalized across sources. These resources remain available separately and do not take part in the main tool ranking.

      1. 1

        Agentic_AI_Engineer

        Agent
        ActiveOpen source ↗

        My complete journey to becoming an Agentic AI Engineer through structured learning, projects, experiments, and production-ready implementations of modern AI systems.

        Jupyter NotebookLicense: MIT
        • a2a-protocol
        • agent-orchestration
        • ai-guardrails
        • fastapi
        • kubernetes
        githubResourcemeasured growth
        18GitHub starsstable