LLM observability: tools for developers

Monitoring, traces, evaluation and quality tracking for LLM applications. An entry can appear under several use cases when its topics or description provide several explicit signals.

Use case

Activity

Sort by

122 entries in this view.

Tool ranking

Ranked by measured growth, normalized across sources. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.

  1. 1

    MCP
    Active

    Production reliability, benchmarking, and evaluation layer for MCP agent runtimes — protocol negotiation, A2A interop, circuit breakers, connection pooling, adaptive streaming, and autoscaling, with measured throughput/latency and a…

    PythonLicense: Apache-2.0
    • a2a-protocol
    • agent-to-agent
    • ai-agents
    • async-python
    • autoscaling

    githubmeasured growthOpen source ↗

    3GitHub starsstable
  2. 2

    Other
    Active

    A command-line tool to scaffold and manage enterprise-ready AI Agents powered by the A2A (Agent-to-Agent) protocol

    GoLicense: Apache-2.0
    • a2a
    • agents
    • ai
    • containerized
    • docker

    githubmeasured growthOpen source ↗

    14GitHub stars+1 (+7.7 %)
  3. 3

    Agent
    Dormant

    Real-time debugging proxy for Agent2Agent (A2A) multi-agent systems

    License: MIT
    • a2a
    • agent2agent
    • debugging
    • debugging-tool
    • developer-tools

    githubmeasured growthOpen source ↗

    3GitHub starsstable
  4. 4

    Skill
    Active

    Axiom is a curated marketplace of shared plugins for Claude Code and Codex.

    PythonLicense: MIT
    • agent-skills
    • anthropic
    • claude
    • claude-code
    • claude-code-marketplace

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/netopsengineer/axiom ~/.claude/skills/axiom

    5GitHub starsstable
  5. 5

    Skill
    Active

    Claude Code plugin marketplace — agentic-engineering (spec-driven shape→decide→execute→measure→eval with adversarial review) + github-keeper (audit/elevate READMEs and make a repo open-source-ready).

    PythonLicense: MIT
    • adr
    • agentic-engineering
    • ai-agents
    • anthropic
    • claude-code

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add GiustoPiedimonte/agentic-engineering-marketplace

    13GitHub starsstable
  6. 6

    MCP
    Dormant

    Open-source Python framework to deploy AI agents via HTTP, A2A, and MCP with built-in observability

    PythonLicense: MIT
    • a2a
    • a2a-client
    • a2a-server
    • agent-evaluation
    • agentic-ai

    githubmeasured growthOpen source ↗

    5GitHub starsstable
  7. 7

    Other
    Dormant

    A Claude Code skill that adds a rubric-based eval layer to any agent project. Framework-agnostic — generates rubric, test cases, judge prompt, and harness. Returns a weighted score plus a judge-leniency signal.

    HTML
    • agent-evaluation
    • anthropic
    • claude-code
    • claude-code-skill
    • evals

    githubmeasured growthOpen source ↗

    13GitHub starsstable
  8. 8

    Agent
    Active

    Amazon Bedrock AgentCore enterprise platform accelerator with AWS CDK, Terraform organization guardrails, MCP/A2A agents, security, memory, and observability.

    PythonLicense: Apache-2.0
    • a2a-protocol
    • agentcore
    • agentcore-agent
    • agentcore-gateway
    • agentcore-identity

    githubmeasured growthOpen source ↗

    6GitHub stars+2 (+50.0 %)
  9. 9

    MCP
    Active

    Query Spanlens LLM observability from Cursor, Claude Desktop, or Continue via MCP.

    ↓ 32 downloads/weekTypeScriptLicense: MIT

      mcpmeasured growthOpen source ↗

      Install claude mcp add mcp-server -- npx @spanlens/mcp-server

      12GitHub starsstable
    • 10

      Agent
      Active

      Provide clear documentation for AgentStack’s MCP protocol, plugins, and ecosystem API with usage examples and tool references.

      • a2a
      • anthropic
      • claude-code
      • crewai
      • deep-learning

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 11

      Agent
      Active

      MemroOS / memroos: memory OS and governance layer for AI agents, agent workflows, dispatch, proof, and context continuity.

      TypeScript
      • a2a
      • agent-governance
      • agent-memory
      • agent-orchestration
      • agent-workflows

      githubmeasured growthOpen source ↗

      7GitHub starsstable
    • 12

      Skill
      Active

      Production-grade AI coding rules for Cursor and Claude Code. 15 rules + 9 doc templates + skills + agents + MCP setup. Drop into any project.

      PythonLicense: MIT
      • ai-agents
      • ai-coding
      • ai-development
      • ai-engineer
      • automation

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/aiagentwithdhruv/ai-dev-stack ~/.claude/skills/ai-dev-stack

      10GitHub stars+1 (+11.1 %)
    • 13

      Skill
      Dormant

      A self-improving harness router for Claude Code.

      PythonLicense: MIT
      • a-b-testing
      • ai-agent
      • ai-orchestration
      • ai-routing
      • ai-workflow

      githubmeasured growthOpen source ↗

      Install /plugin marketplace add SeongwoongCho/adaptive-harness

      8GitHub starsstable
    • 14

      Skill
      Active

      Audit which Claude Code skills you actually use — surface dead installs and hallucinated invocations from your session logs.

      TypeScriptLicense: MIT
      • anthropic
      • audit
      • claude
      • claude-code
      • claude-code-plugin

      githubmeasured growthOpen source ↗

      Install /plugin marketplace add sfrangulov/skill-graveyard

      10GitHub stars+1 (+11.1 %)
    • 15

      Skill
      Active

      Anchor — the production-grade AGENTS.md template for AI coding agents. 51 sections of battle-tested rules. Keep your agents grounded.

      License: MIT
      • agent-instructions
      • agents-md
      • ai-agent-instructions
      • ai-coding
      • ai-coding-agent

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/peva3/anchor ~/.claude/skills/anchor

      8GitHub starsstable
    • 16

      Skill
      Active

      Turn one decision into a judged tournament of solutions, then pick the best — a Claude Code skill that generates candidates, auto-derives the rubric, judges independently, and returns a defensible winner.

      JavaScriptLicense: MIT
      • ai-agents
      • anthropic
      • claude-code
      • claude-code-plugin
      • claude-skill

      githubmeasured growthOpen source ↗

      Install /plugin marketplace add CoriChui/bakeoff

      10GitHub starsstable
    • 17

      Skill
      Dormant

      Local open-source dev tool to debug, secure, and evaluate LLM agents. Provides static analysis, dynamic security checks, and runtime monitoring - integrates with Cursor and Claude Code.

      Python
      • agent-debugging
      • agent-security
      • agent-trace
      • ai-security-tool
      • behavior-analysis

      githubmeasured growthOpen source ↗

      Install /plugin marketplace add cylestio/agent-inspector

      9GitHub starsstable
    • 18

      Other
      Active

      A Go-native framework for LLM agents, with OpenTelemetry observability built in.

      GoLicense: Apache-2.0
      • a2a
      • agents
      • ai
      • anthropic
      • bedrock

      githubmeasured growthOpen source ↗

      10GitHub starsstable
    • 19

      MCP
      Active

      Architecture-first Python scaffold for an auditable multi-agent corporate credit desk using A2A, MCP, deterministic credit policies, model routing, and OpenTelemetry.

      Python
      • a2a-protocol
      • agentic-ai
      • auditability
      • clean-architecture
      • credit-risk

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 20

      Agent
      Dormant

      Contract-based testing for LLM agents: hybrid evaluation, multi-agent + A2A, deterministic replay.

      PythonLicense: MIT
      • a2a
      • agents
      • contracts
      • evaluation
      • llm

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 21

      Skill
      Active

      Marketing Mix Model expert agent plugin for Claude Code - pymc-marketing v0.18.2+

      Python
      • claude-code-agent
      • claude-code-plugin
      • claude-code-skill
      • data-analysis
      • data-analytics

      githubmeasured growthOpen source ↗

      Install /plugin marketplace add Yakoub-ai/agent-mmm

      4GitHub starsstable
    • 22

      Agent
      Dormant

      A Python proof-of-concept for tracing multi-turn Agent-to-Agent (A2A) conversations as a single unified MLflow trace for LLM observability and evaluation.

      Python
      • a2a
      • agent-to-agent
      • ai-agents
      • conversational-ai
      • llm-evaluation

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 23

      Agent
      Active

      A2A green-agent orchestrator for evaluating agents on the AppWorld benchmark, built on the AgentBeats SDK

      Python
      • a2a-protocol
      • agentbeats
      • appworld
      • evaluation
      • llm-agents

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 24

      Agent
      Dormant

      AgentOps: Multi-agent infrastructure remediation platform. A2A protocol, agent coordination, HITL approval, auto-rollback.

      PythonLicense: MIT
      • a2a-protocol
      • ai
      • automation
      • infrastructure
      • llm

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 25

      Other
      Active

      🔑 HomeStream · 家园·流 — 零成本自托管多Agent协作框架，通往AI世界的那把钥匙 | Zero-cost self-hosted multi-agent framework — The key to AI world

      PythonLicense: MIT
      • a2a-protocol
      • agent-framework
      • ai
      • causal-memory
      • event-stream

      githubmeasured growthOpen source ↗

      0GitHub starsstable