LLM observability: tools for developers

Monitoring, traces, evaluation and quality tracking for LLM applications. An entry can appear under several use cases when its topics or description provide several explicit signals.

Use case

Activity

Sort by

122 entries in this view.

Tool ranking

Ranked by measured growth, normalized across sources. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.

  1. 1

    Other
    Active

    A Go-native framework for LLM agents, with OpenTelemetry observability built in.

    GoLicense: Apache-2.0
    • a2a
    • agents
    • ai
    • anthropic
    • bedrock

    githubmeasured growthOpen source ↗

    10GitHub starsstable
  2. 2

    Skill
    Dormant

    Local open-source dev tool to debug, secure, and evaluate LLM agents. Provides static analysis, dynamic security checks, and runtime monitoring - integrates with Cursor and Claude Code.

    Python
    • agent-debugging
    • agent-security
    • agent-trace
    • ai-security-tool
    • behavior-analysis

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add cylestio/agent-inspector

    9GitHub starsstable
  3. 3

    Skill
    Active

    Anchor — the production-grade AGENTS.md template for AI coding agents. 51 sections of battle-tested rules. Keep your agents grounded.

    License: MIT
    • agent-instructions
    • agents-md
    • ai-agent-instructions
    • ai-coding
    • ai-coding-agent

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/peva3/anchor ~/.claude/skills/anchor

    8GitHub starsstable
  4. 4

    Skill
    Dormant

    A self-improving harness router for Claude Code.

    PythonLicense: MIT
    • a-b-testing
    • ai-agent
    • ai-orchestration
    • ai-routing
    • ai-workflow

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add SeongwoongCho/adaptive-harness

    8GitHub starsstable
  5. 5

    Agent
    Active

    MemroOS / memroos: memory OS and governance layer for AI agents, agent workflows, dispatch, proof, and context continuity.

    TypeScript
    • a2a
    • agent-governance
    • agent-memory
    • agent-orchestration
    • agent-workflows

    githubmeasured growthOpen source ↗

    7GitHub starsstable
  6. 6

    MCP
    Dormant

    Open-source Python framework to deploy AI agents via HTTP, A2A, and MCP with built-in observability

    PythonLicense: MIT
    • a2a
    • a2a-client
    • a2a-server
    • agent-evaluation
    • agentic-ai

    githubmeasured growthOpen source ↗

    5GitHub starsstable
  7. 7

    Skill
    Active

    Axiom is a curated marketplace of shared plugins for Claude Code and Codex.

    PythonLicense: MIT
    • agent-skills
    • anthropic
    • claude
    • claude-code
    • claude-code-marketplace

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/netopsengineer/axiom ~/.claude/skills/axiom

    5GitHub starsstable
  8. 8

    MCP
    Active

    Production reliability, benchmarking, and evaluation layer for MCP agent runtimes — protocol negotiation, A2A interop, circuit breakers, connection pooling, adaptive streaming, and autoscaling, with measured throughput/latency and a…

    PythonLicense: Apache-2.0
    • a2a-protocol
    • agent-to-agent
    • ai-agents
    • async-python
    • autoscaling

    githubmeasured growthOpen source ↗

    3GitHub starsstable
  9. 9

    Skill
    Dormant

    An assembly line for AI software development. 35 skills, 11 agent personas, 29 commands. From raw idea to shipped code.

    ShellLicense: MIT
    • agentic-engineering
    • agents
    • claude-code
    • claude-code-plugin
    • claude-code-skills

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add aneja5/forge-skills

    3GitHub starsstable
  10. 10

    Agent
    Active

    Multi-agent SRE on-call investigator that auto-triages Slack/Discord infrastructure alerts via AWS Bedrock AgentCore, fanning out to specialized agents (CloudWatch, EKS, Slack/Discord scanners) for parallel investigation.

    Python
    • a2a-protocol
    • anthropic
    • aws-bedrock
    • aws-lambda
    • bedrock-agentcore

    githubmeasured growthOpen source ↗

    3GitHub starsstable
  11. 11

    Agent
    Dormant

    Real-time debugging proxy for Agent2Agent (A2A) multi-agent systems

    License: MIT
    • a2a
    • agent2agent
    • debugging
    • debugging-tool
    • developer-tools

    githubmeasured growthOpen source ↗

    3GitHub starsstable
  12. 12

    MCP
    Active

    A typed, replayable, budget-aware programming language for LLM agents. Compiles to Python & TypeScript.

    Shell
    • a2a
    • agent-framework
    • agentic-ai
    • ai
    • ai-agents

    githubmeasured growthOpen source ↗

    2GitHub starsstable
  13. 13

    Skill
    Active

    Claude Code skills where every entry ships receipts — accuracy-gated benchmarks against baseline and placebo, rejects published

    JavaScriptLicense: MIT
    • agent-skills
    • ai-agents
    • anthropic
    • benchmark
    • claude

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/sjh9714/skill-receipts ~/.claude/skills/skill-receipts

    2GitHub starsstable
  14. 14

    Skill
    Active

    Multi-agent quality gate skill for Claude Code that researches, reviews, tests and challenges AI-generated work before the final answer.

    • agentic-coding
    • agentic-workflow
    • ai-agent-skills
    • ai-code-review
    • ai-workflows

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/ma-nucho-pro/supervisor-skill-claude ~/.claude/skills/supervisor-skill-claude

    2GitHub starsstable
  15. 15

    Other
    Dormant

    Multi-agent customer support system with Google ADK & Gemini 2.5 Flash Lite. Kaggle capstone demonstrating 11+ concepts. Automates 80%+ queries, <10s response time.

    PythonLicense: MIT
    • a2a-protocol
    • agent-evaluation
    • google-adk
    • multi-agent-system
    • observability

    githubmeasured growthOpen source ↗

    2GitHub starsstable
  16. 16

    Agent
    Dormant

    A Multi-Agent System (MAS) evaluation framework using PydanticAI that generates and evaluates scientific paper reviews through a three-tiered assessment approach: traditional metrics, LLM-as-a-Judge, and graph-based complexity analysis.

    PythonLicense: Apache-2.0
    • a2a-protocol
    • agent-evaluation
    • agentbeats
    • ai-agents
    • benchmarks

    githubmeasured growthOpen source ↗

    2GitHub starsstable
  17. 17

    Agent
    Active

    Generic DeepSeek Harness (dsh) plugins: A2A protocol server, session storage mirror, and Langfuse observability.

    TypeScriptLicense: MIT
    • agent2agent
    • ai-agents
    • cordis
    • deepseek-harness
    • dsh

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  18. 18

    Agent
    Active

    A lightweight Python library that decouples agentic runtime from applications it builds

    PythonLicense: MIT
    • a2a
    • agent-runtime
    • agents
    • observability
    • portability

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  19. 19

    Agent
    Active

    Deterministic crash tests for agent control planes: paired worlds, hard stops, budgets, scopes, approvals, and hash-linked receipts.

    TypeScriptLicense: Apache-2.0
    • a2a
    • agent-evaluation
    • ai-agents
    • benchmark
    • deterministic-replay

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  20. 20

    Agent
    Active

    Agent 降智检测与自愈公评网络 — an immune system for the AI agent society

    TypeScriptLicense: MIT
    • a2a
    • agent
    • agent-guard
    • ai-sre
    • drift-detection

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  21. 21

    Agent
    Dormant

    A safety-first multi-agent mental health companion with real-time distress tracking, triple-layer guardrails, and evidence-based grounding techniques. Built for Kaggle × Google Agents Intensive 2025 Capstone (Agents for Good Track)

    Python
    • a2a-protocol
    • agent-evaluation
    • agents-for-good
    • ai-agents
    • crisis-support

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  22. 22

    Other
    Active

    Local-first Agentic AI Infrastructure Platform with LLM Gateway, Agent DAG Runtime, MCP Tool Hub, A2A Agent Mesh, Local RAG, Tool Sandbox and Observability.

    Python
    • a2a
    • agent-runtime
    • ai-infra
    • deepseek
    • fastapi

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  23. 23

    Agent
    Dormant

    The reliability and interoperability layer for AI agents. A2A-native. MCP-bridged.

    PythonLicense: Apache-2.0
    • a2a
    • agent-framework
    • agents
    • ai
    • ai-safety

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  24. 24

    Skill
    Active

    Skill instalable para Claude Code que mide si tu agente cumple las reglas establecidas. Lee el historial de sesiones de tu disco y devuelve un porcentaje por regla de cumplimiento. Sin instalar nada, sin red, biblioteca estándar.

    PythonLicense: MIT
    • ai-evaluation
    • claude
    • claude-code
    • claude-code-plugin
    • claude-code-skill

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add jleonceo/skill-adherencia-reglas

    1GitHub starsstable

Learning resources

Ranked by measured growth, normalized across sources. These resources remain available separately and do not take part in the main tool ranking.

  1. 1

    tunelab

    Skill
    ActiveOpen source ↗

    Claude Code plugin for LLM fine-tuning, distillation, and evaluation — decide whether you need fine-tuning at all, distill your LLM logs into small local models (MLX/LoRA), evaluate with held-out discipline, and learn the why at every step.

    PythonLicense: MIT
    • claude-code
    • claude-code-plugin
    • distillation
    • fine-tuning
    • llm-evaluation
    github

    Install /plugin marketplace add rchaz/tunelab

    Resourcemeasured growth
    6GitHub starsstable