LLM observability: tools for developers

Monitoring, traces, evaluation and quality tracking for LLM applications. An entry can appear under several use cases when its topics or description provide several explicit signals.

Use case

Activity

Sort by

122 entries in this view.

Tool ranking

Ranked by measured growth, normalized across sources. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.

  1. 1

    Agent
    Active

    Provide clear documentation for AgentStack’s MCP protocol, plugins, and ecosystem API with usage examples and tool references.

    • a2a
    • anthropic
    • claude-code
    • crewai
    • deep-learning

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  2. 2

    Other
    Active

    AgentX python SDK. Build multi-agent AI workforce. Run evaluation. Trace your agent. Full Observability.

    PythonLicense: Apache-2.0
    • a2a
    • agent
    • agent-monitoring
    • evaluation
    • mcp

    githubmeasured growthOpen source ↗

    68GitHub starsstable
  3. 3

    MCP
    Active

    Production reliability, benchmarking, and evaluation layer for MCP agent runtimes — protocol negotiation, A2A interop, circuit breakers, connection pooling, adaptive streaming, and autoscaling, with measured throughput/latency and a…

    PythonLicense: Apache-2.0
    • a2a-protocol
    • agent-to-agent
    • ai-agents
    • async-python
    • autoscaling

    githubmeasured growthOpen source ↗

    3GitHub starsstable
  4. 4

    Skill
    Active

    Make Claude Opus 4.8 behave like Claude Fable 5 — doctrine output style, drift-catching hooks, and an eval loop against golden Fable transcripts. Claude Code plugin.

    ShellLicense: MIT
    • anthropic
    • claude
    • claude-code
    • claude-code-plugin
    • evals

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add rennf93/opus-fable-playbook

    34GitHub stars+1 (+3.0 %)
  5. 5

    Skill
    Active

    Mide si tu agente de IA cumple las reglas que le escribiste. Lee el historial local de Claude Code y devuelve un porcentaje por regla. Sin instalar nada, sin red, biblioteca estándar.

    PythonLicense: MIT
    • agent-evaluation
    • ai-agents
    • claude-code
    • claude-code-skill
    • claude-code-skills

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/jleonceo/adherencia-reglas ~/.claude/skills/adherencia-reglas

    1GitHub starsstable
  6. 6

    Skill
    Active

    Real-time execution trace and cost intelligence for Claude Code

    TypeScriptLicense: MIT
    • agentic-workflow
    • ai-productivity
    • anthropic
    • anthropic-claude
    • claude

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add DeibyGS/claudestat

    34GitHub starsstable
  7. 7

    Skill
    Active

    Open benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic scoring, pre-registered rubric.

    JavaScriptLicense: MIT
    • aeo
    • agent-skills
    • ai
    • ai-seo
    • benchmark

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/aleclindz/seo-skill-bench ~/.claude/skills/seo-skill-bench

    51GitHub stars-5 (-8.9 %)
  8. 8

    Skill
    Active

    Claude Code skills where every entry ships receipts — accuracy-gated benchmarks against baseline and placebo, rejects published

    JavaScriptLicense: MIT
    • agent-skills
    • ai-agents
    • anthropic
    • benchmark
    • claude

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/sjh9714/skill-receipts ~/.claude/skills/skill-receipts

    2GitHub starsstable
  9. 9

    Agent
    Active

    A2A green-agent orchestrator for evaluating agents on the AppWorld benchmark, built on the AgentBeats SDK

    Python
    • a2a-protocol
    • agentbeats
    • appworld
    • evaluation
    • llm-agents

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  10. 10

    Skill
    Active

    A skill creator that proves its skills work. Evidence-driven skill creation for Claude Code and Codex: baseline-tested generation, per-skill regression evals, ecosystem doctor, cross-runtime compile, and an opt-in proactive advisor.

    PythonLicense: MIT
    • agent-skills
    • claude-ai
    • claude-code
    • claude-skills
    • codex

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/tripleyak/SkillForge ~/.claude/skills/SkillForge

    885GitHub starsstable
  11. 11

    Skill
    Active

    Turn one decision into a judged tournament of solutions, then pick the best — a Claude Code skill that generates candidates, auto-derives the rubric, judges independently, and returns a defensible winner.

    JavaScriptLicense: MIT
    • ai-agents
    • anthropic
    • claude-code
    • claude-code-plugin
    • claude-skill

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add CoriChui/bakeoff

    10GitHub starsstable
  12. 12

    MCP
    Active

    Open-source, self-hosted control plane for OpenClaw and Hermes AI-agent fleets on Docker/Kubernetes — REST, CLI, and MCP.

    TypeScriptLicense: Apache-2.0
    • agent-management
    • agent-ops
    • agent-orchestration
    • ai-agent-platform
    • ai-agents

    githubmeasured growthOpen source ↗

    50GitHub stars+1 (+2.0 %)
  13. 13

    Skill
    Active

    YAO = Yielding AI Outcomes. A rigorous engineering, evaluation, governance, and portability system for reusable agent skills.

    PythonLicense: MIT
    • agent-skills
    • ai-agents
    • evaluation
    • governance
    • meta-skill

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/yaojingang/yao-meta-skill ~/.claude/skills/yao-meta-skill

    2 596GitHub stars+37 (+1.4 %)
  14. 14

    Skill
    Active

    Skill instalable para Claude Code que mide si tu agente cumple las reglas establecidas. Lee el historial de sesiones de tu disco y devuelve un porcentaje por regla de cumplimiento. Sin instalar nada, sin red, biblioteca estándar.

    PythonLicense: MIT
    • ai-evaluation
    • claude
    • claude-code
    • claude-code-plugin
    • claude-code-skill

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add jleonceo/skill-adherencia-reglas

    1GitHub starsstable
  15. 15

    Skill
    Dormant

    An assembly line for AI software development. 35 skills, 11 agent personas, 29 commands. From raw idea to shipped code.

    ShellLicense: MIT
    • agentic-engineering
    • agents
    • claude-code
    • claude-code-plugin
    • claude-code-skills

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add aneja5/forge-skills

    3GitHub starsstable
  16. 16

    Skill
    Active

    Audit which Claude Code skills you actually use — surface dead installs and hallucinated invocations from your session logs.

    TypeScriptLicense: MIT
    • anthropic
    • audit
    • claude
    • claude-code
    • claude-code-plugin

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add sfrangulov/skill-graveyard

    10GitHub starsstable
  17. 17

    Agent
    Dormant

    A Python proof-of-concept for tracing multi-turn Agent-to-Agent (A2A) conversations as a single unified MLflow trace for LLM observability and evaluation.

    Python
    • a2a
    • agent-to-agent
    • ai-agents
    • conversational-ai
    • llm-evaluation

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  18. 18

    Skill
    Active

    Production-grade AI coding rules for Cursor and Claude Code. 15 rules + 9 doc templates + skills + agents + MCP setup. Drop into any project.

    PythonLicense: MIT
    • ai-agents
    • ai-coding
    • ai-development
    • ai-engineer
    • automation

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/aiagentwithdhruv/ai-dev-stack ~/.claude/skills/ai-dev-stack

    10GitHub starsstable
  19. 19

    Agent
    Active

    Agent 降智检测与自愈公评网络 — an immune system for the AI agent society

    TypeScriptLicense: MIT
    • a2a
    • agent
    • agent-guard
    • ai-sre
    • drift-detection

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  20. 20

    Skill
    Active

    Dashboard for monitoring claude code sessions.

    PythonLicense: Apache-2.0
    • agents
    • agentskills
    • claude-code
    • claude-code-plugin
    • claude-code-skills

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add JayantDevkar/claude-code-karma

    324GitHub stars+3 (+0.93 %)
  21. 21

    Other
    Active

    A Go-native framework for LLM agents, with OpenTelemetry observability built in.

    GoLicense: Apache-2.0
    • a2a
    • agents
    • ai
    • anthropic
    • bedrock

    githubmeasured growthOpen source ↗

    10GitHub starsstable
  22. 22

    Agent
    Active

    The Operating System for Scalable Enterprise AI Agents - Run, orchestrate, and deploy Compliant Enterprise AI Agents at scale across frameworks, without lock-in, rewrites or fragile glue code. Native support for MCP, A2A. Interface with all…

    PythonLicense: Apache-2.0
    • a2a
    • adk
    • ai-agents
    • aws
    • azure

    githubmeasured growthOpen source ↗

    166GitHub stars+25 (+17.7 %)
  23. 23

    MCP
    Dormant

    Open-source Python framework to deploy AI agents via HTTP, A2A, and MCP with built-in observability

    PythonLicense: MIT
    • a2a
    • a2a-client
    • a2a-server
    • agent-evaluation
    • agentic-ai

    githubmeasured growthOpen source ↗

    5GitHub starsstable
  24. 24

    Agent
    Active

    Deterministic crash tests for agent control planes: paired worlds, hard stops, budgets, scopes, approvals, and hash-linked receipts.

    TypeScriptLicense: Apache-2.0
    • a2a
    • agent-evaluation
    • ai-agents
    • benchmark
    • deterministic-replay

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  25. 25

    Agent
    Active

    Generic DeepSeek Harness (dsh) plugins: A2A protocol server, session storage mirror, and Langfuse observability.

    TypeScriptLicense: MIT
    • agent2agent
    • ai-agents
    • cordis
    • deepseek-harness
    • dsh

    githubmeasured growthOpen source ↗

    1GitHub starsstable