LLM observability: tools for developers

Monitoring, traces, evaluation and quality tracking for LLM applications. An entry can appear under several use cases when its topics or description provide several explicit signals.

Use case

Activity

Sort by

121 entries in this view.

Tool ranking

Ranked by measured growth, normalized across sources. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.

  1. 1

    Skill
    Active

    local-first analytics for AI agent skills

    TypeScript
    • agent-skills
    • analytics
    • bun
    • claude-code
    • cli

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/crafter-station/skill-kit ~/.claude/skills/skill-kit

    77GitHub stars+1 (+1.3 %)
  2. 2

    Agent
    Active

    Generic DeepSeek Harness (dsh) plugins: A2A protocol server, session storage mirror, and Langfuse observability.

    TypeScriptLicense: MIT
    • agent2agent
    • ai-agents
    • cordis
    • deepseek-harness
    • dsh

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  3. 3

    Skill
    Active

    Agent Skill evaluation harness for paired variants, trace artifacts, and runner adapters

    PythonLicense: MIT
    • agent-skills
    • evals
    • python
    • skills

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/adewale/skill-eval-harness ~/.claude/skills/skill-eval-harness

    73GitHub stars+4 (+5.8 %)
  4. 4

    Skill
    Active

    Mide si tu agente de IA cumple las reglas que le escribiste. Lee el historial local de Claude Code y devuelve un porcentaje por regla. Sin instalar nada, sin red, biblioteca estándar.

    PythonLicense: MIT
    • agent-evaluation
    • ai-agents
    • claude-code
    • claude-code-skill
    • claude-code-skills

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/jleonceo/adherencia-reglas ~/.claude/skills/adherencia-reglas

    1GitHub starsstable
  5. 5

    Other
    Active

    AgentX python SDK. Build multi-agent AI workforce. Run evaluation. Trace your agent. Full Observability.

    PythonLicense: Apache-2.0
    • a2a
    • agent
    • agent-monitoring
    • evaluation
    • mcp

    githubmeasured growthOpen source ↗

    68GitHub starsstable
  6. 6

    MCP
    Active

    Vendor-neutral OpenTelemetry tracing for A2A agents and MCP services, with W3C context propagation and privacy-safe telemetry.

    PythonLicense: MIT
    • a2a
    • agent2agent
    • agentic-ai
    • ai-agents
    • clean-architecture

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  7. 7

    Skill
    Active

    Open benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic scoring, pre-registered rubric.

    JavaScriptLicense: MIT
    • aeo
    • agent-skills
    • ai
    • ai-seo
    • benchmark

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/aleclindz/seo-skill-bench ~/.claude/skills/seo-skill-bench

    51GitHub stars-5 (-8.9 %)
  8. 8

    Agent
    Dormant

    An AI-powered multi-agent system that demonstrates clinical triage, OTC medication recommendations, and e-pharmacy integration for respiratory conditions. Built with modular agents that collaborate to provide safe, intelligent healthcare…

    Python
    • agent-to-agent
    • agentic-ai
    • aiagent
    • antigravity-ide
    • cursor-ai

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  9. 9

    MCP
    Active

    8 MCP SMB products — standalone AI servers for SMBs: Guardrails, FinOps, Observability, Router, Trust Score, Memory, ThinkSecure, A2A Lite. 37 tools, TNC credits billing.

    PythonLicense: MIT
    • a2a-protocol
    • ai-governance
    • finops
    • guardrails
    • mcp

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  10. 10

    MCP
    Active

    Open-source, self-hosted control plane for OpenClaw and Hermes AI-agent fleets on Docker/Kubernetes — REST, CLI, and MCP.

    TypeScriptLicense: Apache-2.0
    • agent-management
    • agent-ops
    • agent-orchestration
    • ai-agent-platform
    • ai-agents

    githubmeasured growthOpen source ↗

    50GitHub stars+2 (+4.2 %)
  11. 11

    Skill
    Active

    A skill creator that proves its skills work. Evidence-driven skill creation for Claude Code and Codex: baseline-tested generation, per-skill regression evals, ecosystem doctor, cross-runtime compile, and an opt-in proactive advisor.

    PythonLicense: MIT
    • agent-skills
    • claude-ai
    • claude-code
    • claude-skills
    • codex

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/tripleyak/SkillForge ~/.claude/skills/SkillForge

    884GitHub stars-1 (-0.11 %)
  12. 12

    Skill
    Active

    Skill instalable para Claude Code que mide si tu agente cumple las reglas establecidas. Lee el historial de sesiones de tu disco y devuelve un porcentaje por regla de cumplimiento. Sin instalar nada, sin red, biblioteca estándar.

    PythonLicense: MIT
    • ai-evaluation
    • claude
    • claude-code
    • claude-code-plugin
    • claude-code-skill

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add jleonceo/skill-adherencia-reglas

    1GitHub starsstable
  13. 13

    Skill
    Active

    Real-time execution trace and cost intelligence for Claude Code

    TypeScriptLicense: MIT
    • agentic-workflow
    • ai-productivity
    • anthropic
    • anthropic-claude
    • claude

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add DeibyGS/claudestat

    34GitHub starsstable
  14. 14

    Agent
    Active

    The agent framework where the model never holds the trigger — every consequential action clears your policy first, waits for a human when it matters, and lands on a record you can verify. Build on it, or put it around the agent you already…

    PythonLicense: Apache-2.0
    • a2a
    • agent-framework
    • ai-agents
    • ai-governance
    • ai-safety

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  15. 15

    Skill
    Active

    Make Claude Opus 4.8 behave like Claude Fable 5 — doctrine output style, drift-catching hooks, and an eval loop against golden Fable transcripts. Claude Code plugin.

    ShellLicense: MIT
    • anthropic
    • claude
    • claude-code
    • claude-code-plugin
    • evals

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add rennf93/opus-fable-playbook

    34GitHub stars+1 (+3.0 %)
  16. 16

    Other
    Active

    Local-first Agentic AI Infrastructure Platform with LLM Gateway, Agent DAG Runtime, MCP Tool Hub, A2A Agent Mesh, Local RAG, Tool Sandbox and Observability.

    Python
    • a2a
    • agent-runtime
    • ai-infra
    • deepseek
    • fastapi

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  17. 17

    Skill
    Dormant

    🔍 AI observability skill for Claude Code. Debug LangChain/LangGraph agents by fetching execution traces from LangSmith Studio directly in your terminal.

    License: MIT
    • agent-debugging
    • ai-agents
    • claude-skills
    • claude-skills-creator
    • claude-skills-hub

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/OthmanAdi/langsmith-fetch-skill ~/.claude/skills/langsmith-fetch-skill

    27GitHub starsstable
  18. 18

    MCP
    Active

    AgentGateway — independent third-party profile of a public API surface, by API Evangelist. AgentGateway is an open-source, AI-native proxy and gateway for routing, observing, and governing traffic to and from AI agents, LLM providers, and…

    • agent-to-agent
    • ai-gateway
    • api-gateway
    • apis-json
    • cncf

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  19. 19

    Other
    Dormant

    Official Python SDK for GT8004 — AI agent observability with MCP, A2A, x402 payment tracking. FastAPI, Flask, FastMCP middleware included.

    PythonLicense: MIT
    • a2a
    • ai-agent
    • analytics
    • fastapi
    • flask

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  20. 20

    Agent
    Dormant

    Policy-enforced observability and fail-closed guardrails for MCP/A2A multi-agent systems.

    Rust
    • a2a
    • ai-agents
    • llm-security
    • mcp
    • multi-agent

    githubmeasured growthOpen source ↗

    26GitHub starsstable
  21. 21

    Skill
    Active

    OpenJudge: A Unified Framework for Holistic Evaluation and Quality Rewards

    PythonLicense: Apache-2.0
    • agent
    • agent-skills
    • ai-agent
    • alignment
    • evaluation

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/agentscope-ai/OpenJudge ~/.claude/skills/OpenJudge

    812GitHub stars+7 (+0.87 %)
  22. 22

    Skill
    Active

    Claude Code skills where every entry ships receipts — accuracy-gated benchmarks against baseline and placebo, rejects published

    JavaScriptLicense: MIT
    • agent-skills
    • ai-agents
    • anthropic
    • benchmark
    • claude

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/sjh9714/skill-receipts ~/.claude/skills/skill-receipts

    2GitHub starsstable
  23. 23

    Skill
    Active

    Sentry instrumentation skill for system-behavior tracking

    ShellLicense: MIT
    • agent-skills
    • claude-skills
    • codex-skill
    • instrumentation
    • monitoring

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/tortastudios/sentry-instrumentation ~/.claude/skills/sentry-instrumentation

    24GitHub starsstable
  24. 24

    Other
    Dormant

    Multi-agent customer support system with Google ADK & Gemini 2.5 Flash Lite. Kaggle capstone demonstrating 11+ concepts. Automates 80%+ queries, <10s response time.

    PythonLicense: MIT
    • a2a-protocol
    • agent-evaluation
    • google-adk
    • multi-agent-system
    • observability

    githubmeasured growthOpen source ↗

    2GitHub starsstable
  25. 25

    MCP
    Active

    Deterministic, local-only audit reports for Claude Code AI agent sessions. Rust + SQLite. Zero network calls. MCP server for agents.

    HTMLLicense: MIT
    • agent-monitoring
    • agentic-ai
    • ai-agents
    • ai-engineering
    • audit

    githubmeasured growthOpen source ↗

    22GitHub stars+2 (+10.0 %)