LLM observability: tools for developers

Monitoring, traces, evaluation and quality tracking for LLM applications. An entry can appear under several use cases when its topics or description provide several explicit signals.

Use case

Activity

Sort by

122 entries in this view.

Tool ranking

Ranked by measured growth, normalized across sources. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.

  1. 1

    MCP
    Active

    Governed local-LLM observability: model policy, prompt scanner, capture proxy, 21 tools.

    PythonLicense: MIT

      mcpmeasured growthOpen source ↗

      Install claude mcp add ai-guardian -- uvx ai-guardian-aiops

      0GitHub starsstable
    • 2

      Agent
      Active

      Local-first control plane for durable, governed agent teams: DAG orchestration, approvals, memory, signed skills, trace contracts, and realtime.

      PythonLicense: Apache-2.0
      • a2a-protocol
      • agentic-ai
      • ai-agents
      • fastapi
      • llmops

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 3

      Agent
      Active

      A local A2A event and continuity service for agent applications, with structured history, provenance-safe transcripts, literal search, and natural-language queries.

      PythonLicense: BSD-2-Clause
      • a2a
      • agent-memory
      • ai-agents
      • event-sourcing
      • fastapi

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 4

      Agent
      Active

      Agent 降智检测与自愈公评网络 — an immune system for the AI agent society

      TypeScriptLicense: MIT
      • a2a
      • agent
      • agent-guard
      • ai-sre
      • drift-detection

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 5

      Agent
      Dormant

      The reliability and interoperability layer for AI agents. A2A-native. MCP-bridged.

      PythonLicense: Apache-2.0
      • a2a
      • agent-framework
      • agents
      • ai
      • ai-safety

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 6

      Agent
      Active

      A lightweight Python library that decouples agentic runtime from applications it builds

      PythonLicense: MIT
      • a2a
      • agent-runtime
      • agents
      • observability
      • portability

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 7

      Other
      Active

      Terminal-first validation harness for reviewing PRDs, RFCs, strategy docs, and experiment plans.

      TypeScript
      • ai-agents
      • claude-code
      • claude-code-skills
      • cli
      • evals

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 8

      Agent
      Dormant

      A safety-first multi-agent mental health companion with real-time distress tracking, triple-layer guardrails, and evidence-based grounding techniques. Built for Kaggle × Google Agents Intensive 2025 Capstone (Agents for Good Track)

      Python
      • a2a-protocol
      • agent-evaluation
      • agents-for-good
      • ai-agents
      • crisis-support

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 9

      Agent
      Active

      Deterministic crash tests for agent control planes: paired worlds, hard stops, budgets, scopes, approvals, and hash-linked receipts.

      TypeScriptLicense: Apache-2.0
      • a2a
      • agent-evaluation
      • ai-agents
      • benchmark
      • deterministic-replay

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 10

      Skill
      Active

      Glanceable Claude Code and Codex state in your terminal tabs: white=idle, blue=working, orange=waiting. Multi-terminal (iTerm2, WezTerm, AI Power Term), a live status bar with Anthropic usage limits, /sfl and /nil window save-and-restore,…

      ShellLicense: MIT
      • ai-workflow-observability
      • claude-code
      • claude-code-hooks
      • claude-code-skill
      • claude-code-skills

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/wasulajr/headsup ~/.claude/skills/headsup

      1GitHub starsstable
    • 11

      Agent
      Active

      Generic DeepSeek Harness (dsh) plugins: A2A protocol server, session storage mirror, and Langfuse observability.

      TypeScriptLicense: MIT
      • agent2agent
      • ai-agents
      • cordis
      • deepseek-harness
      • dsh

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 12

      Skill
      Active

      Mide si tu agente de IA cumple las reglas que le escribiste. Lee el historial local de Claude Code y devuelve un porcentaje por regla. Sin instalar nada, sin red, biblioteca estándar.

      PythonLicense: MIT
      • agent-evaluation
      • ai-agents
      • claude-code
      • claude-code-skill
      • claude-code-skills

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/jleonceo/adherencia-reglas ~/.claude/skills/adherencia-reglas

      1GitHub starsstable
    • 13

      MCP
      Active

      Vendor-neutral OpenTelemetry tracing for A2A agents and MCP services, with W3C context propagation and privacy-safe telemetry.

      PythonLicense: MIT
      • a2a
      • agent2agent
      • agentic-ai
      • ai-agents
      • clean-architecture

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 14

      Agent
      Dormant

      An AI-powered multi-agent system that demonstrates clinical triage, OTC medication recommendations, and e-pharmacy integration for respiratory conditions. Built with modular agents that collaborate to provide safe, intelligent healthcare…

      Python
      • agent-to-agent
      • agentic-ai
      • aiagent
      • antigravity-ide
      • cursor-ai

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 15

      MCP
      Active

      8 MCP SMB products — standalone AI servers for SMBs: Guardrails, FinOps, Observability, Router, Trust Score, Memory, ThinkSecure, A2A Lite. 37 tools, TNC credits billing.

      PythonLicense: MIT
      • a2a-protocol
      • ai-governance
      • finops
      • guardrails
      • mcp

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 16

      MCP
      Active

      Open-source, self-hosted control plane for OpenClaw and Hermes AI-agent fleets on Docker/Kubernetes — REST, CLI, and MCP.

      TypeScriptLicense: Apache-2.0
      • agent-management
      • agent-ops
      • agent-orchestration
      • ai-agent-platform
      • ai-agents

      githubmeasured growthOpen source ↗

      50GitHub stars+2 (+4.2 %)
    • 17

      Skill
      Active

      Skill instalable para Claude Code que mide si tu agente cumple las reglas establecidas. Lee el historial de sesiones de tu disco y devuelve un porcentaje por regla de cumplimiento. Sin instalar nada, sin red, biblioteca estándar.

      PythonLicense: MIT
      • ai-evaluation
      • claude
      • claude-code
      • claude-code-plugin
      • claude-code-skill

      githubmeasured growthOpen source ↗

      Install /plugin marketplace add jleonceo/skill-adherencia-reglas

      1GitHub starsstable
    • 18

      Skill
      Active

      Real-time execution trace and cost intelligence for Claude Code

      TypeScriptLicense: MIT
      • agentic-workflow
      • ai-productivity
      • anthropic
      • anthropic-claude
      • claude

      githubmeasured growthOpen source ↗

      Install /plugin marketplace add DeibyGS/claudestat

      34GitHub starsstable
    • 19

      Agent
      Active

      The agent framework where the model never holds the trigger — every consequential action clears your policy first, waits for a human when it matters, and lands on a record you can verify. Build on it, or put it around the agent you already…

      PythonLicense: Apache-2.0
      • a2a
      • agent-framework
      • ai-agents
      • ai-governance
      • ai-safety

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 20

      Skill
      Active

      Make Claude Opus 4.8 behave like Claude Fable 5 — doctrine output style, drift-catching hooks, and an eval loop against golden Fable transcripts. Claude Code plugin.

      ShellLicense: MIT
      • anthropic
      • claude
      • claude-code
      • claude-code-plugin
      • evals

      githubmeasured growthOpen source ↗

      Install /plugin marketplace add rennf93/opus-fable-playbook

      34GitHub stars+1 (+3.0 %)
    • 21

      Other
      Active

      Local-first Agentic AI Infrastructure Platform with LLM Gateway, Agent DAG Runtime, MCP Tool Hub, A2A Agent Mesh, Local RAG, Tool Sandbox and Observability.

      Python
      • a2a
      • agent-runtime
      • ai-infra
      • deepseek
      • fastapi

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 22

      Skill
      Dormant

      🔍 AI observability skill for Claude Code. Debug LangChain/LangGraph agents by fetching execution traces from LangSmith Studio directly in your terminal.

      License: MIT
      • agent-debugging
      • ai-agents
      • claude-skills
      • claude-skills-creator
      • claude-skills-hub

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/OthmanAdi/langsmith-fetch-skill ~/.claude/skills/langsmith-fetch-skill

      27GitHub starsstable
    • 23

      MCP
      Active

      AgentGateway — independent third-party profile of a public API surface, by API Evangelist. AgentGateway is an open-source, AI-native proxy and gateway for routing, observing, and governing traffic to and from AI agents, LLM providers, and…

      • agent-to-agent
      • ai-gateway
      • api-gateway
      • apis-json
      • cncf

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 24

      Other
      Dormant

      Official Python SDK for GT8004 — AI agent observability with MCP, A2A, x402 payment tracking. FastAPI, Flask, FastMCP middleware included.

      PythonLicense: MIT
      • a2a
      • ai-agent
      • analytics
      • fastapi
      • flask

      githubmeasured growthOpen source ↗

      1GitHub starsstable
    • 25

      Agent
      Dormant

      Policy-enforced observability and fail-closed guardrails for MCP/A2A multi-agent systems.

      Rust
      • a2a
      • ai-agents
      • llm-security
      • mcp
      • multi-agent

      githubmeasured growthOpen source ↗

      26GitHub starsstable