LLM observability: tools for developers
Monitoring, traces, evaluation and quality tracking for LLM applications. An entry can appear under several use cases when its topics or description provide several explicit signals.
Use case
Activity
Sort by
121 entries in this view.
Tool ranking
Ranked by known GitHub stars, highest first; archived repositories come after maintained repositories. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.
- 1Active
DeepSeek-Infra
OtherLocal-first Agentic AI Infrastructure Platform with LLM Gateway, Agent DAG Runtime, MCP Tool Hub, A2A Agent Mesh, Local RAG, Tool Sandbox and Observability.
githubmeasured growthOpen source ↗
1GitHub starsstable - 2Active
DriftSentinel
AgentAgent 降智检测与自愈公评网络 — an immune system for the AI agent society
githubmeasured growthOpen source ↗
1GitHub starsstable - 3Active
Skill instalable para Claude Code que mide si tu agente cumple las reglas establecidas. Lee el historial de sesiones de tu disco y devuelve un porcentaje por regla de cumplimiento. Sin instalar nada, sin red, biblioteca estándar.
githubmeasured growthOpen source ↗
Install
/plugin marketplace add jleonceo/skill-adherencia-reglas1GitHub starsstable - 4Active
boundary-bench
AgentDeterministic crash tests for agent control planes: paired worlds, hard stops, budgets, scopes, approvals, and hash-linked receipts.
githubmeasured growthOpen source ↗
1GitHub starsstable - 5Active
pyxen
AgentA lightweight Python library that decouples agentic runtime from applications it builds
githubmeasured growthOpen source ↗
1GitHub starsstable - 6Active
a2a-otel-kit
MCPVendor-neutral OpenTelemetry tracing for A2A agents and MCP services, with W3C context propagation and privacy-safe telemetry.
githubmeasured growthOpen source ↗
1GitHub starsstable - 7Dormant
A safety-first multi-agent mental health companion with real-time distress tracking, triple-layer guardrails, and evidence-based grounding techniques. Built for Kaggle × Google Agents Intensive 2025 Capstone (Agents for Good Track)
githubmeasured growthOpen source ↗
1GitHub starsstable - 8Active
adherencia-reglas
SkillMide si tu agente de IA cumple las reglas que le escribiste. Lee el historial local de Claude Code y devuelve un porcentaje por regla. Sin instalar nada, sin red, biblioteca estándar.
githubmeasured growthOpen source ↗
Install
git clone https://github.com/jleonceo/adherencia-reglas ~/.claude/skills/adherencia-reglas1GitHub starsstable - 9Active
headsup
SkillGlanceable Claude Code and Codex state in your terminal tabs: white=idle, blue=working, orange=waiting. Multi-terminal (iTerm2, WezTerm, AI Power Term), a live status bar with Anthropic usage limits, /sfl and /nil window save-and-restore,…
githubmeasured growthOpen source ↗
Install
git clone https://github.com/wasulajr/headsup ~/.claude/skills/headsup1GitHub starsstable - 10Active
sc-prism-releases
SkillRun many AIs on one board and keep control of all of it. Deterministic code decides who acts — never a model. A privacy floor keeps sensitive work on your machine, your own tests decide what counts as done, and every action lands on a…
githubmeasured growthOpen source ↗
Install
git clone https://github.com/sandhusukhdeep2/sc-prism-releases ~/.claude/skills/sc-prism-releases1GitHub starsstable - 11Dormant
agentops
AgentAgentOps: Multi-agent infrastructure remediation platform. A2A protocol, agent coordination, HITL approval, auto-rollback.
githubmeasured growthOpen source ↗
0GitHub starsstable - 12Active
AI Guardian
MCPGoverned local-LLM observability: model policy, prompt scanner, capture proxy, 21 tools.
mcpmeasured growthOpen source ↗
Install
claude mcp add ai-guardian -- uvx ai-guardian-aiops0GitHub starsstable - 13Active
homestream
Other🔑 HomeStream · 家园·流 — 零成本自托管多Agent协作框架，通往AI世界的那把钥匙 | Zero-cost self-hosted multi-agent framework — The key to AI world
githubmeasured growthOpen source ↗
0GitHub starsstable - 14Active
CodeFlow
Agent面向长程研发任务的 Java Agent Harness，支持 A2A 跨语言协作、可恢复执行、上下文工程与 Eval 驱动开发。
githubmeasured growthOpen source ↗
0GitHub starsstable - 15Dormant
A Python proof-of-concept for tracing multi-turn Agent-to-Agent (A2A) conversations as a single unified MLflow trace for LLM observability and evaluation.
githubmeasured growthOpen source ↗
0GitHub starsstable - 16Active
agentgateway
MCPAgentGateway — independent third-party profile of a public API surface, by API Evangelist. AgentGateway is an open-source, AI-native proxy and gateway for routing, observing, and governing traffic to and from AI agents, LLM providers, and…
githubmeasured growthOpen source ↗
0GitHub starsstable - 17Dormant
An AI-powered multi-agent system that demonstrates clinical triage, OTC medication recommendations, and e-pharmacy integration for respiratory conditions. Built with modular agents that collaborate to provide safe, intelligent healthcare…
githubmeasured growthOpen source ↗
0GitHub starsstable - 18Dormant
OpenTelemetry semantic conventions and instrumentation for agent provenance, derivation lineage, and acceptance criteria evaluation. Fills the Microsoft AI stack observability gap.
githubmeasured growthOpen source ↗
0GitHub starsstable - 19Active
Architecture-first Python scaffold for an auditable multi-agent corporate credit desk using A2A, MCP, deterministic credit policies, model routing, and OpenTelemetry.
githubmeasured growthOpen source ↗
0GitHub starsstable - 20Dormant
agentanvil
AgentContract-based testing for LLM agents: hybrid evaluation, multi-agent + A2A, deterministic replay.
githubmeasured growthOpen source ↗
0GitHub starsstable
Learning resources
Ranked by known GitHub stars, highest first; archived repositories come after maintained repositories. These resources remain available separately and do not take part in the main tool ranking.
- 1ActiveOpen source ↗
Handbook técnico aberto sobre engenharia de IA em produção: ML tradicional, LLMs, RAG, agentes, segurança, observabilidade, FinOps e deployment.
githubResourcemeasured growth1GitHub starsstable