LLM observability: tools for developers

Monitoring, traces, evaluation and quality tracking for LLM applications. An entry can appear under several use cases when its topics or description provide several explicit signals.

Use case

Activity

Sort by

75 entries in this view.

Tool ranking

Ranked by creation date, newest first; undated entries come last. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.

  1. 1

    Other
    Active

    AI-detector auditing toolkit: measures how often a detector flags genuine human writing, how stable its verdict is across seeds, and whether that verdict survives meaning-preserving edits. Detector-in-the-loop measurement harness. Claude…

    PythonLicense: MIT
    • ai-detection
    • ai-detector
    • benchmark
    • claude-code
    • claude-code-skill

    githubmeasured growthOpen source ↗

    18GitHub starsstable
  2. 2

    Skill
    Active

    Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior.

    PythonLicense: Apache-2.0
    • agent-evaluation
    • agent-security
    • agent-skills
    • agentic-ai
    • benchmark

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/NVIDIA/SkillEvaluator ~/.claude/skills/SkillEvaluator

    389GitHub stars+57 (+17.2 %)
  3. 3

    Agent
    Active

    DataBuff is an AI-native APM built on Opentelemetry，with multi-agent troubleshooting out of the box.

    JavaLicense: AGPL-3.0
    • aiops
    • apm
    • application-monitoring
    • devops
    • distributed-tracing

    githubmeasured growthOpen source ↗

    642GitHub stars+32 (+5.2 %)
  4. 4

    MCP
    Active

    Optimize any AI agent’s skills, tools/MCP, and prompts against your own evals.

    PythonLicense: Apache-2.0
    • agent-optimization
    • agent-skills
    • agentic-ai
    • ai-agents
    • claude

    githubmeasured growthOpen source ↗

    47GitHub stars+1 (+2.2 %)
  5. 5

    Skill
    Active

    Agent Skill evaluation harness for paired variants, trace artifacts, and runner adapters

    PythonLicense: MIT
    • agent-skills
    • evals
    • python
    • skills

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/adewale/skill-eval-harness ~/.claude/skills/skill-eval-harness

    73GitHub stars+4 (+5.8 %)
  6. 6

    Agent
    Active

    The agent framework where the model never holds the trigger — every consequential action clears your policy first, waits for a human when it matters, and lands on a record you can verify. Build on it, or put it around the agent you already…

    PythonLicense: Apache-2.0
    • a2a
    • agent-framework
    • ai-agents
    • ai-governance
    • ai-safety

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  7. 7

    Skill
    Active

    Axiom is a curated marketplace of shared plugins for Claude Code and Codex.

    PythonLicense: MIT
    • agent-skills
    • anthropic
    • claude
    • claude-code
    • claude-code-marketplace

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/netopsengineer/axiom ~/.claude/skills/axiom

    5GitHub starsstable
  8. 8

    Other
    Active

    Local-first Agentic AI Infrastructure Platform with LLM Gateway, Agent DAG Runtime, MCP Tool Hub, A2A Agent Mesh, Local RAG, Tool Sandbox and Observability.

    Python
    • a2a
    • agent-runtime
    • ai-infra
    • deepseek
    • fastapi

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  9. 9

    Skill
    Active

    Anchor — the production-grade AGENTS.md template for AI coding agents. 51 sections of battle-tested rules. Keep your agents grounded.

    License: MIT
    • agent-instructions
    • agents-md
    • ai-agent-instructions
    • ai-coding
    • ai-coding-agent

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/peva3/anchor ~/.claude/skills/anchor

    8GitHub starsstable
  10. 10

    Other
    Active

    Open-source observability tool that uses AI agents to self-heal your software

    TypeScriptLicense: Apache-2.0
    • ai
    • llm
    • memory
    • observability
    • opentelemetry

    githubmeasured growthOpen source ↗

    1 404GitHub stars+2 (+0.14 %)
  11. 11

    Skill
    Active

    Glanceable Claude Code and Codex state in your terminal tabs: white=idle, blue=working, orange=waiting. Multi-terminal (iTerm2, WezTerm, AI Power Term), a live status bar with Anthropic usage limits, /sfl and /nil window save-and-restore,…

    ShellLicense: MIT
    • ai-workflow-observability
    • claude-code
    • claude-code-hooks
    • claude-code-skill
    • claude-code-skills

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/wasulajr/headsup ~/.claude/skills/headsup

    1GitHub starsstable
  12. 12

    MCP
    Active

    74 open-source Agent Skills for Claude Code and Codex: AI SEO, AEO and GEO, code review with an A-F ship grade, CI gates, AI evals, design systems, conversion copy, Instagram growth, iOS and Android app shipping, creator rights, and…

    JavaScriptLicense: MIT
    • agent-orchestration
    • agent-skill
    • agent-skills
    • ai-agents
    • ai-coding

    githubmeasured growthOpen source ↗

    129GitHub stars-4 (-3.0 %)
  13. 13

    Other
    Active

    A Go-native framework for LLM agents, with OpenTelemetry observability built in.

    GoLicense: Apache-2.0
    • a2a
    • agents
    • ai
    • anthropic
    • bedrock

    githubmeasured growthOpen source ↗

    10GitHub starsstable
  14. 14

    Skill
    Active

    A test runner for agentskills.io-style AI agent skills

    TypeScriptLicense: MIT
    • agent-evals
    • agent-skills
    • agentskills
    • ai-agents
    • cli

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/darkrishabh/agent-skills-eval ~/.claude/skills/agent-skills-eval

    719GitHub stars+13 (+1.8 %)
  15. 15

    Skill
    Active

    Real-time execution trace and cost intelligence for Claude Code

    TypeScriptLicense: MIT
    • agentic-workflow
    • ai-productivity
    • anthropic
    • anthropic-claude
    • claude

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add DeibyGS/claudestat

    34GitHub stars+1 (+3.0 %)
  16. 16

    MCP
    Active

    Query Spanlens LLM observability from Cursor, Claude Desktop, or Continue via MCP.

    ↓ 32 downloads/weekTypeScriptLicense: MIT

      mcpmeasured growthOpen source ↗

      Install claude mcp add mcp-server -- npx @spanlens/mcp-server

      12GitHub starsstable
    • 17

      Agent
      Active

      MemroOS / memroos: memory OS and governance layer for AI agents, agent workflows, dispatch, proof, and context continuity.

      TypeScript
      • a2a
      • agent-governance
      • agent-memory
      • agent-orchestration
      • agent-workflows

      githubmeasured growthOpen source ↗

      7GitHub starsstable
    • 18

      Skill
      Active

      YAO = Yielding AI Outcomes. A rigorous engineering, evaluation, governance, and portability system for reusable agent skills.

      PythonLicense: MIT
      • agent-skills
      • ai-agents
      • evaluation
      • governance
      • meta-skill

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/yaojingang/yao-meta-skill ~/.claude/skills/yao-meta-skill

      2 577GitHub stars+35 (+1.4 %)
    • 19

      Skill
      Active

      Skills, prompts, and instructions for building AI agents on top of Dynatrace production context

      JavaScriptLicense: Apache-2.0
      • agent-skills
      • ai-agents
      • claude-code
      • devops
      • dql

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/Dynatrace/dynatrace-for-ai ~/.claude/skills/dynatrace-for-ai

      132GitHub stars+4 (+3.1 %)
    • 20

      MCP
      Active

      AgentGateway — independent third-party profile of a public API surface, by API Evangelist. AgentGateway is an open-source, AI-native proxy and gateway for routing, observing, and governing traffic to and from AI agents, LLM providers, and…

      • agent-to-agent
      • ai-gateway
      • api-gateway
      • apis-json
      • cncf

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 21

      Agent
      Active

      Provide clear documentation for AgentStack’s MCP protocol, plugins, and ecosystem API with usage examples and tool references.

      • a2a
      • anthropic
      • claude-code
      • crewai
      • deep-learning

      githubmeasured growthOpen source ↗

      0GitHub starsstable
    • 22

      MCP
      Active

      Open-source, self-hosted control plane for OpenClaw and Hermes AI-agent fleets on Docker/Kubernetes — REST, CLI, and MCP.

      TypeScriptLicense: Apache-2.0
      • agent-management
      • agent-ops
      • agent-orchestration
      • ai-agent-platform
      • ai-agents

      githubmeasured growthOpen source ↗

      50GitHub stars+2 (+4.2 %)
    • 23

      Skill
      Active

      Vendor-neutral OpenTelemetry skills for AI coding agents, grounded in upstream sources

      ShellLicense: Apache-2.0
      • agent-skills
      • ai
      • observability
      • opentelemetry

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/ollygarden/opentelemetry-agent-skills ~/.claude/skills/opentelemetry-agent-skills

      97GitHub stars+4 (+4.3 %)
    • 24

      Skill
      Active

      Agent skills for Arize — datasets, experiments, and traces via the ax CLI

      PythonLicense: MIT
      • agent-skills
      • ai-agents
      • ai-observability
      • arize
      • claude-code

      githubmeasured growthOpen source ↗

      Install git clone https://github.com/Arize-ai/arize-skills ~/.claude/skills/arize-skills

      47GitHub starsstable

    Learning resources

    Ranked by creation date, newest first; undated entries come last. These resources remain available separately and do not take part in the main tool ranking.

    1. 1

      awesome-agentic-devops

      MCP
      ActiveOpen source ↗

      Curated + scored map of official MCP servers and agents for DevOps, Cloud, SRE, and Platform Engineering — every entry rated on production access, approval gates, and audit evidence.

      PythonLicense: MIT
      • agent-skills
      • agentic-ai
      • agentic-devops
      • ai-agents
      • awesome
      githubResourcemeasured growth
      77GitHub starsstable