LLM observability: tools for developers

Monitoring, traces, evaluation and quality tracking for LLM applications. An entry can appear under several use cases when its topics or description provide several explicit signals.

Use case

Activity

Sort by

117 entries in this view.

Tool ranking

Ranked by measured growth, normalized across sources. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.

  1. 1

    Skill
    Dormant

    An assembly line for AI software development. 35 skills, 11 agent personas, 29 commands. From raw idea to shipped code.

    ShellLicense: MIT
    • agentic-engineering
    • agents
    • claude-code
    • claude-code-plugin
    • claude-code-skills

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add aneja5/forge-skills

    3GitHub starsstable
  2. 2

    Skill
    Active

    Audit which Claude Code skills you actually use — surface dead installs and hallucinated invocations from your session logs.

    TypeScriptLicense: MIT
    • anthropic
    • audit
    • claude
    • claude-code
    • claude-code-plugin

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add sfrangulov/skill-graveyard

    10GitHub starsstable
  3. 3

    Agent
    Dormant

    A Python proof-of-concept for tracing multi-turn Agent-to-Agent (A2A) conversations as a single unified MLflow trace for LLM observability and evaluation.

    Python
    • a2a
    • agent-to-agent
    • ai-agents
    • conversational-ai
    • llm-evaluation

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  4. 4

    Skill
    Active

    Production-grade AI coding rules for Cursor and Claude Code. 15 rules + 9 doc templates + skills + agents + MCP setup. Drop into any project.

    PythonLicense: MIT
    • ai-agents
    • ai-coding
    • ai-development
    • ai-engineer
    • automation

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/aiagentwithdhruv/ai-dev-stack ~/.claude/skills/ai-dev-stack

    10GitHub starsstable
  5. 5

    Agent
    Active

    Agent 降智检测与自愈公评网络 — an immune system for the AI agent society

    TypeScriptLicense: MIT
    • a2a
    • agent
    • agent-guard
    • ai-sre
    • drift-detection

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  6. 6

    Other
    Active

    A Go-native framework for LLM agents, with OpenTelemetry observability built in.

    GoLicense: Apache-2.0
    • a2a
    • agents
    • ai
    • anthropic
    • bedrock

    githubmeasured growthOpen source ↗

    10GitHub starsstable
  7. 7

    MCP
    Dormant

    Open-source Python framework to deploy AI agents via HTTP, A2A, and MCP with built-in observability

    PythonLicense: MIT
    • a2a
    • a2a-client
    • a2a-server
    • agent-evaluation
    • agentic-ai

    githubmeasured growthOpen source ↗

    5GitHub starsstable
  8. 8

    Agent
    Active

    Deterministic crash tests for agent control planes: paired worlds, hard stops, budgets, scopes, approvals, and hash-linked receipts.

    TypeScriptLicense: Apache-2.0
    • a2a
    • agent-evaluation
    • ai-agents
    • benchmark
    • deterministic-replay

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  9. 9

    Agent
    Active

    Generic DeepSeek Harness (dsh) plugins: A2A protocol server, session storage mirror, and Langfuse observability.

    TypeScriptLicense: MIT
    • agent2agent
    • ai-agents
    • cordis
    • deepseek-harness
    • dsh

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  10. 10

    Skill
    Active

    Multi-agent quality gate skill for Claude Code that researches, reviews, tests and challenges AI-generated work before the final answer.

    • agentic-coding
    • agentic-workflow
    • ai-agent-skills
    • ai-code-review
    • ai-workflows

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/ma-nucho-pro/supervisor-skill-claude ~/.claude/skills/supervisor-skill-claude

    2GitHub starsstable
  11. 11

    Agent
    Active

    A local A2A event and continuity service for agent applications, with structured history, provenance-safe transcripts, literal search, and natural-language queries.

    PythonLicense: BSD-2-Clause
    • a2a
    • agent-memory
    • ai-agents
    • event-sourcing
    • fastapi

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  12. 12

    Other
    Active

    Local-first Agentic AI Infrastructure Platform with LLM Gateway, Agent DAG Runtime, MCP Tool Hub, A2A Agent Mesh, Local RAG, Tool Sandbox and Observability.

    Python
    • a2a
    • agent-runtime
    • ai-infra
    • deepseek
    • fastapi

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  13. 13

    Agent
    Active

    Multi-agent SRE on-call investigator that auto-triages Slack/Discord infrastructure alerts via AWS Bedrock AgentCore, fanning out to specialized agents (CloudWatch, EKS, Slack/Discord scanners) for parallel investigation.

    Python
    • a2a-protocol
    • anthropic
    • aws-bedrock
    • aws-lambda
    • bedrock-agentcore

    githubmeasured growthOpen source ↗

    3GitHub starsstable
  14. 14

    MCP
    Active

    Query Spanlens LLM observability from Cursor, Claude Desktop, or Continue via MCP.

    ↓ 32 downloads/weekTypeScriptLicense: MIT

      mcpmeasured growthOpen source ↗

      Install claude mcp add mcp-server -- npx @spanlens/mcp-server

      12GitHub starsstable
    • 15

      MCP
      Active

      Governed local-LLM observability: model policy, prompt scanner, capture proxy, 21 tools.

      PythonLicense: MIT

        mcpmeasured growthOpen source ↗

        Install claude mcp add ai-guardian -- uvx ai-guardian-aiops

        0GitHub starsstable
      • 16

        Agent
        Active

        A lightweight Python library that decouples agentic runtime from applications it builds

        PythonLicense: MIT
        • a2a
        • agent-runtime
        • agents
        • observability
        • portability

        githubmeasured growthOpen source ↗

        1GitHub starsstable
      • 17

        Agent
        Active

        Amazon Bedrock AgentCore enterprise platform accelerator with AWS CDK, Terraform organization guardrails, MCP/A2A agents, security, memory, and observability.

        PythonLicense: Apache-2.0
        • a2a-protocol
        • agentcore
        • agentcore-agent
        • agentcore-gateway
        • agentcore-identity

        githubmeasured growthOpen source ↗

        6GitHub stars+1 (+20.0 %)
      • 18

        Skill
        Active

        Claude Code plugin marketplace — agentic-engineering (spec-driven shape→decide→execute→measure→eval with adversarial review) + github-keeper (audit/elevate READMEs and make a repo open-source-ready).

        PythonLicense: MIT
        • adr
        • agentic-engineering
        • ai-agents
        • anthropic
        • claude-code

        githubmeasured growthOpen source ↗

        Install /plugin marketplace add GiustoPiedimonte/agentic-engineering-marketplace

        13GitHub starsstable
      • 19

        Skill
        Active

        Measure prompt and skill improvements with blind A/B comparison.

        PythonLicense: MIT
        • agent-skills
        • ai-tools
        • claude-code
        • claude-code-plugin
        • claude-code-skill

        githubmeasured growthOpen source ↗

        Install /plugin marketplace add shinpr/rashomon

        18GitHub starsstable
      • 20

        Other
        Active

        Terminal-first validation harness for reviewing PRDs, RFCs, strategy docs, and experiment plans.

        TypeScript
        • ai-agents
        • claude-code
        • claude-code-skills
        • cli
        • evals

        githubmeasured growthOpen source ↗

        1GitHub starsstable
      • 21

        Agent
        Dormant

        The reliability and interoperability layer for AI agents. A2A-native. MCP-bridged.

        PythonLicense: Apache-2.0
        • a2a
        • agent-framework
        • agents
        • ai
        • ai-safety

        githubmeasured growthOpen source ↗

        1GitHub starsstable
      • 22

        MCP
        Active

        A typed, replayable, budget-aware programming language for LLM agents. Compiles to Python & TypeScript.

        Shell
        • a2a
        • agent-framework
        • agentic-ai
        • ai
        • ai-agents

        githubmeasured growthOpen source ↗

        2GitHub starsstable
      • 23

        Other
        Dormant

        A Claude Code skill that adds a rubric-based eval layer to any agent project. Framework-agnostic — generates rubric, test cases, judge prompt, and harness. Returns a weighted score plus a judge-leniency signal.

        HTML
        • agent-evaluation
        • anthropic
        • claude-code
        • claude-code-skill
        • evals

        githubmeasured growthOpen source ↗

        13GitHub starsstable
      • 24

        Agent
        Active

        Local-first control plane for durable, governed agent teams: DAG orchestration, approvals, memory, signed skills, trace contracts, and realtime.

        PythonLicense: Apache-2.0
        • a2a-protocol
        • agentic-ai
        • ai-agents
        • fastapi
        • llmops

        githubmeasured growthOpen source ↗

        0GitHub starsstable
      • 25

        Agent
        Active

        MemroOS / memroos: memory OS and governance layer for AI agents, agent workflows, dispatch, proof, and context continuity.

        TypeScript
        • a2a
        • agent-governance
        • agent-memory
        • agent-orchestration
        • agent-workflows

        githubmeasured growthOpen source ↗

        7GitHub starsstable