LLM observability: tools for developers

Monitoring, traces, evaluation and quality tracking for LLM applications. An entry can appear under several use cases when its topics or description provide several explicit signals.

Use case

Activity

Sort by

118 entries in this view.

Tool ranking

Ranked by creation date, newest first; undated entries come last. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.

  1. 1

    Skill
    Dormant

    An assembly line for AI software development. 35 skills, 11 agent personas, 29 commands. From raw idea to shipped code.

    ShellLicense: MIT
    • agentic-engineering
    • agents
    • claude-code
    • claude-code-plugin
    • claude-code-skills

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add aneja5/forge-skills

    3GitHub starsstable
  2. 2

    MCP
    Active

    Open-source, self-hosted control plane for OpenClaw and Hermes AI-agent fleets on Docker/Kubernetes — REST, CLI, and MCP.

    TypeScriptLicense: Apache-2.0
    • agent-management
    • agent-ops
    • agent-orchestration
    • ai-agent-platform
    • ai-agents

    githubmeasured growthOpen source ↗

    50GitHub stars+1 (+2.0 %)
  3. 3

    Skill
    Dormant

    A self-improving harness router for Claude Code.

    PythonLicense: MIT
    • a-b-testing
    • ai-agent
    • ai-orchestration
    • ai-routing
    • ai-workflow

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add SeongwoongCho/adaptive-harness

    8GitHub starsstable
  4. 4

    Agent
    Dormant

    OpenTelemetry semantic conventions and instrumentation for agent provenance, derivation lineage, and acceptance criteria evaluation. Fills the Microsoft AI stack observability gap.

    Python
    • a2a-protocol
    • agentic-ai
    • ai-agents
    • observability
    • opentelemetry

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  5. 5

    Skill
    Active

    Production-grade AI coding rules for Cursor and Claude Code. 15 rules + 9 doc templates + skills + agents + MCP setup. Drop into any project.

    PythonLicense: MIT
    • ai-agents
    • ai-coding
    • ai-development
    • ai-engineer
    • automation

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/aiagentwithdhruv/ai-dev-stack ~/.claude/skills/ai-dev-stack

    10GitHub starsstable
  6. 6

    Skill
    Active

    Vendor-neutral OpenTelemetry skills for AI coding agents, grounded in upstream sources

    ShellLicense: Apache-2.0
    • agent-skills
    • ai
    • observability
    • opentelemetry

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/ollygarden/opentelemetry-agent-skills ~/.claude/skills/opentelemetry-agent-skills

    97GitHub stars+1 (+1.0 %)
  7. 7

    Skill
    Active

    Agent skills for Arize — datasets, experiments, and traces via the ax CLI

    PythonLicense: MIT
    • agent-skills
    • ai-agents
    • ai-observability
    • arize
    • claude-code

    githubestimated momentumOpen source ↗

    Install git clone https://github.com/Arize-ai/arize-skills ~/.claude/skills/arize-skills

    47GitHub stars—
  8. 8

    MCP
    Active

    AURA is a production-tested SRE agent platform you can deploy in minutes. AURA handles the guardrails, APIs, state management, streaming, and failure handling required to put AI to work safely on production infrastructure.

    RustLicense: Apache-2.0
    • a2a
    • ai-agent-framework
    • ai-agents
    • ai-sre
    • aiops

    githubmeasured growthOpen source ↗

    339GitHub stars+81 (+31.4 %)
  9. 9

    MCP
    Dormant

    Open-source Python framework to deploy AI agents via HTTP, A2A, and MCP with built-in observability

    PythonLicense: MIT
    • a2a
    • a2a-client
    • a2a-server
    • agent-evaluation
    • agentic-ai

    githubmeasured growthOpen source ↗

    5GitHub starsstable
  10. 10

    Agent
    Dormant

    AgentOps: Multi-agent infrastructure remediation platform. A2A protocol, agent coordination, HITL approval, auto-rollback.

    PythonLicense: MIT
    • a2a-protocol
    • ai
    • automation
    • infrastructure
    • llm

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  11. 11

    Skill
    Active

    local-first analytics for AI agent skills

    TypeScript
    • agent-skills
    • analytics
    • bun
    • claude-code
    • cli

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/crafter-station/skill-kit ~/.claude/skills/skill-kit

    77GitHub stars+1 (+1.3 %)
  12. 12

    MCP
    Active

    Log Claude Code sessions to Opik, the open-source LLM observability and evaluation platform, built by Comet. Tracing, evaluation, and skills for observable AI applications.

    GoLicense: Apache-2.0
    • ai-agent
    • claude-ai
    • claude-code-plugin
    • llm-observation
    • mcp-server

    githubmeasured growthOpen source ↗

    22GitHub stars+1 (+4.8 %)
  13. 13

    Agent
    Dormant

    Policy-enforced observability and fail-closed guardrails for MCP/A2A multi-agent systems.

    Rust
    • a2a
    • ai-agents
    • llm-security
    • mcp
    • multi-agent

    githubmeasured growthOpen source ↗

    26GitHub starsstable
  14. 14

    Agent
    Dormant

    A Python proof-of-concept for tracing multi-turn Agent-to-Agent (A2A) conversations as a single unified MLflow trace for LLM observability and evaluation.

    Python
    • a2a
    • agent-to-agent
    • ai-agents
    • conversational-ai
    • llm-evaluation

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  15. 15

    Skill
    Active

    Measure prompt and skill improvements with blind A/B comparison.

    PythonLicense: MIT
    • agent-skills
    • ai-tools
    • claude-code
    • claude-code-plugin
    • claude-code-skill

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add shinpr/rashomon

    18GitHub starsstable
  16. 16

    Skill
    Active

    Dashboard for monitoring claude code sessions.

    PythonLicense: Apache-2.0
    • agents
    • agentskills
    • claude-code
    • claude-code-plugin
    • claude-code-skills

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add JayantDevkar/claude-code-karma

    324GitHub stars+3 (+0.93 %)
  17. 17

    Skill
    Active

    A skill creator that proves its skills work. Evidence-driven skill creation for Claude Code and Codex: baseline-tested generation, per-skill regression evals, ecosystem doctor, cross-runtime compile, and an opt-in proactive advisor.

    PythonLicense: MIT
    • agent-skills
    • claude-ai
    • claude-code
    • claude-skills
    • codex

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/tripleyak/SkillForge ~/.claude/skills/SkillForge

    885GitHub starsstable
  18. 18

    Skill
    Dormant

    🔍 AI observability skill for Claude Code. Debug LangChain/LangGraph agents by fetching execution traces from LangSmith Studio directly in your terminal.

    License: MIT
    • agent-debugging
    • ai-agents
    • claude-skills
    • claude-skills-creator
    • claude-skills-hub

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/OthmanAdi/langsmith-fetch-skill ~/.claude/skills/langsmith-fetch-skill

    27GitHub starsstable
  19. 19

    MCP
    Active

    Self improving agents through iterations

    PythonLicense: Apache-2.0
    • agent-learning
    • agent-memory
    • agentic-workflows
    • ai-agents
    • claude-code

    githubmeasured growthOpen source ↗

    104GitHub starsstable
  20. 20

    Other
    Dormant

    Multi-agent customer support system with Google ADK & Gemini 2.5 Flash Lite. Kaggle capstone demonstrating 11+ concepts. Automates 80%+ queries, <10s response time.

    PythonLicense: MIT
    • a2a-protocol
    • agent-evaluation
    • google-adk
    • multi-agent-system
    • observability

    githubmeasured growthOpen source ↗

    2GitHub starsstable
  21. 21

    Skill
    Active

    Build tested agent skills and govern their lifecycle through a user-defined marketplace: evidence, discovery, updates, rollback, quarantine, and 17-platform distribution.

    PythonLicense: MIT
    • agent-skills
    • ai-agents
    • claude-code
    • codex
    • cursor

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/FrancyJGLisboa/agent-skills-platform ~/.claude/skills/agent-skills-platform

    2 376GitHub starsstable
  22. 22

    Skill
    Active

    Build tested agent skills and govern their lifecycle through a user-defined marketplace: evidence, discovery, updates, rollback, quarantine, and 17-platform distribution.

    PythonLicense: MIT
    • agent-skills
    • ai-agents
    • claude-code
    • codex
    • cursor

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/FrancyJGLisboa/agent-skill-creator ~/.claude/skills/agent-skill-creator

    2 367GitHub stars+15 (+0.64 %)
  23. 23

    Agent
    Active

    A2A green-agent orchestrator for evaluating agents on the AppWorld benchmark, built on the AgentBeats SDK

    Python
    • a2a-protocol
    • agentbeats
    • appworld
    • evaluation
    • llm-agents

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  24. 24

    Other
    Active

    A command-line tool to scaffold and manage enterprise-ready AI Agents powered by the A2A (Agent-to-Agent) protocol

    GoLicense: Apache-2.0
    • a2a
    • agents
    • ai
    • containerized
    • docker

    githubmeasured growthOpen source ↗

    14GitHub stars+1 (+7.7 %)
  25. 25

    Agent
    Active

    The Operating System for Scalable Enterprise AI Agents - Run, orchestrate, and deploy Compliant Enterprise AI Agents at scale across frameworks, without lock-in, rewrites or fragile glue code. Native support for MCP, A2A. Interface with all…

    PythonLicense: Apache-2.0
    • a2a
    • adk
    • ai-agents
    • aws
    • azure

    githubmeasured growthOpen source ↗

    166GitHub stars+25 (+17.7 %)