LLM observability: tools for developers

Monitoring, traces, evaluation and quality tracking for LLM applications. An entry can appear under several use cases when its topics or description provide several explicit signals.

Use case

Activity

Sort by

74 entries in this view.

Tool ranking

Mixed ranking: measured growth takes priority; entries without two snapshots are estimated. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.

  1. 1

    Skill
    Active

    Measure prompt and skill improvements with blind A/B comparison.

    PythonLicense: MIT
    • agent-skills
    • ai-tools
    • claude-code
    • claude-code-plugin
    • claude-code-skill

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add shinpr/rashomon

    18GitHub starsstable
  2. 2

    MCP
    Active

    A typed, replayable, budget-aware programming language for LLM agents. Compiles to Python & TypeScript.

    Shell
    • a2a
    • agent-framework
    • agentic-ai
    • ai
    • ai-agents

    githubmeasured growthOpen source ↗

    2GitHub starsstable
  3. 3

    Skill
    Active

    Research-backed, eval-driven skills for AI agents

    PythonLicense: Apache-2.0
    • agent-skills
    • ai-agents
    • codex
    • design
    • evals

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/ZSeven-W/craft-skills ~/.claude/skills/craft-skills

    145GitHub stars+9 (+6.6 %)
  4. 4

    Skill
    Active

    🪢 Langfuse documentation -- Langfuse is the open source LLM Engineering Platform. Observability, evals, prompt management, playground and metrics to debug and improve LLM apps

    MDXLicense: MIT
    • agent-skills
    • agents
    • genai
    • hacktoberfest
    • langfuse

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/langfuse/langfuse-docs ~/.claude/skills/langfuse-docs

    240GitHub stars+4 (+1.7 %)
  5. 5

    Agent
    Active

    Local-first control plane for durable, governed agent teams: DAG orchestration, approvals, memory, signed skills, trace contracts, and realtime.

    PythonLicense: Apache-2.0
    • a2a-protocol
    • agentic-ai
    • ai-agents
    • fastapi
    • llmops

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  6. 6

    Skill
    Active

    Skills, prompts, and instructions for building AI agents on top of Dynatrace production context

    JavaScriptLicense: Apache-2.0
    • agent-skills
    • ai-agents
    • claude-code
    • devops
    • dql

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/Dynatrace/dynatrace-for-ai ~/.claude/skills/dynatrace-for-ai

    135GitHub stars+4 (+3.1 %)
  7. 7

    Agent
    Active

    MemroOS / memroos: memory OS and governance layer for AI agents, agent workflows, dispatch, proof, and context continuity.

    TypeScript
    • a2a
    • agent-governance
    • agent-memory
    • agent-orchestration
    • agent-workflows

    githubmeasured growthOpen source ↗

    7GitHub starsstable
  8. 8

    Other
    Active

    A command-line tool to scaffold and manage enterprise-ready AI Agents powered by the A2A (Agent-to-Agent) protocol

    GoLicense: Apache-2.0
    • a2a
    • agents
    • ai
    • containerized
    • docker

    githubmeasured growthOpen source ↗

    14GitHub stars+1 (+7.7 %)
  9. 9

    Skill
    Active

    Open-source infrastructure that turns scattered SKILL.md files into curated, retrieval-ready agent-skill corpora—with retrieval and evaluation tooling included.

    PythonLicense: Apache-2.0
    • agent-memory
    • agent-skills
    • ai-agents
    • benchmark
    • dataset

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/EverMind-AI/SkillCorpus ~/.claude/skills/SkillCorpus

    525GitHub stars+199 (+61.0 %)
  10. 10

    MCP
    Active

    74 open-source Agent Skills for Claude Code and Codex: AI SEO, AEO and GEO, code review with an A-F ship grade, CI gates, AI evals, design systems, conversion copy, Instagram growth, iOS and Android app shipping, creator rights, and…

    JavaScriptLicense: MIT
    • agent-orchestration
    • agent-skill
    • agent-skills
    • ai-agents
    • ai-coding

    githubmeasured growthOpen source ↗

    129GitHub stars-4 (-3.0 %)
  11. 11

    Skill
    Active

    Make Claude write clearly, for everyone.

    PythonLicense: 0BSD
    • claude-code
    • claude-code-plugin
    • controlled-language
    • llm-evaluation
    • output-style

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add stefanobaghino/simple-output-styles

    16GitHub starsstable
  12. 12

    Skill
    Active

    Marketing Mix Model expert agent plugin for Claude Code - pymc-marketing v0.18.2+

    Python
    • claude-code-agent
    • claude-code-plugin
    • claude-code-skill
    • data-analysis
    • data-analytics

    githubmeasured growthOpen source ↗

    Install /plugin marketplace add Yakoub-ai/agent-mmm

    4GitHub starsstable
  13. 13

    Agent
    Active

    The agent framework where the model never holds the trigger — every consequential action clears your policy first, waits for a human when it matters, and lands on a record you can verify. Build on it, or put it around the agent you already…

    PythonLicense: Apache-2.0
    • a2a
    • agent-framework
    • ai-agents
    • ai-governance
    • ai-safety

    githubmeasured growthOpen source ↗

    1GitHub starsstable
  14. 14

    Skill
    Active

    🐙 ADLC Team Skills — Agentic SDLC for Engineering Teams

    ShellLicense: MIT
    • adr
    • agent-skills
    • agentic-sdlc
    • ai-coding
    • developer-tools

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/tikalk/adlc-team-skills ~/.claude/skills/adlc-team-skills

    132GitHub stars+1 (+0.76 %)
  15. 15

    MCP
    Active

    A Model Context Protocol (MCP) server for Langfuse, enabling AI agents to query Langfuse trace data for enhanced debugging and observability

    PythonLicense: MIT
    • agent-skills
    • ai-agents
    • claude-code
    • codex
    • developer-tools

    githubmeasured growthOpen source ↗

    105GitHub starsstable
  16. 16

    MCP
    Active

    Architecture-first Python scaffold for an auditable multi-agent corporate credit desk using A2A, MCP, deterministic credit policies, model routing, and OpenTelemetry.

    Python
    • a2a-protocol
    • agentic-ai
    • auditability
    • clean-architecture
    • credit-risk

    githubmeasured growthOpen source ↗

    0GitHub starsstable
  17. 17

    Skill
    Active

    Mini Program Engineering Skill Suite is an Agent Skill suite for evidence‑first mini‑program development. It helps agents bring WeChat or other mini‑program projects from vague intent to reliable engineering work: project intake, product…

    PythonLicense: MIT
    • agent-skills
    • ai-agents
    • alipay
    • ci-cd
    • claude-code

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/NocodeMrLi/mini-program-engineering-skill-suite ~/.claude/skills/mini-program-engineering-skill-suite

    45GitHub stars+6 (+15.4 %)
  18. 18

    Skill
    Active

    Build tested agent skills and govern their lifecycle through a user-defined marketplace: evidence, discovery, updates, rollback, quarantine, and 17-platform distribution.

    PythonLicense: MIT
    • agent-skills
    • ai-agents
    • claude-code
    • codex
    • cursor

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/FrancyJGLisboa/agent-skills-platform ~/.claude/skills/agent-skills-platform

    2 376GitHub starsstable
  19. 19

    Skill
    Active

    Run many AIs on one board and keep control of all of it. Deterministic code decides who acts — never a model. A privacy floor keeps sensitive work on your machine, your own tests decide what counts as done, and every action lands on a…

    • a2a
    • agent-orchestration
    • agent-skills
    • ai-agents
    • ai-governance

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/sandhusukhdeep2/sc-prism-releases ~/.claude/skills/sc-prism-releases

    1GitHub starsstable
  20. 20

    Other
    Active

    Open-source observability tool that uses AI agents to self-heal your software

    TypeScriptLicense: Apache-2.0
    • ai
    • llm
    • memory
    • observability
    • opentelemetry

    githubestimated momentumOpen source ↗

    1 404GitHub stars—

Learning resources

Ranked by measured growth, normalized across sources. These resources remain available separately and do not take part in the main tool ranking.

  1. 1

    Agentic_AI_Engineer

    Agent
    ActiveOpen source ↗

    My complete journey to becoming an Agentic AI Engineer through structured learning, projects, experiments, and production-ready implementations of modern AI systems.

    Jupyter NotebookLicense: MIT
    • a2a-protocol
    • agent-orchestration
    • ai-guardrails
    • fastapi
    • kubernetes
    githubResourcemeasured growth
    18GitHub starsstable
  2. 2

    awesome-agentic-devops

    MCP
    ActiveOpen source ↗

    Curated + scored map of official MCP servers and agents for DevOps, Cloud, SRE, and Platform Engineering — every entry rated on production access, approval gates, and audit evidence.

    PythonLicense: MIT
    • agent-skills
    • agentic-ai
    • agentic-devops
    • ai-agents
    • awesome
    githubResourcemeasured growth
    77GitHub starsstable
  3. 3

    awesome-llm-observability

    Skill
    ActiveOpen source ↗

    50+ curated LLM observability tools PLUS 26 Agent Skills (several with runnable, unit-tested scripts) to build, evaluate, debug, secure & monitor reliable LLM apps. Tracing, evals, guardrails, LLMOps.

    Python
    • ai-agents
    • ai-engineering
    • ai-observability
    • awesome
    • awesome-list
    github

    Install git clone https://github.com/ContextJet-ai/awesome-llm-observability ~/.claude/skills/awesome-llm-observability

    Resourcemeasured growth
    33GitHub stars+2 (+6.5 %)
  4. 4

    trigger_tree

    Skill
    ActiveOpen source ↗

    Documentation-discovery telemetry for Claude Code — heat/cold maps, health grade, evidence-backed router fixes. 100% local, zero tokens. /tt

    PythonLicense: MIT
    • ai-agents
    • claude-code
    • claude-code-plugin
    • docs-as-code
    • documentation
    github

    Install /plugin marketplace add Hedde/trigger_tree

    Resourcemeasured growth
    14GitHub starsstable