Die Werkzeugbeschreibungen sind auf Englisch.

LLM-Beobachtbarkeit

Monitoring, Traces, Bewertung und Qualität von LLM-Anwendungen.

Anwendungsfall

Aktivität

Sortieren nach

123

Werkzeug-Rangliste

Ranked by known GitHub stars, highest first; archived repositories come after maintained repositories. Rohwerte behalten ihr eigenes Zeitfenster; eine Änderung erfordert mindestens zwei Messungen in 7 Tagen.

  1. 1

    Sonstige
    Aktiv

    The fastest path to AI-powered full stack observability, even for lean teams.

    GoLizenz: GPL-3.0
    • ai
    • alerting
    • cncf
    • data-visualization
    • database

    githubgemessenes WachstumQuelle öffnen ↗

    80 402GitHub-Sterne+91 (+0.11 %)
  2. 2

    Sonstige
    Aktiv

    SigNoz is an open-source, OpenTelemetry-native observability platform for your team and their AI agents. Get logs, metrics, and traces in one tool with features like APM, distributed tracing, log management, infra monitoring, etc. Combined…

    TypeScript
    • apm
    • application-monitoring
    • distributed-tracing
    • go
    • good-first-issue

    githubgemessenes WachstumQuelle öffnen ↗

    31 992GitHub-Sterne+62 (+0.19 %)
  3. 3

    Agent
    Aktiv

    The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models…

    PythonLizenz: Apache-2.0
    • agentops
    • agents
    • ai
    • ai-governance
    • apache-spark

    githubgemessenes WachstumQuelle öffnen ↗

    27 768GitHub-Sterne+80 (+0.29 %)
  4. 4

    Sonstige
    Aktiv

    Mastra is the modern TypeScript framework for AI-powered applications and agents.

    TypeScript
    • agents
    • ai
    • chatbots
    • evals
    • javascript

    githubgemessenes WachstumQuelle öffnen ↗

    27 624GitHub-Sterne+121 (+0.44 %)
  5. 5

    Sonstige
    Aktiv

    eBPF-based Networking, Security, and Observability

    GoLizenz: Apache-2.0
    • bpf
    • cncf
    • cni
    • containers
    • ebpf

    githubgemessenes WachstumQuelle öffnen ↗

    25 041GitHub-Sterne+27 (+0.11 %)
  6. 6

    Sonstige
    Aktiv

    Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI…

    TypeScriptLizenz: MIT
    • ci
    • ci-cd
    • cicd
    • evaluation
    • evaluation-framework

    githubgemessenes WachstumQuelle öffnen ↗

    24 738GitHub-Sterne+131 (+0.53 %)
  7. 7

    Bibliothek
    Aktiv

    Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

    PythonLizenz: Apache-2.0
    • automation
    • data
    • data-engineering
    • data-ops
    • data-science

    githubgemessenes WachstumQuelle öffnen ↗

    23 759GitHub-Sterne+66 (+0.28 %)
  8. 8

    Sonstige
    Aktiv

    A high-performance observability data pipeline.

    RustLizenz: MPL-2.0
    • agent
    • cloud-native
    • data-transformation
    • datadog
    • etl

    githubgemessenes WachstumQuelle öffnen ↗

    22 507GitHub-Sterne+45 (+0.20 %)
  9. 9

    Sonstige
    Aktiv

    Open source observability platform for logs, metrics, traces, frontend monitoring, pipelines and LLM observability. A sophisticated, simple and highly performant alternative to Datadog, Splunk, and Elasticsearch with 140x lower storage…

    TypeScriptLizenz: AGPL-3.0
    • analytics
    • apm
    • datadog
    • elasticsearch
    • grafana

    githubgemessenes WachstumQuelle öffnen ↗

    21 608GitHub-Sterne+115 (+0.54 %)
  10. 10

    Skill
    Aktiv

    The container platform tailored for Kubernetes multi-cloud, datacenter, and edge management ⎈ 🖥 ☁️

    Go
    • agent-skills
    • ai
    • cloud-native
    • cncf
    • devops

    githubgemessenes WachstumQuelle öffnen ↗

    Installieren git clone https://github.com/kubesphere/kubesphere ~/.claude/skills/kubesphere

    17 037GitHub-Sterne+9 (+0.05 %)
  11. 11

    Agent
    Aktiv

    eBPF-powered network observability for Kubernetes. Indexes L4/L7 traffic with full K8s context, decrypts TLS without keys. Queryable by AI agents via MCP and humans via dashboard.

    GoLizenz: Apache-2.0
    • cloud-native
    • devops
    • docker
    • ebpf
    • golang

    githubgemessenes WachstumQuelle öffnen ↗

    12 066GitHub-Sterne+7 (+0.06 %)
  12. 12

    Sonstige
    Aktiv

    the LLM vulnerability scanner

    PythonLizenz: Apache-2.0
    • ai
    • llm-evaluation
    • llm-security
    • security-scanners
    • vulnerability-assessment

    githubgemessenes WachstumQuelle öffnen ↗

    9 079GitHub-Sterne+43 (+0.48 %)
  13. 13

    MCP
    Aktiv

    🫖 Status page with uptime monitoring & API monitoring as code 🫖

    TypeScriptLizenz: AGPL-3.0
    • alerting
    • incident-management
    • mcp
    • mcp-server
    • monitoring

    githubgemessenes WachstumQuelle öffnen ↗

    9 052GitHub-Sterne+28 (+0.31 %)
  14. 14

    Skill
    Aktiv

    YAO = Yielding AI Outcomes. A rigorous engineering, evaluation, governance, and portability system for reusable agent skills.

    PythonLizenz: MIT
    • agent-skills
    • ai-agents
    • evaluation
    • governance
    • meta-skill

    githubgemessenes WachstumQuelle öffnen ↗

    Installieren git clone https://github.com/yaojingang/yao-meta-skill ~/.claude/skills/yao-meta-skill

    2 577GitHub-Sterne+54 (+2.1 %)
  15. 15

    Skill
    Aktiv

    Build tested agent skills and govern their lifecycle through a user-defined marketplace: evidence, discovery, updates, rollback, quarantine, and 17-platform distribution.

    PythonLizenz: MIT
    • agent-skills
    • ai-agents
    • claude-code
    • codex
    • cursor

    githubgemessenes WachstumQuelle öffnen ↗

    Installieren git clone https://github.com/FrancyJGLisboa/agent-skill-creator ~/.claude/skills/agent-skill-creator

    2 367GitHub-Sterne+30 (+1.3 %)
  16. 16

    Skill
    Aktiv

    The Fable Workflow: how Claude Fable 5 worked, distilled into skills any model can run, with the eval that keeps it honest. Think / act / prove.

    PythonLizenz: MIT
    • agent-skills
    • ai-agents
    • claude
    • claude-code
    • claude-md

    githubgemessenes WachstumQuelle öffnen ↗

    Installieren git clone https://github.com/Sahir619/fable-method ~/.claude/skills/fable-method

    2 272GitHub-Sterne+19 (+0.84 %)
  17. 17

    MCP
    Aktiv

    A Go framework for building production agent systems with graph workflows, tools, memory, A2A, AG-UI, MCP, evaluation, and observability.

    GoLizenz: Apache-2.0
    • a2a
    • a2a-protocol
    • ag-ui
    • agent
    • agent-framework

    githubgemessenes WachstumQuelle öffnen ↗

    1 759GitHub-Sterne+24 (+1.4 %)
  18. 18

    Sonstige
    Aktiv

    Open-source observability tool that uses AI agents to self-heal your software

    TypeScriptLizenz: Apache-2.0
    • ai
    • llm
    • memory
    • observability
    • opentelemetry

    githubgemessenes WachstumQuelle öffnen ↗

    1 404GitHub-Sterne+2 (+0.14 %)
  19. 19

    Skill
    Aktiv

    A skill creator that proves its skills work. Evidence-driven skill creation for Claude Code and Codex: baseline-tested generation, per-skill regression evals, ecosystem doctor, cross-runtime compile, and an opt-in proactive advisor.

    PythonLizenz: MIT
    • agent-skills
    • claude-ai
    • claude-code
    • claude-skills
    • codex

    githubgemessenes WachstumQuelle öffnen ↗

    Installieren git clone https://github.com/tripleyak/SkillForge ~/.claude/skills/SkillForge

    884GitHub-Sterne+3 (+0.34 %)
  20. 20

    Skill
    Aktiv

    OpenJudge: A Unified Framework for Holistic Evaluation and Quality Rewards

    PythonLizenz: Apache-2.0
    • agent
    • agent-skills
    • ai-agent
    • alignment
    • evaluation

    githubgemessenes WachstumQuelle öffnen ↗

    Installieren git clone https://github.com/agentscope-ai/OpenJudge ~/.claude/skills/OpenJudge

    809GitHub-Sterne+10 (+1.3 %)
  21. 21

    Sonstige
    Aktiv

    AI SRE AgenticOps for Kubernetes and cloud infrastructure.

    Python
    • agenticops
    • ai
    • aiops
    • aisre
    • cloud

    githubgemessenes WachstumQuelle öffnen ↗

    782GitHub-Sterne+1 (+0.13 %)
  22. 22

    Skill
    Aktiv

    A test runner for agentskills.io-style AI agent skills

    TypeScriptLizenz: MIT
    • agent-evals
    • agent-skills
    • agentskills
    • ai-agents
    • cli

    githubgemessenes WachstumQuelle öffnen ↗

    Installieren git clone https://github.com/darkrishabh/agent-skills-eval ~/.claude/skills/agent-skills-eval

    719GitHub-Sterne+15 (+2.1 %)
  23. 23

    Agent
    Aktiv

    DataBuff is an AI-native APM built on Opentelemetry，with multi-agent troubleshooting out of the box.

    JavaLizenz: AGPL-3.0
    • aiops
    • apm
    • application-monitoring
    • devops
    • distributed-tracing

    githubgemessenes WachstumQuelle öffnen ↗

    634GitHub-Sterne+26 (+4.3 %)
  24. 24

    Skill
    Aktiv

    Open-source infrastructure that turns scattered SKILL.md files into curated, retrieval-ready agent-skill corpora—with retrieval and evaluation tooling included.

    PythonLizenz: Apache-2.0
    • agent-memory
    • agent-skills
    • ai-agents
    • benchmark
    • dataset

    githubgemessenes WachstumQuelle öffnen ↗

    Installieren git clone https://github.com/EverMind-AI/SkillCorpus ~/.claude/skills/SkillCorpus

    424GitHub-Sterne+204 (+92.7 %)
  25. 25

    Skill
    Aktiv

    Multi-tier framework for evaluating AI agent skills with quality gates, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior.

    PythonLizenz: Apache-2.0
    • agent-evaluation
    • agent-security
    • agent-skills
    • agentic-ai
    • benchmark

    githubgemessenes WachstumQuelle öffnen ↗

    Installieren git clone https://github.com/NVIDIA/SkillEvaluator ~/.claude/skills/SkillEvaluator

    389GitHub-Sterne+63 (+19.3 %)