工具说明为英文。

技术工具雷达: Data

雷达筛选条件

近期变化

Of 20 comparable tools in Data Engineering, 20 are increasing, 0 are stable, and 0 are declining. arrow has the largest measured increase: +256866 crates.io downloads over 90 days. These movements describe only changes in metrics collected over the period.

测量周期：2026-08-26 至 2026-09-01

工具排名

按实测增长排序，并在不同来源间归一化。 原始值保留各自的数据窗口；计算变化需要 7 天内至少两次测量。

  1. 1

    其他
    活跃

    End-to-end streaming data platform on AWS: Kafka → Iceberg lake → Kimball model → Athena, orchestrated with Airflow 3. Terraform, Glue PySpark, dbt.

    Python
    • apache-airflow
    • apache-iceberg
    • athena
    • aws
    • data-engineering

    github实测增长打开来源 ↗

    106GitHub 星标+51 (+92.7 %)
  2. 2

    其他
    活跃

    Graph-Native Infrastructure for Context and Accountable AI Systems

    Python许可证: MIT
    • agent-memory
    • ai
    • ai-governance
    • ai-infrastructure
    • artificial-intelligence

    github实测增长打开来源 ↗

    11 532GitHub 星标+698 (+6.4 %)
  3. 3

    库
    活跃

    SDK for robotics teams to verify the quality of their data used for AI model training.

    Python许可证: Apache-2.0
    • airflow
    • data-engineering
    • data-pipeline
    • data-quality
    • dataset-curation

    github实测增长打开来源 ↗

    217GitHub 星标+71 (+48.6 %)
  4. 4

    技能
    活跃

    World's first open-source, agentic video production system. 12 production pipelines, 100+ tools, 700+ agent skill and production-knowledge files. Turn your AI coding assistant into a full video production studio.

    Python许可证: AGPL-3.0
    • agent
    • agentic-ai
    • ai
    • claude
    • copilot

    github实测增长打开来源 ↗

    安装 git clone https://github.com/calesthio/OpenMontage ~/.claude/skills/OpenMontage

    52 517GitHub 星标+2 213 (+4.4 %)
  5. 5

    库
    未知最近提交：暂无数据

    Rust implementation of Apache Arrow

    • arrow

    crates实测增长打开来源 ↗

    安装 cargo add arrow

    17 280 524crates.io 90 天下载量+256 866 (+1.5 %)
  6. 6

    库
    未知最近提交：暂无数据

    DataFusion is an in-memory query engine that uses Apache Arrow as the memory model

    • sql
    • query
    • arrow

    crates实测增长打开来源 ↗

    安装 cargo add datafusion

    4 754 796crates.io 90 天下载量+64 216 (+1.4 %)
  7. 7

    技能
    活跃

    Fast, efficient, battle-tested at Alibaba's scale. Hybrid architecture code review tool: deterministic pipelines + LLM Agent, precise line-level comments, built-in multi-language ruleset (NPE, thread-safety, XSS, SQL injection), OpenAI &…

    Go许可证: Apache-2.0
    • agent
    • agent-skills
    • code-review
    • code-review-assistant
    • harness

    github实测增长打开来源 ↗

    安装 git clone https://github.com/alibaba/open-code-review ~/.claude/skills/open-code-review

    21 735GitHub 星标+316 (+1.5 %)
  8. 8

    其他
    活跃

    Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.

    TypeScript
    • agent
    • agentic-ai
    • agentic-framework
    • agentic-workflow
    • ai

    github实测增长打开来源 ↗

    154 043GitHub 星标+524 (+0.34 %)
  9. 9

    其他
    活跃

    Summer 2027 software engineering, data science, AI, quant, product management, and hardware internship postings. Updated daily by Simplify and Pitt CSC.

    Python
    • data-science
    • fall-2026
    • github
    • internship
    • internships

    github实测增长打开来源 ↗

    47 002GitHub 星标+196 (+0.42 %)
  10. 10

    其他
    活跃

    Data-engineering benchmark for coding agents (DuckDB + Snowflake dbt tasks).

    Python许可证: Apache-2.0

      github实测增长打开来源 ↗

      64GitHub 星标+11 (+20.8 %)
    • 11

      其他
      活跃

      Open source observability platform for logs, metrics, traces, frontend monitoring, pipelines and LLM observability. A sophisticated, simple and highly performant alternative to Datadog, Splunk, and Elasticsearch with 140x lower storage…

      TypeScript许可证: AGPL-3.0
      • analytics
      • apm
      • datadog
      • elasticsearch
      • grafana

      github实测增长打开来源 ↗

      21 594GitHub 星标+119 (+0.55 %)
    • 12

      应用
      活跃

      SparkyFitness: Built for Families. Powered by AI. Track food, fitness, water, and health — together.

      TypeScript
      • ai
      • artificial-intelligence
      • fitness
      • fitness-app
      • fitness-tracker

      github实测增长打开来源 ↗

      5 713GitHub 星标+79 (+1.4 %)
    • 13

      MCP
      活跃

      Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

      Python许可证: MIT
      • asr
      • audio
      • chinese
      • emotion-recognition
      • funasr

      github实测增长打开来源 ↗

      20 106GitHub 星标+86 (+0.43 %)
    • 14

      其他
      活跃

      Apache Superset is a Data Visualization and Data Exploration Platform

      Python许可证: Apache-2.0
      • analytics
      • apache
      • apache-superset
      • asf
      • bi

      github实测增长打开来源 ↗

      74 566GitHub 星标+103 (+0.14 %)
    • 15

      其他
      活跃

      Tech internships & CS jobs tracker - 180 open software engineering internships from 4,300 employer job boards, auto-updated every 30 min with visa-sponsorship and H-1B sponsor data.

      Python许可证: MIT
      • 2027
      • career
      • computer-science
      • faang
      • h1b

      github实测增长打开来源 ↗

      698GitHub 星标+26 (+3.9 %)
    • 16

      应用
      活跃

      A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.

      Python许可证: Apache-2.0
      • artificial-intelligence
      • dag
      • data-science
      • data-visualization
      • dataflow

      github实测增长打开来源 ↗

      22 576GitHub 星标+81 (+0.36 %)
    • 17

      智能体
      活跃

      The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models…

      Python许可证: Apache-2.0
      • agentops
      • agents
      • ai
      • ai-governance
      • apache-spark

      github实测增长打开来源 ↗

      27 759GitHub 星标+78 (+0.28 %)
    • 18

      其他
      活跃

      Harness Open Source is an end-to-end developer platform with Source Control Management, CI/CD Pipelines, Hosted Developer Environments, and Artifact Registries.

      Go许可证: Apache-2.0
      • build-automation
      • build-pipelines
      • ci
      • ci-cd
      • code-repository

      github实测增长打开来源 ↗

      38 196GitHub 星标+72 (+0.19 %)
    • 19

      其他
      活跃

      dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

      Rust许可证: Apache-2.0
      • analytics
      • business-intelligence
      • data-modeling
      • dbt-viewpoint
      • elt

      github实测增长打开来源 ↗

      13 750GitHub 星标+60 (+0.44 %)
    • 20

      其他
      活跃

      LLM speculative inference server for heterogeneous hardware & consumer GPUs

      C++许可证: Apache-2.0
      • cuda
      • cuda-kernels
      • dflash
      • heterogeneous-computing
      • kernel

      github实测增长打开来源 ↗

      2 825GitHub 星标+37 (+1.3 %)
    • 21

      库
      活跃

      A software update framework for macOS

      Objective-C
      • framework
      • macos
      • software-update
      • update

      github实测增长打开来源 ↗

      9 613GitHub 星标+52 (+0.54 %)
    • 22

      其他
      活跃

      Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for…

      Python许可证: Apache-2.0
      • agent-framework
      • agentic-ai
      • agentic-rag
      • agents
      • ai

      github实测增长打开来源 ↗

      26 377GitHub 星标+60 (+0.23 %)
    • 23

      其他
      活跃

      2026–2027 tech internships in the US, updated daily. Software engineering, data science, security, product, and design intern roles.

      许可证: MIT
      • careers
      • computer-science
      • hiring
      • internship
      • internships

      github实测增长打开来源 ↗

      93GitHub 星标+8 (+9.4 %)
    • 24

      其他
      活跃

      Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

      Python许可证: Apache-2.0
      • airflow
      • apache
      • apache-airflow
      • automation
      • dag

      github实测增长打开来源 ↗

      46 672GitHub 星标+64 (+0.14 %)

    学习与参考资源

    按实测增长排序，并在不同来源间归一化。 这些资源可单独访问，不参与主要排名。

    1. 1

      data-engineering-zoomcamp

      其他
      活跃打开来源 ↗

      Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼

      Jupyter Notebook
      • course
      • data-engineering
      • dbt
      • docker
      • free
      github资源实测增长
      45 131GitHub 星标+171 (+0.38 %)