Data radar: tools and trends

Momentum based on available 7-day snapshots for Data: measured with two snapshots, estimated otherwise. Each value shows its own source window. This page reads stored snapshots only.

Radar filters

What’s changing

Of 20 comparable tools in Data Engineering, 20 are increasing, 0 are stable, and 0 are declining. arrow has the largest measured increase: +256866 crates.io downloads over 90 days. These movements describe only changes in metrics collected over the period.

Measured period: 2026-08-26 to 2026-09-01

Tool ranking

Ranked by measured growth, normalized across sources. Raw values keep their source-specific window, shown under each tool; a change requires at least two snapshots from the last 7 days.

  1. 1

    Other
    Active

    End-to-end streaming data platform on AWS: Kafka → Iceberg lake → Kimball model → Athena, orchestrated with Airflow 3. Terraform, Glue PySpark, dbt.

    Python
    • apache-airflow
    • apache-iceberg
    • athena
    • aws
    • data-engineering

    githubmeasured growthOpen source ↗

    106GitHub stars+51 (+92.7 %)
  2. 2

    Other
    Active

    Graph-Native Infrastructure for Context and Accountable AI Systems

    PythonLicense: MIT
    • agent-memory
    • ai
    • ai-governance
    • ai-infrastructure
    • artificial-intelligence

    githubmeasured growthOpen source ↗

    11 532GitHub stars+698 (+6.4 %)
  3. 3

    Library
    Active

    SDK for robotics teams to verify the quality of their data used for AI model training.

    PythonLicense: Apache-2.0
    • airflow
    • data-engineering
    • data-pipeline
    • data-quality
    • dataset-curation

    githubmeasured growthOpen source ↗

    217GitHub stars+71 (+48.6 %)
  4. 4

    Skill
    Active

    World's first open-source, agentic video production system. 12 production pipelines, 100+ tools, 700+ agent skill and production-knowledge files. Turn your AI coding assistant into a full video production studio.

    PythonLicense: AGPL-3.0
    • agent
    • agentic-ai
    • ai
    • claude
    • copilot

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/calesthio/OpenMontage ~/.claude/skills/OpenMontage

    52 517GitHub stars+2 213 (+4.4 %)
  5. 5

    Library
    UnknownLast commit: unavailable

    Rust implementation of Apache Arrow

    • arrow

    cratesmeasured growthOpen source ↗

    Install cargo add arrow

    17 280 524crates.io downloads over 90 days+256 866 (+1.5 %)
  6. 6

    Library
    UnknownLast commit: unavailable

    DataFusion is an in-memory query engine that uses Apache Arrow as the memory model

    • sql
    • query
    • arrow

    cratesmeasured growthOpen source ↗

    Install cargo add datafusion

    4 754 796crates.io downloads over 90 days+64 216 (+1.4 %)
  7. 7

    Skill
    Active

    Fast, efficient, battle-tested at Alibaba's scale. Hybrid architecture code review tool: deterministic pipelines + LLM Agent, precise line-level comments, built-in multi-language ruleset (NPE, thread-safety, XSS, SQL injection), OpenAI &…

    GoLicense: Apache-2.0
    • agent
    • agent-skills
    • code-review
    • code-review-assistant
    • harness

    githubmeasured growthOpen source ↗

    Install git clone https://github.com/alibaba/open-code-review ~/.claude/skills/open-code-review

    21 735GitHub stars+316 (+1.5 %)
  8. 8

    Other
    Active

    Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.

    TypeScript
    • agent
    • agentic-ai
    • agentic-framework
    • agentic-workflow
    • ai

    githubmeasured growthOpen source ↗

    154 043GitHub stars+524 (+0.34 %)
  9. 9

    Other
    Active

    Summer 2027 software engineering, data science, AI, quant, product management, and hardware internship postings. Updated daily by Simplify and Pitt CSC.

    Python
    • data-science
    • fall-2026
    • github
    • internship
    • internships

    githubmeasured growthOpen source ↗

    47 002GitHub stars+196 (+0.42 %)
  10. 10

    Other
    Active

    Data-engineering benchmark for coding agents (DuckDB + Snowflake dbt tasks).

    PythonLicense: Apache-2.0

      githubmeasured growthOpen source ↗

      64GitHub stars+11 (+20.8 %)
    • 11

      Other
      Active

      Open source observability platform for logs, metrics, traces, frontend monitoring, pipelines and LLM observability. A sophisticated, simple and highly performant alternative to Datadog, Splunk, and Elasticsearch with 140x lower storage…

      TypeScriptLicense: AGPL-3.0
      • analytics
      • apm
      • datadog
      • elasticsearch
      • grafana

      githubmeasured growthOpen source ↗

      21 594GitHub stars+119 (+0.55 %)
    • 12

      App
      Active

      SparkyFitness: Built for Families. Powered by AI. Track food, fitness, water, and health — together.

      TypeScript
      • ai
      • artificial-intelligence
      • fitness
      • fitness-app
      • fitness-tracker

      githubmeasured growthOpen source ↗

      5 713GitHub stars+79 (+1.4 %)
    • 13

      MCP
      Active

      Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.

      PythonLicense: MIT
      • asr
      • audio
      • chinese
      • emotion-recognition
      • funasr

      githubmeasured growthOpen source ↗

      20 106GitHub stars+86 (+0.43 %)
    • 14

      Other
      Active

      Apache Superset is a Data Visualization and Data Exploration Platform

      PythonLicense: Apache-2.0
      • analytics
      • apache
      • apache-superset
      • asf
      • bi

      githubmeasured growthOpen source ↗

      74 566GitHub stars+103 (+0.14 %)
    • 15

      Other
      Active

      Tech internships & CS jobs tracker - 180 open software engineering internships from 4,300 employer job boards, auto-updated every 30 min with visa-sponsorship and H-1B sponsor data.

      PythonLicense: MIT
      • 2027
      • career
      • computer-science
      • faang
      • h1b

      githubmeasured growthOpen source ↗

      698GitHub stars+26 (+3.9 %)
    • 16

      App
      Active

      A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.

      PythonLicense: Apache-2.0
      • artificial-intelligence
      • dag
      • data-science
      • data-visualization
      • dataflow

      githubmeasured growthOpen source ↗

      22 576GitHub stars+81 (+0.36 %)
    • 17

      Agent
      Active

      The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models…

      PythonLicense: Apache-2.0
      • agentops
      • agents
      • ai
      • ai-governance
      • apache-spark

      githubmeasured growthOpen source ↗

      27 759GitHub stars+78 (+0.28 %)
    • 18

      Other
      Active

      Harness Open Source is an end-to-end developer platform with Source Control Management, CI/CD Pipelines, Hosted Developer Environments, and Artifact Registries.

      GoLicense: Apache-2.0
      • build-automation
      • build-pipelines
      • ci
      • ci-cd
      • code-repository

      githubmeasured growthOpen source ↗

      38 196GitHub stars+72 (+0.19 %)
    • 19

      Other
      Active

      dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

      RustLicense: Apache-2.0
      • analytics
      • business-intelligence
      • data-modeling
      • dbt-viewpoint
      • elt

      githubmeasured growthOpen source ↗

      13 750GitHub stars+60 (+0.44 %)
    • 20

      Other
      Active

      LLM speculative inference server for heterogeneous hardware & consumer GPUs

      C++License: Apache-2.0
      • cuda
      • cuda-kernels
      • dflash
      • heterogeneous-computing
      • kernel

      githubmeasured growthOpen source ↗

      2 825GitHub stars+37 (+1.3 %)
    • 21

      Library
      Active

      A software update framework for macOS

      Objective-C
      • framework
      • macos
      • software-update
      • update

      githubmeasured growthOpen source ↗

      9 613GitHub stars+52 (+0.54 %)
    • 22

      Other
      Active

      Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for…

      PythonLicense: Apache-2.0
      • agent-framework
      • agentic-ai
      • agentic-rag
      • agents
      • ai

      githubmeasured growthOpen source ↗

      26 377GitHub stars+60 (+0.23 %)
    • 23

      Other
      Active

      2026–2027 tech internships in the US, updated daily. Software engineering, data science, security, product, and design intern roles.

      License: MIT
      • careers
      • computer-science
      • hiring
      • internship
      • internships

      githubmeasured growthOpen source ↗

      93GitHub stars+8 (+9.4 %)
    • 24

      Other
      Active

      Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

      PythonLicense: Apache-2.0
      • airflow
      • apache
      • apache-airflow
      • automation
      • dag

      githubmeasured growthOpen source ↗

      46 672GitHub stars+64 (+0.14 %)

    Learning resources

    Ranked by measured growth, normalized across sources. These resources remain available separately and do not take part in the main tool ranking.

    1. 1

      data-engineering-zoomcamp

      Other
      ActiveOpen source ↗

      Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼

      Jupyter Notebook
      • course
      • data-engineering
      • dbt
      • docker
      • free
      githubResourcemeasured growth
      45 131GitHub stars+171 (+0.38 %)