Data engineering · Library

SDK for robotics teams to verify the quality of their data used for AI model training.

Source

GitHub

Language

Python

Domains

Data engineering

High-performance AI pipeline engine with a C++ core and 50+ Python-extensible nodes. Build, debug, and scale LLM workflows with 13+ model providers, 8+ vector databases, and agent orchestration, all from your IDE. Includes VS Code extension, TypeScript/Python SDKs, and Docker deployment.

Source

GitHub

Language

Python

Domains

Data engineering

Uses

Retrieval-augmented generation (RAG), Workflows

With dbt, data analysts and engineers can build analytics the way engineers build applications.

Source

PyPI

Domains

Data engineering

Programmatically author, schedule and monitor data pipelines

Source

PyPI

Domains

Data engineering

Uses

Workflows

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

Source

GitHub

Language

Python

Domains

Data engineering, Rust

Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learn...

Source

GitHub

Language

Java

Domains

Data engineering

LakeSoul is an end-to-end, realtime cloud-native Lakehouse framework for fast data ingestion, concurrent updates, incremental analytics, multimodal data processing and vector search — powering next-generation BI and AI workloads.

Source

GitHub

Language

Rust

Domains

Data engineering

Uses

Retrieval-augmented generation (RAG)