Pipeline

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

:zap: Workflow Automation Platform. Orchestrate & Schedule code in any language, run anywhere, 600+ plugins. Alternative to Airflow, n8n, Rundeck, VMware vRA, Zapier ...

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. All in a modern, AI-native editor.

A high-performance observability data pipeline.

Tired of pushing to test your .gitlab-ci.yml?

Kubernetes-native platform to run massively parallel data/streaming jobs

Kubernetes operator to manage installation, updation and uninstallation of tektoncd projects (pipeline, …)

A DSL for data-driven computational pipelines

XLS: Accelerated HW Synthesis

A blazing fast tool for building data pipelines: read, process and output events. Our community: https://t.me/file_d_community

Write your pipelines in C# !

Documentation for Kestra — an event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.

Notebooks, code samples, sample apps, and other resources that demonstrate how to use, develop and manage machine learning and generative AI workflows using Google Cloud Vertex AI.