APACHE AIRFLOW: BUILDING SCALEABLE DYNAMIC WORKFLOWS

Analysis automation and analytic services is the future of data engineering! Apache Airflow's DSL makes it natural to build complex DAGs of tasks dynamically, and ING has been leveraging this feature creating anarray of services as dynamic workflows. In this talk, we'll explain the mechanics of dynamic pipeline generation.