Skip to content

cloudymoma/dingo-composer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Google Cloud Composer 3 (Apache Airflow)

Manage dataproc serverless jobs.

We use this spark job in this sample. You could build your own jars.

Before you start, update the makefile, deploy.sh and spark-batches.py with your own environment settings accordingly.

make composer - Create a Composer 3 in your project

make deploy - Deploy the DAG to composer

feel free to explore and play with spark-batches.py for testing.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published