flights_simple

The 'flights_simple' project was derived from work done by datakickstart-dabs project. It is meant to show a simple project that includes what is needed to demonstrate best practices for developer workflow and CI/CD (Continuous Integration / Continuous Deployment). You will find bundle config files databricks.yml in the project directory which references bundle resources in the resources/ directory. GitHub Action to run tests and deploy are found in the .githubs/workflows directory.

Getting started

1. Install the Databricks CLI from https://docs.databricks.com/dev-tools/cli/databricks-cli.html

2. Authenticate to your Databricks workspace:

   $ databricks configure
   

3. Go to root project directory then deploy a development copy of this project, type:

   $ databricks bundle deploy --target dev
   

   (Note that "dev" is the default target, so the --target parameter is optional here.)

   This deploys everything that's defined for this project. You can find the jobs by opening your workpace and clicking on Workflows.

4. Similarly, to deploy a production copy, type:

   $ databricks bundle deploy --target prod
   

5. To run a job or pipeline, use the "run" comand:

   $ databricks bundle run notebook_validation_job
   

6. Optionally, install developer tools such as the Databricks extension for Visual Studio Code from https://docs.databricks.com/dev-tools/vscode-ext.html. Or read the "getting started" documentation for Databricks Connect for instructions on running the included Python code from a different IDE.

7. For documentation on the Databricks asset bundles format used for this project, and for CI/CD configuration, see https://docs.databricks.com/dev-tools/bundles/index.html.