datakickstart_dabs

The 'datakickstart_dabs' project was generated by using the default-python template then modified to have multiple project folders that are deployed separately. Since they are deployed separately, you will find bundle config files databricks.yml in the project directory, such as in simple_project or in complex_project.

Getting started

1. Install the Databricks CLI from https://docs.databricks.com/dev-tools/cli/databricks-cli.html

2. Authenticate to your Databricks workspace:

   $ databricks configure
   

3. Go to a project directory then deploy a development copy of this project, type:

   cd simple_project
   $ databricks bundle deploy --target dev
   

   (Note that "dev" is the default target, so the --target parameter is optional here.)

   This deploys everything that's defined for this project. For example, the default template would deploy a job called [dev yourname] datakickstart_dabs_job to your workspace. You can find that job by opening your workpace and clicking on Workflows.

4. Similarly, to deploy a production copy, type:

   $ databricks bundle deploy --target prod
   

5. To run a job or pipeline, use the "run" comand:

   $ databricks bundle run datakickstart_dabs_job
   

6. Optionally, install developer tools such as the Databricks extension for Visual Studio Code from https://docs.databricks.com/dev-tools/vscode-ext.html. Or read the "getting started" documentation for Databricks Connect for instructions on running the included Python code from a different IDE.

7. For documentation on the Databricks asset bundles format used for this project, and for CI/CD configuration, see https://docs.databricks.com/dev-tools/bundles/index.html.