notebook-2-app...

...is a foolproof web-based databricks-notebook to web-application conversion platform.

Watch the notebook-2-app Youtube Video!

Bridging the first two of these ↑ stages, notebook-2-app provides a user-friendly pipeline to automate the transformation of non-linear, loosely-structured Python notebook-style code, into maintainable, modular, robust, and efficient applicaiton-code.

This includes...

• Specific notebook-only code substitutions, e.g. replacing figure.show() in a Dash app, with code to start a Dash server (app = Dash() ... app.run())
• Re-wiring the notebook to maintain data sources, if it reads data from a workspace file (e.g. /Workspace/dir/data.csv)
• Identifying & removing secrets (e.g. API keys) and placing them in a seperate .env file
• Suggesting the insertion of proper documentation & commentry within the code
• Moving pip dependancies into a centralised requirements.txt file
• Homogenising identifiers' naming conventions e.g. to snake_case
• (Freedom to insert custom transformations!)

Under the hood

Python notebooks give data scientists the freedom to quickly & loosely build prototypes with a dataset. They aren't - however - conducive to building modular, robust, input-validating applications. This tool helps data scientists get application-ready code, fast.

Walk-through

After entering a username...

...notebook-2-app contacts the Databricks' workspace's REST API...

...allowing selection of a python notebook directly from the workspace.

Tick cells containing code needed for the application...

...and notebook-2-app automatically suggests changes, to make secure & robust application-code...

...plus feedback on code quality.

Simply download the resultant app-code package...

...to instantly run the standalone app locally...

...or deploy it from a dev-ops' repository to an Azure App-Service, using the generated deploy.yml.

Porting notebook- to app-code

notebook-2-app focuses on three phases of code-conversion...

• Syntactically; for neatness. Notebook-code is passed through black as an initial clean-up.
• Operationally; for platform-compatibility. E.g. replacing figure.show() in a Dash app, with code to start a Dash server (app = Dash() ... app.run())
• Structurally; for modularity. Code is seperated into different functions & modules, for maintainability.
  
  
  
  
  

To use this software...

• Download a zip of this repository
• Populate /backend/server_code/secrets.py with values for your Databricks Workspace
• Specify your COMPANY_EMAIL_ADDRESS_ENDING in /frontend/resources/js/task-ui-functions.js
• Execute python -m flask --app ./server.py run
  (Can use _debug.cmd on Windows for local development/testing; runs SASS transpiler & starts server)

This is a prototype software product, entirely open to receiving feedback & improvement ideas!