Bootstrap Patterns (Cloud and Local)

[StylusEaterChef]

Example Request

This example request includes usage of the following Chef products (select one
or more):

• Chef Automate
  • Includes Chef Server, Client, Backend, ChefDK and Supermarket
• Chef Workstation
  • Includes Chef Workstation only
• Habitat
  • Includes Habitat, Habitat Builder (on-prem and SaaS)
• InSpec
  • Includes InSpec, Ad-hoc Automate jobs, Audit Cookbook

Problem Statement

We need some basic patterns for bootstrapping Chef Infra and leveraging related tools both locally (we have one for Vagrant) and in the cloud.

External References

• AWS (CloudFormation)
• Azure (ARM)
• GCP (Deployment Manager)
• Pulumi
• Terraform

Additional Notes

Try to introduce cloud native examples at first using the native tools available with the platform. Create examples using Terraform and Pulumi.

[jmassardo]

Here's some prior art to build from: https://blog.chef.io/bootstrapping-nodes-in-bulk/

[StylusEaterChef]

Existing Azure Docs (Updated 02/22/20): https://docs.microsoft.com/en-us/azure/developer/chef/

[danielcbright]

Wanted to leave a suggestion here re: layout of how to include any docs/examples for this.

On the main README, perhaps we could add a new subsection like so:

Infra

Bootstrapping

• Cloud bootstrapping
• Targeted ad-hoc bootstrapping
• etc...

I'm not sure exactly what the categories should be, but I feel like we are starting to get enough content that we can start breaking things down a bit, thoughts?

[StylusEaterChef]

Boneyard project: https://github.com/chef-boneyard/chef-provisioning

• problem with this was we were using Cookbooks and even nested Cookbooks to get things provisioned
• can we pull any useful approaches from this more complex approach and makes things more simple?

[danielcbright]

@StylusEaterChef I think chef-provisioning has been largely replaced by tools like Terraform, ARM Templates, etc... and that most users already have a method of which they want to spin up their "instances/nodes/vms/etc..." I'm going to dive deeper into the link you shared though because I think we could get some valuable information out of that in regards to how to bootstrap nodes.

[StylusEaterChef]

@StylusEaterChef I think chef-provisioning has been largely replaced by tools like Terraform, ARM Templates, etc... and that most users already have a method of which they want to spin up their "instances/nodes/vms/etc..." I'm going to dive deeper into the link you shared though because I think we could get some valuable information out of that in regards to how to bootstrap nodes.

I totally agree people have a method in which to spin up their "instances/nodes/vms/etc" and that many use Terraform, ARM templates and etc.

What I've seen in the past with Terraform is that people want to build their code quickly (who doesn't?) and then when they attempt to map things to the enterprise environment they run into massive friction because their "quick build" required an "over privileged" user/role to provision an entire setup and most enterprise infrastructures don't work well with that setup.

What I've seen with native provisioning through tools like ARM, CloudFormation or Cloud Deployment Manager is that some people don't like that they can't readily achieve "multi-cloud" or "hybrid" cloud without learning multiple tools.

In practice, I've found some people when using their preferred solution usually end up breaking their infrastructure apart into logical "chunks" much like one would nest "Russian dolls" with an inner core of functionality that expands out with each of their layers adopting their own least privileged roles and "teams" dealing with each individual component. This is usually later mapped into a pipeline with those roles or an over-privileged role.

I definitely don't claim to have the "best" answer for what we should but I do have an opinion on it. I'd like to start with leveraging native tools, then look at what permissions would be required for building each solution so we could have a discussion about required permissions and how those look for each platform? This should help us going forward when we want to bring other tools into the mix because we'll have a great view into the security requirements top to bottom.

What are your thoughts?