Provider namespaces

[mjuenema]

Terraform 0.13 (earlier?) introduced the concept of provider namespaces. The basic idea is that if two providers have the same name they can be distinguished by their namespace as shown in the example from Terraform Provider Requirements. The question is how to reflect this in Terraform.

terraform {
  required_providers {
    # In the rare situation of using two providers that
    # have the same type name -- "http" in this example --
    # use a compound local name to distinguish them.
    hashicorp-http = {
      source  = "hashicorp/http"
      version = "~> 2.0"
    }
    mycorp-http = {
      source  = "mycorp/http"
      version = "~> 1.0"
    }
  }
}

I came across this while I was re-writing makecode.py to query the Terraform Provider Registry. The registry lists multiple "null" providers (https://registry.terraform.io/search/providers?q=null). In it's current state, makecode.py will actually overwrite a previously created file (terrascript/providers/null.py in this case) if it processes another provider with the same name as it is not aware of namespaces. This is clearly a bug.

Takin the "null" providers as an example, in order to make Terrascript support namespaces, the module layout probably would have to be restructured like this.

Namespace/Provider	Path to Python module	Python Import
hashicorp/null	terrascript/providers/hashicorp/null.py	import terrascript.providers.hashicorp.null
mildred/null	terrascript/providers/mildred/null.py	import terrascript.providers.mildred.null
paultyng/null	terrascript/providers/paultyng/null.py	import terrascript.providers.paultyng.null

Obviously this breaks backwards compatibility where people are used to write the import statement without a namespace level.
A possible work-around for this would be to continue making some providers available without the namespace level. Preference would be based on Terraform Registry tier levels: "official" is higher than "verified" which is higher than "community", e.g.

• import terrascript.providers.null (short-cut for import terrascript.providers.hashicorp.null)
• import terrascript.providers.hashicorp.null
• import terrascript.providers.mildred.null
• import terrascript.providers.paultyng.null

I am happy to discuss this here.

[mjuenema]

I've been working on this and propose that namespaces will be part of release 0.10.0. This release will then only support Terraform 0.13.x and (presumably) later.

The list of providers that are going to be supported can be found here. Don't use this branch yet as it is still quite a mess!!!

There are some quirks (bugs?) with the Terraform Provider Registry API where it does not include all providers. For example, https://registry.terraform.io/v1/providers/hashicorp?limit=100 does not include the azure provider although it is available from the web interface: https://registry.terraform.io/providers/hashicorp/azurerm/latest.

Thus the rewritten makecode.py will need some additional code to accept a hard-coded list of providers in addition to the ones fetched from the Terraform Provider Registry API. This is work in progress...

[ilons]

I think that namespaces should be a major, as it is a change breaking backwards compatibility.

Regarding the list of providers, is there some kind of pagination maybe? Or is some providers missing even paginated?

[mjuenema]

The API does serve results paginated. I figured out that the azure provider is actually the azurerm provider.

For backwards compatibility I intend to make "official" and "partner" providers also available without namespace.

[ilons]

For backwards compatibility I intend to make "official" and "partner" providers also available without namespace.

Yeah, that might be good enough.
I still think it would be a good idea to release a major with this breaking, explicit is better than implicit :)

[mjuenema]

See also issue #138.