Multi-Repository Endpoints

[LexManos]

The further facilitate better administration I am working on re-designing how repositories are defined.
The idea is that I can setup repositories for our parnters, and have their data isolated from ours in the backend.
But served to the public as one cohesive repository.

To my understanding, Sonatype has the following design:

• /repo-releases
• /repo-snapshots
• /repo-public

The public is a merged view of snapshot and releases. Down to directory listings, and maven metadatas.

I propose that we follow the same design, where we disallow - in repository names, and use that to specify the view type.
But instead of having a 'public' view, we simply treat requests for the repo without a - as the 'merged' repo.
For example, we have two files in the 'forge' repository:

• forge/releases/net/minecraftforge/fg/1.0/fg-1.0.jar
• forge/snapshots/net/minecraftforge/fg/1.0-SNAPSHOT/fg-1.0-0000000.0000.jar

If there is a request for /forge/net/minecraftforge/fg/ it would list both sub-folders.
But if there is a request for /forge-snapshots/net/minecraftforge/fg/ it would only list the one snapshot version.
To this end, I also propose we special case the root names releases and snapshots. So that no repository may have that name.
If we do encounter one of those names, we should treat it as if requesting that view of all available repos.
This way consumers can have one endpoint for the maven repository without having to know what we do in the backend.

For example:

• forge/releases/net/minecraftforge/fg/1.0/fg-1.0.jar
• forge/snapshots/net/minecraftforge/fg/1.0-SNAPSHOT/fg-1.0-0000000.0000.jar
• sponge/releases/org/spongepowered/sponge/1.0/sponge-1.0.jar
• fsponge/snapshots/org/spongepowered/sponge/1.0-SNAPSHOT/sponge-1.0-0000000.0000.jar

Requesting /releases/ would list both net and org, and be able to serve direct requests for the 1.0 artifacts for both groups.
Requesting /snapshots/ would be the same, but serve the 1.0-SNAPSHOT artifacts.
Requesting / would serve all 4 artifacts.

For publishing, there are two options. Either we can support publishing to the root repo forge which then would attempt to extract the artifact version via the folder structure and place it in the correct sub-repo. Or we do not allow publishing to the merged repos and rely on those uploading to specify the correct repo.

Personally, I like the idea of being able to publish to the root and have the backend hidden. However that imposes quite complex operations on the servers end. As we would need to attempt to find the 'version' of the file being published, and then pick the correct sub-repo. So the first pass should probably require publishing to an explicitly defined repo.

At the very least i'd love to be able to support publishing to /releases/ and /snapshots/ and having it auto pick the correct top level repo based on path prefixes.

[TheCurle]

What would you suggest for established artifacts which contain a - dash character?

Would there be some other replacement syntax where an underscore _ is used, and replaced with - for a search if not found?

[LexManos]

If you're referring to the top group directory that includes a - such as https://repo1.maven.org/maven2/ant-contrib/
I don't think that would be an issue. We should have a known set of repos on the server's end. And we just disallow repo names to have -
For example:

ant {
  prefixes [
    /ant-contrib/
    /ant-doxygen/
    /ant4eclipse/
  ]
}

And then the requests would be translated as follows:
/ant-contrib/foo -> /ant/ant-contrib/foo, because we only know of private and releases view, the 'contrib' would not match and we'd fall back to / as the requested repo.

Nothing on maven central has a top level folder ending in -releases or -snapshots so I think we'll be fine to make this somewhat arbitrary limitation.

[TheCurle]

With the inclusion of a "whitelist" of repo names, then this system seems fine to me.

[LexManos]

There is a concern for mirroring things, say we have a repo named pub and we're mirroring maven central.
Since there are artifacts that are published under the 'pub' root folder there wouldn't be a way too distinguish between the repo name and the first part of the group.
In these cases, Honestly I think we should just say screw it and call it bad server config. Letting it return 404. {Because the file wouldnt exist in the pub repo}
Leave it up to the server admin to pick repo names that don't collide with things they are mirroring.