[PULP-1118] Add better error handling for repover duplicate content

[pedro-psb]

Added a proper error class for duplicate content handling and some more logging to inform exactly what are the conflicting content.

When duplicates are detected, we do some extra work to collect duplicate content.
A simple performance test shows it's not too bad:

In [1]: import pulpcore.plugin.repo_version_utils as ut

In [2]: content_qs = Package.objects.all()

In [3]: unique_keys = Package.repo_key_fields

In [4]: content_qs.count()
Out[4]: 24547

In [5]: ut.count_duplicates(content_qs, unique_keys)
Out[5]: 7701

In [6]: %timeit ut.count_duplicates(content_qs, unique_keys)
35.1 ms ± 154 μs per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [7]: %timeit ut.collect_duplicates(content_qs, unique_keys)
61.4 ms ± 266 μs per loop (mean ± std. dev. of 7 runs, 10 loops each)

Closes: #7184

📜 Checklist

• Commits are cleanly separated with meaningful messages (simple features and bug fixes should be squashed to one commit)
• A changelog entry or entries has been added for any significant changes
• Follows the Pulp policy on AI Usage
• (For new features) - User documentation and test coverage has been added

See: Pull Request Walkthrough

[dralley]

It would seem like this assumes that a RepositoryVersion creation failed due to duplicates always? Do we want to assume that? Should the error be more specific?

[pedro-psb]

Yeah, makes sense. I'll make more specific one

[dralley]

Is there are particular reason to separate this into its own function if it's only used in one place?

[pedro-psb]

Just for the abstraction. At a glance, it's trivial do read the main validate function and understand what it does, and dig into the functions if you are interested in implementation details.
If the call cost is not relevant (i believe it isnt here), then it is just a matter of style. I can undo that, if you prefer..

[dralley]

It's just a matter of style, but if only one line is doing work here, I'd rather just inline it personally

I do not consider it a blocker if you have strong feelings about it or if there are plans to re-use it in the future, though.

[pedro-psb]

Ok, I'll inline the log one. I'll keep the count and collect duplicates because it was at least convenient for testing them in isolation.

[dralley]

same

[dralley]

This case is at least more sensible than the other one though, since there's more than one line that's actually doing something

[mdellweg]

I think the logic is messed up a bit.
If you have two types in the loop and the first one has duplicates the second is fine, this will not raise.