Skip to content

Comments

Data Migration & UAT Fixes#690

Merged
xzzy merged 24 commits intodbca-wa:mainfrom
oakdbca:data-migration-1
Feb 10, 2026
Merged

Data Migration & UAT Fixes#690
xzzy merged 24 commits intodbca-wa:mainfrom
oakdbca:data-migration-1

Conversation

@oakdbca
Copy link
Member

@oakdbca oakdbca commented Feb 10, 2026

  • Rework some of the CS adapter and handler to match other migration structures
  • Implement any missing transformations to prepare for dev data verification

Bug fixes for:

13979 | CS Delisting change 3/5: introduce a condition (based on Change Type) to alter the final actions displayed
13980 | CS Delisting change 4/5: validation required when Change Type = DeList is selected

…ransformations for effective_to, listing_date, and lodgement_date
…d optimize species/community resolution logic
…s_and_communities to conservation_status models
…ased on Change Type) to alter the final actions displayed. Fixed a case difference between environments 'DeList' locally and 'Delist' in UAT. Code will now normalise code to lower case for comparisons.
… Change Type = DeList is selected. Added case normalisation for front end and backend comparisons
@oakdbca oakdbca requested a review from a team February 10, 2026 04:03
Comment on lines +269 to +279
# We need to map this Taxonomy PK to the Species internal PK
species_pk = None
species_taxonomy_id = None

if row.get("species_id"):
# Pipeline transformed species_id to Taxonomy PK
species_taxonomy_id = row["species_id"]
species_pk = tax_to_species_pk_map.get(species_taxonomy_id)

# community_migrated_from_id pipeline should return Community PK or None
community_pk = row.get("community_migrated_from_id")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The migration silently creates ConservationStatus records with a null species_id if a Taxonomy record lacks an associated Species, bypassing model-level safeguards by using bulk_create().
Severity: MEDIUM

Suggested Fix

Before creating the ConservationStatus object, check if species_pk is None after the lookup. If it is, log an error and continue to the next row. This will prevent the creation of inconsistent records and restore the explicit validation behavior of the previous code.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: boranga/components/data_migration/handlers/conservation_status.py#L267-L279

Potential issue: The data migration handler for `ConservationStatus` resolves species
via a `taxonomy_id`. If a `taxonomy_id` from legacy data exists but has no corresponding
`Species` record, the `tax_to_species_pk_map.get()` call returns `None`. This `None`
value is then assigned to `species_pk` and used in the `ConservationStatus` object
creation. Because the handler uses `bulk_create()`, it bypasses the model's `save()`
method and its `assign_or_create_species()` safeguard. This results in the silent
creation of `ConservationStatus` records with a valid `species_taxonomy_id` but a `null`
`species_id`, leading to data inconsistency. The previous implementation would have
logged an error and skipped such records.

Did we get this right? 👍 / 👎 to inform future reviews.

Copy link
Collaborator

@xzzy xzzy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sensitivity check Completed

@xzzy xzzy merged commit 0246537 into dbca-wa:main Feb 10, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants