SPIKE - come up with rough auth flows to use with tombstone migration flow

[argush3]

There is a need to provide the option of performing auth related operations that is separate from the tombstone flow. In this ticket, we want to figure out which auth prefect flows are required and what each prefect flow will do in the context of migrating corporations to the modernized system.

The tombstone flow already provides the ability to do most of the functionality that we need in the separate auth flows we are looking at building. The auth functionality currently available in the tombstone flow include:

• create an entity
• create entity contact
• create affiliation to one or more accounts
• deletion of entity + affiliations

The reason why we are looking at providing an option to perform auth operations outside of the tombstone flow include the following:

• From testing larger migration batches with parallel tombstone pipelines running, the auth api is timing out from the number of calls. This might be mitigated a bit if we talk to auth team. There is still a need to have separate flows so we have more control over auth operations.
• Calling auth as part of the corp migration slows down the tombstone pipeline. This does not mattter that much for smaller batches but is problematic for larger batches.
• Separating auth specific operations provides more flexibility in how we hit auth api and how we package the different auth api calls
• Ability to potentially run auth flows a bit ahead of migration depending on the requirements of batch
• Ability to perform one off auth operations more easily. This is something we need to do quite frequently in dev/test environments and sometimes in prod.

The new auth flows will need to be able to handle the following use cases. Note that all flows should be able to work as manual mode or with USE_MIGRATION_FILTER:

• Creating an auth entity and any of the following: entity contact, affiliation and send uninvited email(can’t add if affiliation is being added). Note that we should be able to turn things on/off with the minimal being just creating the auth entity. Can probably leverage a lot of logic in tombstone flow. The send uninvited email logic is currently being implemented in tombstone flow via Tombstone pipeline - Add option to send unaffiliated email #32437.
• Deleting auth info. Can probably leverage a lot of logic in batch delete flow. We will probably want to have it such that we can delete just the unaffiliated invites or affiliations for specific businesses. Or if we want, everything associated with an entity in auth.
• Ability to just send an unaffiliated email with the assumption that the auth entity and auth entity contact info is in auth already. This is a possible case where we migrate the auth entity and entity contact info a bit before the migration and right after the migration we hit the the unaffiliated email endpoint

Note that we still intend to keep the existing auth operations we have in place via the tombstone and batch delete flows. These are useful for loading of small batches, one off migrations and loading small test datasets for QA purposes.

Tasks

• Come up with required auth flows detailing what each flow does
• Think about how the flows will track info. Reference how tombstone flow uses corp_processing and colin freeze flow uses colin_tracking. It probably makes sense that all auth flows reference an auth specific tracking table. Maybe colin_tracking is a good starting point.
• Review suggested auth flows with team
• Cut placeholder tickets for dev work

[argush3]

Hey team! Please add your planning poker estimate with Zenhub @ketaki-deodhar @Rajandeep98

[ketaki-deodhar]

Auth-Create-Flow

auth-create-flow: Auth flow should be able to do one or all from the following depending on the configs.

• create an entity (can be turned on/off based on UPDATE_ENTITY config which already is been used)
• create entity contact
• create affiliation to one or more accounts (can be turned on/off based on AFFILIATE_ENTITY config which already is been used)

Create tracking table for auth flows: auth_processing

• Columns can be id, corp_num, corp_type_cd, in_early_adopter, environment, create_date, last_modified, claimed_at, flow_name, flow_run_id, processed_status, last_error, mig_batch_id, account_ids, affiliated_email_sent (flag)
• Discuss with argus if we need in_early_adopter for auth flows or any other column are needed to keep track like if it was only updating entity beforehand and sending unaffiliated email right after migration (check affiliated_email_sent (flag))???????

Flow tasks:

• Add new configs for auth batches (recommended) or maybe we can use the ones for tombstone too

• Work with count and batches logic same as other flows and with USE_MIGRATION_FILTER

• Reserving for the flow, claiming the batch etc functionality in ExtractTrackingService can be used to reserve the corps.

• load business data as needed. To load business data, look at get_tombstone_data -> get_corp_snapshot_filings_queries . We might not even need all the filing data.

• Leverage the same logic to create entity from update_auth function in tombstone flow

• Specific tasks to create entity:
  o This can be turned on or off based on the UPDATE_ENTITY config
  o Leverage the same logic to create entity from update_auth function in tombstone flow
  o Create entity with AuthService.create_entity
  o Send email based on SEND_UNAFFILIATED_EMAIL or set affiliated_email_sent to false to send it the day of the migration

• Specific tasks to affiliate entity:
  o This can be turned on or off based on the AFFILIATE_ENTITY config
  o Leverage the same logic to affiliate entity from update_auth function in tombstone flow
  o Affiliate entity with AuthService.create_affiliation for all accounts

Auth-Create-Flow

auth-delete-flow: Delete unaffiliated and/or affiliated records from Auth

• We can pretty much follow the similar implementation of batch_delete_flow
• auth_api_delete function deletes based on:
  o AFFILIATE_ENTITY flag - delete affiliations for businesses if set to true
  o DELETE_AUTH_RECORDS flag - delete entities

Auth-Invite-Flow

auth-invite-flow: send an unaffiliated email with the assumption that the auth entity and auth entity contact info is in auth already.

• Get businesses based on affiliated_email_sent (flag) from auth_processing
• get required data for businesses (identifier and email address is required to send email)
• logic to send unaffiliated email should be pretty much same as in corps_tombstone_flow base on SEND_UNAFFILIATED_EMAIL flag

Auth-Affiliation-Flow

auth-affiliation-flow: create or delete affiliations.

• set create_plan or delete_plan based on the operation to perform
• get eligible businesses from auth_processing and corp accounts for the businesses
• affiliate or delete affiliation

Auth-Contact-Flow

auth-contact-flow: upsert entity contact email (no entity creation, no affiliations, no invite).

• get eligible businesses from auth_processing and contact details the businesses
• update contact email

Above are all the high level descriptions and initial thoughts for all the flows but we can use Separate auth flows implementation PR as a base. We will cut the tickets based on this PR in upcoming sprints.