boto3 automation

[drernie]

1. Define the Roles
   1. RAJEE is the S3-compatible data endpoint that actually serves objects.
   2. RAJA is the authentication service that mints a JWT from (bucket, key).
   3. boto3 must talk only to RAJEE, but consult RAJA just-in-time for auth.

⸻

2. Create the boto3 Client Pointing Only to RAJEE
   1. Configure the client with endpoint_url = RAJEE.
   2. Decide whether RAJEE expects:
      • JWT only → disable SigV4 signing, or
      • SigV4 + JWT → keep normal signing.
   3. Ensure path-style addressing if RAJEE requires it.

⸻

3. Identify the Two Necessary Hook Points in botocore
   1. before-parameter-build (per operation)
      • You have clean access to Bucket and Key.
      • Safe place to call external services.
   2. before-send (per operation)
      • The HTTP request is fully formed.
      • Safe place to mutate headers before the request leaves.

These two hooks share a request context.

⸻

4. What Happens at before-parameter-build
   1. Read Bucket and Key from the operation parameters.
   2. Call RAJA with those values.
   3. Receive a JWT.
   4. Store the JWT in the request’s context (not in globals).

This runs once per attempt and is outside the HTTP transport path.

⸻

5. What Happens at before-send
   1. Retrieve the JWT from the request context.
   2. Attach it as an HTTP header (e.g., Authorization: Bearer …).
   3. Let botocore proceed to send the request to RAJEE.

⸻

6. Why This Placement Is Correct
   1. You do not block or interfere with botocore’s retry/HTTP machinery.
   2. Retries naturally mint fresh JWTs.
   3. Multipart, redirects, and internal retries remain correct.
   4. boto3 usage stays completely normal for callers.

⸻

7. Register the Hooks for Object Operations
   1. Attach both hooks to GetObject.
   2. Repeat for HeadObject, PutObject, DeleteObject, etc., as needed.
   3. Alternatively, attach to all S3 operations and branch only when Bucket/Key exist.

⸻

8. Runtime Flow for a GetObject Call
   1. Your code calls get_object(Bucket, Key).
   2. botocore triggers before-parameter-build.
   3. RAJA is called → JWT minted → stored in context.
   4. botocore builds the HTTP request to RAJEE.
   5. botocore triggers before-send.
   6. JWT header is attached.
   7. Request is sent to RAJEE.
   8. Response flows back normally through boto3.

⸻

9. The Key Mental Model
   • boto3 thinks it is talking to a normal S3 endpoint.
   • RAJEE receives a normal S3 request with a JWT header.
   • RAJA is invisible to boto3’s normal flow, invoked only via event hooks.
   • Callers of the client never know this is happening.