Deploy grants tagger with sage #4
Description
We might need to update the README but worth checking that deployment works for you first @ArneRobben. Note that you need a couple of steps for deploying custom containers like the wellcome bert mesh model which are out of scope for this tool but should probably neverthless be either in the documentatinon or as a example scripts
sage deploy IMAGE_URI text-classification ROLE --endpoint-name test-wellcome-bert-mesh
You need to first build and push a container to pass the IMAGE_URI
Build
API
#!/usr/bin/env python
from transformers import AutoModel, AutoTokenizer
from fastapi import FastAPI
from pydantic import BaseModel
class Input(BaseModel):
text: str
app = FastAPI()
tokenizer = AutoTokenizer.from_pretrained(
"Wellcome/WellcomeBertMesh"
)
model = AutoModel.from_pretrained(
"Wellcome/WellcomeBertMesh",
trust_remote_code=True
)
@app.post("/invocations")
async def predict(input: Input):
inputs = tokenizer([input.text], padding="max_length")
labels = model(**inputs, return_labels=True)
return labels[0]
@app.get("/ping")
def health():
return "\n"
Dockerfile
FROM python:3.8.10-slim-buster
COPY requirements.txt .
RUN pip install -r requirements.txt
RUN pip install torch==2.0.0 --index-url https://download.pytorch.org/whl/cpu
COPY api.py .
COPY run_api.sh /usr/bin/serve
RUN chmod 755 /usr/bin/serve
EXPOSE 8080
Build
#!/bin/bash
# Amd64 machine (most common)
#docker build -t custom-container:latest .
# Apple M1 machine or non-amd64 machine
docker buildx build --platform linux/amd64 -t custom-container:latest .
Push
Note you need to create a ECR repository, in this examples this is wellcome-custom-containers
aws ecr get-login-password --region eu-west-1 | docker login --username AWS --password-stdin WELLCOME_ACCOUNT_ID.dkr.ecr.eu-west-1.amazonaws.com
docker tag custom-container:latest WELLCOME_ACCOUNT_ID.dkr.ecr.eu-west-1.amazonaws.com/wellcome-custom-containers
docker push WELLCOME_ACCOUNT_ID.dkr.ecr.eu-west-1.amazonaws.com/wellcome-custom-containers
Test
sage predict test-wellcome-bert-mesh "This grant is about malaria"