Our Github
Mission: Reduce car related injuries and deaths
Our project for https://hackdavis.io/event 2024.
- Use Intel Developer Cloud to train AI 🚀
- Performance gains (reduce PyTorch train time by 4 minutes) with bfloat16 and ipex ⚡
- Tensorflow performance gains (train time going from 2:23s to 1:14s) Almost twice as fast with ipex!
- Use ipex with PyTorch in IDC
- Make three different OpenCV models for lane detection, human obstacle detection, and driver alertness detection
Use a forward facing camera and OpenCV to recognize lanes and alert the driver if they do not stay within their lane.
Use a driver facing camera and OpenCV to detect if the driver is awake and paying attention to the road
Use PyTorch and Intel Developer Cloud Notebook to detect pedestrians walking infront and next to the car and alert the driver if they get too close. We were able to use Intel's ipex, PyTorch plugin, and bfloat16 to reduce the training time by 4 entire minutes. We used ipex from Intel AI and the Intel PyTorch plugin to leverage Intel AMX.
import torch
import intel_extension_for_pytorch as ipex
from engine import train_one_epoch, evaluate
import datetime
# ...
params = [p for p in model.parameters() if p.requires_grad]
optimizer = torch.optim.SGD(
params,
lr=0.005,
momentum=0.9,
weight_decay=0.0005
)
lr_scheduler = torch.optim.lr_scheduler.StepLR(
optimizer,
step_size=3,
gamma=0.1
)
model = get_model_instance_segmentation(num_classes)
model, optimizer = ipex.optimize(model, optimizer=optimizer, dtype=torch.float32)
model = model.to(device)Optimization Experiment:
Start time: 2024-04-27 16:27:36.983459
End time: 2024-04-27 16:47:38.238154
datetime.timedelta(seconds=1201, microseconds=254695)
Start time: 2024-04-27 17:46:16.191538
End time: 2024-04-27 18:02:06.233684
datetime.timedelta(seconds=950, microseconds=42146)
Start time: 2024-04-27 17:02:22.302090
End time: 2024-04-27 17:18:17.402227
datetime.timedelta(seconds=955, microseconds=100137)
As you can see, intel ipex and bfloat16 saved us 4 entire minutes for this model. Imagine a much larger model and the type of time and compute cost savings that could be achived.
We submitted a custom bfloat16 model to the intel leaderboard. Here is the model fine tuned from Gemma
We used LLMs to generate and extend data that we used to fine tune our text directions Gemma model. Hallucinations for directions in this case could cause people to go to places that do not exist. This could be fixed by using existing databases of locations (Like a mapping API) to insure that the LLM is always directing someone to a real place.
Use an Arduino and an ultrasonic distance sensor to alert the driver if they are too close to anything, including another car
