🛡️ FoodGuard

AI-Powered Food Fraud Detection System

Detect AI-generated, compressed, and tampered food images using deep learning forensics.

---

📌 Overview

FoodGuard is a 4-class deep learning system that classifies food images as:

Class	Description
🟢 Real	Genuine, unedited food photographs
🔴 Perfect AI	High-quality AI-generated food images (no post-processing)
🟡 Compressed AI	AI-generated images degraded by JPEG compression & resizing
🟠 Edited AI	Real images tampered via AI inpainting (e.g., cockroach, mold inserted)

Goal: Achieve ≤ 5% False Positive Rate — genuine food photos must NOT be wrongly flagged.

---

🏗️ System Architecture

graph TB
    subgraph Data Collection
        K1["Kaggle: Food-101<br/>~101K images"]
        K2["Kaggle: Indian Food<br/>~4K images"]
        K3["Kaggle: Food Image Dataset<br/>~86K images"]
    end

    subgraph AI Generation
        G1["RealVisXL V4.0<br/>Text-to-Image"]
        G2["SDXL Inpainting<br/>Fraud Objects"]
    end

    K1 & K2 & K3 --> CSV["build_csv.py<br/>dataset_index.csv"]
    CSV --> ORG["organize_4class_dataset.py"]
    G1 --> C1["class1: Perfect AI<br/>600 imgs"]
    G1 --> C2["class2: Compressed AI<br/>600 imgs"]
    G1 --> C3["class3: Degraded AI<br/>400 imgs"]
    G2 --> C4["class4: Edited Real<br/>500+ imgs"]

    ORG --> DS["dataset_4class/<br/>train / val / test<br/>70% / 15% / 15%"]
    C1 & C2 & C3 & C4 --> DS

    DS --> TR["train_4class_detector.py<br/>EfficientNet-B3 + AMP"]
    TR --> CK["checkpoints/<br/>food_ai_detector.pth"]
    CK --> INF["inference.py<br/>Threshold-Calibrated Prediction"]

    style DS fill:#1a1a2e,stroke:#e94560,color:#fff
    style TR fill:#0f3460,stroke:#e94560,color:#fff
    style INF fill:#16213e,stroke:#00d2ff,color:#fff

Loading

---

🔬 Training Pipeline

flowchart LR
    A["📂 Load Dataset<br/>ImageFolder"] --> B["🔄 Transforms<br/>512×512, Normalize"]
    B --> C["🧠 EfficientNet-B3<br/>Pretrained ImageNet"]
    C --> D["📉 Weighted CE Loss<br/>[1.2, 1.0, 1.0, 1.0]"]
    D --> E["⚡ AdamW + AMP<br/>lr=3e-4"]
    E --> F["📊 Cosine Scheduler<br/>20 Epochs"]
    F --> G{"Val Accuracy<br/>Improved?"}
    G -- Yes --> H["💾 Save Best Model"]
    G -- No --> I["Continue Training"]
    H --> J["🎯 Threshold Calibration<br/>Target FPR ≤ 5%"]
    J --> K["✅ Final Evaluation<br/>Confusion Matrix + Metrics"]

Loading

---

🧬 Model Architecture

graph LR
    IMG["Input Image<br/>512 × 512 × 3"] --> EN["EfficientNet-B3<br/>~12M params"]
    EN --> GAP["Global Avg Pool<br/>1536-d"]
    GAP --> FC["Linear<br/>1536 → 4"]
    FC --> SM["Softmax"]
    SM --> R["P(real)"]
    SM --> P["P(perfect_ai)"]
    SM --> C["P(compressed_ai)"]
    SM --> E["P(edited_ai)"]

    R --> TH{"P(real) > θ ?"}
    TH -- Yes --> REAL["✅ REAL"]
    TH -- No --> AI["⚠️ AI Detected<br/>argmax of AI classes"]

    style REAL fill:#00c853,stroke:#00c853,color:#fff
    style AI fill:#ff1744,stroke:#ff1744,color:#fff

Loading

---

🗂️ Project Structure

FoodGuard/
├── config/
│   └── category_mapping.yaml      # Food category mapping
├── src/
│   ├── data/
│   │   ├── dataset_loader.py      # Generic food dataset loader
│   │   ├── detector_dataset.py    # 4-class detector dataset
│   │   ├── augmentations.py       # Training transforms
│   │   └── ela.py                 # Error Level Analysis (forensic)
│   └── models/
│       ├── food_classifier.py     # Base food classifier
│       ├── dual_stream_detector.py# RGB + FFT dual-stream model
│       ├── focal_loss.py          # Focal Loss implementation
│       └── trainer.py             # Training loop manager
├── scripts/
│   ├── build_csv.py               # Build master dataset CSV
│   ├── build_detector_csv.py      # Build 4-class detector CSV
│   ├── organize_4class_dataset.py # Organize into train/val/test
│   ├── generate_ai_images.py      # AI food image generation (SDXL)
│   ├── generate_fraud_inpainting.py # Fraud object inpainting
│   ├── generate_fraud_simple.py   # Overlay-based fraud fallback
│   └── validate_csv.py            # Dataset validation checks
├── train_4class_detector.py       # 🚀 Main training script
├── evaluate.py                    # Model evaluation & metrics
├── inference.py                   # Single-image inference
├── requirements.txt               # Python dependencies
└── README.md

---

🚀 Quick Start

1. Install Dependencies

pip install -r requirements.txt

2. Prepare Dataset

# Build the unified CSV from raw datasets
python scripts/build_csv.py

# Organize into 4-class train/val/test splits (70/15/15)
python scripts/organize_4class_dataset.py

3. Train the Model

python train_4class_detector.py

Training uses mixed-precision (AMP) on CUDA automatically. Checkpoints are saved to checkpoints/food_detector/.

4. Run Inference

python inference.py path/to/food_image.jpg

Sample Output:

============================================================
Image: test_burger.jpg
============================================================
Prediction:  REAL
Confidence:  94.32%
Is Fake:     NO

Class Probabilities:
  real           :  94.32% █████████████████████████████████████████████
  perfect_ai     :   3.21% █
  compressed_ai  :   1.87% 
  edited_ai      :   0.60% 
============================================================

---

🎯 AI Image Generation

FoodGuard generates its own training data using RealVisXL V4.0 (SG161222/RealVisXL_V4.0):

graph TD
    subgraph "Text-to-Image Generation"
        P["Curated Food Prompts<br/>+ Quality Modifiers"] --> SDXL["RealVisXL V4.0<br/>25 steps, cfg=5-7.5"]
        SDXL --> RAW["Class 1: Raw AI<br/>512×512 PNG"]
        SDXL --> COMP["Class 2: Compressed<br/>JPEG q=40-85, resize"]
        SDXL --> DEG["Class 3: Degraded<br/>Blur + Noise"]
    end

    subgraph "Inpainting Generation"
        REAL["Real Food Image<br/>from clean pool"] --> MASK["Irregular Mask<br/>2-4% area, center-biased"]
        MASK --> INP["SDXL Inpainting<br/>cfg=4.5, strength=0.99"]
        INP --> EDIT["Class 4: Edited<br/>Fraud object inserted"]
    end

    style RAW fill:#e3f2fd,stroke:#1565c0,color:#000
    style COMP fill:#fff3e0,stroke:#e65100,color:#000
    style DEG fill:#fce4ec,stroke:#c62828,color:#000
    style EDIT fill:#ffebee,stroke:#b71c1c,color:#000

Loading

Fraud Objects: cockroach, housefly, mosquito, bee, ant, worm, human hair, mold, plastic fragment, paper piece, metal shard.

---

📊 Data Sources

Dataset	Source	Images	Cuisine Coverage
Food-101	Kaggle (ETH Zurich)	~101,000	Western, International
Indian Food Dataset	Kaggle	~4,000	Indian (biryani, paneer, etc.)
Food Image Dataset	Kaggle (UECFOOD256 + AIcrowd)	~86,000	Japanese, Mixed
AI-Generated	RealVisXL V4.0 (local)	~2,000	Multi-cuisine
AI-Inpainted Fraud	SDXL Inpainting (local)	~550	Multi-cuisine

Total Real Images: ~191,000+  |  Sampled for Training: 5,000 (balanced prototype)

---

⚙️ Training Configuration

Parameter	Value	Rationale
Backbone	EfficientNet-B3	Best accuracy-per-param; fits 12GB VRAM
Image Size	512 × 512	Preserves forensic artifacts vs 224×224
Batch Size	16	Max for 512×512 on 12GB
Optimizer	AdamW	Better weight decay for fine-tuning
Learning Rate	3e-4	Standard for timm fine-tuning
Scheduler	Cosine Annealing (T=20)	Smooth decay, no sudden drops
Loss	Cross-Entropy	Weights: [1.2, 1.0, 1.0, 1.0] — penalizes FP on real
AMP	Enabled	~2× speed, ~40% less VRAM
Epochs	20	With early stopping
Target FPR	≤ 5%	Calibrated via threshold sweep

---

🧪 Why 4 Classes, Not Binary?

graph TD
    B["Binary Classifier<br/>Real vs Fake"] --> L1["❌ Loses WHY it's fake"]
    B --> L2["❌ Misses subtle edits<br/>95% real, 5% inpainted"]

    F["4-Class Classifier"] --> W1["✅ Detects fully AI-generated"]
    F --> W2["✅ Handles compressed AI<br/>social media sharing"]
    F --> W3["✅ Catches subtle inpainting<br/>deliberate fraud"]
    F --> W4["✅ Maintains low FPR<br/>on genuine photos"]

    style B fill:#ffcdd2,stroke:#c62828,color:#000
    style F fill:#c8e6c9,stroke:#2e7d32,color:#000

Loading

---

🗺️ Roadmap

• Real food data collection (3 Kaggle datasets, ~191K images)
• AI image generation pipeline (RealVisXL V4.0)
• Fraud inpainting pipeline (SDXL Inpainting)
• Data processing & 4-class organization
• Training script with AMP and threshold calibration
• Model training & hyperparameter tuning
• Threshold calibration for ≤5% FPR
• Evaluation (confusion matrix, per-class metrics)
• Grad-CAM explainability visualization
• Dual-stream model (RGB + FFT frequency analysis)
• REST API deployment (FastAPI)

---

🔧 Tech Stack

Technology	Purpose
PyTorch	Deep learning framework
timm	EfficientNet model zoo
HuggingFace Diffusers	SDXL text-to-image & inpainting
RealVisXL V4.0	Photorealistic image generation
scikit-learn	Metrics & evaluation
Pillow	Image I/O and processing
xformers	Memory-efficient attention for SDXL
CUDA AMP	Mixed-precision training
matplotlib / seaborn	Visualization

---

🌍 Real-World Applications

• Food Delivery Apps — Detect fraudulent complaint images (fake contaminants for refunds)
• Restaurant Reviews — Filter AI-manipulated food photos
• Food Safety Agencies — Verify authenticity of food complaint evidence
• Social Media — Flag AI-generated food content for transparency

---

👥 Team

Member	Role
Raj	Architecture, AI generation, training pipeline, inference
Rahul	Data validation, category mapping, evaluation scripts
Aman	Dataset management, augmentation testing, dual-stream model

---

Project
Built with 🔬 PyTorch and ☕ caffeine