PID-OpenCV Ball Balancing Platform

Control system leveraging PID & computer vision to balance a ball on a platform.

WIP: PID tuning is ongoing. Final results will be added soon.

Contents

• Overview
• Architecture
  • Firmware
  • Hardware
• Prerequisites / Installation
• Contact

Overview

I built this project to learn and apply control theory and system design (PID), and embedded image processing (openCV).

The system's main function is to balance a ball by using computer vision to extract its coordinates relative to a user-selected region of interest (ROI). These coordinates are then sent to a control module that computes the correctional Servo angles for both axes. The system's firmware was coded in a mix of Python and Arduino.

Built With

• 
• 
• 
• 

Architecture

System

The system's design is modular, it conists of three sections:

Vision

This module processes the webcam stream, and detects, tracks, and returns the ball's positional coordinates.

• ROI Selection, HUD Creation
• Object Binarization, Gaussian Blur, Colour Masking, Contour Detection /li>
• ROI-Relative Position Extraction (coords)

Control

The control module computes the motor correction angles with the ball's positional coordinates via the PID algorithm/equation.

• Compute correctional angles with PID on each axis (2 controllers)
• Convert to servo-writeable angles

Actuation

The actuation module receives the correction angles, applies smoothing, and writes to each axis motor.

• Read incoming serial data from Python-side (correction angles)
• Smooth and write angles to each motor (X and Y)

Firmware

System was written in Python and Arduino, using pyserial to connect both sides.

Code workflow consists of user selection of the ROI (region of interest), ball binarization, detection, and position extraction. Positions are then sent to the control script to calculate the correctional angles which are sent to the Arduino side via serial to be smoothed and applied to each motor.

• The PID parameters along with any scaling/smoothing factors are tuned to optimized the system.

As seen in the Python directory within Firmware, there are code modules responsible for each step (vision, pid, serial, etc.).

Here's an example of a vision module function:

binarize_ball()

# this function is used to "binarize" the ball, converting the ball region into a clean binary mask by highlighting the ball based on its color (orange in my case).


def _binarize_ball(self, roi_bgr):
    # colour masking
    blurred = cv2.GaussianBlur(roi_bgr, (5,5), 0)
    hsv = cv2.cvtColor(blurred, cv2.COLOR_BGR2HSV)

    lower = np.array([0, 120, 120])
    upper = np.array([25, 255, 255])
    mask = cv2.inRange(hsv, lower, upper)

    # cleaning 
    mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, self.kernel)
    mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, self.kernel)

    return mask

A snippet from the Arduino (actuation) module:

handleServoCommand()

// this functions takes the command (angles) sent by the Python side via serial, applies smoothing, and writes them to the motors. 

void handleServoCommand(String command) {
  int spaceIndex = command.indexOf(' ');
  if (spaceIndex == -1) return;  

  int angleX = command.substring(0, spaceIndex).toInt();
  int angleY = command.substring(spaceIndex + 1).toInt();

  smoothedX = alpha * angleX + (1 - alpha) * smoothedX;
  smoothedY = alpha * angleY + (1 - alpha) * smoothedY;

  
  SERVO_X.write(int(smoothedX));
  SERVO_Y.write(int(smoothedY));
}

Hardware

The system's hardware can be broken down into electrical and mechanical components.

• The electric components feature an ELEGOO Uno R3 MCU for central control (serial, angle writing), and two 20kg DS3218 Servo motors; all powered by 6V 2A power from an AC Adapter.

• The mechanical components were mostly custom designed and 3D-Printed with Fusion360. These include a rising bracket/mount for the Servos, L-brackets to connect the Servo arms to the platform, and the system's central base. The acrylic platform was purchased seperately and drilled through for connection.

Prerequisites / Installation

Prerequisites

• Arduino IDE: with board support for your MCU
• Python environment
• pip for Python

Installation & Usage

1. Clone the repository:

git clone https://github.com/jeevan9s/pid-cv-balance-sys.git
cd pid-cv-balance-sys

2. Install Python dependencies:

pip install -r requirements.txt

3. Upload the Arduino sketch to your microcontroller using the Arduino IDE. Make sure the correct board and port are selected.

4. Run the main script:

python main.py

5. Select an ROI manually with your cursor and tune parameters in config file according to the observed behaviour.

thanks for reading!

Contact

email    LinkedIn