A simple distributed task scheduler written in Go. It's designed to handle large volume of tasks and distribute them across multiple workers for execution.
- Go
- gRPC & protocol buffers
- PostgreSQL
- Docker
- Scheduler - the front-end server of the system that receives tasks from clients and schedules them for execution.
- Coordinator - responsible for picking up tasks and distributing them across available workers to execute.
- Workers - responsible for executing tasks, they register and communicate with coordinator using heartbeats.
- Database - responsible for storing tasks and their information, like task id, their life cycle times, like scheduled at, picked at, completed at, etc.
The scheduler communicates with clients via HTTP, while the coordinator and workers use gRPC for faster, strongly-typed, and efficient communication with low latency.
First, spin up a cluster using this command -
docker compose --build --scale worker=3
This command builds the required Docker images and then starts up a cluster with 3 workers.
To schedule a task, send a POST request to localhost:8081/schedule with a JSON body including:
"command": A string representing the command to be executed."scheduled_at": A string representing the current time in ISO 8601 format. This request returns a response including the"task_id", which can be used to query the status of the task.
curl -X POST localhost:8081/schedule -d '{"command":"pwd","scheduled_at":"2025-10-02T17:10:00+05:30"}'
To get the status of a scheduled task, send a GET request to localhost:8081/status?task_id=<task-id> where <task-id> is the ID of the task returned by the schedule request.
I built this as a learning project to understand task scheduling, Go concurrency, and gRPC communication. Inspired by this article, which guided the architecture and implementation.