Java - Protocol Buffers

This application consist in a REST api with a single endpoint that consumes a JSON object and save its data into files using Google Protocol buffers, the data is rolled over every 24 hours at midnight.

This app uses mainly the following technologies:

• Java
• Maven
• Grizzly
• JAX-RS
• JSON
• Google Protocol Buffers
• Jetty Utils
• Bash
• Docker
• Docker compose

Build process:

This application uses maven to make the build.

The code generation of the Protocol Buffers is integrated to the build process using a maven plugin. The proto files are located in 'src/main/java/proto' and the generated code goes to the 'generated-sources' inside the 'target' folder.

Storage

The binaries of the messages are stored is files. Files are stored under a folder called 'pb-data', that is created at the same level as the root application folder. It does not matter if the application is running in the local machine or in a docker container, for the container cases a volume is used to share data.

Files are rolled over every 24 hours at midnight, that means that at midnight a new file is created with the new date.

File names has the format:

person_messages_yyyy_mm_dd.bin

Example:

person_messages_2019_01_30.bin

If a new request comes in the same day, the data is stored in the same file but if it comes in a new day, a new file is created with the new date.

Example:

../pb-data/person_messages_2019_01_30.bin
../pb-data/person_messages_2019_01_31.bin

For that functionality is used the RolloverFileOutputStream provided by jetty utils.

Docker Info

There is Dockerfile provided to build and run the application, it is a multi-stage build definition.

The first stage called 'builder' make the application build using maven, and it is based on alpine with maven and the JDK.

The second stage called 'deploy' launch the application and it is based on alpine with the JRE. To run the application it executes a bash script called "launch_app.sh" that can be found under the "scripts" folder and it just execute the application jar.

Docker compose Info

The docker-compose file contains a single service, it starts a container with the application running.

It use the Dockerfile provided to build the image that is used to run the container.

It define a volume between the container and the host machine to store the data generated by the application. The path in the host machine is a folder called 'pb-data' that is located at the same level as the root application folder.

The application is exposed in the '8080' port.

Running application

Running app using docker compose:

Prerequisites:

Docker, docker-compose 3.4 or higher.

Go to the root application folder and execute the following command:

docker-compose up --build

Running app locally

Prerequisites:

Java 8 or higher, Maven 3.x (tested with 3.6.0), Protobuf 3 (tested with 3.6.1).

Go to the root application folder.

To build application:

mvn clean package

To run application:

mvn exec:java

Or:

java -jar target/simple-app-1.0.0-SNAPSHOT.jar

How to use

Create a request to the following URL:

http://localhost:8080/api/person

The request body must contain a JSON Object with the following format:

{"name": "<name>", "id": "<number>"}

A successful request will store the data, see the Storage section for more details.

It is provided a script to test the application, it creates a request to the api using curl, execute the script with the following command from the root application folder.

./scripts/test_request.sh

To do

• Create a volume to share the .m2 data with the container so it do not have to download all the dependencies every time the application is built.

• Refactoring the protobuf utils code to be more generic in case there are going to be more types of messages in the future.

• Add more unit test.