HugeGraph PD Development Guide

This document provides comprehensive guidance for developing, testing, and contributing to HugeGraph PD.

Development Environment Setup
Building from Source
Testing
Development Workflows
Code Style and Standards
Debugging
Contributing

Development Environment Setup

Prerequisites

Ensure you have the following tools installed:

Tool	Minimum Version	Recommended	Purpose
JDK	11	11 or 17	Java runtime and compilation
Maven	3.5.0	3.8+	Build tool and dependency management
Git	2.0+	Latest	Version control
IDE	N/A	IntelliJ IDEA	Development environment

Verify Installation

# Check Java version
java -version
# Expected: openjdk version "11.0.x" or later

# Check Maven version
mvn -version
# Expected: Apache Maven 3.5.0 or later

# Check Git version
git --version
# Expected: git version 2.x

Clone Repository

# Clone HugeGraph repository
git clone https://github.com/apache/hugegraph.git
cd hugegraph

# PD module location
cd hugegraph-pd

IDE Setup (IntelliJ IDEA)

Import Project

Open IntelliJ IDEA
File → Open → Select hugegraph-pd directory
Wait for Maven to download dependencies (may take 5-10 minutes)

Configure Code Style

File → Settings → Editor → Code Style
Ensure Enable EditorConfig support is checked
Apply and OK

Enable Annotation Processing

Required for Lombok support:

File → Settings → Build, Execution, Deployment → Compiler → Annotation Processors
Check Enable annotation processing
Apply and OK

Configure JDK

File → Project Structure → Project
Project SDK: Select JDK 11 or 17
Project language level: 11
Apply and OK

Building from Source

Full Build

Build all PD modules from the hugegraph-pd directory:

cd hugegraph-pd
mvn clean install -DskipTests

Output:

JARs in each module's target/ directory
Distribution package: hg-pd-dist/target/hugegraph-pd-<version>.tar.gz

Build Time: 2-5 minutes (first build may take longer for dependency download)

Module-Specific Builds

Build individual modules:

# Build gRPC module only (regenerate proto stubs)
mvn clean compile -pl hg-pd-grpc

# Build core module only
mvn clean install -pl hg-pd-core -am -DskipTests

# Build service module only
mvn clean install -pl hg-pd-service -am -DskipTests

# Build distribution package only
mvn clean package -pl hg-pd-dist -am -DskipTests

Maven Flags:

-pl <module>: Build specific module
-am: Also build required dependencies (--also-make)
-DskipTests: Skip test execution (faster builds)
-Dmaven.test.skip=true: Skip test compilation and execution

Clean Build

Remove all build artifacts:

mvn clean

# This also removes:
# - *.tar, *.tar.gz files
# - .flattened-pom.xml (CI-friendly versioning)

Build from Project Root

Build PD from HugeGraph root directory:

cd /path/to/hugegraph

# Build PD and dependencies
mvn clean package -pl hugegraph-pd -am -DskipTests

Testing

Test Organization

PD tests are located in hg-pd-test/src/main/java/ (non-standard location):

hg-pd-test/src/main/java/org/apache/hugegraph/pd/
├── BaseTest.java               # Base test class with common setup
├── core/                       # Core service tests
│   ├── PartitionServiceTest.java
│   ├── StoreNodeServiceTest.java
│   └── ...
├── client/                     # Client library tests
├── raft/                       # Raft integration tests
└── PDCoreSuiteTest.java        # Test suite (runs all tests)

Running Tests

All Tests

# Run all PD tests
mvn test

# Run all tests with coverage report
mvn test jacoco:report
# Coverage report: hg-pd-test/target/site/jacoco/index.html

Module-Specific Tests

# Core module tests
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-core-test

# Client module tests
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-client-test

# Common module tests
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-common-test

# REST API tests
mvn test -pl hugegraph-pd/hg-pd-test -am -P pd-rest-test

Single Test Class

# Run specific test class
mvn -pl hugegraph-pd/hg-pd-test test -Dtest=PartitionServiceTest -DfailIfNoTests=false

# Run specific test method
mvn -pl hugegraph-pd/hg-pd-test test -Dtest=PartitionServiceTest#testSplitPartition -DfailIfNoTests=false

Test from IDE

IntelliJ IDEA:

Open test class (e.g., PartitionServiceTest.java)
Right-click on class name or test method
Select Run 'PartitionServiceTest'

Test Coverage

View test coverage report:

# Generate coverage report
mvn test jacoco:report

# Open report in browser
open hg-pd-test/target/site/jacoco/index.html

Target Coverage:

Core services: >80%
Utility classes: >70%
Generated gRPC code: Excluded from coverage

What Integration Tests Cover:

Raft cluster formation and leader election
Partition allocation and balancing
Store registration and heartbeat processing
Metadata persistence and recovery
gRPC service interactions

Debugging Raft Issues

Enable detailed Raft logging in conf/log4j2.xml:

<Loggers>
    <!-- Enable Raft debug logging -->
    <Logger name="com.alipay.sofa.jraft" level="DEBUG"/>

    <!-- Specific Raft components -->
    <Logger name="com.alipay.sofa.jraft.core.NodeImpl" level="DEBUG"/>
    <Logger name="com.alipay.sofa.jraft.storage.impl" level="DEBUG"/>
</Loggers>

Raft State Inspection:

# Check Raft data directory
ls -lh pd_data/raft/

# Raft logs
ls -lh pd_data/raft/log/

# Raft snapshots
ls -lh pd_data/raft/snapshot/

Common Raft Issues:

Issue	Symptom	Solution
Split-brain	Multiple leaders	Check `peers-list` consistency, network partitioning
Leader election failure	Constant candidate state	Check network latency, increase election timeout
Log replication lag	Followers behind leader	Check follower disk I/O, network bandwidth
Snapshot transfer failure	Followers can't catch up	Check snapshot directory permissions, disk space

Code Style and Standards

Code Formatting

HugeGraph PD follows Apache HugeGraph code style.

IDE Configuration:

IntelliJ IDEA: File → Settings → Editor → Code Style
Ensure Enable EditorConfig support is checked

Key Style Rules:

Indentation: 4 spaces (no tabs)
Line length: 100 characters (Java), 120 characters (comments)
Braces: K&R style (opening brace on same line)
Imports: No wildcard imports (import java.util.*)

License Headers

All source files must include Apache License header.

Check License Headers:

mvn apache-rat:check

# Output: target/rat.txt (lists files missing license headers)

Add License Header: Manually add to new files:

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

Naming Conventions

Type	Convention	Example
Classes	PascalCase	`PartitionService`, `StoreNodeService`
Interfaces	PascalCase (prefix with `I` optional)	`MetadataStore` or `IMetadataStore`
Methods	camelCase	`getPartition()`, `registerStore()`
Variables	camelCase	`storeId`, `partitionCount`
Constants	UPPER_SNAKE_CASE	`MAX_RETRY_COUNT`, `DEFAULT_TIMEOUT`
Packages	lowercase	`org.apache.hugegraph.pd.core`

JavaDoc

Public APIs must include JavaDoc comments.

Example:

/**
 * Get partition by partition code.
 *
 * @param graphName   the graph name
 * @param partitionId the partition ID
 * @return the partition metadata
 * @throws PDException if partition not found or Raft error
 */
public Partition getPartitionByCode(String graphName, int partitionId) throws PDException {
    // Implementation...
}

Required for:

All public classes and interfaces
All public and protected methods
Complex private methods (optional but recommended)

Error Handling

Use custom PDException for PD-specific errors.

Example:

if (store == null) {
    throw new PDException(ErrorType.STORE_NOT_FOUND,
                         "Store not found: " + storeId);
}

Exception Hierarchy:

PDException: Base exception for all PD errors
RaftException: Raft-related errors (from JRaft)
GrpcException: gRPC communication errors

Debugging

Local Debugging in IDE

Run PD from IDE

Create run configuration in IntelliJ IDEA:
- Run → Edit Configurations
- Add New Configuration → Application
- Main class: org.apache.hugegraph.pd.HgPdApplication (in hg-pd-service)
- Program arguments: --spring.config.location=file:./conf/application.yml
- Working directory: hugegraph-pd/hg-pd-dist/target/hugegraph-pd-<version>/
- JRE: 11 or 17
Set breakpoints in code
Click Debug (Shift+F9)

Debug Tests

Open test class (e.g., PartitionServiceTest.java)
Set breakpoints
Right-click on test method → Debug 'testMethod'

Remote Debugging

Debug PD running on a remote server.

Start PD with Debug Port:

bin/start-hugegraph-pd.sh -j "-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005"

Connect from IDE:

Run → Edit Configurations → Add New → Remote JVM Debug
Host: PD server IP
Port: 5005
Debugger mode: Attach
Click Debug

Logging

Increase log verbosity for troubleshooting.

Edit conf/log4j2.xml:

<Loggers>
    <!-- Enable DEBUG logging for PD -->
    <Logger name="org.apache.hugegraph.pd" level="DEBUG"/>

    <!-- Enable DEBUG for specific package -->
    <Logger name="org.apache.hugegraph.pd.core.PartitionService" level="DEBUG"/>
</Loggers>

View Logs:

# Real-time log monitoring
tail -f logs/hugegraph-pd.log

# Search logs
grep "ERROR" logs/hugegraph-pd.log
grep "PartitionService" logs/hugegraph-pd.log

Performance Profiling

Use JVM profiling tools to identify performance bottlenecks.

Async-profiler (recommended):

# Download async-profiler
wget https://github.com/jvm-profiling-tools/async-profiler/releases/download/v2.9/async-profiler-2.9-linux-x64.tar.gz
tar -xzf async-profiler-2.9-linux-x64.tar.gz

# Profile running PD process
./profiler.sh -d 60 -f /tmp/pd-profile.svg <PD_PID>

# View flamegraph
open /tmp/pd-profile.svg

JProfiler:

Download JProfiler from https://www.ej-technologies.com/products/jprofiler/overview.html
Attach to running PD process
Analyze CPU, memory, and thread usage

Contributing

Contribution Workflow

Fork Repository:
- Fork https://github.com/apache/hugegraph on GitHub

Clone Your Fork:

git clone https://github.com/YOUR_USERNAME/hugegraph.git
cd hugegraph

Create Feature Branch:

git checkout -b feature/your-feature-name

Make Changes:
- Write code
- Add tests
- Update documentation

Run Tests:

mvn test -pl hugegraph-pd/hg-pd-test -am

Check Code Style:
```
mvn apache-rat:check
```

Commit Changes:

git add .
git commit -m "feat(pd): add new feature description"

Commit Message Format:

<type>(<scope>): <subject>

<body>

<footer>

Types: feat, fix, docs, style, refactor, test, chore

Example:

feat(pd): add partition auto-splitting

- Implement partition split threshold detection
- Add split operation via Raft proposal
- Update partition metadata after split

Closes #123

Push to Fork:

git push origin feature/your-feature-name

Create Pull Request:
- Go to https://github.com/apache/hugegraph
- Click New Pull Request
- Select your fork and branch
- Fill in PR description (what, why, how)
- Submit PR

Pull Request Guidelines

PR Title Format:

[PD] Brief description of changes

PR Description Template:

## What changes were proposed in this pull request?
<!-- Describe the changes in detail -->

## Why are the changes needed?
<!-- Explain the motivation and context -->

## How was this patch tested?
<!-- Describe the testing process -->

## Related Issues
<!-- Link to related issues: Closes #123 -->

Before Submitting:

Tests pass locally
Code style is correct (mvn apache-rat:check)
JavaDoc added for public APIs
Documentation updated (if applicable)
Commit messages follow convention

Code Review Process

Automated Checks:
- CI builds and tests PR
- Code style validation
- License header check
Reviewer Feedback:
- Address reviewer comments
- Push updates to same branch
- PR automatically updates
Approval:
- At least 1 committer approval required
- All CI checks must pass
Merge:
- Committer merges PR
- Delete feature branch

Additional Resources

Documentation

Architecture Documentation - System design and components
API Reference - gRPC APIs and examples
Configuration Guide - Configuration options and tuning

Community

Mailing List: dev@hugegraph.apache.org
GitHub Issues: https://github.com/apache/hugegraph/issues
GitHub Discussions: https://github.com/apache/hugegraph/discussions

Useful Commands

# Quick build (no tests)
mvn clean install -DskipTests -pl hugegraph-pd -am

# Run specific test
mvn test -pl hugegraph-pd/hg-pd-test -am -Dtest=PartitionServiceTest

# Generate coverage report
mvn test jacoco:report -pl hugegraph-pd/hg-pd-test -am

# Check license headers
mvn apache-rat:check -pl hugegraph-pd

# Package distribution
mvn clean package -DskipTests -pl hugegraph-pd/hg-pd-dist -am

# Clean all build artifacts
mvn clean -pl hugegraph-pd

Summary

This guide covers:

Setup: Environment configuration and IDE setup
Building: Maven commands for full and module-specific builds
Testing: Running tests and viewing coverage reports
Development: Adding gRPC services, metadata stores, and modifying core logic
Debugging: Local and remote debugging, logging, profiling
Contributing: Workflow, PR guidelines, and code review process

For questions or assistance, reach out to the HugeGraph community via mailing list or GitHub issues.

Happy coding!

FilesExpand file tree

development.md

Latest commit

History

development.md

File metadata and controls

HugeGraph PD Development Guide

Table of Contents

Development Environment Setup

Prerequisites

Verify Installation

Clone Repository

IDE Setup (IntelliJ IDEA)

Import Project

Configure Code Style

Enable Annotation Processing

Configure JDK

Building from Source

Full Build

Module-Specific Builds

Clean Build

Build from Project Root

Testing

Test Organization

Running Tests

All Tests

Module-Specific Tests

Single Test Class

Test from IDE

Test Coverage

Debugging Raft Issues

Code Style and Standards

Code Formatting

License Headers

Naming Conventions

JavaDoc

Error Handling

Debugging

Local Debugging in IDE

Run PD from IDE

Debug Tests

Remote Debugging

Logging

Performance Profiling

Contributing

Contribution Workflow

Pull Request Guidelines

Code Review Process

Additional Resources

Documentation

Community

Useful Commands

Summary