ElasticProxyServer Features Overview

📋 Project Overview

ElasticProxyServer is an enterprise-grade Elasticsearch transparent proxy server built on Java 17 + Spring Boot WebFlux, providing comprehensive proxy forwarding, security control, multi-tenant support, monitoring and auditing capabilities. It adopts a reactive programming model to support high-concurrency, low-latency data access scenarios.

🎯 Core Value Proposition

🛡️ Enterprise Security - 7 categories of search protection metrics with comprehensive multi-layer security protection
🏢 Multi-Tenant Architecture - Complete isolation with full multi-tenant support
⚡ High-Performance Proxy - Non-blocking reactive architecture based on WebFlux
🔍 Intelligent Querying - Unified Search API simplifying complex query operations
📊 Comprehensive Monitoring - Enterprise-grade monitoring, auditing and performance analysis
🔄 Zero-Downtime Operations - Configuration hot reload and self-healing capabilities

🏗️ System Architecture

Overall Architecture Diagram

┌─────────────────────────────────────────────────────────────┐
│                    ElasticProxyServer                        │
├─────────────────────────────────────────────────────────────┤
│  Client Requests  │
│      ↓           │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │              Web Filter Chain (WebFlux)                │ │
│ │  IP Access Control → Rate Limiting → Tenant Auth → Logging │ │
│ └─────────────────────────────────────────────────────────┘ │
│      ↓           │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │                   Routing Layer                         │ │
│ │  Unified Search API  │  Transparent Proxy  │  CCS  │  Admin │ │
│ └─────────────────────────────────────────────────────────┘ │
│      ↓           │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │               Query Analysis & Security                 │ │
│ │  Query Complexity Analysis  │  Permission Validation  │  Index ACL │ │
│ └─────────────────────────────────────────────────────────┘ │
│      ↓           │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │              Backend Connection Management              │ │
│ │  ES Connection Pool  │  Failover  │  Load Balance  │  Tenant Isolation │ │
│ └─────────────────────────────────────────────────────────┘ │
│      ↓           │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │                Monitoring & Auditing                    │ │
│ │  Performance Monitor  │  Query Logs  │  Security Audit  │  Metrics │ │
│ └─────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
           ↓
    Elasticsearch Clusters

Technology Stack

Component Category	Technology	Version	Purpose
Base Framework	Spring Boot	3.3.2	Application foundation
Web Framework	Spring WebFlux	-	Reactive web framework
Network Layer	Reactor Netty	-	Non-blocking network communication
Caching	Caffeine	3.1.8	High-performance in-memory cache
Rate Limiting	Bucket4j	8.9.0	Token bucket rate limiting algorithm
Monitoring	Micrometer	-	Application metrics monitoring
ES Client	Elasticsearch Java Client	8.11.4	ES connection and operations
Configuration	Spring Cloud	2023.0.4	Configuration hot reload support
Build Tool	Maven	3.x	Project build management

🚀 Core Features

1. 🛡️ Enterprise Security Control

IP Access Control

CIDR Network Support: Complete support for IPv4/IPv6 CIDR format (e.g., 192.168.1.0/24)
Whitelist/Blacklist: Flexible access control policies
Dynamic Updates: Configuration hot reload support without service restart

proxy:
  allowList: ["127.0.0.1", "10.0.0.0/24", "192.168.0.0/16"]
  blockList: ["192.168.100.0/24"]  # Block specific subnets

Intelligent Rate Limiting

Token Bucket Algorithm: High-performance rate limiting based on Bucket4j
Multi-Time Dimensions: Support for second, minute, and hour-level rate limiting
Client Isolation: Independent rate limiting quotas calculated per IP address

proxy:
  limits:
    rateLimit: 100/minute  # 100 requests per minute

Query Security Analysis

Complexity Assessment: Intelligent analysis of query complexity to prevent resource abuse
Depth Limitation: Control query DSL nesting depth to avoid stack overflow
Dangerous Query Interception: Automatically identify and block high-risk queries like prefix wildcards and regex

2. 🏢 Multi-Tenant Architecture Support

Complete Tenant Isolation

Independent Authentication: Each tenant uses dedicated HTTP Basic authentication credentials
Backend Isolation: Tenant-specific Elasticsearch connection pools and credentials
Permission Isolation: Tenant-level index access control and operation restrictions

proxy:
  multi-tenant:
    enabled: true
    tenants:
      tenant-logs:
        name: "Log Analysis Tenant"
        basicAuth:
          username: "logs-user"
          password: "logs-secret-2024"
        allowedIps: ["10.1.0.0/24", "192.168.1.0/24"]
        elasticsearch:
          cluster: "logs"
        limits:
          allowedIndices: ["logs-*", "app-logs-*"]

High-Performance Multi-Tenant Implementation

Smart Caching: Tenant context caching (15-minute TTL)
Connection Pool Management: Independent WebClient connection pools per tenant
Configuration Inheritance: Tenants can inherit global configuration with specific overrides

3. 🔍 Unified Search API

Intelligent Query Simplification

Simplify complex Elasticsearch DSL queries into intuitive JSON condition combinations:

Traditional ES Query:

{
  "query": {
    "bool": {
      "filter": [
        {"term": {"orderId": "ORDER_12345"}},
        {"range": {"date": {"gte": "2023-01-01", "lt": "2023-12-31"}}},
        {"match": {"description": "iPhone mobile phone"}}
      ]
    }
  }
}

Unified Search API:

{
  "conditions": [
    {"key": "orderId", "value": "ORDER_12345"},
    {"key": "date", "value": "2023-01-01,2023-12-31"},
    {"key": "description", "value": "iPhone mobile phone"}
  ],
  "relationshipType": "AND"
}

Intelligent Type Recognition

Auto Inference: Automatically identify query types based on field names and value formats
Exact Matching: Order IDs, user IDs, and other identifiers automatically use Term queries
Range Queries: Dates, prices, and numerical values automatically support range filtering
Text Search: Descriptions, comments, and content automatically enable tokenized search

Query Result Caching

High-Performance Cache: Caffeine-based in-memory cache with 80-95% response time improvement
Smart Cache Keys: MD5 hash generated based on index path, query conditions, pagination parameters
Cache Management: TTL settings, maximum entry limits, manual cleanup support

4. 🌐 Cross-Cluster Search Support

CCS Alias Routing

Cluster Aliases: Support access to different ES clusters through aliases
Transparent Routing: Automatically route /logs:index-*/_search to logs cluster
Scroll Support: Complete support for cross-cluster scroll operations

elasticsearch:
  clusters:
    logs:
      url: http://es-logs:9200
      username: "elastic"
      password: "secret"
    metrics:
      url: http://es-metrics:9200
      username: "elastic" 
      password: "secret"

Enhanced Scroll Operations

Automatic Tracking: Intelligent tracking of scroll_id to cluster mapping relationships
Transparent Operations: Subsequent scroll requests automatically route to correct cluster
Memory Management: Built-in cleanup mechanisms to prevent memory leaks

5. ⚡ High Availability Support

Failover Mechanisms

Multi-Node Support: Automatic detection and switching of ES nodes
Intelligent Retry: Exponential backoff retry algorithms
Health Detection: Real-time monitoring of node status

elasticsearch:
  url: http://node1:9200,http://node2:9200  # Multi-node configuration

Connection Pool Optimization

Configurable Parameters: Comprehensive tunability for connections, timeouts, idle time
Performance Tuning: Runtime connection pool parameter adjustment support

6. 📊 Comprehensive Monitoring & Auditing

ES Monitoring Log Integration

Asynchronous Writing: Non-blocking ES log writing without affecting main request performance
Daily Indexing: Automatic creation of elastic-proxy-logs-YYYY-MM-DD indices
Complete Recording: Includes request details, query analysis, performance metrics

{
  "requestId": "550e8400-e29b-41d4-a716-446655440000",
  "timestamp": "2025-10-25T10:30:45.123Z",
  "clientIp": "192.168.1.100",
  "method": "POST",
  "path": "/test1/_search",
  "statusCode": 200,
  "duration": 156,
  "queryAnalysis": {
    "allowed": true,
    "analysis": {
      "complexity": 48,
      "depth": 4,
      "aggregations": "passed"
    }
  }
}

Multi-Tenant Monitoring Enhancement

Tenant Identification: All monitoring data includes complete tenant information
Dual-Write Mode: Support for tenant-specific indices + global index dual-write
Audit Trails: Complete tenant operation audit chains

Performance Metrics Collection

Prometheus Integration: Standardized metrics exposure
Real-Time Monitoring: Key indicators like response time, request count, error rate
Custom Metrics: Support for business-related custom monitoring metrics

7. 🔄 Operations-Friendly Features

Configuration Hot Reload

Zero-Downtime Updates: Update configuration through /actuator/refresh endpoint
Full Component Support: All configuration items support hot reload
Cache Cleanup: Automatic cleanup of related caches when configuration updates

# Trigger hot reload after configuration changes
curl -X POST http://localhost:8000/actuator/refresh

Health Checks

Multi-Dimensional Checks: ES connectivity, service status, resource usage
Probe Support: Kubernetes liveness/readiness probe support
Detailed Diagnostics: Provide detailed health status information

Management Endpoints

Actuator Integration: Complete Spring Boot Actuator support
Tenant Management: Dedicated tenant configuration query and connection test endpoints
Cache Management: Unified search cache statistics and cleanup endpoints

📈 Performance Characteristics

Reactive Architecture Advantages

Feature	Traditional Sync Architecture	WebFlux Reactive Architecture	Performance Gain
Concurrency	Thread pool limitations	Event-driven, no thread blocking	10-20x
Memory Usage	One thread per connection	Shared event loop	5-10x optimization
Latency	I/O blocking delays	Non-blocking async processing	50-80% reduction
Throughput	Limited by thread count	CPU-intensive scaling	3-5x improvement

Cache Performance Optimization

Query Cache: 60-80% cache hit rate, 80-95% response time improvement
Tenant Cache: 15-minute TTL, avoiding repeated authentication overhead
Connection Pool: Reuse long connections, reducing connection establishment overhead

Monitoring Performance Impact

Monitoring Overhead: <1% CPU, <10MB memory
Log Recording: Asynchronous processing, zero blocking
Metrics Collection: Memory-efficient counters and histograms

🛠️ Deployment & Configuration

Quick Start

# 1. Build application
mvn -DskipTests clean package

# 2. Start service
java -jar target/elasticproxy-server-1.5.3.jar \
  --server.port=8000 \
  --elasticsearch.url=http://localhost:9200

Docker Deployment

# Standard mode
./start-docker.sh standard -d

# Multi-tenant mode
./start-docker.sh multi-tenant -d

Core Configuration Example

server:
  port: 8000

elasticsearch:
  url: http://localhost:9200
  username: "elastic"
  password: "secret"
  requestTimeoutSeconds: 30

proxy:
  useElasticsearchMonitoring: true
  allowList: ["127.0.0.1", "10.0.0.0/24"]
  limits:
    rateLimit: 100/minute
    maxResultWindow: 10000
    maxQueryDepth: 5
  
  unified-search:
    enabled: true
    enableQueryCache: true
    queryCacheTtlSeconds: 300
    allowedIndices: ["logs-*", "metrics-*"]
    
  multi-tenant:
    enabled: false  # Disabled by default

📋 Use Cases

1. Enterprise Data Platform

Unified Data Access: Provide unified ES access interface for different business lines
Security Compliance: Meet enterprise-grade security and audit requirements
Performance Optimization: Improve data access efficiency through caching and query optimization

2. SaaS Multi-Tenant Platform

Tenant Isolation: Complete independent tenant data access control
Resource Management: Resource usage restrictions and monitoring by tenant
Flexible Billing: Tenant usage-based billing based on monitoring data

3. Data Security Gateway

Query Protection: Prevent malicious queries from affecting ES cluster performance
Access Auditing: Complete data access audit trails
Permission Control: Fine-grained data access permission control

4. Development & Testing Environment

Simplified Querying: Lower development barriers through Unified Search API
Environment Isolation: ES access isolation for different environments
Debugging Support: Detailed query logs and performance analysis

🔮 Development Roadmap

v1.6.0 Planned Features

Data Masking: Automatic sensitive data masking processing
Query Optimization: AI-driven query performance optimization suggestions
Plugin System: Extensible plugin architecture

🤝 Technical Support

Version Information

Current Version: v1.5.3
Java Version: 17+
Spring Boot: 3.3.2
Minimum ES Version: 7.0+

Compatibility Guarantee

✅ Backward Compatible: All new features are disabled by default, no impact on existing deployments
✅ Progressive Upgrade: Support phased enablement of new features
✅ Zero-Downtime Operations: Configuration hot reload and smooth upgrades

Enterprise Support

🔧 Technical Consulting: Architecture design and performance optimization recommendations
📊 Monitoring Integration: Integration with existing enterprise monitoring systems
🎓 Training Services: Team skill training and best practice sharing
🚀 Custom Development: Customized feature modules based on business requirements

More Information

🌐 For more details, please visit elasticproxy.com website

ElasticProxyServer - Making Elasticsearch access more secure, intelligent, and efficient! 🚀

Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
config		config
target		target
.DS_Store		.DS_Store
docker-compose.yml		docker-compose.yml
docker-usage-guide.md		docker-usage-guide.md
docker-usage-guide_zh.md		docker-usage-guide_zh.md
readme.md		readme.md
readme_zh.md		readme_zh.md
start.bat		start.bat
start.sh		start.sh
start_docker.sh		start_docker.sh
start_multi_tenant.bat		start_multi_tenant.bat
start_multi_tenant.sh		start_multi_tenant.sh
stop_docker.sh		stop_docker.sh

YogooSoft/elasticproxy

Folders and files

Latest commit

History

Repository files navigation