Skip to content

[pixels-index] optimize rocksdb index and mainindex #1252

@AntiO2

Description

@AntiO2
  1. 1024 * 1024 * 1024 * 101 * tableNum MainIndexCache Entries would take more than 1TB memory
Image
  1. Set fixedLengthPrefix to enable bloom filter

  2. MainIndexCache admit stall

Code Base (Using HashMap):

Image

After(Using LinkedHashMap):

Image
  1. MainIndexBuffer doesn't flush, which caused stall

Solution: When load pixels file, flush the corresponding index

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions