PHORBAS Store

A PHORBAS store is an associative 1:1 key to binary content data storage abstraction. Key must be a string, although often hex encoded from cryptographic hashing via PHORBAS Opaque. Content is binary (ArrayBuffer), direclty supporting encrypted content. The store handles the principles of Persistent, Binary content, Addressable, and Store.

• PHORBAS Opaque handles the principle of Hashed, Opaque, and Addressable.
• PHORBAS HAMT handles the principle of Replicable.

API & Use

import {kbc_js_map} from '@phorbas/store'

// use one of the many backend storage adapters
let stg = kbc_js_map()

// keys are opaque strings; some backend adapters may route based on parts of keys
let some_key = `some_key_${Date.now().toString(36)}`

await stg.bkc_exists(some_key) // 0

// store binary content addressed by key
let binary_body = crypto.getRandomValues(new Uint32Array(1024))
await stg.bkc_store(some_key, binary_body)


// key now exists
await stg.bkc_exists(some_key) // 1

// and we can fetch the binary content, returned in an ArrayBuffer
let body_result = await stg.bkc_fetch(some_key)

The backend storage API is fundamentally asynchronous, designed for use with @phorbas/opaque. The kbc prefix is for "keyed binary content".

• async kbc_exists(key) returns 1 if exists, otherwise 0

• async kbc_fetch(key) returns ArrayBuffer if exists, otherwise null|undefined

• async kbc_store(key, body) returns error | null, where error is null upon success.

• async * kbc_stream_exists(key_aiter, {signal}) is an async iterable streaming version of kbc_stream_exists

• async * kbc_stream_fetch(key_aiter, {signal}) is an async iterable streaming version of kbc_stream_fetch

• async * kbc_stream_store(key_body_aiter, {signal}) is an async iterable streaming version of kbc_store

Storage Implementations

JavaScript, Browser, and WinterCG

Unit Test	Integ Test	Technology / Library	Implementation
✅	✅	JS Map new Map()	impl: code/js_map.jsy test_js_map.jsy integ--core/integ-core.test.mjs
✅	✅	JS Fetch API globalThis.fetch()	impl: code/web/fetch.jsy integ_fetch.jsy integ--core/integ-web.test.mjs
✅		Browser DOM Storage globalThis.localStorage or globalThis.sessionStorage	impl: code/web/web_storage.jsy test_web_storage.jsy
✅		Browser DOM Element document.createElement()	impl: code/web/web_dom.jsy test_web_dom.jsy
✅		Browser IndexedDB API globalThis.indexedDB.open()	impl: code/web/web_db.jsy test_web_db.jsy
✅	✅	Browser Service Worker Cache API globalThis.caches.open()	impl: code/web/web_cache.jsy test_web_cache.jsy integ_web_cache_fetch.jsy

Local

Unit Test	Integ Test	Technology / Library	Implementation
	✅	LMDB node-lmdb	impl: code/local/lmdb.jsy integ--level/integ-lmdb.test.mjs
	✅	LevelDB Level ecosystem	impl: code/adapter/level.jsy integ--level/integ-level.test.mjs
✅	✅	Node FS Promises API import * as fsp from 'node:fs/promises'	impl: code/local/fsp.jsy test_local_fsp.jsy integ--local/integ-fs-promises.test.mjs
✅	✅	Node FS API import * as fs from 'node:fs'	impl: code/local/fs.jsy test_local_fs.jsy integ--local/integ-fs.test.mjs
✅	✅	Web FS & FS Promises via lightning-fs from isomorphic-git	impl: code/local/fsp.jsy integ_web_lightning-fs.js

NoSQL

| Integ Test | Technology / Library | Implementation | |------------|------------|----------------------|----------------| | ⚠️ | ArangoDB
arangojs | impl: code/nosql/arangojs.jsy
integ--arangodb/integ-arango.test.mjs
note: Binary values encode in base64 | ✅ | CouchDB
nano | impl: code/nosql/couchdb.jsy
integ--couchdb/integ-couchdb.test.mjs | ✅ | PouchDB
pouchdb | impl: code/nosql/pouchdb.jsy
test/nosql/integ_pouchdb.jsy
integ--couchdb/integ-pouchdb.test.mjs | ✅ | Consul KV
node-consul | impl: code/nosql/consulkv.jsy
integ--consulkv/integ-consul.test.mjs | ✅ | AWS DynamoDB
aws-sdk | impl: code/nosql/dynamodb.jsy
integ--dynamodb/integ-dynamodb.test.mjs | ✅ | Memcache
memjs | impl: code/nosql/memjs.jsy
integ--memcache/integ-memjs.test.mjs | ✅ | MongoDB
mongojs | impl: code/nosql/mongojs.jsy
integ--mongodb/integ-mongodb.test.mjs | ✅ | FerretDB
mongojs | impl: code/nosql/mongojs.jsy
integ--mongodb/integ-mongodb.test.mjs | ✅ | Redis
ioredis | impl: code/nosql/ioredis.jsy
integ--redis/integ-ioredis.test.mjs | ✅ | Valkey
ioredis | impl: code/nosql/ioredis.jsy
integ--redis/integ-ioredis.test.mjs | ✅ | RethinkDB
rethinkdb | impl: code/nosql/rethinkdb.jsy
integ--rethinkdb/integ-rethinkdb.test.mjs

S3 NoSQL

Integ Test	Technology / Library	Implementation
✅	MinIO minio	impl: code/nosql/minio.jsy integ--core/integ-minio.test.mjs
✅	AWS S3 or compatible aws4fetch	impl: code/nosql/s3_aws4fetch.jsy test/unittest.web.mjs integ--core/integ-aws4fetch.test.mjs

SQL and Knex adapter

Integ Test	Technology / Library	Implementation
✅	SQLite import {DatabaseSync} from 'node:sqlite'	impl: code/sql/better_sqlite3.jsy integ--sqlite/integ-sqlite-nodejs.test.mjs
✅	SQLite better-sqlite3	impl: code/sql/better_sqlite3.jsy integ--sqlite/integ-better-sqlite3.test.mjs
✅	SQLite node-sqlite3	impl: code/sql/sqlite.jsy integ--sqlite/integ-sqlite3.test.mjs
✅	SQLite knex.js	impl: code/sql/knex.jsy integ--sqlite3/integ-knex-sqlite.test.mjs
✅	PostgreSQL knex.js	impl: code/sql/knex.jsy integ--postgres/integ-postgres-knex.test.mjs
✅	CockroachDB knex.js	impl: code/sql/knex.jsy integ--postgres/integ-cockroach-knex.test.mjs
❌	CrateDB knex.js	impl: code/sql/knex.jsy note: Crate SQL lacks BLOB in-table support
✅	MariaDB knex.js	impl: code/sql/knex.jsy integ--maria-mysql/integ-knex-maria-mysql.test.mjs note: Uses MEDIUMBLOB
✅	MySQL knex.js	impl: code/sql/knex.jsy integ--maria-mysql/integ-knex-maria-mysql.test.mjs note: Uses MEDIUMBLOB
✅	Microsoft SQL Server knex.js	impl: code/sql/knex.jsy integ--mssql-server/integ-mssql.test.mjs

Keyv adapter

Integ Test	Technology / Library	Implementation
✅	JavaScript Map keyv and new Map()	impl: code/adapter/keyv.jsy integ--local-filesystem/integ-keyv-map.test.jsy
⚠️	One file keyv and keyv-file	impl: code/adapter/keyv.jsy integ--local-filesystem/integ-keyv-file.test.jsy Stored in a single JSON file with binary values encode in base64
✅	Memcache keyv and keyv-memcache	impl: code/adapter/keyv.jsy integ--memcache/integ-keyv-memjs.test.mjs
✅	MongoDB keyv and @keyv/mongo	impl: code/adapter/keyv.jsy integ--mongodb/integ-keyv-mongodb.test.mjs
✅	SQLite keyv and @keyv/sqlite	impl: code/adapter/keyv.jsy integ--sqlite3/integ-keyv-sqlite.test.mjs

About PHORBAS

The PHORBAS project implements a persistent binary data storage scheme that accomodates opaque (encrypted) information. It is inspired by binary content addressable data storage, replication features of CouchDB, the opaque data storage of Tahoe-LAFS, the eventual consistency of CRDTs, and persistent Hash array mapped trie data structures.

Phorbas appears in in Greek mythology and is connected to "giving pasture" -- in this case, safe pasture to for data storage.

P - Persistent
H - Hashed
O - Opaque
R - Replicable
B - Binary content
A - Addressable
S - Store

Due to the opaque nature of the data storage, the storage is not queryable like a database. It is similar to IPFS with a less ambitious goal of being a pratical data storage scheme for data for embedding into other projects.

License

2-Clause BSD