PW database

PW Database

Perfect World stores all items, quests, npcs, recipes, etc as raw structures in binary files. They're not meant to be edited inplace. The regular strategy for editing PW is to load the entire file into memory, parse it, modify it, then save it under the same filename. That's what Mirage's patcher does both serverside and clientside, it doesn't provide any GUI though. The changes come from JSON files created by the web editor and this document describes how they're created.

script/DB.mjs allows keeping track of changes made to any object, also allows maintaining history, dumping changes into JSON, loading them, calling external hooks on any object modification. It is usable both in browser as well as serverside Node.js, where it can be used to validate user-submitted changes, parse them and postprocess them.

Example JS code

const db = new DB();

/* add new array "items" with 2 objects
db.register_type("items", [{ id: 7, field1: "value", field2: "value2" }, { id: 8, field: "value2" }]);

const obj = db.items[7];
console.log(obj.id + ': ' + obj.field2); // 7: value2

const cb = db.register_commit_cb((obj, diff, prev) => {
	console.log(obj);
	console.log(diff);
	console.log(prev);
});

db_open(obj);
db.field2 = "new_value";
db_commit(obj); // -> cb is called:
/* (obj, diff, prev) => {
 * 	console.log(obj); // { id: 7, field1: "value", field2: "new_value" }
 * 	console.log(diff); // { field2: "new_value" }
 * 	console.log(prev); // { field2: "value" }
 * });
 
db.unregister_commit_cb(cb);

Internals

This database relies heavily on javascript objects. On db.open() a deep copy of the obj is created to be used for generating diffs. At db.commit() all object fields (including nested ones) are compared to the deep copy and a diff object gets generated. All commit callbacks are called with this diff, then the copy obj gets updated.

DB doesn't store every diff object separately. It stores "changesets", usually a 5-min worth of changes to a specific object. Singular diffs generated on commit are provided to the commit_cb, then merged with the latest changeset. Changeset contains only new values of changed fields, although the first entry in the changeset array is always a full copy of the original, unchanged object. The changeset array for a single object could look as follows:

[0] = { < full original object > };
[1] = { field2: "another_value" };
[2] = { field: "new_value" };
[3] = { field2: "better_value" };

We can tell the final object will have field == "new_value" and field2 == "better_value". If we wanted to know the value of field before changeset #2, we would have to walk through the entire array, starting from 0 and applying all the changesets along the way. This is not a problem though, because history is usually presented through commit_cb (where previous value for each field is available) or though the entire history view - in which we walk through all changesets anyway.

What makes a new changeset is DB.new_generation(); It bumps an internal generation counter inside DB and clears all deep object copies made at open() so far. (Those can be created again when db.open() is called afterwards).

To sum up, an object commited at least once can have the following kept in memory:

• deep copy made at last open() within the current changeset generation
• array of changesets, with the first entry being the full, deep copy of the original object
• a few fixed-size metadata values

All db objects use _db field to store metadata - the original object must not use _db for a field name anywhere, otherwise it will be overriden, or the DB might even refuse to work.

To be able to dump the changes to JSON easily, DB also keeps an array of all changesets of all generations.