Unchecked history tree growth

[disgustingwall]

The main problem with the system currently is that the history tree can become infinitely large without having any content stored in it.

This problem is especially noticeable when two servers are set to pull each other's trees and merge them: one server will merge the other's tree, making a node with only references to two other nodes in it - the foreign server's head node and the server's own previous node. The second server will then detect that a new head node is available, and it will create a similarly empty node in its own tree. The first server will then recognize that a new node is available, and the cycle will repeat.

This could be solved in one of two ways:

• Each server could use spare resources to compress its own tree: adjacent sparse nodes could be compressed into a single node and the tree could be rebuilt. The server could aim to have each node be a natural file size, IE less than or equal to some IPFS or local filesystem block size, or perhaps some multiple of 512 bytes.
• The receiving node could check if content included in the foreign tree has already been included in its own tree. This would take time, delay the inclusion of content to the server's tree, and an exhaustive search would be extremely resource intensive and likely fruitless. This is likely a poor solution in isolation.

[disgustingwall]

An elegant solution could be to make the API that returns the most recent mailbox return a list of recent head mailboxes instead of only one, such that nodes can still refer to cached, but outdated, mailboxes while still compressing mailboxes.