Proposal for dynamic network difficulty adjustment

[PeterSurda]

There is a need to raise the difficulty in order to better protect against SPAM. Unfortunately this is at least partially not backwards compatible, because there is no mechanism for updating the network difficulty, the recipient metadata, or chan difficulty.

The main constraint is that I want this to not require a central instance, but at the same time not reduce anonymity.

The core of the proposal is a new network object, a difficulty vote. It will not contain any recipient, sender, encryption or signatures, only some random padding. It could have a fixed size. In metadata, it would contain

• either a creation time or a duration, to allow calculating the original difficulty
• current difficulty
• proposed difficulty

The client will verify the PoW as it does now, and then use a sliding window to calculate the current difficulty. There needs to be some wiggle room to avoid under/overshoots as the inventory contents are transient.

What this will allow is to have people who want to become "miners" and "bump" the network difficulty, making it more costly for the spammers. It would auto-adjust based on how much proof of work the miners contribute.

Deployment would happen in phases:

• Each of your addresses should track last announced difficulty. We already track last time the pubkey was sent. This is needed so that we know whether it's needed to announce an update or not.
• Allow pubkey objects to update metadata (difficulty). I.e. if you receive an updated pubkey from someone, you update the demanded difficulty. This currently doesn't work, after you receive a pubkey, the difficulty is never updated.
• Have pubkey objects to be auto generated when difficulty changes even if there is no request for them and the old ones haven't expired, but leave a bit of wiggle room (delay, range), to counter deanonymisation and DoS attempts.
• Add an option in keys.dat to enable generating difficulty vote and configure the target difficulty. Default off. Simultaneously, accept difficulty vote objects and calculate difficulty, just to see how it works. The resulting difficulty can be shown inside the network stats tab, and/or exposed via API.
• Allow tracking of objects that don't have sufficient difficulty for network-wide difficulty. Currently, they would keep getting redownloaded. We also need to be able to track them without having to store them fully in the inventory, so that we also reduce our storage of spam. For example, we'd track the inventory vector and expiration. We can use a new table in messages.dat for that.
• There should be new options in keys.dat to
  • Use the difficulty vote to calculate which difficulty to announce for your own addresses (i.e. your own spam filter). Default off.
  • Use the difficulty vote to calculate which objects to accept in inventory (i.e. a network-wide spam filter). Default off.
• Deprecate user-configurable difficulty and have the global one calculated from the difficulty votes to be used instead. This will make your own spam filter mandatory.
• Force migrate chans, because chans don't have a difficulty metadata and they are most affected by spam
• Force using difficulty for inventory filter (i.e. network-wide spam filter)

The difficulty check for accepting inventory objects should be more relaxed than for the local spam filter, to make network-wide transitions smoother.