[Phase-1] HIP 33: Registry protection against puppet and farming attacks

Following intense discussion at the PoH Governance groups, I kickstart the ideation phase for the HIP that would prevent farming and other attacks.

4 Likes

What if someone has a big family?

Multiple registrations allowed + signing up my non technical parents OK?

We need to find mechanisms that protect those kind of cases.

1 Like

Two points about this:

  1. To continue with this discussion, let’s build a working definition of what farming and puppettering means. I’ll start and let’s work it from there.

Profile farming is the act in which a person recruits un consenting humans to register in proof of Humanity for the purpose of collecting their accrued UBIs and/or increase their influence in the DAO.

Puppettering is the act in which a person takes control of other people’s account with or without their knowledge and/or consent to perform attacks on the platform. Such actions include the act of vouchallenge, voting on behalf of the person, or stream their accrued UBIs to another account.

  1. I think that with this HIP we’re entering a murky terrain and this will become increasingly complex for the jurors to just vote automatically and for challengers invent or omit provision of evidence as (in my opinion, and cases where there were ridiculous allegations such as witches or zombies) has been in the past. It will also dictate what challengers must present as evidence to be considered it proper. For this reason I think that, in the body of the HIP there should be precise instruction on what juror duty is and how and what constitute proper ruling. These instructions, for example, must indicate that jurors must include a small text that indicate the exact reason why a decision was made (something very rarely seen at least in Humanity Court).

Identifying the Problem

As we have seen with the Cryptoevangelist cases and to a lesser extent with the recent removal of 120 or so puppet profiles from the registry, the current registry policy’s wording is sufficient to ensure obvious puppet profiles will be voted out when challenged given reasonable evidence. Some might disagree with the way in which Kleros jurors go about interpreting rules in general, but the fact is that the current rules are sufficient as things stand. This is not to say they could not benefit from clarification (the registry policy is very poorly worded) but I think this is a matter of lesser importance and can be tackled in another HIP if we decide to go forward with it at all.

One very obvious problem however is the lack of incentives for removing a profile that is found to be invalid after it has already been fully registered. Many of the Cryptoevangelist profiles were able to be challenged during their registration period due to the sloppy methods of the puppet master, and importantly, many of these challenges were only successful due to the ridiculous self-incriminating evidence the puppet master volunteered themselves. On the other hand, the more recently registered 120 or so puppet profiles were not detected during registration, or at least there wasn’t sufficient evidence to challenge them, and the necessary evidence only became available later.

EDIT: The removal of the 120 profiles mentioned above was only possible because it was subsidized by the Kleros Cooperative I believe. I don’t think this has been officially announced but that’s what it looks like when tracking the transactions. This is obviously not a sustainable or scalable solution.

A Clean Solution?

So how can we go about incentivizing removals? The most obvious way I can think of is to reward the removal of invalid profiles with freshly minted UBI… except that creates perverse incentives! I could create a profile, ask for removal by providing “proof” that the profile is a puppet (e.g. in the form of a recording of the fake puppet master scheming with me), claim the reward, and maybe even start over again. In other words, in order to prevent submitters from gaming the system, we need to make sure we design a negative sum game.

One possibility would be to change the way ETH deposits are handled and only return them when the profile expires or is removed by the user. This is obviously not an acceptable solution since it virtually eliminates the possibility of crowd funding which is a necessity for a large part of the world population.

A hopefully more palatable (but similar in spirit) option would be to require forcefully removed profiles to pay a fee in order to renew their profile. This requirement would of course be tied to the human and not the address. But even then, potential puppet masters might be able to exploit this new mechanism to their advantage if they are ever found out or start seeing removals requests for their puppet profiles: they could attempt to front run the removers and claim the rewards for removing their puppets themselves. The puppet profiles would have to pay a fee larger than the reward in order to re-enter the registry, but that would not be the puppet master’s problem at that point. We could also add a rule that the puppet master should be removed too (assuming the real puppet can even be found) but that is unlikely to be much of a deterrent either.

No matter how I think this through, I cannot find a clean solution to encourage removals without compromising ease of access (with long-term deposits) or game theoretic soundness (by encouraging the puppet master to challenge their own profiles).

A Pragmatic Solution

So here’s one less clean but pragmatic idea: elect trusted removers, or prosecutors if you will. A small group of say, 5, prosecutors would be given the exclusive ability to request removals and get rewarded for successful removals (anyone else could still request removals, but they would not be rewarded). If the removal was made based on information disclosed by a third party —a whistleblower or a private investigator— they would be entitled to half of the prosecutor’s profits (i.e. rewards minus transaction fees). These rewards would initially be held in an escrow supervised by Kleros so as to guarantee fair redistribution.

In order to discourage corruption within the ranks of the prosecutors, each would have to put down a deposit of say, 1 ETH, which would not be redeemable for say, 6 months, after stepping down from their function. Furthermore, rewards from prosecutions would also be withheld for 6 months before they become redeemable by the prosecutor who earned them. The purpose of holding up these funds is as you might have guessed to introduce a new kind of challenge which would allow anyone with sufficient evidence to request the removal of a prosecutor for corruption and to claim the deposit as a reward if successful.

As an added benefit, this mechanism could potentially encourage people to try and set up fake bribes as a trap for dishonest prosecutors, and the threat of being trapped in this way should in turn discourage less ethically inclined prosecutors from accepting bribes.

Another necessary aspect of this solution in my opinion is to retain some form of punishment for profiles participating in attacks. The punishment does not have to be as severe as what I pondered in the previous section since we are no longer attempting to build a clean game theoretic solution, but I still consider some form of punishment desirable since there is otherwise little incentive not to commit the crime other than transaction fees. What I propose is simply to deprive humans who have participated in an attack from receiving UBI for 1 year. Unlike requiring a fee to re-register, this does not infringe their fundamental right to be part of the registry in any way.

Additional Considerations

Forceful removals should lead to a court case by default

Since forceful removal will become profitable for prosecutors and costly for the removed, another change that should be made is that all forceful removal cases should go to court by default. This will ensure that a rogue prosecutor cannot simply try removing random profiles in the hope that their owners are not paying attention, which let’s face it, is going to be the case for most PoH profile owners.

Primary document for removal cases

I think we should clarify in the registry policy that the validity of forceful removals must be decided based on the rules (primary document) at the time the profile was created, i.e. the same rules that would have been applicable if the profile had been challenged then.

Anonymity layers and removals

If I’m wrong about this point, my whole proposal falls flat…

One thing to keep in mind with removals is the issue of how they interact with anonymized PoH profiles. There is no such thing at the moment but having an anonymized PoH layer (let’s call it aPoH) has, I believe, been a goal since PoH’s inception, and a belief that removals could not be reflected in an anonymity layer seems to have been one of the reasons why removals were neglected. I see two points to consider here:

  1. Is it technically feasible to create an anonymity/pseudonymity layer that can track profile removals? I think this probably depends on how the layer works. If it works by having PoH users register once with a zk-proof that they own a profile in PoH which is not already in aPoH, then it would not be possible. However, if aPoH were to work by requiring a zk-proof that the user owns a non-expired PoH profile every time it is used or every so often (every day, week, or month), tracking removals would not be an issue, but this comes at extra computational/gas cost of course.
  2. What are the anonymity implications of reflecting PoH removals in aPoH and should an anonymity layer ever try to track removals in the base layer? For instance, say I suspect a very active pseudonymized profile is, Elon Musk, and notice a technical error in his PoH profile which was not caught during registration. I now request removal of the profile on that basis and once removed observe that the suspected pseudonymous profile has stopped all activity. This would be a strong indication that my suspicion was correct. This attack requires a rather unlikely set of circumstances however, so for all but the most extreme threat models, allowing removal tracking should be an acceptable compromise.

To conclude this tangent, I’m not convinced that having some reliance on removals to keep the registry clean is necessarily incompatible with anonymized profiles but then I am not very knowledgeable in zk-anything.

Implementation

Implementing such a proposal would require updating the PoH and UBI smart contracts. One potentially delicate issue is that the PoH contract should be allowed to mint UBI.

To implement a UBI ban as punishment, users registering a profile under a different address would have to reference their previous address and any penalty from the previous address would apply to the new one. Failure to reference the old address would render the profile invalid and therefore likely to be challenged.

tl;dr

  • Elect trusted prosecutors who can make a profit from removals.
  • Share profits with investigators or whistleblowers who provide the necessary evidence.
  • Punish participants in attacks by removing their UBI stream for one year.

So there you go. Hopefully this made some sense and isn’t some schizo rant :slight_smile:

3 Likes

This is mostly just mostly just a personal reply to Luis. To anyone else, feel free to skip this post (unless you feel yourself in agreement with Luis’ above post).

To continue with this discussion, let’s build a working definition of what farming and puppettering means. I’ll start and let’s work it from there.

As I’ve stated already, I don’t think that is a priority. I also don’t see any benefit to introducing a distinction between puppeteering and farming, nor do I agree with the one you made.

this will become increasingly complex for the jurors to just vote automatically and for challengers invent or omit provision of evidence as (in my opinion, and cases where there were ridiculous allegations such as witches or zombies) has been in the past.

Jurors have already proven that they are capable of interpreting the rules with the necessary flexibility to stop profile farming when provided with enough evidence. The examples you refer to where two challengers (if I recall correctly) made accusations of the parties involved in a case being witches or zombies was clearly a form of trolling and inconsequential to the outcome of the cases so I’m not sure why you bring that up.

It will also dictate what challengers must present as evidence to be considered it proper.

Once again, this is not needed. Jurors are free to make up their own mind as to what constitutes valid or convincing evidence and have shown they do a good job at it.

For this reason I think that, in the body of the HIP there should be precise instruction on what juror duty is and how and what constitute proper ruling. These instructions, for example, must indicate that jurors must include a small text that indicate the exact reason why a decision was made (something very rarely seen at least in Humanity Court).

This cannot be defined by a Proof of Humanity proposal and is already defined in the General Court rules anyway. And I’ll say it again, jurors have never failed to maintain the Sybil resistance of the registry with the current set of rules. It is the economic incentives specific to PoH in its current iteration which have been the issue.

Regarding your very last point, I totally agree that jurors should write a justification to go along with their vote, however this can barely be said to be relevant to this HIP in particular. But to give you some insight on that point anyway, I’m quite convinced that the juror justification functionality is completely bugged out and has been for months now. It is unfortunately only implemented on the web side of things and not on the smart contract side as it should be but this design flaw will hopefully be fixed with court v2, or I will lobby for it very strongly, I can assure you. There was also talk of requiring jurors to justify their vote under threat of penalty in court v2. I think this is a good idea and I will lobby for it too if necessary.

1 Like

@clesaege @william Wouldn’t mind your opinions on this if you find the time :slight_smile:

Here are some possible modifications I’ve thought of since by the way:

  • Add a dynamic registration fee that is a fraction of the gas cost for the registration transaction and put it in a removal reward pool. This would get rid of the need to mint UBI and the fee could just be stored in ETH which is a more reliable asset (IMO). By making the fee a fraction of the gas fee already required for registration, we can simultaneously ensure that this extra fee will only be a small burden for new registrants and that on average the removal reward pool stays large enough to pay for removal transaction fees as gas conditions evolve. To determine this new registration fee fraction, we could use the heuristic <number of profiles removed or needing removal> / <number of profiles registered> * <gas cost of removal> / <gas cost of registration> * C, where C would be a constant between 1 (realistically, at least 1.1) and 2, used for rewards (on top of a transaction fee refund that is) and as a buffer in case the number of profiles to remove suddenly increases.
  • Luis rightly pointed out that some people might be coerced into becoming puppets. In that case, they should not be penalized by removing their right to UBI for one year. This might require having an extra parameter when asking for removal: willing/unwilling participant. I personally think people who are scammed into participating as puppets should still be fined since I think lack of due diligence needs to be penalized to discourage people from handing over their voting rights.

With the current setup using a fraction of the gas fee could bring some funds in the pool. But I’m a bit afraid that having a fee would discourage people from registering. The registry could be seems as a way to get money and not a public good (even if the fee is actually used for public good). Also participants would judge POH way more harshly if they need to pay (as people are more exigent when they pay, even for small amounts).
In the next version on a rollup, we could have pretty low fees and having a protocol fee would make a huge difference.

Well if they could show they were coerce, I don’t think they should be penalized. But is it that important or a small detail? As I don’t believe we ever saw any coercion in the registry yet.

Thanks Clément. Actually I was more so wondering if you had opinions on my initial proposal. With regards to anonymity layers for instance.

Regarding the additional fee, I was thinking it could probably be around 10% of the transaction fee, which would make it barely noticeable. Actually, if we assume there have been 200 puppet profiles up until now, taking into account that the gas cost of registering seems to be about the same as the gas cost to request a removal, and setting C to a fairly conservative 1.5, the formula I suggested becomes 200 / 10000 * 1.5 = 3%. So for instance, with transactions costing around 40$ at the moment, that’s an extra 1.20$. But some people might still not like the idea, I don’t know.

Regarding not penalizing victims of coercion, this would definitely add extra complexity for a perhaps rare edge case but I’m getting the feeling this might have worse optics still than simply requesting a small fee, if that’s something we want to be concerned about.

Indeed, how removals would interact with an anonymity layer depends on how that anonymity layer works. In an extreme case one could imagine that every time you wanted to use a PoH profile in some anonymized way, you issue a zk-proof that you control a currently valid profile. This is possible, but if you are using the profile to interact with an on-chain application that application needs to spend (a potentially substantial amount of) gas to verify the proof for each interaction.

An alternative approach is to have people with PoH profiles issue zk-proofs that some new address belongs to a unique member of PoH (without revealing who) which creates a sort of aPoH in that you have a Sybil resistant list of addresses that correspond to the members of PoH but in a pseudoanonymized way. This saves on having to spend gas to issue a new zk-proof for every time that one wants to interact in an anonymous, Sybil-resistant fashion with a dapp, but because one can tie together the actions of any given address on aPoH it has a somewhat less complete level of anonymity compared to issuing a new zk-proof for every action; particularly your Elon Musk example would be an issue here but not in the model where you issue a new zk-proof for every action. Note that for some applications having an pseudo-anonymized list of addresses would be more straightforward to integrate with than the approach of checking a zk-proof with every action. For example, having a module in the next version of the Kleros Court that only draws jurors on PoH/limits jurors to a maximum of one voter per PoH profile would be more straightforward if this just involves random draws from a (pseudo-anonymized) list.

Moreover, in the second model, you could only have removals take effect in the anonymized list after some point where participants are required to issue a new zk-proof that they still have a valid profile (which the removed people are no longer capable of generating). So you would probably want the aPoH to have a fairly short period of validity - e.g. the following month or whatever the old aPoH is seen as invalid and in order to get a new address on the new aPoH everyone needs to issue a new zk-proof. Notably, one would probably want the period of validity of a given anonymized list to be significantly shorter than the frequency with which people need to renew their profiles for the public PoH.

Note that one could conceivably have both of these models in parallel, where some applications require to issue a proof of inclusion on PoH each time you use them and others use the list based approach.

3 Likes

Thanks William. Glad to have confirmation that the idea can at least work in principle.

For the Kleros jury selection use case, renewal every month seems reasonable, although kind of bad from a UX perspective, especially for small jurors who only rarely get cases and therefore only rarely interact with any Kleros UI. Perhaps some background desktop application could automatically renew the aPoH profile every month. It could also be used for revealing hidden votes while we’re at it. And ideally, this application would be able to submit these transactions from a low-value secondary hot wallet address while the user could keep their precious PNK in a hardware wallet. EDIT: Desktop or mobile application. Mobile would probably be better for most people, but I seem to recall you were already considering that for vote reveals.

1 Like

First a few things about me and a some disclaimers.
.My first time writing here, sorry if I misinterpret the dynamics.
.I am not a developer, so some of my opinions or “solutions” may not be even possible!
.Last but not least, I am one of the Kleros’ fellows of the 5th batch, currently working in improving PoH in a specific way. Exactly this problem.

Now my view:
Is it possible that the real problem here is to check if every human is really in control of the wallet they submitted? I mean, if we could automatize a robust way to check that, most of the problems related to “puppeteering or farming” would disappear, right?
I have been thinking about that perfect way to check it and of course have not reached it. But the best ideas I came through were these.

  1. To require a new video after a randomized (to avoid preparation of the farmer) period of time. To give the person a fixed (but short) period of time to submit that video. The mentioned video should have the same human of the registration one, but this time saying a new phrase. There may be a large amount of possible phrases to be said and the one in each case should be randomly selected (to avoid a pre-recorded video).
    In the case that the person does not upload that video, the profile could be moved to a phase where it does not get UBIs nor it can vote. Some kind of “pause phase”?. Another possibility is that the person at this point could choose to submit a new deposit to remain registered, but that can be used as a bounty and the profile could be challenged or so.

  2. Would to increase the amount of vouches needed fix these weaknesses? Is it possible to change the “weight” of the vouch of some profiles according to their behaviour? (for example: stronger vouchers would be those profiles who were never challenged or something like that)

Sorry if my English is not the best.

3 Likes
  1. That’s a good idea, I think @clesaege had suggested something similar. The video ‘proof of liveliness’ can include a recent blockhash.

  2. The social graph of vouchers is under-utilized. The gasless vouches work well to add submissions to POH, but only one gasless vouch is recorded on chain. This can be modified ofcourse. Vouching on chain requires gas fees which is cost prohibitive on L1, but POH is under development for Gnosis Chain and other chains in the future. On chains with low transaction fee environments, higher vouch requirements are more feasible.

6 Likes