In a system for moderating offensive user content, how do you decide on a threshold for automatic removal?

Question

I'm writing a forum application and want to automatically moderate offensive posts using a flagging system similar to the one on StackOverflow in which users are given the ability to report problematic posts as falling into one of three categories:

* Abusive speech
* Off topic
* Spam

If a particular post receives a certain number of flags, I would like the system to automatically delete the post.

How should I decide what this threshold should be?

Answer 1

By 'delete', I would assume you mean 'deactivate and make invisible to normal users', so that you can still access the post and make determinations as to whether the flagging system is being used appropriately.

I would say, decide what the threshold should be by picking a number between three and five depending on how large and active a user base you have, and adjusting the number upward or downward based on how you see it playing out in practice.

Answer 2

I'd make it some percentage of the views after a certain number of views.

1. Views > 5
2. Greater than 1/5 ratio of offensive flags to views