Reward for editing a wiki page

Question

I want to award points whenever someone edits a wiki page. Not all changes are equal, however. Creating the first revision of a document from a blank page is surely more valuable than inserting a missed comma.

What are the mechanism that I can use to determine when a change between two document revisions is significant? Would a percent difference suffice or are there better methods? How would one even calculate percent difference between two texts? What would be the threshold of significance? 5% change? 10%?

The programmer in me says that you can give points in proportion to the percent change. For example, out of possible 10 points you would get 3 point for a 10% change, 6 points for 20% change, and say all 10 points for anything over 30% change. The writer in me says that it's probably not as clear cut.

In general I'm open to other solutions to the problem of giving a fair reward based on a level of contribution. This is very important to me, so please forgive the open ended nature of the question.

EDIT: Please consider other models except stackoverflow. The model of a single wiki with many authors is very different from a model of a question with many answers. Thanks to everyone who answered so far.

Answer 1

I would steer clear of automated points awards and go with a system by which other users reward the author/editors via a voting system, similar to this site. Awarding points by how much of the document was changed only awards people for making massive edits, not for improving articles.

Answer 2

This is a very difficult problem to solve mechanically. If your wiki is going to be large enough to have an active user-base which participates actively, consider allowing users to up-vote and down-vote each others changes. Sound familiar?

Answer 3

Interesting question. One thing that comes to mind is how the "Related" questions over there on the right appear. Jeff has mentioned that in order to find related question, they parse out all common words like the and and. Here are two hundred of them. http://www.teacherjoe.us/Vocab200.html I would base my percent changes based off of that.

Going down this road is a tricky one though, what happens if a user inserts spam? Are you going to then take away points? I think a voting system like what is used here might be more useful.

Answer 4

Edits are very context sensitive; it would be difficult to come up with an algorithm that determined the relative quality of an edit, because the computer would have to "understand" what it was reading. I don't think the relative amount of change is a valid indicator; you will get a score, but it won't reflect the actual value of the edit.

Answer 5

People will game such a mechanic by creating spam pages and replacing page content with copypasta. As soon as you put a moderation / metamodetation scheme in place to prevent this, you could just use that to produce the scoring directly - cf. Stack Overflow.

Answer 6

I know instant gratification is good, but if you truly want to reward users on how meaningful their change is give each page a rank. New pages rank at 0 so creating a page gives you no points. Then allow people that view the page to give the page an up vote or down vote. So then you get a a ranking for the page. Lets say 60% of users vote up on the page. Now when someone makes a change to that page the former success rate is recorded. You then tally all incoming votes to a new total. If the approval rate increases say to 65%, you can award the changer 5 points. If it drops to 50% the changer gets -10 points.

No matter what your system is keep it secret. If Google page rank has taught us anything its that an unknown game is harder to play than a game where you know all the rules.

Good Luck