Link terms on page to Wikipedia articles in pure JavaScript

Question

While browsing I came across this blog post about using the Wikipedia API from JavaScript, to link a single search term to it's definition. At the end of the blog post the author mentions possible extensions including:

A plugin which auto links terms to Wikipedia articles.

This fits the bill perfectly for a project requirement I'm working on, but sadly I lack the programming skills to extend the original source code. What I'd like is to have a pure JavaScript snippet I can add to a webpage, that links all the terms on that webpage that have an article on an internal wiki to that wiki.

I know this might be asking for much, but the code looks like it's nearly there, and I'd be willing to add a bounty if anyone will do the remaining work for that virtual credit.. ;) I also suspect this might be of value to a few others, as I've seen similar requests but no working implementation (that's a mere JavaScript (and therefore portable) library/snippet include).

Here's a sample of the original source code, I hope anyone is able to add to this or point me to what I'd need to add if I were to implement this myself (in which case I'll share the code if I manage to put something together).

<script type="text/javascript"><!--
var spellcheck = function (data) {
    var found = false; var url=''; var text = data [0];
    if (text != document.getElementById ('spellcheckinput').value)
        return;
    for (i=0; i<data [1].length; i++) {
        if (text.toLowerCase () == data [1] [i].toLowerCase ()) {
            found = true;
            url ='http://en.wikipedia.org/wiki/' + text;
            document.getElementById ('spellcheckresult').innerHTML = '<b style="color:green">Correct</b> - <a target="_top" href="' + url + '">link</a>';
        }
    }
    if (! found)
        document.getElementById ('spellcheckresult').innerHTML = '<b style="color:red">Incorrect</b>';
};

var getjs = function (value) {
    if (! value)
        return;
    url = 'http://en.wikipedia.org/w/api.php?action=opensearch&amp;search='+value+'&amp;format=json&amp;callback=spellcheck';
    document.getElementById ('spellcheckresult').innerHTML = 'Checking ...';
    var elem = document.createElement ('script');
    elem.setAttribute ('src', url);
    elem.setAttribute ('type','text/javascript');
    document.getElementsByTagName ('head') [0].appendChild (elem);
};--></script>
<form action="#" method="get" onsubmit="return false"> 
<p>Enter a word - <input id="spellcheckinput" onkeyup="getjs (this.value);" type="text"> <span id="spellcheckresult"></span></p></form>

---

Update
As pointed out in the comments, both the time it would take to link all words and how to handle multiple word spanning article names were concerns of mine as well..

I'd think starting with single word articles would already cover a large percentage of the use cases, with maybe some performance benefits gained when skipping the 500 most common words in the English language, but still I'm uncertain how feasible this approach will be..

On the upside however this would all be client side, and some delay in linking terms is fully acceptable.

Alternatively searching for terms the mouse is hovering over / selected might be acceptable as well, but I'm unsure if this would decrease or increase complexity..

---

Update 2

'Pointy' explained below that this functionality could be achieved by altering some fairly standard highlighting scripts, after having obtained a list of article topics from api.php?action=query&list=allpages.
To reinterate: we're using an internal wiki, so the list of articles is likely limited, non ambiguous and domain specific enough to overcome some of the expected problems in matching words.

Since we've had some good suggestions so far, and a few workable ideas, I'm starting a bounty to see if I can get a few answers on this..

Answer 1

Perhaps something like this might help:

Assuming very simple HTML/Text like so:

<div id="theText">Testing the auto link system here...</div>

And two very small scripts.

dictionary.js sets up your list of your terms. My thought was that this could be generated in php by querying the articles database if you wanted. It also can be loaded cross domain (as it sets window.termsRE). If you don't need to generate the list from the database, you could also manually put it with termlinker.js.

This code that generates the RegExp assumes that your terms array contains properly formatted strings to match using Regular Expressions, so be sure to use \\ to escape []\.?*+|(){}^&

// dictionary.js - define some terms
var terms = ['testing', 'auto link'];
window.termsRE = new RegExp("\\b("+terms.join("|")+")\\b",'gi');

termlinker.js is just a simple regexp search replace on the defined terms. It could be an inline <script> too. requires that the dictionary.js has been loaded before you run it.

// termlinker.js - add some tags
var element = document.getElementById("theText");

element.innerHTML = element.innerHTML.replace(termsRE, function(term) {
  return "<a href='http://en.wikipedia.org/wiki/"+escape(term)+"'&gt;"+term+"&lt;/a&gt;";
}); 

This simply searches for any words in the terms array and replaces them with a link to the term. Of course, it will also match properties and values inside HTML tags, which could break your markup a little.

All thrown together you get this (jsbin preview)

---

Using the API

Based off of the "minimum case" from before, here is the code sample for using the API to receive the list of words directly and the jsbin preview

// Utility Function
RegExp.escape = function(text) {
  if (!arguments.callee.sRE) {
    var specials = [
      '/', '.', '*', '+', '?', '|',
      '(', ')', '[', ']', '{', '}', '\\'
    ];
    arguments.callee.sRE = new RegExp(
      '(\\' + specials.join('|\\') + ')', 'g'
    );
  }
  return text.replace(arguments.callee.sRE, '\\$1');
};

// JSONP Callback for receiving the API
function receiveAPI(data) {
  var terms = [];
  if (!data || !data['query'] || !data['query']['allpages']) return false;  
  var pages = data.query.allpages
  for (var x in pages) {
    terms.push(RegExp.escape(pages[x].title));
  }
  window.termsRE = new RegExp("\\b("+terms.reverse().join("|")+")\\b",'gi');
  linkterms();
}  

function linkterms() {
  var element = document.getElementById("theText");

  element.innerHTML = element.innerHTML.replace(termsRE, function(term) {
    return "<a href='http://en.wikipedia.org/wiki/"+escape(term)+"'&gt;"+term+"&lt;/a&gt;";
  });
}


// the apfrom=testing can be removed, it is only there so that
// we can get some useful terms near "testing" to work with.
// we are limited to 500 terms for the purpose of this demo:
url = 'http://en.wikipedia.org/w/api.php?action=query&amp;list=allpages&amp;aplimit=500&amp;format=json&amp;callback=receiveAPI' + '&apfrom=testing';
var elem = document.createElement('script');
elem.setAttribute('src', url);
elem.setAttribute('type','text/javascript');
document.getElementsByTagName('head')[0].appendChild (elem);