Best programming aids for a quadriplegic programmer

Question

Before you jump to conclusions, yes, this is programming related. It covers a situation that comes under the heading of, "There, but for the grace of God, go you or I." This is brand new territory for me so I'm asking for some serious help here.

A young man, Honza Ripa, in a nearby town did the classic Dumb Thing two weeks after graduating from High School -- he dove into shallow water in the Russian River and had a C-4/C-5 break, sometimes called a Swimming Pool break. In a matter of seconds he went from an exceptional golfer and wrestler to a quadriplegic. (Read the story ... all of us should have been so lucky as to have a girlfriend like Brianna.) That was 10 months ago and he has regained only tiny amounts of control of his right index finger and a couple of other hand/foot motions, none of them fine-grained.

His total control of his computer (currently running Win7, but we can change that as needed) is via voice command. Honza's not dumb. He had a 3.7 GPA with AP math and physics.

The Problems:

1. Since all of his input is via voice command, he is concerned that the predominance of special characters in programming will require vast amount of verbose commands. Does anyone know of any well done voice input system specifically designed for programmers? I'm thinking about something that might be modal--e.g. you say "Python input" and it goes into a macro mode for doing class definitions, etc. Given all of the RSI in programmer-land there's got to be something out there. What OS(es) does it run on?

2. I am planning on teaching him Python, which is my preferred language for programming and teaching. Are there any applications / whatevers that are written in Python and would be a particularly good match for engaging him mentally while supporting his disability? One of his expressed interests is in stock investing, but that not might be a good starting point for a brand-new programmer.

3. There are a lot of environments (Flash, JavaScript, etc) that are not particularly friendly to people with accessibility challenges. I vaguely remember (but cannot find) a research project that basically created an overlay system on top of a screen environment and then allowed macro command construction on top of the screen image. If we can get/train this system, we may be able to remove many hurdles to using the net.

4. I am particularly interested in finding open source Python-based robotics and robotic prostheses projects so that he can simultaneously learn advanced programming concepts while learning to solve some of his own immediate problems.

I've done a ton of googling on this, but I know there are things I'm missing. I'm asking the SO community to step up to the plate here. I know this group has the answers, so let me hear them! Overwhelm me with the opportunities that any of us might have/need to still program after such a life-changing event.

Update: I just registered computingforquads.org and I'll be creating pages for all sorts of solutions to all sorts of problems. Thanks for you help so far and keep those answers coming!

Answer 1

It's worth looking at the Dasher Project, which makes it possible to enter text reasonably quickly even for the severly disabled. Dasher is built on a probabilistic model of languages, so that more likely utterances are easier to enter into the system. The demonstration system comes with a fairly impressive collection of natural languages. It should be easy to get a large corpus of programs written in Python, load Dasher with the corpus, and create a special-purpose version for entering Python programs.

Answer 2

hello

I have sports injuries, and I cannot type more than few characters without serious pain.

Instead, I use emacs together with Dragon NaturallySpeaking. I have written macros and scripts to help it get by. The system is not perfect, but it works. I program mostly in C++, but I also use python.

If you want to, I will help you with that. I have to warn you, it takes few months to get used to speech to text software and train it. moreover, I am not native English speaker, am sure that gets in the way

Do not despair, there are solutions.

here a link to emacs and Dragon files (unfortunately have not documented yet)

http://code.google.com/p/asadchev/source/browse/#svn/trunk/home/Dragon

http://code.google.com/p/asadchev/source/browse/#svn/trunk/emacs/elisp

also, if you need more info, my number is 515 230 9363 (United States, Iowa). I will be glad to help you if I can

Answer 3

I know someone in a village in India who is a paraplegic, who uses Dragon Speech to Text software to write on her computer. I don't know how well suited it is for a programmer (she is not a programmer), but it's a start.

You might also want to look into Natural Point. It's an eye controlled mouse, which might help Honza

Hope this helps

Answer 4

This isn't part of any professional software, but when I saw this, I've thought it would be good for text entry using eye movement tracking or minimal mouse movement. See Ken Perlin's Processing page, and look at the applets for "pen input".

Answer 5

iPython with completion

On the python side, iPython shows parameters, functions, etc, and has command completion. Perhaps it could also be customized to respond to the various input devices as well?

http://ipython.scipy.org/moin/

Answer 6

One thing that may help (i got it from this question) is http://shorttalk-emacs.sourceforge.net/ . It seems to be an interface between emacs and speech recognition. And regarding languages, i would recommend using Lua over Python. It has a more natural English flow to it.