How do you monitor, manage, and run untrusted third-party code in a hosted service?

Question

Let's say I'm running a hosted service that wishes to allow plugins written by third-party clients.

Perhaps a gaming service provider that provides infrastructure but allows clients to develop their own game referees. Or, a coding competition site that allows coders to submit code to be run as their solution to some coding problem.

How would you harden/lock-down/sandbox this user code from doing potential harm to the server that intends to run it?

How would you monitor and restrict resource usage (CPU, memory mostly)?

This is a good start for Python but I'm wondering if anyone here has more specific experiences they can share regardless of language (Python, Lua, Ruby, etc.).

chroot, jails, rexec, mangling __import__, etc. Let me know what you've done.

Thanks!

Answer 1

RestrictedPython (shown in the link you provided) looks promising. I can't say I've actually tried to do such a thing, however.

Another option that might work is building an extremely minimal Linux distribution, then replicating it in a virtualized environment for each user. Use the virtual machine's monitoring mechanisms to restrict CPU and memory usage by suspending the VM once resources reach a certain level. If you built the distro to be small enough, there would be very little impact on resources by the VM itself, but I would imagine you would still need a fairly large amount of RAM for a solution like that (just to be on the safe side).

Just my 2 cents. I'm not a Python expert, though.

Answer 2

It's probably impossible to be 100% secure, but chroot'ing is a good start. I use several scripts on my servers run in chroot'd jails. One script was the victim of an attack (I'd overlooked a rather clever loophole - oops!) and although the infiltrater broke the script, the jail stopped them from doing any further damage.

If you're going down the minimal python option, have a look at Minimal Python.

Answer 3

You might want to do things like the following:

1. Compile the incoming code to pick out ALL import statements and restrict what can be imported.

2. Check for use of the _import_ and reload functions, also.

3. Define an API for these modules that allows them to run as a separate process, wrapped in a lightweight WSGI server you provide. Fork them as subprocesses and interact through your server API. If they crash or hang, you'll detect this through timeouts.

Answer 4

Lua has the best sandboxing and watchdogging that I've seen to date. My host language is Python. Thus, I've decided to go with Lunatic Python.

Answer 5

FWIW, Apache/mod_wsgi version 3.0 has ability to run its daemon mode processes in a chroot environment.

If only interested in trying to protect the main operating system then that is one possible component in being able to do it.

If you also want to stop users from interfering with other users code and there can be arbitrary numbers of users that can change dynamically then that is harder though as Apache/mod_wsgi doesn't yet have a feature to dynamic daemon process groups and instead relies on static configuration.