ansaurus

Question

Web-ifing a python command line script ?

Answer 1

A:

Your task sounds interesting. :-) A scenario that just came into mind: You continuosly scrape the resources with your home-brew scripts, and push the results into your persistent database and a caching system -- like Redis -- simultanously. Your caching system/layer serves as primary data source when serving client requests. Redis f.e. is a high-performant key-value-store that is capable to handle 100k connections per second. Though only the n latest (say f.e. 50k entries) matter the caching system will only hold these entries and let you solely focus on developing the server-side API (handling connections, processing requests, reading from Redis) and the frontend. The communication between frontend and backend-API could be driven by WebSocket connections. A pretty new part of the HTML5 spec. Though, however, already supported by many browser versions released these days. Alternatively you could fallback on some asynchronous Flash Socket-stuff. Websockets basically allow for persistent connections between a client and a server; you can register event listener that are called for every incoming data/-packet -- no endless polling or other stuff.

kishkash 2010-07-20 12:10:22

Answer 2

+2 A:

A sketch for a solution:

Create an HTML file which contains the layout for your web page, with a dedicated DIV for the output of the script:

<html>
<body>
<div id="scriptoutput"></div>
<script type="text/javascript" src="localhost:8000/runscript"/>
</body>
</html>

This HTML file can be served using any server you wish.

Now, write a simple http server which runs the script and converts each line to a javascript command (example in python):

import os
f = os.popen('ping 127.0.0.1')
for i in range(30):
  ln = f.readline() 
  print "document.getElementById('scriptoutput').innerHTML += '<pre>%s</pre><br/>';" % ln

You can use any CGI/WSGI server for the task, or if performance is not crucial, even use Python's own BaseHTTPServer class.

This would do the trick, as most browsers parse and execute Javascript scripts as they are received (and not only when the request is completed) - note that no polling or server-side storage is needed!

adamk 2010-07-20 12:23:16

I proved python2.7 server.py script running and modified to serve script directory without parameters and the html code as index.html.I run the python code and got error: File "D:/test/javascr.py", line 4, in <module> ln = f.readline()AttributeError: 'tuple' object has no attribute 'readline'And browser showed blank page. something must be missing. Where to put python script name to run for example, say simple print "Hello world"?

Tony Veijalainen 2010-07-20 13:17:20

sorry, it should have been popen instead of popen2 - fixed.

adamk 2010-07-21 04:39:54

Answer 3

A:

Hello, I hope that I understand your need properly.

The idea behind Ajax is to update the content of the page without reloading the whole page. I think it should correspond to your need. You may have to modify your commands if you want to webify them. You may need to get their print logs "on the fly".

Here are some ideas:

Write a very simple page with the possibility to execute commands (menu, form ...)
When the user ask for a command execution, send an ajax query to the server which executes the command.

Your commands need to be modified in order to redirect the sys.stdout to something that stores the print logs into a database. You can do it by assigning to sys.stdout an object with a write function.

class MyDbLogger:
    def __init__(self, ...):
        """Some initialization"""
        ...


 def write(self, s):
    """write into the database"""
    ...


dbout = MyDbLogger(...)
sys.stdout = dbout

The client will poll the server regurlarly to get the content into the database and then write it into the page.
Comet is certainly the technology to investigate in order to have a real-time behavior. That'll avoid the client to poll the server on a regular basis. That can be an improvement to #4 but may be a little more difficult to implement.

I hope it helps

luc 2010-07-20 12:31:10

ansaurus

tags:

views:

answers:

Web-ifing a python command line script ?

related questions