views:

533

answers:

3

I'm writing web application where I need to push data from server to the connected clients. This data can be send from any other script from web application. For example one user make some changes on the server and other users should be notified about that.

So my idea is to use unix socket (path to socket based on user #ID) to send data for corresponding user (web app scripts will connect to this socket and write data). The second part is ThreadingTCPServer which will accept user connections and push data from unix socket to user over TCP socket connection.

Here is the workflow:

  1. Used connect to the TCP Server
  2. Django script open unixsocket and write data to it.
  3. TCP Server read data from unix socket and send it to open connection with user.

I hope you understand my idea :)

So, I have 2 questions:

1.What do you think about my idea in general? Is it good or bad solution? Any recommendations are welcome.

2.Here is my code.

import SocketServer
import socket
import netstring
import sys, os, os.path
import string
import time

class MyRequestHandler(SocketServer.BaseRequestHandler):
    def handle(self):
        try:
            print "Connected:", self.client_address

            user = self.request.recv(1024).strip().split(":")[1]
            user = int(user)
            self.request.send('Welcome #%s' % user)

            self.usocket_path = '/tmp/u%s.sock' % user
            if os.path.exists(self.usocket_path):
                os.remove(self.usocket_path)

            self.usocket = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
            self.usocket.bind(self.usocket_path)
            self.usocket.listen(1)

            while 1:
                usocket_conn, addr = self.usocket.accept()
                while 1:
                    data = usocket_conn.recv(1024)
                    if not data: break
                    self.request.send(data)
                    break
                usocket_conn.close()
                time.sleep(0.1)

        except KeyboardInterrupt:
            self.request.send('close')
            self.request.close()

myServer = SocketServer.ThreadingTCPServer(('', 8081), MyRequestHandler)
myServer.serve_forever()

and I got an exception

  File "server1.py", line 23, in handle
    self.usocket.listen(1)
  File "<string>", line 1, in listen
error: (102, 'Operation not supported on socket')
A: 

This is too complex. Use the socketserver module, please.

S.Lott
myServer = SocketServer.ThreadingTCPServer(('', 8081), MyRequestHandler)Isn't it?
Vladimir Prudnikov
That's what http://docs.python.org/library/socketserver.html#asynchronous-mixins says
S.Lott
A: 

I think You should not use unix sockets. If Your app will (someday) become popular or mission-critical, You won't be able to just add another server to add scalability or to make it redundant and fail-safe.

If, on the other hand, You will put the data into f.e. memcached (and user's "dataset number" as the separate key) You'll be able to put data into memcached from multiple servers and read it from multiple servers. If user will disconnect and connect back from some other server, You'll still be able to get the data for him.

You could also use a database (to make it more fail-safe), or a mix of database and memcached if You like, but I have seen an app using unix sockets in the way You are trying to, and the programmer regreted it later. The table could have userid, data and timestamp columns and You could remember the last timestamp for the given user.

Reef
The maximum amount of connected users is, let's say, 1000 users at the same time. I think it's no so much for one server to handle 1000 connections.Database is used for that, of course. Thanks for memcached solution, i'll think about it.. but not sure if it is possible to "listen" for new data instead of checking in every second for example.
Vladimir Prudnikov
Do You really, really need the data instantly? In most cases, You don't. I admit, sometimes You do, but not as often as people think. BTW: You can get more than 1000 open sockets if You increase ulimit.
Reef
A: 

You should use Twisted, or, more specifically, Orbited, to push data to your clients. The sample code you posted has a number of potential problems, and it would be a lot harder to figure out what they all are than for you to just use a pre-built piece of code to do what you need.

Glyph