Twisted HTTP Proxy Channel set to None on LostConnection but loseConnection hasn't been called yet

Question

Ok I have been writing a proxy to take http GET requests and translate them into HTTP POST requests (because a lot of media players for python only support GET). So I know am working on caching those results that way I only download a url once, I moved a lot of code from the super class to the sub class and changed it so that I could send the same thing to multiple server connections (e.g if a server connection was made I would just attach it to a download instance if available).

Anyways, it mostly works except the last part for some reason the http channel is set to None before I call loseConnection (this is without any errors like the internet going out etc), this results in an error that a None object has no attribute transport that I am calling loseConnection on.

Here is the code

    self.connectionDone = True
    print self.producers
    for p in self.producers:
        print p
        print p.channel
        print dir(p)
        p.channel.transport.loseConnection()

    self.transport.loseConnection()

and the traceback of self.channel being set to none (traceback module's print_stack on each setattr call)

Note: Look at the last few lines, I think that is where the problem is.

   File "Sharky.py", line 581, in <module>
     reactor.run()
   File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\internet\wxreactor.py", line 145, in run
     self.wxapp.MainLoop()
   File "C:\Program Files\Python 2.6.2\lib\site-packages\wx-2.8-msw-unicode\wx\_core.py", line 8007, in MainLoop
     wx.PyApp.MainLoop(self)
   File "C:\Program Files\Python 2.6.2\lib\site-packages\wx-2.8-msw-unicode\wx\_core.py", line 7303, in MainLoop
     return _core_.PyApp_MainLoop(*args, **kwargs)
   File "C:\Program Files\Python 2.6.2\lib\site-packages\wx-2.8-msw-unicode\wx\_core.py", line 14640, in <lambda>
     lambda event: event.callable(*event.args, **event.kw) )
   File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\internet\_threadedselect.py", line 243, in _interleave
     getattr(self, '_process_' + msg)(*args)
   File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\internet\_threadedselect.py", line 209, in _process_Notify
     _logrun(selectable, _drdw, selectable, method, dct)
   File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\python\log.py", line 84, in callWithLogger
     return callWithContext({"system": lp}, func, *args, **kw)
   File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\python\log.py", line 69, in callWithContext
     return context.call({ILogContext: newCtx}, func, *args, **kw)
   File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\python\context.py", line 59, in callWithContext
     return self.currentContext().callWithContext(ctx, func, *args, **kw)
   File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\python\context.py", line 37, in callWithContext
     return func(*args,**kw)
   File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\internet\_threadedselect.py", line 303, in _doReadOrWrite
     self._disconnectSelectable(selectable, why, method == "doRead")
   File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\internet\posixbase.py", line 253, in _disconnectSelectable
     selectable.connectionLost(f)
   File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\internet\tcp.py", line 519, in connectionLost
     protocol.connectionLost(reason)
   File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\web\http.py", line 1725, in connectionLost
     request.connectionLost(reason)
   File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\web\http.py", line 1287, in connectionLost
     self.channel = None
   File "C:\Documents and Settings\Admin\My Documents\Mercurial\sharky\ProxyServer.py", line 172, in __setattr__
     traceback.print_stack()

This is the none attribute exception I get

Traceback (most recent call last):
  File "C:\Program Files\Python 2.6.2\lib\site-packages\wx-2.8-msw-unicode\wx\_core.py", line 7303, in MainLoop
    return _core_.PyApp_MainLoop(*args, **kwargs)
  File "C:\Program Files\Python 2.6.2\lib\site-packages\wx-2.8-msw-unicode\wx\_core.py", line 14640, in <lambda>
    lambda event: event.callable(*event.args, **event.kw) )
  File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\internet\_threadedselect.py", line 243, in _interleave
    getattr(self, '_process_' + msg)(*args)
  File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\internet\_threadedselect.py", line 209, in _process_Notify
    _logrun(selectable, _drdw, selectable, method, dct)
--- <exception caught here> ---
  File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\python\log.py", line 84, in callWithLogger
    return callWithContext({"system": lp}, func, *args, **kw)
  File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\python\log.py", line 69, in callWithContext
    return context.call({ILogContext: newCtx}, func, *args, **kw)
  File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\python\context.py", line 59, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\python\context.py", line 37, in callWithContext
    return func(*args,**kw)
  File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\internet\_threadedselect.py", line 303, in _doReadOrWrite
    self._disconnectSelectable(selectable, why, method == "doRead")
  File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\internet\posixbase.py", line 253, in _disconnectSelectable
    selectable.connectionLost(f)
  File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\internet\tcp.py", line 677, in connectionLost
    Connection.connectionLost(self, reason)
  File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\internet\tcp.py", line 519, in connectionLost
    protocol.connectionLost(reason)
  File "C:\Program Files\Python 2.6.2\lib\site-packages\twisted\web\http.py", line 489, in connectionLost
    self.handleResponseEnd()
  File "C:\Documents and Settings\Admin\My Documents\Mercurial\sharky\ProxyServer.py", line 103, in handleResponseEnd
    p.channel.transport.loseConnection()
exceptions.AttributeError: 'NoneType' object has no attribute 'transport'

Answer 1

Connections may be lost without the internet going out. All it takes is for one side of the connection to call shutdown() or close(). Have you ruled that out? And even if you have, for the code to be correct, it needs to handle that possibility anyway, because it might happen at some other time. See Request.notifyFinish.