Message480

Author thimm
Recipients smart-list-disabled
Date 2006-05-08.09:03:13
Content
W/o keep-alive smart's concurrent downloading kills a web server, as each
package/metadata file opens up a new concurrent httpd process on the other side.

A typical smart session on a rather often updated system shows up to 15 httpd
processes simultaneously serving this one IP.

There are keep-alive solutions for urllib2, for instance urlgrabber's
keepalive.py that can be used as a handler for urllib2. This looks easy enough
for me to try patching up smart with it, if it's considered useful.

But most probably keep-alive is not enough as smart deliberately fires up
package retrievals in parallel, and keep-alive is only of help for reusing
connections. If a connection is still in use, you end up creating a new one.
Therefore a serialization procedure is neccessary when the packages come from
the same host (or at least the same channel).
History
Date User Action Args
2006-05-08 09:03:17thimmsetmessageid: <1147078995.87.0.862541933723.issue147@labix.org>
2006-05-08 09:03:17thimmsetrecipients: + smart-list-disabled
2006-05-08 09:03:17thimmlinkissue147 messages
2006-05-08 09:03:16thimmcreate