Home Original page

"Do this, and come back when you're done"

Donn Cave donn at drizzle.com
Sat Dec 13 12:57:37 EST 2003 
Quoth Kamus of Kadizhar <yan at NsOeSiPnAeMr.com>:
| I have the following function which generates MD5 hashes for files on a 
| local and remote server.  The remote server has a little applet that 
| runs from inetd and generates an MD5 hash given the file name.
|
| The problem is that it takes 2+ minutes to generate the MD5 hash, so 
| this function takes about 5 minutes every time it is called.  Since the 
| first MD5 hash is generated on a remote machine, the local machine does 
| nothing but wait for half that time.
|
| Is there any way to rewrite each half of the function to run in the 
| background, so to speak, and then have a master process that waits on 
| the results?  This would cut execution time in half more or less.

Yes.  I may be missing something here, because the followups
I have seen strike me as somewhat misguided, if they're not
just fooling with you.  You already have two independent threads
or processes here, one on each machine.  All you need to do is
take the results from the remote machine AFTER the local computation.
Move the line that says "remoteMD5hash = Socket.recv(256)" to
after the block that ends with "localMD5hash = hasher.hexdigest()".
No?

	Donn Cave, donn at drizzle.com
-----------------------------------
| # checkMD5
| def checkMD5(fileName, localDir):
|     # get remote hash
|     Socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
|     Socket.connect((MD5server,888))
|     #throw away ID string
|     Socket.recv(256)
|     Socket.send(fileName+'\n')
|     remoteMD5hash = Socket.recv(256)
|
|     # get local hash
|     try:
|        file=open(makeMovieName(localDir,fileName), 'r')
|     except IOError:
|        localMD5hash = '0'
|     else:
|        hasher = md5.new()
|        while True:
|           chunk = file.read(1024)
|           if not chunk:
|              break
|           hasher.update(chunk)
|        localMD5hash = hasher.hexdigest()
|     if Debug: print "local:",localMD5hash, "remote:",remoteMD5hash
|     return localMD5hash.strip() == remoteMD5hash.strip()
|
| -Kamus
More information about the Python-list mailing list