Home Original page

fastest way to merge lists.

Alex alex at somewhere.round.here
Sat Jan 8 22:53:41 EST 2000 
If speed is a big issue for you, might consider using Aaron Watters'
kjbuckets extension.  It provides a set type, and is supposed to be much
faster than using dictionaries.  (I believe the claim, but I've never
tested it.  I use it mainly so I don't have to write d[k]=None all the
time.)  I use it all the time for just the sort of thing you describe.

See
http://www.pythonpros.com/arw/kjbuckets/

Also, if you don't go that route, there may be some mileage in using
operator.setitem.  I suppose I should time it myself, since I'm
suggesting it, but I can't be bothered.  Something like

import operator
d={}
l=list1+list2+list3+list4+list5 
map(operator.setitem,len(l)*[d],l,len(l)*[1])

...might be worth looking into.

Oh, and another thing you could do is sort the whole lot, and run over
them to pick out the first instance of each string:

l=list1+list2+list3+list4+list5 
l.sort()
last=None
merged=[]
for element in l:
  if element!=last:
    merged.append(element)
    last=element

Alex.
More information about the Python-list mailing list