Home Original page

while loop with f.readline()

Jeff Epler jepler at inetnebr.com
Thu Feb 15 19:44:03 EST 2001 
On Thu, Feb 15, 2001 at 05:04:19PM -0700, Sean Reifschneider wrote:
> On Thu, Feb 15, 2001 at 01:22:39PM +0000, Jeff Epler wrote:
> >I'd say, "xreadlines will never hold more than a few KB of the file
> >in memory", but that's wrong if you have a file with a multi-megabyte line.
> 
> Say "xreadlines()" will never hold more than a single line of the file in
> memory.  Note the implications if you feed it a file with multi-megabyte
> line(s).

Well, the full explanation is that it uses 'readlines(sizehint)'
internally, and it never holds more than one chunk as returned by
readlines().

So
	for line in file.xreadlines(): pass
has the memory behavior of
	while 1:
		lines = file.readlines(sizehint)
		if not lines: break
		for line in lines: pass
but the flow control of 
	for line in file.readlines(): pass

Jeff
PS I don't recall what value of sizehint is used in 2.1, it was changed
when the patch was integreated, after some benchmarking on python-dev
More information about the Python-list mailing list