Home Original page

how to read the last line of a huge file???

John O'Hagan research at johnohagan.com
Sun Jan 30 00:37:54 EST 2011 
On Sat, 29 Jan 2011, Aahz wrote:
> In article <mailman.1412.1296196161.6505.python-list at python.org>,
> 
> John O'Hagan <research at johnohagan.com> wrote:
[...]
> >
> >def lastline(filename):
> >    offset = 0
> >    line = ''   
> >    with open(filename) as f:
> >        while True:
> >            offset -= 1
> >            f.seek(offset, 2)
> >            nextline = f.next()            
> >            if nextline == '\n' and line.strip():
> >                return line          
> >            else:
> >                line = nextline
> 
> It's a Bad Idea to mix direct file operations with the iterator API.

I didn't know that; from the docs on file objects:

"As a consequence of using a read-ahead buffer, combining next() with other file 
methods (like readline()) does not work right. However, using seek() to 
reposition the file to an absolute position will flush the read-ahead buffer."

You are right in general, but the second sentence is why I got away with it in 
this case.

> Use f.read() instead of f.next().

Which actually ends up improving the code as well:

def lastline(filename):
    offset = 0
    with open(filename) as f:
        while 1:
            f.seek(offset, 2)
            if f.tell() == 0:
                return f.read().strip()
            line = f.read()
            if line.strip() and line[0] == '\n':
                return line.strip()
            offset -= 1

although Tim Chase's solution covers files with very long lines.


John
More information about the Python-list mailing list