I am reluctant to post this because (a) I might have made some dumb mistake in my code, simple as it is, and (b) the problem might be well-known or even hopeless because the cgi module may not be WSGI compliant, but if this isn't going to be fixed then at least the problem should be described in the cgi module documentation.

I run an HTTPServer with a CGIHTTPRequestHandler.
I open an HTML file with a trivial form that simply POSTs an upload of a single file.
I browse, select a file, and click submit.
The web request never completes -- the browser just waits for a response.
Interrupting the server with ^C^C produces a backtrace that indicates it is stuck trying to decode the file contents.

The attached zip file contains:
    cgi-server.py
    cgi-post.html
    cgi-bin/cgi-test.py
    exception.txt
The latter is the backtrace output from when I interrupt the server.

This is on OS X 10.5 with either Python3.1 or Python3.2 from the repository.

This doesn't look like a documentation bug to me - handling of uploaded files via CGI *should* work, even if CGI is not the best way to do that.

The example works for me if I make this change:

--- Lib/cgi.py	(revision 81862)
+++ Lib/cgi.py	(working copy)
@@ -608,7 +608,7 @@
         parser = email.parser.FeedParser()
         # Create bogus content-type header for proper multipart parsing
         parser.feed('Content-Type: %s; boundary=%s\r\n\r\n' % (self.type, ib))
-        parser.feed(self.fp.read())
+        parser.feed(self.fp.read(self.length))
         full_msg = parser.close()
         # Get subparts
         msgs = full_msg.get_payload()


However this seems iffy to me because the content length presumably counts bytes whereas self.fp seems to be a text file, but since most HTTP clients don't close the connection, without some kind of boundary on the read() call it just hangs forever.

Also someone pointed out to me offline that this change may be needed, separately (though I haven't confirmed this yet):

--- Lib/cgi.py	(revision 81862)
+++ Lib/cgi.py	(working copy)
@@ -233,6 +233,7 @@
         lines = []
         while 1:
             line = fp.readline()
+            line = line.decode()
             if not line:
                 terminator = lastpart # End outer loop
                 break

Could this be related to issue 4953?

There are 2 different problems :
- handling of data by cgi.FieldStorage (issue 4953) : fixed since version 3.2
- in http.server.CGIHTTPRequestHandler, for POST requests on Windows, before opening the subprocess in run_cgi() all data is read by a *single* call to self.rfile.read(). If not all bytes are read by this single call, which is the case for a large file upload, only the read data are processed

The attached patch modifies http.server :
- if all data are read in the first call to self.rfile.read() (that is, if its length is equal to the Content-length header), process them
- if not, store all data in a temporary file (not in memory) and set the stdin argument of subprocess.Popen to this temporary file

With this patch, the tests provided by Mitchell all work on my PC (Windows XP Pro SP3)

components: + Windows, - Library (Lib)
title: cgi handling of POSTed files is broken in Windows -> [Windows] cgi handling of POSTed files is broken in Windows

nosy: + orsenthil

title: urlparse -> cgi handling of POSTed files is broken