Home Original page

Setting referrer using urllib

Phil Mayes nospam at bitbucket.com
Wed Jul 26 02:17:04 EDT 2000 
Thomas Weholt wrote in message <397bff4b.18231775 at news.online.no>...
>Hi,
>
>I`m trying to access various search interfaces on online sites using
>urllib, contrstructing a url, opening the location and reading the
>output, but many of them, well, most of them, return some permission
>denied response. Somebody told me this was due to the fact that urllib
>didn't have referrer set correctly, so the site wouldn't allow it.

Sounds unlikely, but let me point out that the HTTP spec spells
referrer incorrectly with three Rs not 4, so:
    h = httplib.HTTP(domain)
    h.putheader('Referer', ref)

But I think that for a robust solution you will also need to support
some or all of:
  * https (secure sockets)
  * 301/302 redirection
  * <META> redirection
  * cookies

Get some software that captures all TCP/IP data, then run Netscape and
your program and see where they differ.
--
Phil Mayes    pmayes AT olivebr DOT com
Olive Branch Software - home of Arranger
http://www.olivebr.com/
Check out our PalmPilot news page at http://www.olivebr.com/pilot/news.htm
More information about the Python-list mailing list