«
»

Python proxy client connections requiring authentication using urllib2 ProxyHandler

October 27, 2008, 11:25 am

I’ve been playing around with trying to pass HTTP connections through a proxy server requiring authentication from Python and after a few hours of hunting around I’ve managed to find enough bits of information to cobble together into a working example.  Since I couldn’t find such an example myself I’ve put this one together.  It looks simple now, which it is (thanks Python!), but unfortunately I couldn’t find any documentation that presented it this clearly.

Just substitute the appropriate variable values and it should run as is.

?View Code PYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

# urllib2_proxy_handler.py
#
# Author: Wayne Koorts
# Date:   27/10/2008
#
# Example for using urllib2.urlopen() with a proxy server requiring authentication
 
import urllib2
 
uri = "http://www.python.org"
http_proxy_server = "someproxyserver.com"
http_proxy_port = "3128"
http_proxy_realm = http_proxy_server # Worked in my (limited) testing environment.
http_proxy_user = "username"
http_proxy_passwd = "password"
 
# Next line = "http://username:password@someproxyserver.com:3128"
http_proxy_full_auth_string = "http://%s:%s@%s:%s" % (http_proxy_user,
                                                      http_proxy_passwd,
                                                      http_proxy_server,
                                                      http_proxy_port)
 
def open_url_no_proxy():
    urllib2.urlopen(uri)
 
    print "Apparent success without proxy server!"
 
def open_url_installed_opener():
    proxy_handler = urllib2.ProxyHandler({"http": http_proxy_full_auth_string})
 
    opener = urllib2.build_opener(proxy_handler)
    urllib2.install_opener(opener)
    urllib2.urlopen(uri)
 
    print "Apparent success through proxy server!"
 
if __name__ == "__main__":
    open_url_no_proxy()
    open_url_installed_opener()

Tested using an Ubuntu 8.04 Server Edition proxy server with a mostly out-of-the-box Squid installation using ncsa_auth authentication.

-Wayne

Tags: , , , ,
Category: Geeky, Software Development  |  Comment (RSS)  |  Trackback

Leave a Reply