Goodbye URLLIB2, I’m not going to miss you!

Hot on the trail of another monster chunk of code writing, I found I was stuck in a trap I made for myself. I was at the end of my understanding of Python, URLLIB2 and IMC eAPI’s. I was trying to HTTP POST a chunk of xml into IMC’s Configuration Template library. This was quite perplexing….I tried every thing I could and no matter how I changed the programming, I would still get the dreaded 500 Internal Server error….you know what I’m talking about. I feel you cringing right now!

So, after about a billion Google searches I started seeing the stuff called Requests. Developed by a guy named Kenneth Reitz, it is my new favorite plaything.

Take a look at this sample of URLLIB2 code to get the POST working.

cj = CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
resp =
print c_url
print cj._cookies
# get to the cookie
foo = cj._cookies[‘’][‘/imc/’][‘JSESSIONID’].value
foo1 = “JSESSIONID=%s” % (foo)
# Authenticate
authhandler = urllib2.HTTPDigestAuthHandler()
authhandler.add_password(“iMC RESTful Web Services”, server, user, passw)
opener = urllib2.build_opener(authhandler)
a2 = urllib2.install_opener(opener)

# Now build header to send HTTP POST for controller file
#agent = “Apache-Httpclient/4.1 (java 1.5)”
pagehandle=urllib2.Request(my_url,c_data) #Add values forces POST
pagehandle.add_header(‘Content-Type’,’application/xml; charset=utf-8′)
pagehandle.add_header(‘USer-Agent’,’Apache-Httpclient/4.1 (java 1.5)’)
pagehandle.add_header(‘Cookie’, foo1)
c_result = urllib2.urlopen(pagehandle)

If you ask me, it was good for its time but very confusing with the openers and handlers.

Now here is the same code using requests..

s = requests.session() # This keeps the session open

# Cookie Factory
r = s.get(my_url)
cook = r.headers[‘set-cookie’]
# Strip out the JSESSIONID
x1, x2, x3 = cook.split(‘;’)
# Set up Authentication header info

# POST with requests (Probably don’t need all these headers…but they don’t hurt)

headers = {‘Accept’: ‘application/xml’, ‘host’: ‘’, ‘Content-Type’: ‘application/xml; charset=utf-8’, ‘Accept-encoding’: ‘application/xml’, ‘Connection’: ‘Keep-Alive’, ‘User-Agent’: ‘Apache-HttpClient/4.1 (java 1.5)’, ‘Cookie’: x1, ‘Cookie2’: ‘$Version=1’}

# This sends the controller xml data to the IMC server

r =, data=c_data, auth=auth, headers=headers)

From here I can use: r.headers, r.return_code because everything the remote site sent back is in the variable “r”.

Thank you Mr. Reitz!!!

P.S. In the end it was a xml tag that I had given a wrong name. Correct name wasand I had… humbling!