Is it possible to read Wikipedia using Python requests library?

To read a content from a given URL I do the following:

import requests
proxies = {'http':''}
url = ''
r = requests.get(url, proxies = proxies)
print r.text.encode('utf-8')

And it works fine! I get the content.

However, if I use another URL:

url = ''

It does not work. I get an error message that starts with:

requests.exceptions.ConnectionError: ('Connection aborted.', error(10060

Is Wikipedia blocking automatic requests?


I tried to set a user agent in the following way:

headers = {'User-Agent':'Mozilla/5.0'}
r = requests.get(url, proxies = proxies, headers = headers)

Unfortunately it does not help. I still get the same error.


Now I am confused. If I try to get content from with setting proxy, I get it. If I do not set proxy, I get content generated by proxy. This behavior I can understand. Now, if I try to get content from Wikipedia, I get the same error message independently on whether I set or do not set proxy. So, I do not understand where this error message comes from Wikipedia or proxy (both options cannot be true).


The problem was resolved by replacing:

proxies = {'http':''}

with the following line:

proxies = {'http':'', 'https':''}

Need Your Help

The last UNION in SQL ignores existing INDEX

sql stored-procedures indexing query-optimization union

This is a refined problem I have been struggling with for a couple of days now.