urllib

Loading url with cyrillic symbols

Hi guys, I have to load some url with cyrillic symbols. My script should work with this: http://wincode.org/%D0%BF%D1%80%D0%BE%D0%B3%D1%80%D0%B0%D0%BC%D0%BC%D0%B8%D1%80%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5/ If I'll use this in browser it would replaced into normal symbols, but urllib code fails with 404 error. How to decode correctly th...

how to encode a url with urllib or urllib2

I want a url like example.com/page.html to somthing like example.com/a$xDzf9D84qGBOeXkXNstw%3D%3D106 ...

Problem with python urllib

I'm getting an error when ever I try to pull down a web page with urllib.urlopen. I've disabled windows firewall and my AV so its not that. I can access the pages in my browser. I even reinstalled python to rule out it being a broken urllib. Any help would be greatly appreciated. >>> import urllib >>> h = urllib.urlopen("http://www.goog...

Does urllib or urllib2 in Python 2.5 support https?

Thanks for the help in advance. I am puzzled that the same code works for python 2.6 but not 2.5. Here is the code import cgi, urllib, urlparse, urllib2 url='https://graph.facebook.com' req=urllib2.Request(url=url) p=urllib2.urlopen(req) response = cgi.parse_qs(p.read()) And here is the exception I got Traceback (most recent call l...

Problem with urllib

I wrote this code: import urllib proxies = {'http': 'http://112.65.135.54:8080/'} opener = urllib.FancyURLopener(proxies) r = opener.open("http://www.python.org/") print r.read() and when I execute it this program works fine, and send for me source code of python.org But when i use this: import urllib proxies = {'http': 'http://80...

Cookies with urllib

This will probably seem like a really simple question, and I am quite confused as to why this is so difficult for me. I would like to write a function that takes three inputs: [url, data, cookies] that will use urllib (not urllib2) to get the contents of the requested url. I figured it'd be simple, so I wrote the following: def fetch(u...

Errno socket error in python

i wrote this code : import random import sys import urllib openfile = open(sys.argv[1]).readlines() c = random.choice(openfile) i = 0 while i < 5: i=i+1 c = random.choice(openfile) proxies = {'http': c} opener = urllib.FancyURLopener(proxies).open("http://whatismyip.com.au/").read() ::: I put 3 proxy in a txt file . : http://211...

python urllib post question

hello ALL im making some simple python post script but it not working well. there is 2 part to have to login. first login is using 'http://mybuddy.buddybuddy.co.kr/userinfo/UserInfo.asp' this one. and second login is using 'http://user.buddybuddy.co.kr/usercheck/UserCheckPWExec.asp' i can login first login page, but i couldn't login...

Looping through a directory on the web and displaying its contents (files and other directories) via Python

In the same vein as http://stackoverflow.com/questions/2593399/process-a-set-of-files-from-a-source-directory-to-a-destination-directory-in-pyth I'm wondering if it is possible to create a function that when given a web directory it will list out the files in said directory. Something like... files[] for file in urllib.listdir(dir): ...

Python 2.6 -> Python 3 (ProxyHandler)

Hallo, I wrote a script that works with a proxy (py2.6x): proxy_support = urllib2.ProxyHandler({'http' : 'http://127.0.0.1:80'}) But in py3.11x there is no urllib2 just a urllib... and that doesnt support the ProxyHandler How can I use a proxy with urllib? Isnt Python 3 newer then Python 2? Why did they remove urllib2 in a newer vers...

Why can't I get Python's urlopen() method to work?

Why isn't this simple Python code working? import urllib file = urllib.urlopen('http://www.google.com') print file.read() This is the error that I get: Traceback (most recent call last): File "C:\workspace\GarchUpdate\src\Practice.py", line 26, in <module> file = urllib.urlopen('http://www.google.com') File "C:\Python26\lib\u...

I'm getting an error when trying to open a website url with Python 3.1, urllib & json: operation was attempted on something that is not a socket

I'm getting an error when trying to open a website url with Python 3.1, urllib & json urllib.error.URLError: Here's the code. The first website loads fine. The second one import json import urllib.request import urllib.parse import util # This one works fine response = urllib.request.urlopen('http://python.org/') html = response....

Urllib's urlopen breaking on some sites (e.g. StackApps api): returns garbage results

I'm using urllib2's urlopen function to try and get a JSON result from the StackOverflow api. The code I'm using: >>> import urllib2 >>> conn = urllib2.urlopen("http://api.stackoverflow.com/0.8/users/") >>> conn.readline() The result I'm getting: '\x1f\x8b\x08\x00\x00\x00\x00\x00\x04\x00\xed\xbd\x07`\x1cI\x96%&/m\xca{\x7fJ\... I'm...

Downloading a picture via urllib and python.

So I'm trying to make a Python script that downloads webcomics and puts them in a folder on my desktop. I've found a few similar programs on here that do something similar, but nothing quite like what I need. The one that I found most similar is right here (http://bytes.com/topic/python/answers/850927-problem-using-urllib-download-imag...

FancyURLopener hangs when not passing a proxy or passing an ISA one

Hi, I'm trying to create a script to automate some actions involving chained GET and POST HTTP requests. I'm using Python 3.1 on a Windows XP SP3 machine, and I need to interact to a web server located on our intranet (so no proxy should be involved). Mind that I'm quite new to Python so coding class inheritance, overriding and such is...

Python 2 vs. Python 3 - urllib formats

I'm getting really tired of trying to figure out why this code works in Python 2 and not in Python 3. I'm just trying to grab a page of json and then parse it. Here's the code in Python 2: import urllib, json response = urllib.urlopen("http://reddit.com/.json") content = response.read() data = json.loads(content) I thought the equiv...

Make urllib retry multiple times

My Python application makes a lot of HTTP requests using the urllib2 module. This application might be used over very unreliable networks where latencies could be low and dropped packets and network timeouts might be very common. Is is possible to override a part of the urllib2 module so that each request is retried an X number of times ...

Django request XML file with SSL IO error

I'm making a Django website and am fairly new. In this webapp I need to use this API which will spit out an xml file with the requested data from the database. Basically the API URL is: https://adminuser:[email protected]/database.getdata?arg=1&amp;arg2=0 So in my python views.py I have: def fetch_xml(url): import urllib ...

Help with HTML parsing and sending requests to a web server

Hello, I'm working on a small project and I've run into a small problem. The script I have needs to fetch a website and find a specific value in the source HTML file. The value is like this: id='elementID'> <fieldset> <input type='hidden' name='hash' value='e46c945fe32a3' /> </fieldset> Now I'm been trying to use the Elemen...

Help with urllib + proxy in Python

Hey ppl... My program isn't running properly as should be... I'm getting only the error message (except part) of the urlopen with the proxy... why? At least, one of the proxy was tested and work correctly... please, some one take a look on the code here: http://pastebin.com/cBfv5H8J edit: the code doesn't work on the first try part,...