curl

Few Google Checkout Questions

I am planning to integrate a Google Checkout payment system on a social networking website. The idea is that members can buy "tokens" for real money (which are sort of the website currency) and then they can buy access to some extra content on the website etc. What I want to do is create a Google Checkout button that takes a member to t...

how to query restful rails app with curl?

here's my problem.. resource: user method: create I call curl like this: curl -X POST -H 'Content-type: text/xml' -d '<xml><login>john</login><password>123456</password></xml>' http://0.0.0.0:3000/users but the params hash in rails look like this: {"xml"=> {"login"=>"luca", "password"=>"123456"}} I want it to look like this: {"lo...

How do I gurantee that utf-8 characters are scraped accurately using CURL in php?

Hello, I am scraping webpages (using php's curl) that have accented characters (like "é"). In the source of those webpages, those characters are written using utf-8 (they are not html encoded.) However, when the result is produced using the following code, I get question marks instead of the accented characters. $ch = curl_init(); $ti...

How to switch from POST to GET in PHP CURL

I have tried switching from a previous Post request to a Get request. Which assumes its a Get but eventually does a post. I tried the following in PHP : curl_setopt($curl_handle, CURLOPT_POSTFIELDS, null); curl_setopt($curl_handle, CURLOPT_POST, FALSE); curl_setopt($curl_handle, CURLOPT_HTTPGET, TRUE); What am I missing? Additional ...

Login to a site and then POST to a page in it

I have to login to a page using three parameters, after that I have to POST two parameters to another page inside the site I've just logged in. So far I've got a cookie with this: curl -c cookie.txt -d "username=username&pwd=pwd&domain=mydomain" http://myurl ...inside of the cookie I have a JSESSION id. I use the cookie as follows: ...

managing curl output in php

How do i hide the output from curl in php? Or modify it? My code as it stands is the following; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL,$url); curl_setopt($ch, CURLOPT_POST, 1); curl_setopt($ch, CURLOPT_USERPWD, PSSWDINFO); $result= curl_exec ($ch); curl_close ($ch); The problem is that is spews out th...

Can I use cURL to grab a html table?

Hi all, I'm trying to use cURL to grab an external web page to put into my own website, it's basically a "ladder" of a sports team, I contacted them, but they do not have a RSS feed of the ladder, so I'm trying to obtain the ladder by other means, is it possible to grab everything between < table > and < / table > using cURL? I can grab...

Curl tutorials & resources

Could anyone share good resources/tutorials about php curl? ...

Searching Twitter

How can I make a simple search on Twitter through PHP, Java, or CURL JSON? ...

Can't get cookies with php curl from www.dramexchange.com

I can get cookie from the others but not from this site www.dramexchange.com? Anybody know why? Maybe someone can do it with php and curl? :> $ch = curl_init(); curl_setopt($ch, CURLOPT_COOKIEJAR, "cookieFileName"); curl_setopt($ch, CURLOPT_URL,"http://www.dramexchange.com"); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); ob_start(); ...

How is a POST request from cURL in PHP like/unlike a POST request from loadVars in Flash?

All, Sorry - this is probably a very strange question. I'm working on a Flash RIA. One of the things it does is call an ASP page (that resides on another domain) to retrieve some data. However, that ASP page requires users to log-in to that site before they're allowed to call that ASP page. So, my first attempt at getting this to wor...

PHP: UTF 8 characters encoding

Hey I am scraping a list of RSS feeds by using cURL, and then I am reading and parsing the RSS data with SimpleXML. The sorted data is then inserted into a mySQL database. However, as notice on http://dansays.co.uk/research/MNA/rss.php I am having several issues with characters not displaying correctly. Examples: ‘Guitar Hero: Van ...

Send HTTP request as fast as possible

In order to test my program, I want to send about 50 http requests as fast as possile. Currently I'm using bash script which calls curl & to send each request like curl "http://my.host.com/test.php?param=1" & curl "http://my.host.com/test.php?param=2" & curl "http://my.host.com/test.php?param=100" & but it seems not fast enough. I gue...

php - multiple requests with curl (NOT 'multi-threaded')

If i want to do multiple things (that require cookies) with curl, for example: login to (my own) blog then automatically submit a blog post do I do this in one curl instance before curl close or do i close first session and then start second one for second task? (Sorry if it's dumb question but i can't quite get it. Generally all exa...

NTLM proxy without password?

I work on a corporate windows network (which I log in to) with a HTTP proxy. When I use Internet Explorer it magically uses the proxy without me needing to type in my password. Certain other programs seem to manage this too, like JavaWebStart has a "use browser settings" option. However when I use scripts/programs like curl or wget to f...

characters changed in a Curl request.

When I look at the XML data feed i get with the below code, special characters are correct in the XML code. However when Curl returns the data, characters like "ó" and "ä" are converted into resp. "ó" and "ä". This conversion happens to all special characters, these 2 are just an example. $myvar = curl_init(); $myURL = "http://someurl...

php ssl curl : object moved error

Im developing a php script to scrape this website and email me the data to me. It seems to be logging in correctly because when the script runs, it seems to redirect and give me a message saying Object moved Here and the here is linked to the default.aspx page which is what exactly happens when I manually login. Below is my script: ...

How to set a timeout on PHP5 curl calls? Published CURL options do not seem to work...

We've written a script that pulls data from an external server. If the server goes down we don't want our server waiting for the data since we process a lot of data and we don't want it bogged down. To address this, we're trying to timeout our curl calls if they take more than a couple hundred milliseconds. I found some documentatio...

Dynamic screen scraping with PHP and getting past javascript

I have a website that provides a price comparison for students textbooks. I wrote a ruby script to go class by class and grab all the textbook information and store it in a database that the website can query for book information. The problem is that the bookstore keeps changing the books needed for each class so I need to figure out a w...

trying to parse weird formatted xml in php

I am trying to chop XML data into usable strings to reuse them later on in my script. I am receiving the data via a Curl request and his goes great. now chopping the data kills me.. this a part of the XML I am receiving (the whole data part is about 90 lines) <professions> <skill key="IT Specialist" maxage="40" group="IT" worked="...