curl

Why is the console output returning to the beginning of the line? OR Why is my concatenation not concatenating?

I was inspired by another question to write a script (or rather a one-liner) to grab random Wikipedia pages. Here's what I've got so far: # Grab the HTTP header response from Wikipedia's random page link curl 'http://en.wikipedia.org/wiki/Special:Random' -sI # Search STDIN for the Location header and grab its content perl -wnl -e '/Lo...

CURL Progress - Only Show Percentage

Hello all, Is there an option for the CURL command to show only a percentage output to shell rather than all of this, for example: % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 12.4M 100 12.4M 0 0 4489k 0 0...

CURL Progress Bar: How to pipe and extract numbers only using grep?

This is what I have so far: [my1@graf home]$ curl -# -o f1.flv 'http://osr.com/f1.flv' | grep -o '*[0-9]*' ####################################################################### 100.0% I wish to use grep and only extract the percentage from that progress bar that CURL outputs. I think my regex is not correct and I am also not sure ...

How can I attach a file with a PHP cURL XML Call

I'm using the Amazon AIMS API to upload a an inventory file and I'm having an issue with the cURL call to upload the file. The documentation is very limited, so there is no example code that helps out in this. This is what I have so far of the cURL call: // $FILENAME is filename of the CSV file being uploaded: $inventory = fopen($FILE...

Payment Gateway using cURL with SSL?

I am processing credit cards using a payment gateway. To POST the data to their servers, I am using cURL in PHP. I have an SSL certificate issued to my domain, to ensure all POST'ed data is encrypted. Because the SSL certificate is already installed, do I still need to use the SSL options for cURL? If so, which of the options do I ne...

Problem with CURL posting data

I'll keep this nice and short. I'm trying to post a value to a remote form. This code is working perfectly: $url = "http://eirestudio.net/blog/"; // URL to POST FORM. $post_fields = "s=test"; $ch = curl_init(); // Initialize a CURL session. curl_setopt($ch, CURLOPT_URL, $url); // Pass URL as parameter. curl_setopt($ch, CURLOPT_POST...

How to go about trawling through pagination results with PHP/Curl?

Say for example I wanted grab the results from the first 3 pages in Google. So basically I want the top 30 results. I can go about grabbing the first 10 on page 1, but how do I tell Curl that once page one is done, "click" the second page and so on? ...

How do you get libcurl libraries to work with InstantRails?

I'm working on a windows machine and trying to get the curb plugin to work, first thing I realized was I needed the curl library installed on my machine, so that's what I'm looking to do. I simply downloaded a curl library, a curllib library, and have them located at c:/curl and c:/curllib respectively. I setup my environmental variabl...

has anyone succeeded in passing multiple params to the gmail contacts api?

The api (http://code.google.com/apis/gdata/docs/2.0/reference.html#Queries) states that I can pass multiple params to retrieve my contacts, a la... http://www.google.com/m8/feeds/contacts/foo%40gmail.com/full?max-results=500&alt=json However, it appears to only pay attention to the first param and ignore any subsequent. I'm using CU...

how to pass the url value in curl php

$description = "some test data and url"; $description .="http://www.mydata.com?test=1&user=4&destination=645&source=stackoverflow"; curl_setopt($sch, CURLOPT_URL, "myserverurl"); curl_setopt($sch, CURLOPT_HEADER, 0); curl_setopt($sch, CURLOPT_POST, true); curl_setopt($sch, CURLOPT_RETURNTRANSFER , 1); curl_setop...

trying to use curl to download a series of files

I'm trying to use curl to download a seried of files in the following format: http://example.com/001.jpg .. http://example.com/999.jpg So I used this command: time curl "http://example.com/[0-9][0-9][0-9].jpg" -o "#1#2#3.gif" But some of the files don't exist, and that command will create the files on my end bu...

how to use htmlparsing and curl in JAVA for this task...?

I'm trying to write a program that takes company names from a text file and searches them on a search engine website (SEC's Edgar search). Each search usually comes up with 1-10 unique search result links and so I want to use curl to click on the link with the relevant company name. The link page has a brief summary with the term "state ...

download a file from ftp using curl and php

I'm trying to download a file from an ftp server using curl and php but I can't find any documentation to help $curl = curl_init(); curl_setopt($curl, CURLOPT_URL,"ftp://$_FTP[server]"); curl_setopt($curl, CURLOPT_FTPLISTONLY, 1); curl_setopt($curl, CURLOPT_USERPWD, "$_FTP[username]:$_FTP[password]"); curl_setopt($curl, CURLOPT_RETURNTR...

Rails: How to get the servers response HTTP headers?

Hi, I want to autodetect the pingback-url of remote websites, so I need to parse the HTTP response headers sent by the remote server. I don't need and don't want the contents of the remote website, I'm only looking for something like this: X-Pingback: http://www.techcrunch.com/xmlrpc.php Similar to using curl: curl -I "your url" I...

How can I get an HTML page as a string via PHP?

I am fetching some info via PHP from a webpage using simple_php_dom and curl. The problem is that the page is not built correctly so the DOM object contains erroneous info. How can I get the HTML file as a string in a PHP var so that I can run a regular expression through it? Curl doesn't work as it is ignoring the bad part. simple_htm...

received HTTP headers incorrect/unexpected for XSL file type

I'm using PHP+curl to retrieve an XSL file, but first I want to do some minimum validation that the file is really a XSL file. I make sure the file exists, has status code 200 OK, and that it has the correct file extension, but then I want to check the header for the Content-Type. I do this using curl_getinfo() function. All fine. But...

To use cURL FTP does both servers need the PHP cURL library installed?

I'm trying to wrap my brain about how to do this. We need to provide some files within a directory from our servers to our clients' servers via a PHP/Web interface using FTP. I've looked at the FTP capabilities built in to PHP and some custom classes, but someone suggested cURL might be a better option. We will have the FTP login credent...

cURL failing when called from PHP exec()

I'm making a call to download an XML file and write it to a file using exec() and the curl command. I tried using curl_exec(), but it wasn't recognized by my PHP system. Somehow, the cURL call now fails. It didn't before, but one day I visit my site and– WHAM. Bye bye XML parser. Things that did work: Calling cURL from the command li...

javascript or PHP option to detect AIM status

The website is http://www.ipalaces.org/support/ The code I use for the status indicators is <img src="http://big.oscar.aol.com/imperialpalaces?on_url=http://www.ipalaces.org/support/widget/status_green.gif&amp;off_url=http://www.ipalaces.org/support/widget/status_offline.gif"&gt; which is a neat thing that big.oscar.aol.com lets yo...

How to Get Body content of HTTPS using CURL

Hi The following code will retrieve the body content of a url retrieved using CURL in php but not https. Can anyone tell me how I edit the code as I need to get the data returned not just the header. From the test I did here is the result. You can see it has a content-length, I just don't know how to access it. Thanks Stephen Error...