How can I call curl from a Perl CGI script?

Question

I have a CGI (perl) script that is attempting to call curl using the open command:

@curl = ('/usr/bin/curl', '-S','-v','--location', $url, 
                          '-H', 'Content-Type:'.$content_type,
                          '-H', "Authorization: $authorization",
                          '-H', "X-Gdata-Key:$gdata_key",
                          '-H', "Content-Length:$content_length",
                          '-H','GData-Version:2',
                          '--data',"\@$filename");

And then executed like so:

open CURL, "-|", @curl;

The program works perfectly from the command line, but when I try to run it in the browser, the page ends up timing out.

What do I need to change on my server or in my script to get this to work properly?

Answer 1

You should check if the open succeeded and also attempt to close the pipe explicitly, check for error. In case of error, die with the error message. Then find the error message in the server error log.

Answer 2

In addition to the Sinan's suggestion, the timeout you are getting might point to a long running process - always an issue when run under CGI. Please look at other solutions like a Queue Manager. I use Beanstalk for these situations. But I have heard good things about Gearman and The Schwartz

I also learnt a lot about running processes that take a lot of time under CGI from this article

Answer 3

After looking at the error log and seeing the error

[Mon Nov 30 14:59:07 2009] [error] slurp_filename(
'/var/www/vhosts/mydomain.net/httpdocs /youtube/youtube.pl') / opening: (2)
No such file or directory at /usr/lib64/perl5/vendor_perl/5.8.6/
x86_64-linux-thread-multi/ModPerl/RegistryCooker.pm line 540

I thought it had something to do with the fact that I'm passing in my XML to curl as a file instead of as a string. Here is the new command that works with the xml passed as a string instead:

@curl = ('/usr/bin/curl', '-S','-v','--location', $url, '-H',
'Content-Type:'.$content_type,'-H',"Authorization: $authorization",'-H',
"X-Gdata-Key:$gdata_key",'-H',"Content-Length:$content_length",'-H',
'GData-Version:2','--data',"$xml");

And I am still using the command below to open/call curl:

open CURL, "-|", @curl;

It now runs successfully in the browser and returns me the values I am requesting with it.