Hey everyone, I have a quick question. I want to build a web app for myself that will allow me to post stuff to Craigslist and retrieve postings from Craigslists. I am not doing this to spam Craigslist, I am just doing it to learn. My question is: Is this possible? If so, will knowing PHP be enough? Or will I also need to know how to use the cURL library? Thanks in advance for the help and advice.
A:
PHP itself is just a language. It's the various modules that make it useful. This does include the cURL module. So yes, you'll need to know cURL as well.
Ignacio Vazquez-Abrams
2010-06-03 23:45:36
+2
A:
I'm pretty sure Craigslist looks down on scrapers.
You will need to use cURL, or file_get_contents() if your host allows it.
cURL will be easier however, as you won't have to deal with context for POST stuff.
If you just want to learn and don't want to get into trouble (legal or ethical), scrape a site where you have permission of the site's owner. Or even easier, make your own site and then scrape it yourself (via HTTP).
alex
2010-06-03 23:46:06
In other words: don't do it, you'll just get in trouble. Posting and retrieving from Craigslist via an app is, *YES IT IS*, useless. You can just go to the website itself to achieve the exact same result.
animuson
2010-06-03 23:50:27
That isn't true in the slightest. You can't get a callback if you post from the site itself; you can't preload the sites forms with data; you can't easily post multiple items from the site. All of this can be done with tools that I believe are totally legal. Look -- if you don't like people using tools -- fine; but your rather ridiculous inane comment doesn't add much to this conversation.
Evan Carroll
2010-06-04 00:15:32
@Evan Carroll: As you posted above, you can't do anything for a third party, which means you can only use it for yourself. If you can only use it for yourself, you might as well just use the website, rendering your app useless. This kind of app just *mimics* functionality elsewhere, which is useful for a phone, but not for another website.
animuson
2010-06-04 00:37:08
In the same sense that Microsoft Word twiddles bytes in the same way as ed therefore they are both equally useful - yes. And, it isn't that third parties can't use your tool -- it is that you can't use your tool for third parties.
Evan Carroll
2010-06-04 00:41:03
A:
For a most readable English, Craigslist most likely does NOT allows you to post from your app there, unless they give you "privileges" to do so.
Regarding your question about the necessary knowledge to accomplish something like that:
Yes, it's possible to post from one app to another just using php!
Zuul
2010-06-03 23:46:52
That's poor wording, it sounds like you're saying that Craigslist does allow you to do that.
animuson
2010-06-03 23:47:35
This isn't true, as I read it, Craigslist TOS do not prohibit tools that assist in posting -- they do however prohibit "Posting Agents" or those that post for third parties.
Evan Carroll
2010-06-04 00:08:39
@Evan Carroll, Am I really speaking French? Doesn't my answer stats just that? (posting from one's app to Craigslist, it's no assisting, it's like an "personal Agent")
Zuul
2010-06-04 00:17:01
@Zuul, no you're speaking nonsense. If you use [Dragon Naturally Speaking](http://www.nuance.com/naturallyspeaking/) to assist in making a post you're not violating the rules against automation. If however, you have your friend use Dragon you're violating the rules (third-party). If you use a program that helps post regularly you're violating the rules, or is capable of posting in bulk or bypassing the captcha (automation) you're violating the rules. At least, this is how I'm reading their TOS.
Evan Carroll
2010-06-04 00:23:51
I'm just stating my interpretation - I don't see anything in the TOS that precludes using a PHP application to prepopulate Craigslist's posting forms any more than I see verbiage that would preclude a paste of raw text into those same forms from the cliboard. There is a difference between full automation, bulk spamming, third party postings, and innocuous tools that generate HTML and meld a WYSIWYG editor into the `<textarea>`.
Evan Carroll
2010-06-04 00:35:30
A:
A few comments:
- There are a few posts relevant to accessing Craigslist on SO. I'd check those out if you haven't already.
- I don't believe Craigslist has an official API. That would mean your exercises may violate some of their terms of use. Make sure and read through that first.
- PHP isn't a requirement, if you decide to try interacting with their service. Pick your poison: PHP, ASP, CFM, Perl, .NET (any flavor), Java, etc.
- cURL, like PHP, is only one method of interacting with an external service. Depending on the API, you can expect some languages to have their own libraries for interaction (e.g. SOAP, XML-RPC, AMF, Restful HTTP, etc.).
If you're looking to interact with an external service (for educational purposes) maybe something like Del.icio.us may be of more use.
Good luck.
Inkspeak
2010-06-03 23:51:55
cURL isn't a "method" it is a library, PHP is a language that uses cURL. Perl tends to prefer libwww, but there are cURL bindings too.
Evan Carroll
2010-06-04 00:09:38
@Evan Carroll, "method", as in behavior or opportunity. cURL is not unique to PHP. I was thinking more generically.
Inkspeak
2010-06-04 00:13:29
A:
I would highly suggest you reference Craiglickr in your pursuit. While I have no desire to help you port it to PHP, it is the only open source tool that I personally know of that has anything to do with Cragslist. The short answer: yes, it is possible in PHP. No it won't be easy. And, PHP's lack of an event-driven framework will make it especially challenging.
Evan Carroll
2010-06-04 00:19:42