I'm coming up fast on my 4-year anniversary as a work-from-home freelancer, so here's my story:
I started off by quitting my last full-time job and establishing a consulting relationship with that company. This lasted for about a year and a half before they went bankrupt.
Their collapse left me with no clients (it had been a sweet enough deal that I hadn't needed to look for other work alongside them) and I ended up going to one of the big headhunting firms and did a bit of on-site contracting through them.
A bit over a year ago, I stumbled across rentacoder and did some work through them for a couple months, but, like you said, it tends to be very poorly-paid there and this was really just bringing in enough money to scrape by until a former coworker hooked me up with a local web designer who needed some programming done for one of his projects and was willing to pay decently for them. This guy then hooked me up with a few of his other clients and I've mostly been working within that group of clients this year.
I haven't been active on rentacoder in about 10 months, but buyers from those projects have continued coming back to ask me to do more for them. Two of these rentacoder clients have agreed to pay my current normal rates, while a third can't afford them (he runs a business in the Philippines, so he doesn't bring much in), but we've worked out an arrangement for him to provide marketing assistance in exchange for a lower rate.
So, yes, as others have said, relationships and word of mouth are the primary keys to surviving as a freelancer. That and always make sure you have more than one active client at any given time, so that you won't be left high and dry if they go out of business.