Nonprofits are also a great place to get started. They ALWAYS have problems to solve. If you come up with a realistic and positive solution to a non-profit problem, you'll not only have the solution to market yourself with, but also the contacts that the non-profit brings (they have to have a board of directors, community members at large, etc) Extra bonus if they let you put your "company" name on it.
The bad news is that this is "expensive" to you as it takes money away as opposed to doing paying work. But on the other hand, sitting idle with no work is --more-- "expensive".
I got my start that way, then picked up (low) paying jobs to begin. I rocked the low-paying jobs and word-of-mouth spread until I'm now backed up 6-8 months on projects. I've never had to pay to advertise, nor do I rely on my website to pick up clients out of the blue. They're not the type I want to work with.
A word of warning: If you've never operated without a paycheck and worry about that thought, freelancing ain't the best gig in town. It's not just a matter of always being able to find clients, it's being able to collect from the clients you've done work for too. You'd be suprised how many clients think that they're hiring you forever for the price of a basic website. You tell them the project is done and they look at you funny when you ask to get paid. Then they start coming up with add-ons, excuses, or "bugs" that go on for weeks. Or, better yet, they decide what you gave them is not what they ask for and just disappear. Regardless, your rent isn't getting paid. In addition to programming skills, you need solid business documents, accounting knowledge, customer service skills, a website, and a legal advisor.
I decided that the risk isn't worth the strain it puts on my family. So I've changed my model to freelancing 4-7 hours a night after a 40-hour a week job at a web startup. Granted, it's a high risk startup, but it's a steady paycheck. In turn, I've committed to my clients to only do one project at a time which allows me to focus my more limited time on the project at hand. Everybody (including my family) wins. Only you know what the best solution will be. Good luck.