My software business was in a very, very small niche market centered on computer aided design of the magnetic layer in hard disk drives (www.micromagnetica.com - please note that I am in the process of closing down my business as the number of potential customers has shrunk to the point of making the business not viable. The web site reflects this point). I have been in business for 10 years and have done pretty well. My competition was a series of commercial and open source programs (mostly university or government sponsored), so, although the market was small, I was able to create a unique product that sold well.
Pitfalls:
1) Putting your needs above the customer - Customer comes first - always listen to your customer's needs and make sure your development follows their needs rather than yours. Every programmer has a list of things they want to learn or do. Don't use this list a guide for your development unless it solves an issue or helps create functionality that the customer wants/needs. This one point can make or break your company.
2) Not clarifying your business idea - Put together a business plan - it will help clarify what you are doing. Read the book, "The Art of the Start", by Guy Kawasaki to get the business perspective of starting a business. If you need money then you can use this to help secure financing from either angel investors or venture capitalists. Otherwise, it will help clarify what you are doing.
3) Not marketing yourself - Do this the following:
a) Find a good name for your company and secure your domain name. Even though a bad choice for company name won't kill you (my first company was called "Euxine Technologies" and it doesn't get much worse than that), but my product sold itself and was not encumbered by the name.
b) Put together a web site as soon as possible with a good description of your product. Google will eventually find you and traffic will start flowing to your site.
c) As soon as you have a working prototype create a mechanism where potential enthusiastic customers can download it and start helping you find bugs. You can make this the full version with a limited time or a limited version with no time limit. I have done both and both work. Make sure that users know it is a beta (or alpha) version of the software. The most important part of creating the beta user relationship is they will ask for features that you did not think about and this could take development along an otherwise unforeseen (and lucrative) path. This will also give you a way to keep your hand on the pulse of potential users.
d) If your product is applicable to a particular industry go to relevant conferences
(either get a booth or make contact with potential customers) and sell your product through demonstrations, flyers, and the distribution of free limited versions of your software on CD.
4) Not Branding yourself - come up with a logo that you will use to identify you and your product. This logo will show up on your web, your business stationary, and business cards.
5) Not Managing your money - initially there is going to be a long spell before the money starts coming in. Be very frugal with your seed money. The money will not start coming in the moment your deem the software is ready to sell. There could be a time-lag of at least a couple of months between when people show interest in your software and when the sale comes in. This will depend on how much your software costs. The more costly the software the longer the time-lag.
Once you start making sales, there will be seasonal variations in how much money comes in. Always try and keep at least 6 months worth of money in the bank to cover salary and operating costs.
6) Not knowing who your customers are - Once you start selling software, make sure you know who your customers are - they might be different from what you thought they were. When I started my software company, I thought my customers would be all R&D engineers who were doing research in magnetic layers. After a while it became clear that most of my users were the subset of this group that couldn't program, but understood the physics behind the software.
7) Not acting in a professional manner - When interacting with customers be professional - act and dress in a professional manner.