First, It depends on your audience. (This is where analytics come in handy)
For instance, a design focused blog might be able to avoid caring about Internet Explorer 6, as they can expect a large number of their users to be running the latest browser.
On the other hand, a website that focuses on business and accounting might need to support Internet Explorer 6, as some businesses refuse to upgrade.
Second, you must take into account that Google, many websites, and several countries have decided that Internet Explorer 6 is too horrible to support anymore. This means that you are in good company with not supporting Internet Explorer 6, and most likely you won't have too many problems.
Finally, you have to think if it is worth your time to develop for older browsers. It comes down to a cost per hour type thing, where if you could possibly be making more money doing other things than making sure your website works in older versions of browsers, you probably want to do those things instead.