Yes, you don't need deadlines. You do need goals, and to keep focus on those. That's really hard to achieve if the goals are big. To keep focus, you need to understand what's most important at the moment.
Using time boxing with more specific goals helps with this, as you need to understand on what you will be focusing in the next time block. These are not deadlines as you don't have to meet all those mini goals, but its where all the effort needs to go into. As I said, its a matter of focus.
I do weekly sprints. While in some projects/clients we need to have an overall picture, there are some cases where we just go on with the sprints, based on: what is the very next we need to build/release.
I focus all my efforts in the current goals. Now, that doesn't mean trying to go faster by skipping stuff, and going slower at the end. It does however, means being practical about how you are developing.
I think the issue you have with those projects is probably more a combination of:
- Priorities. Before getting into the week's code, consider what is the next important thing you need to add to the product. Do that as if you were the client, so think of terms of users/business/budget.
- Trying to improve the whole code base at once. Remember the goal at all times. This doesn't mean you go on working with the code and never improving it. Instead it means, that you only improve the code you are working on.
- Trying to do all improvements at once. Even if you only improve the code you are working on, if it is missing a lot of things, then you do only a few of the improvements.
Update: As was pointed in John's answer, Duke Nukem Forever is a good example. It actually went in both approaches, at some time it had a supposed deadline, and during another very long period of time didn't have a deadline anymore. Regardless, it hasn't been released after many years.
It reminds you that you can fail in both cases, using deadlines or not. If you have something too big in your hands, break it into pieces you can release independently. That may not be possible in all scenarios, but in that case you have to ask yourself if you have the time/money to invest in such a long project.
Update 2: As I mentioned in comment on Alphaneo's answer:
The presence of a deadline alone won't guarantee that you get a team continually focused. Consider if you grab untrained devs and put them all together on a big project (as in long in time) and set a deadline. They will get on it, and at some point in time will loose focus. Later on the deadline is coming, and they will rush.
Now grab the same team, and every 2 weeks set some goals with clear priorities. You don't have to make them deadlines, some goals that we want to achieve, but some of which might be missed. Review what have been achieved, give feedback, question why something wasn't met, give congrats successes.
Which one of both approaches above do you think will give better results?