How is it so?
It's all a matter of how you define success. I'll define success as "The stakeholders happy with the results."
Typically the way your stakeholders measure the results are along the lines of:
- Did they get the deliverable (program/system) that they needed (this might not be what they originally spec'ed out)? This also includes functionality and quality aspects (bugs, speed and the overall look).
- Did they get the deliverable in a timely fashion? Did it show up before it was REALLY needed on on schedule?
- Did they they pay what they expected to for the deliverable? Time could be involved in this? Also remember that accuracy of the cost estimate is important also; too little can be as bad as too much. "Too much" is obvious. "Too little" also means you mis-estimated the project, and whomever was funding you now has a budget surplus they have to handle (and in large organizations, if you leave money on the table this often means that you will loose that for next year).
Re: Finding #1) Having no schedule knocks out #2 and probably also #3 (because resource time has a cost too). No schedule = unlimited budget (until someone control of the purse strings figures out what is going on). If your project comes to an end, and the users got what they needed, then it's a success. But by that same measurement criteria, Duke Nukem Forever was a success right up until May 8, 2009
Also, those in charge are often afraid to admit that a project was a failure or even a only a partial-success because it might reflect badly on them and their future career. therefore, even if it's 500% over budget and 3 years late. They always define it as a success.
Re: Finding #2) The most important thing is that you have a plan. The methodology is just a common language within your organisation. UML (Unified Modeling Language) is really just a collection of tools for expressing your plan. It doesn't matter if you are using SCRUM, XP, or just POW (Plain Old Waterfall), you still have to do planning, it just changes when and how often you are doing the planing. If you've got no schedule to worry about, the planning (or non planning) can happen on the fly; and rework because of a lack of plans doesn't matter.
Re: Finding #3) I find that a specious statement; because I know it to be not true. I've seen developers involved in the estimates with all manner of requirements (both in and out of the USA). What you are probably seeing here is segmentation of roles. "We can't send that geek Charley to go and meet with the client, send Bill the B.A. to gather the requirements. He can't do Fizz-Buzz, but he's got a three piece suit."
At the same time, there is a philosophy that some mangers have of giving impossible deadlines to improve performance; and the fear that if we let the workers tell them how long something is really going to take that they will automatically inflate the estimates unreasonably. They believe that Parkinson's Law applies everywhere. This is false as it was only intended to apply to bureaucracies (like government, and administration in a large corporation) and has never actually been scientifically tested. It continues to be perpetuated because it's funny, not because it's been shown to be true.
Why you need schedules, UML, ..etc.
Unless you are in a miracle project, where you have no time and budget limitations, then yes: you need a schedule, you need requirements, you need a plan, etc.
And even if you are on one of these miracle projects, you still risk that someone higher up will wise up and shut you down.