The reason you need this is that for many tests you often need to initialize state before each test so that the tests can all make assumptions about the start state they're running in.
Suppose your test class wraps, say database access. After each test you'd want to remove whatever changes your tests have made to the db - if you didn't do that, each test runs against a slightly modified database. Additionally, any given test may see a different set of changes if some subset of the previous tests have failed. For example, suppose test1 does an insert, test2 checks that you're accurately reading the table size. Day 1, test1 fails, and 0 is correct. Day 2, test1 succeeds, and 1 is correct?
BTW, junit also supports @BeforeClass if you want to do a global setup, and setup and teardowns are optional.