Because there are certain objects that you should perform cleanup on, otherwise it will cause issues for your application.
The constantly used example is using a SqlConnection:
SqlConnection conn = new SqlConnection(connString);
try
{
conn.Open();
throw new ArgumentException();
}
catch(SqlException ex)
{
}
In this case, the SqlConnection is left open with no way of closing it because you handle a SqlException, but ArgumentException is thrown. If you used a finally block, this wouldn't happen because the finally block code would execute:
try
{
conn.Open();
throw new ArgumentException();
}
catch(SqlException ex)
{
}
finally
{
conn.Dispose();
}