Per the spec, it is used in age calculations. If you don't know what time the server thinks it is, you won't be able to calculate the "age" of a resource. Here's the relevant text from the spec:
Summary of age calculation algorithm, when a cache receives a response:
age_value
is the value of Age: header received by the cache with
this response.
date_value
is the value of the origin server's Date:
header
request_time
is the (local) time when the cache made the request
that resulted in this cached response
response_time
is the (local) time when the cache received the
response
now
is the current (local) time
apparent_age = max(0, response_time - date_value);
corrected_received_age = max(apparent_age, age_value);
response_delay = response_time - request_time;
corrected_initial_age = corrected_received_age + response_delay;
resident_time = now - response_time;
current_age = corrected_initial_age + resident_time;