As per rwong's comment, the system function H gives you the phase and magnitude response of the system at a particular frequency. This means if the input to the system is cos[ωn] = cos[2πfn], the output will be a(f)cos[2πfn + Φ(f)], where a(f) = |H(f)| and Φ(f) = phase(H(f)). In your case the magnitude is 1 since the signal isn't being scaled in any way, only shifted in time. And the phase shift is -ω, where ω is the angular frequency of the sinusoidal input to the system.
I hope the following isn't too rudimentary for Stack Overflow, but maybe going over the basic basics of time series analysis will be helpful to minibear and others.
If you have a system with an impulse response of h[n] = δ[n-1] (where δ[n] is a delta function), as in your example, this means you're delaying the input by 1 time step. Think about what this means in terms of the phase of a sinusoid. The fastest changing sinusoid has a digital frequency of 0.5 (i.e. a period of 2 samples) -- e.g. cos[πn]. This is the series [1,-1,...]. If you delay this signal by 1 you get the series [-1,1,...], i.e. cos[πn - π] = cos[π(n - 1)], i.e. the input signal phase shifted by -π radians (-180 degrees). Look at a longer period signal with a digital frequency of 0.25 (i.e. a period of 4 samples) -- e.g. cos[0.5πn]. This is the series [1,0,-1,0,...]. A unit delay yields the series [0,1,0,-1,...], i.e. cos[0.5πn - 0.5π] = cos[0.5π(n - 1)], i.e. the input signal phase shifted by -π/2 radians (-90 degrees). Similarly, you can work out that an input of cos[0.25πn] yields an output of cos[0.25πn - 0.25π] = cos[0.25π(n - 1)], i.e. the input phase shifted by -π/4 radians (-45 degrees), etc, etc.
It's pretty clear that if the input angular frequency is ω (e.g. 0.5π), the output will be phase shifted by Φ = -ω. Think about the signal as a train going around the unit circle on a counterclockwise route, with its time series values corresponding to stops on this route. An angular frequency of 0.5π means it makes 4 stops at the following radian values: 0, 0.5π, π, 1.5π. Then it returns to 0 and repeats the cycle over and over. If this train gets delayed by a stop, that corresponds to a shift of -0.5π radians on the scheduled route.
Getting back to H(f), I hope it makes sense why it equals exp(-i2πf) = exp(-iω). Similarly, if your system has a delay of 2, then h[n] = δ[n-2] and H(f) = exp(-i4πf) = exp(-2iω) -- which is a delay of 2 stops on the unit circle. That's all the frequency response of a system/filter tells you, i.e. how much a system scales and delays each input sinusoid as a function of frequency.
FIR systems (i.e. finite impulse response, corresponding to a moving average model [MA]) are the simplest since they're just a sum of delta (i.e. scale and delay) functions on the feed forward path. IIR systems (i.e. infinite impulse response, corresponding to an autoregressive model [AR]) are more interesting to analyze since they have a feedback path.