Using LINQ to create an IEnumerable<> of delta values

Question

I've got a list of timestamps (in ticks), and from this list I'd like to create another one that represents the delta time between entries.

Let's just say, for example, that my master timetable looks like this:

1. 10
2. 20
3. 30
4. 50
5. 60
6. 70

What I want back is this:

1. 10
2. 10
3. 20
4. 10
5. 10

What I'm trying to accomplish here is detect that #3 in the output table is an outlier by calculating the standard deviation. I've not taken statistics before, but I think if I look for the prevalent value in the output list and throw out anything outside of 1 sigma that this will work adequately for me.

I'd love to be able to create the output list with a single LINQ query, but I haven't figured it out yet. Currently I'm just brute forcing it with a loop.

Answer 1

LINQ is not really designed for what you're trying to do here, because it usually evaluates value by value, much like an extremely efficient combination of for-loops. You'd have to know your current index, something you don't, without some kind of workaround.

Answer 2

If you are running .NET 4.0, this should work fine:

var deltas = list.Zip(list.Skip(1), (current, next) => next - current);

Apart from the multiple enumerators, this is quite efficient; it should work well on any kind of sequence.

Here's an alternative for .NET 3.5:

var deltas = list.Skip(1)
                 .Select((next, index) => next - list[index]);

Obviously, this idea will only be efficient when the list's indexer is employed. Modifying it to use ElementAt may not be a good idea: quadratic run-time will occur for non IList<T> sequences. In this case, writing a custom iterator is a good solution.

EDIT: If you don't like the Zip + Skip(1) idea, writing an extension such as this (untested) maybe useful in these sorts of circumstances:

public class CurrentNext<T>
{
    public T Current { get; private set; }
    public T Next { get; private set; }

    public CurrentNext(T current, T next)
    {
        Current = current;
        Next = next;
    }
}

...

public static IEnumerable<CurrentNext<T>> ToCurrentNextEnumerable<T>(this IEnumerable<T> source)
{
    if (source == null)
        throw new ArgumentException("source");

    using (var source = enumerable.GetEnumerator())
    {
        if (!enumerator.MoveNext())
            yield break;

        T current = enumerator.Current;

        while (enumerator.MoveNext())
        {
            yield return new CurrentNext<T>(current, enumerator.Current);
            current = enumerator.Current;
        }
    }
}

Which you could then use as:

var deltas = list.ToCurrentNextEnumerable()
                 .Select(c=> c.Next - c.Current);

Answer 3

This should do the trick:

static IEnumerable<int> GetDeltas(IEnumerable<int> collection)
{
    int? previous = null;

    foreach (int value in collection)
    { 
        if (previous != null)
        {
            yield return value - (int)previous;
        }
        previous = value;
    }
}

Now you can call your collection like this:

var masterTimetable = GetMasterTimeTable();

var deltas = GetDeltas(masterTimetable);

It's not really LINQ, but will effectively do the trick.

Answer 4

Not that I recommend this, but totally abusing LINQ the following would work:

var vals = new[] {10, 20, 30, 50, 60, 70};

int previous = 0;
var newvals = vals.Select(i =>
                            {
                                int dif = i - previous;
                                previous = i;
                                return dif;
                            });
foreach (var newval in newvals)
{
    Console.WriteLine(newval);
}

Answer 5

You can use Ani's answer:-

var deltas = list.Zip(list.Skip(1), (current, next) => next - current);

With a super-simple implementation of the Zip extension method:-

public static IEnumerable<TResult> Zip<TFirst, TSecond, TResult>(
  this IEnumerable<TFirst> first,
  IEnumerable<TSecond> second,
  Func<TFirst, TSecond, TResult> func)
{
  var ie1 = first.GetEnumerator();
  var ie2 = second.GetEnumerator();

  while (ie1.MoveNext() && ie2.MoveNext())
    yield return func(ie1.Current, ie2.Current);
}

That'll work with 3.5.

Answer 6

It looks like there are sufficient answers to get you going already, but I asked a similar question back in the spring:

How to zip one ienumerable with itself

In the responses to my question, I learned about "Pairwise" and "Pairwise"

As I recall, explicitly implementing your own "Pairwise" enumerator does mean that you iterate through you list exactly once whereas implementing "Pairwise" in terms of .Zip + .Skip(1) means that you will ultimately iterate over your list twice.

In my post I also include several examples of geometry (operating on lists of points) processing code such as Length/Distance, Area, Centroid.

Answer 7

One liner for you:

int[] i = new int[] { 10, 20, 30, 50, 60, 70 };
IEnumerable<int> x = Enumerable.Range(1, i.Count()-1).Select(W => i[W] - i[W - 1]);