How to transform one list to another semi-aggregated list via LINQ?

Question

I'm a Linq beginner so just looking for someone to let me know if following is possible to implement with Linq and if so some pointers how it could be achieved.

I want to transform one financial time series list into another where the second series list will be same length or shorter than the first list (usually it will be shorter, i.e., it becomes a new list where the elements themselves represent aggregation of information of one or more elements from the 1st list). How it collapses the list from one to the other depends on the data in the first list. The algorithm needs to track a calculation that gets reset upon new elements added to second list. It may be easier to describe via an example:

List 1 (time ordered from beginning to end series of closing prices and volume):

{P=7,V=1}, {P=10,V=2}, {P=10,V=1}, {P=10,V=3}, {P=11,V=5}, {P=12,V=1}, {P=13,V=2}, {P=17,V=1}, {P=15,V=4}, {P=14,V=10}, {P=14,V=8}, {P=10,V=2}, {P=9,V=3}, {P=8,V=1}

List 2 (series of open/close price ranges and summation of volume for such range period using these 2 param settings to transform list 1 to list 2: param 1: Price Range Step Size = 3, param 2: Price Range Reversal Step Size = 6):

{O=7,C=10,V=1+2+1}, {O=10,C=13,V=3+5+1+2}, {O=13,C=16,V=0}, {O=16,C=10,V=1+4+10+8+2}, {O=10,C=8,V=3+1}

In list 2, I explicitly am showing the summation of the V attributes from list 1 in list 2. But V is just a long so it would just be one number in reality. So how this works is opening time series price is 7. Then we are looking for first price from this initial starting price where delta is 3 away from 7 (via param 1 setting). In list 1, as we move thru the list, the next step is upwards move to 10 and thus we've established an "up trend". So now we build our first element in list 2 with Open=7,Close=10 and sum up the Volume of all bars used in first list to get to this first step in list 2. Now, next element starting point is 10. To build another up step, we need to advance another 3 upwards to create another up step or we could reverse and go downwards 6 (param 2). With data from list 1, we reach 13 first, so that builds our second element in list 2 and sums up all the V attributes used to get to this step. We continue on this process until end of list 1 processing.

Note the gap jump that happens in list 1. We still want to create a step element of {O=13,C=16,V=0}. The V of 0 is simply stating that we have a range move that went thru this step but had Volume of 0 (no actual prices from list 1 occurred here - it was above it but we want to build the set of steps that lead to price that was above it).

Second to last entry in list 2 represents the reversal from up to down.

Final entry in list 2 just uses final Close from list 1 even though it really hasn't finished establishing full range step yet.

Thanks for any pointers of how this could be potentially done via Linq if at all.

Answer 1

My first thought is, why try to use LINQ on this? It seems like a better situation for making a new Enumerable using the yield keyword to partially process and then spit out an answer.

Something along the lines of this:

public struct PricePoint
{
    ulong price;
    ulong volume;
}

public struct RangePoint
{
    ulong open;
    ulong close;
    ulong volume;
}

public static IEnumerable<RangePoint> calculateRanges(IEnumerable<PricePoint> pricePoints)
{
    if (pricePoints.Count() > 0)
    {
        ulong open = pricePoints.First().price;
        ulong volume = pricePoints.First().volume;

        foreach(PricePoint pricePoint in pricePoints.Skip(1))
        {
            volume += pricePoint.volume;
            if (pricePoint.price > open)
            {
                if ((pricePoint.price - open) >= STEP)
                {
                    // We have established a up-trend.
                    RangePoint rangePoint;
                    rangePoint.open = open;
                    rangePoint.close = close;
                    rangePoint.volume = volume;

                    open = pricePoint.price;
                    volume = 0;

                    yield return rangePoint;
                }
            }
            else
            {
                if ((open - pricePoint.price) >= REVERSAL_STEP)
                {
                    // We have established a reversal.
                    RangePoint rangePoint;
                    rangePoint.open = open;
                    rangePoint.close = pricePoint.price;
                    rangePoint.volume = volume;

                    open = pricePoint.price;
                    volume = 0;

                    yield return rangePoint;
                }
            }
        }

        RangePoint lastPoint;
        lastPoint.open = open;
        lastPoint.close = pricePoints.Last().price;
        lastPoint.volume = volume;
        yield return lastPoint;
    }
}

This isn't yet complete. For instance, it doesn't handle gapping, and there is an unhandled edge case where the last data point might be consumed, but it will still process a "lastPoint". But it should be enough to get started.