Filtering two arrays to avoid Inf/NaN values

Question

I have two arrays of doubles of the same size, containg X and Y values for some plots.

I need to create some kind of protection against Inf/NaN values. I need to find all that pairs of values (X, Y) for which both, X and Y are not Inf nor NaN

If I have one array, I can do it using lambdas:

var filteredValues = someValues.Where(d=> !(double.IsNaN(d) || double.IsInfinity(d))).ToList();

Now, for two arrays I use following loop:

List<double> filteredX=new List<double>();
List<double> filteredY=new List<double>();

for(int i=0;i<XValues.Count;i++)
{
   if(!double.IsNan(XValues[i]) &&
         !double.IsInfinity(XValues[i]) && 
         !double.IsNan(YValues[i]) && 
         !double.IsInfinity(YValues[i]) )
     {
       filteredX.Add(XValues[i]);
       filteredY.Add(YValues[i]);
     }
}

Is there any way of filtering two arrays at the same time using LINQ/Lambdas, as it was done for single array?

EDIT:
unfortunately I can use only .Net 3.5

Answer 1

C# 4.0 introduces the Enumerable.Zip extension method to accomplish iterating over enumerables "in parallel" as you describe.

I haven't used it myself, but it should be something like:

var filteredValues = 
   XValues.Zip(YValues, (x,y) => new { X = x, Y = y})
        .Where( o =>
          !(double.IsNan(o.X) || double.IsNan(o.Y) || double.IsInfinity(o.X) || double.IsInfinity(o.Y)))
            .ToList();

(Sorry for the funny indentation, wanted it to be more readable on SO)

Answer 2

Slight correction for Mark's original answer:

var filteredValues = XValues.Zip(YValues, (x,y) => new { x, y })
        .Where(p => !(double.IsNan(p.x) || double.IsNan(p.y) || 
                      double.IsInfinity(p.x) || double.IsInfinity(p.y)))
        .ToList();

Alternatively, you might want to make it slightly neater:

Func<double, bool> valid = z => !double.IsNan(z) && !double.IsInfinity(z);
var filteredValues = XValues.Zip(YValues, (x,y) => new { x, y })
        .Where(p => valid(p.x) && valid(p.y))
        .ToList();

If you then need the results back into two lists, you can do:

var filteredX = filteredValues.Select(p => p.x).ToList();
var filteredY = filteredValues.Select(p => p.y).ToList();

Answer 3

You can try to do this:

doubleArray1.Zip(doubleArray2, (x, y) => Tuple.Create(x, y))
            .Where(tup => !double.IsNaN(tup.Item1) &&
            !double.IsNaN(tup.Item2) &&
            !double.IsInfinity(tup.Item1) &&
            !double.IsInfinity(tup.Item1));

Alternately, you could make a method for filtering and zipping at the same time, the major benefit is that you're not limited to C# 4:

public static IEnumerable<Tuple<TOne, TTwo>> DualWhere<TOne, TTwo>(this IEnumerable<TOne> one, IEnumerable<TTwo> two, Func<TOne, TTwo, bool> predicate)
{
    var oneEnumerator = one.GetEnumerator();
    var twoEnumerator = two.GetEnumerator();
    while (oneEnumerator.MoveNext() && twoEnumerator.MoveNext())
    {
        if (predicate(oneEnumerator.Current, twoEnumerator.Current))
            yield return Tuple.Create(oneEnumerator.Current, twoEnumerator.Current);
    }
    oneEnumerator.Dispose();
    twoEnumerator.Dispose();
}

Edit: The latter should work with C# 3.5.

Answer 4

Here's a solution that will work in C# 3 and .NET 3.5

List<double> list1 = new List<double>() { 1.2, 3.8, double.NaN, 17.8 };
List<double> list2 = new List<double>() { 9.4, double.PositiveInfinity, 10.4, 26.2 };

var query = from x in list1.Select((item, idx) => new { item, idx })
            where !double.IsNaN(x.item) && !double.IsInfinity(x.item)
            join y in list2.Select((item, idx) => new { item, idx })
            on x.idx equals y.idx
            where !double.IsNaN(y.item) && !double.IsInfinity(y.item)
            select new { X = x.item, Y = y.item };

Iterating over the query would produce pairs containing 1.2 & 9.4 and 17.8 & 26.2. The middle two pairs would be discarded because one contains NaN and the other contains infinity.

foreach (var pair in query)
{
    Console.WriteLine("{0}\t{1}", pair.X, pair.Y);
}

Answer 5

OK, so you can't use .NET 4.0 and therefore can't use the Zip extension.

Or can you?

public static IEnumerable<TResult> Zip<TFirst, TSecond, TResult>(
    this IEnumerable<TFirst> first,
    IEnumerable<TSecond> second,
    Func<TFirst, TSecond, TResult> resultSelector)
{
    using (var eFirst = first.GetEnumerator())
    using (var eSecond = second.GetEnumerator())
    {
        while (eFirst.MoveNext() && eSecond.MoveNext())
            yield return resultSelector(eFirst.Current, eSecond.Current);
    }
}

See for yourself :)

static void Main(string[] args)
{
    var x = new double[] { 0.0, 1.0, 2.0, double.NaN, 4.0, 5.0 };
    var y = new double[] { 0.5, 1.5, double.PositiveInfinity, 3.5, 4.5, 5.5 };

    // note: using KeyValuePair<double, double> --
    // you could just as easily use your own custom type
    // (probably a simple struct)
    var zipped = x.Zip(y, (a, b) => new KeyValuePair<double, double>(a, b))
        .Where(kvp => IsValid(kvp.Key) && IsValid(kvp.Value))
        .ToList();

    foreach (var z in zipped)
        Console.WriteLine("X: {0}, Y: {1}", z.Key, z.Value);
}

static bool IsValid(double value)
{
    return !double.IsNaN(value) && !double.IsInfinity(value);
}

Output:

X: 0, Y: 0.5
X: 1, Y: 1.5
X: 4, Y: 4.5
X: 5, Y: 5.5