Problematic behavior of Linq Union?!

Question

Hi,

consider the following example:

    public IEnumerable<String> Test ()
    {
        IEnumerable<String> lexicalStrings = new List<String> { "test", "t" };
        IEnumerable<String> allLexicals = new List<String> { "test", "Test", "T", "t" };

        IEnumerable<String> lexicals = new List<String> ();
        foreach (String s in lexicalStrings)
            lexicals = lexicals.Union (allLexicals.Where (lexical => lexical == s));

        return lexicals;
    }

I'd hoped for it to produce "test", "t" as output, but it does not (The output is only "t"). I'm not sure, but may have to do something with the deferred processing. Any ideas how to get this to work or for a good alternative?

Edit: Please note that this is just a simplified example. lexicalStrings and allLexicals are different types in the original code. So I cannot directly combine these.

Edit2 the problem to solve looks more like this:

    public IEnumerable<Lexical> Test ()
    {
        IEnumerable<String> lexicalStrings = new List<String> { "test", "t" };
        IEnumerable<Lexical> allLexicals = new List<Lexical> { ... };

        IEnumerable<Lexical> lexicals = new List<Lexical> ();
        foreach (String s in lexicalStrings)
            lexicals = lexicals.Union (allLexicals.Where (lexical => lexical.Text == s));

        return lexicals;
    }

Answer 1

Is this what you are trying to achieve?

lexicals.Union( allLexicals ).Distinct( StringComparer.OrdinalIgnoreCase )

EDIT:

Or better yet as @Dave suggested:

lexicals.Intersect( allLexicals, StringComparer.OrdinalIgnoreCase )

EDIT 2:

If they are different types one of them must implement IEqualityComparer to the other. Then pass this class to the Intersect method:

lexicals.Intersect( allLexicals, new MyCustomTComparer() )

Answer 2

public IEnumerable<Lexical> Test ()
{
    var lexicalStrings = new List<String> { "test", "t" };
    var allLexicals = new List<Lexical> { ... };

    var lexicals = new List<Lexical> ();
    foreach (string s in lexicalStrings)
    {
        lexicals.AddRange(allLexicals.Where (lexical => lexical.Text == s));
    }

    return lexicals;
}

Answer 3

You are using wrong operation as other answer explaining. But still it is interesting why your code works incorrectly despite looking fine.

let's modify your app a bit:

        IEnumerable<String> lexicalStrings = new List<String> { "test", "t" };
        IEnumerable<String> allLexicals = new List<String> { "test", "Test", "T", "t" };

        IEnumerable<String> lexicals = new List<String>();
        foreach (String s in lexicalStrings)
        {
            lexicals = lexicals.Union(
                allLexicals.Where(
                lexical =>
                {
                    Console.WriteLine(s);
                    return lexical == s;
                }
                )
            );
        }
        Console.WriteLine();
        foreach (var item in lexicals)
        {
        }

what output do you expect? here is it:

t
t
t
t
t
t
t
t

interesting, is not it?

now let's modify it again:

    IEnumerable<String> lexicalStrings = new List<String> { "test", "t" };
    IEnumerable<String> allLexicals = new List<String> { "test", "Test", "T", "t" };

    IEnumerable<String> lexicals = new List<String>();
    foreach (String s in lexicalStrings)
    {
        string ls = s;
        lexicals = lexicals.Union(
            allLexicals.Where(
            lexical =>
            {
                Console.WriteLine(ls);
                return lexical == ls;
            }
            )
        );
    }            
    foreach (var item in lexicals)
    {                
    }

now the output and results are fine:

test
test
test
test
t
t
t
t

Why does it happen? You use closure - the use of outer var in inner lambda. Since you do not actually iterate your sequence the current value of s doesn't get into the lambda. foreach exits and all inner copies of s hold value of last iteration. In case of inner variable they hold values copies that are created for every iteration. This conflict comes from inner lazyness of LINQ. If you do things like List.AddRange inside loop result will be fine, because List.AddRange forces iteration.