Optimising Searching of a 2-dimensional Array with LINQ

Question

I have a 2-dimensional array of objects (predominantly, but not exclusively strings) that I want to filter by a string (sSearch) using LINQ. The following query works, but isn't as fast as I would like.

I have changed Count to Any, which led to a significant increase in speed and replaced Contains by a regular expression that ignores case, thereby elimiating the call to ToLower. Combined this has more than halved the execution time.

What is now very noticeable is that increasing the length of the search term from 1 to 2 letters triples the execution time and there is another jump from 3 to 4 letters (~50% increase in execution time). While this is obviously not surprising I wonder whether there is anything else that could be done to optimise the matching of strings?

Regex rSearch = new Regex(sSearch, RegexOptions.IgnoreCase);
rawData.Where(row => row.Any(column => rSearch.IsMatch(column.ToString())));

In this case the dataset has about 10k rows and 50 columns, but the size could vary fairly significantly.

Any suggestions on how to optimise this would be greatly appreciated.

Answer 1

One optimisation is to use Any instead of Count - that way as soon as one matching column has been found, the row can be returned.

rawData.Where(row => row.Any(column => column.ToString()
                                             .ToLower().Contains(sSearch)))

You should also be aware that ToLower is culture-sensitive. If may not be a problem in your case, but it's worth being aware of. ToLowerInvariant may be a better option for you. It's a shame there isn't an overload for Contains which lets you specify that you want a case-insensitive match...

EDIT: You're using a regular expression now - have you tried RegexOptions.Compiled? It may or may not help...