To check if String contains an element from a List (of Strings) - Is there a better way to write this code?

Question

For the following block of code:

        For I = 0 To listOfStrings.Count - 1
            If myString.Contains(lstOfStrings.Item(I)) Then
                Return True
            End If
        Next
        Return False

The output is:

Case 1:

myString: C:\Files\myfile.doc

listOfString: C:\Files\, C:\Files2\

Result: True

Case 2:

myString: C:\Files3\myfile.doc

listOfString: C:\Files\, C:\Files2\

Result: False

At any time the list (listOfStrings) may contain several items (minimum 20) and it has to be checked against a thousands of Strings (like myString).

Is there a better (more efficient) way to write this code?

Answer 1

Instead of a list of strings could you use regular expressions? Even if you had to use multiple regular expressions, I would think that testing a couple (hundred?) would be better than doing thousands of string comparisons.

Answer 2

I'm not sure if it's more efficient, but you could think about using at Lambda Expressions.

Answer 3

With LINQ, and using C# (I don't know VB much these days):

bool b = listOfStrings.Any(s=>myString.Contains(s));

or (shorter and more efficient, but arguably less clear):

bool b = listOfStrings.Any(myString.Contains);

If you were testing equality, it would be worth looking at HashSet etc, but this won't help with partial matches unless you split it into fragments and add an order of complexity.

---

update: if you really mean "StartsWith", then you could sort the list and place it into an array ; then use Array.BinarySearch to find each item - check by lookup to see if it is a full or partial match.

Answer 4

There were a number of suggestions from an earlier similar question "Best way to test for existing string against a large list of comparables".

Regex might be sufficient for your requirement. The expression would be a concatenation of all the candidate substrings, with an OR "|" operator between them. Of course, you'll have to watch out for unescaped characters when building the expression, or a failure to compile it because of complexity or size limitations.

Another way to do this would be to construct a trie data structure to represent all the candidate substrings (this may somewhat duplicate what the regex matcher is doing). As you step through each character in the test string, you would create a new pointer to the root of the trie, and advance existing pointers to the appropriate child (if any). You get a match when any pointer reaches a leaf.

Answer 5

Based on your patterns one improvement would be to change to using StartsWith instead of Contains. StartsWith need only iterate through each string until it finds the first mismatch instead of having to restart the search at every character position when it finds one.

Also, based on your patterns, it looks like you may be able to extract the first part of the path for myString, then reverse the comparison -- looking for the starting path of myString in the list of strings rather than the other way around.

string[] pathComponents = myString.Split( Path.DirectorySeparatorChar );
string startPath = pathComponents[0] + Path.DirectorySeparatorChar;

return listOfStrings.Contains( startPath );

EDIT: This would be even faster using the HashSet idea @Marc Gravell mentions since you could change Contains to ContainsKey and the lookup would be O(1) instead of O(N). You would have to make sure that the paths match exactly. Note that this is not a general solution as is @Marc Gravell's but is tailored to your examples.

Sorry for the C# example. I haven't had enough coffee to translate to VB.

Answer 6

If speed is critical, you might want to look for the Aho-Corasick algorithm for sets of patterns.

It's trie with failure links, i.e. complexity is O(n+m+k), where n is the length of the input text, m the cumulative length of the patterns and k the number of matches. You just have to modify the algorithm to terminate after the first match is found.

Answer 7

Have you tested the speed?

i.e. Have you created a sample set of data and profiled it? It may not be as bad as you think.

This might also be something you could spawn off into a separate thread and give the illusion of speed!