string mask and offset with regex

Question

I have a string on which I try to create a regex mask that will show N number of words, given an offset. Let's say I have the following string:

"The quick, brown fox jumps over the lazy dog."

I want to show 3 words at the time:

offset 0: "The quick, brown"
offset 1: "quick, brown fox"
offset 2: "brown fox jumps"
offset 3: "fox jumps over"
offset 4: "jumps over the"
offset 5: "over the lazy"
offset 6: "the lazy dog."

I'm using Python and I've been using the following simple regex to detect 3 words:

>>> import re
>>> s = "The quick, brown fox jumps over the lazy dog."
>>> re.search(r'(\w+\W*){3}', s).group()
'The quick, brown '

But I can't figure out how to have a kind of mask to show the next 3 words and not the beginning ones. I need to keep punctuation.

Answer 1

One slant would be to split the string and select slices:

words = re.split(r"\s+", s)
for i in range(len(words) - 2):
    print ' '.join(words[i:i+3])

This does, of course, assume that you either have only single spaces between words, or don't care if all whitespace sequences are folded into single spaces.

Answer 2

The prefix-matching option

You can make this work by having a variable-prefix regex to skip the first offset words, and capturing the word triplet into a group.

So something like this:

import re
s = "The quick, brown fox jumps over the lazy dog."

print re.search(r'(?:\w+\W*){0}((?:\w+\W*){3})', s).group(1)
# The quick, brown 
print re.search(r'(?:\w+\W*){1}((?:\w+\W*){3})', s).group(1)
# quick, brown fox      
print re.search(r'(?:\w+\W*){2}((?:\w+\W*){3})', s).group(1)
# brown fox jumps 

Let's take a look at the pattern:

 _"word"_      _"word"_
/        \    /        \
(?:\w+\W*){2}((?:\w+\W*){3})
             \_____________/
                group 1

This does what it says: match 2 words, then capturing into group 1, match 3 words.

The (?:...) constructs are used for grouping for the repetition, but they're non-capturing.

References

• regular-expressions.info/Capturing Groups, Non-capturing Groups
  • Repeating a Capturing Group vs Capturing a Repeated Group

---

Note on "word" pattern

It should be said that \w+\W* is a poor choice for a "word" pattern, as exhibited by the following example:

import re
s = "nothing"
print re.search(r'(\w+\W*){3}', s).group()
# nothing

There are no 3 words, but the regex was able to match anyway, because \W* allows for an empty string match.

Perhaps a better pattern is something like:

\w+(?:\W+|$)

That is, a \w+ that is followed by either a \W+ or the end of the string $.

---

The capturing lookahead option

As suggested by Kobi in a comment, this option is simpler in that you only have one static pattern. It uses findall to capture all matches (see on ideone.com):

import re
s = "The quick, brown fox jumps over the lazy dog."

triplets = re.findall(r"\b(?=((?:\w+(?:\W+|$)){3}))", s)

print triplets
# ['The quick, brown ', 'quick, brown fox ', 'brown fox jumps ',
#  'fox jumps over ', 'jumps over the ', 'over the lazy ', 'the lazy dog.']

print triplets[3]
# fox jumps over 

How this works is that it matches on zero-width word boundary \b, using lookahead to capture 3 "words" in group 1.

    ______lookahead______
   /      ___"word"__    \
  /      /           \    \
\b(?=((?:\w+(?:\W+|$)){3}))
     \___________________/
           group 1

References

• regular-expressions.info/Lookarounds

Answer 3

No need for regex

>>> s = "The quick, brown fox jumps over the lazy dog."
>>> for offset in range(7):
...     print 'offset {0}: "{1}"'.format(offset, ' '.join(s.split()[offset:][:3]))
... 
offset 0: "The quick, brown"
offset 1: "quick, brown fox"
offset 2: "brown fox jumps"
offset 3: "fox jumps over"
offset 4: "jumps over the"
offset 5: "over the lazy"
offset 6: "the lazy dog."

Answer 4

We have two orthogonal issues here:

1. How to split the string.
2. How to build groups of 3 consecutive elements.

For 1 you could use regular expressions or -as others have pointed out- a simple str.split should suffice. For 2, note that you want looks very similar to the pairwise abstraction in itertools's recipes:

http://docs.python.org/library/itertools.html#recipes

So we write our generalized n-wise function:

import itertools

def nwise(iterable, n):
    """nwise(iter([1,2,3,4,5]), 3) -> (1,2,3), (2,3,4), (4,5,6)"""
    iterables = itertools.tee(iterable, n)
    slices = (itertools.islice(it, idx, None) for (idx, it) in enumerate(iterables))
    return itertools.izip(*slices)

And we end up with a simple and modularized code:

>>> s = "The quick, brown fox jumps over the lazy dog."
>>> list(nwise(s.split(), 3))
[('The', 'quick,', 'brown'), ('quick,', 'brown', 'fox'), ('brown', 'fox', 'jumps'), ('fox', 'jumps', 'over'), ('jumps', 'over', 'the'), ('over', 'the', 'lazy'), ('the', 'lazy', 'dog.')]

Or as you requested:

>>> # also: map(" ".join, nwise(s.split(), 3))
>>> [" ".join(words) for words in nwise(s.split(), 3)]
['The quick, brown', 'quick, brown fox', 'brown fox jumps', 'fox jumps over', 'jumps over the', 'over the lazy', 'the lazy dog.']