I'm working on a Python program that makes heavy use of eggs (Plone). That means there are 198 directories full of Python code I might want to search through while debugging. Is there a good way to search only the .py files in only those directories, avoiding unrelated code and large binary files?
find `echo $PYTHONPATH | sed "s/:/ /g"` -name "*.py" | xargs grep PATTERN
joeforker
2009-05-14 20:07:25
that's nice. Hadn't thought of that.
Steve B.
2009-05-14 20:14:12
fortunately none of the entries in $PYTHONPATH contain a ' '.
joeforker
2009-05-14 20:32:19
find DIRECTORY -name '*.py' -exec grep PATTERN {} +
Marius Gedminas
2009-09-17 20:54:51
In all these examples, in place of DIRECTORY you can have more than one directory if necessary (eg: find /usr/local /opt -name ...)
Bryan Oakley
2009-09-17 21:09:19
find DIRECTORY -name "*.py" -print | xargs grep -iRn --color PATTERN
Maddy
2009-10-21 12:26:48
the "xargs grep" will first find all files and then grep them.the "exec grep" will not display the matching file name as there is only one argument every time.=> "find... | xargs -n2 grep PATTERN"It will start grepping as soon as it finds two files and it will show the filename.
Eric Darchis
2010-07-14 16:02:43
I like the -n to include line numbers. Surprisingly, the `find | xargs` version is slightly faster.
joeforker
2009-05-14 20:27:34
This version is 26 times slower than the | xargs or standalone grep solution because it executes grep 16,836 times instead of once.
joeforker
2009-05-14 20:31:25
but if you end it with a + instead of \;, then it's equivalent to the xargs solution, except doesn't break if your pathnames have spaces in them.
Marius Gedminas
2009-09-17 20:56:05
+8
A:
You may want to look at ack, a grep substitute, "aimed at programmers with large trees of heterogeneous source code" (from the website)
Brian Agnew
2009-05-14 20:15:21
+2
A:
I second the suggestion to use ack, it's a faster and more capable tool than grep.
limi
2009-05-14 21:33:50
Thanks. I will have to try it, but luckily grep only takes 2 seconds to run the accepted answer.
joeforker
2009-05-14 22:45:15
+3
A:
I also use ack a lot these days. I did tweak it a bit to find all the relevant file types:
# Add zcml to the xml type:
--type-add
xml=.zcml
# Add more files the plone type:
--type-add
plone=.dtml,.zpt,.kss,.vpy,.props
# buildout config files
--type-set
buildout=.cfg
# Include our page templates to the html type so we can limit our search:
--type-add
html=.pt,.zpt
# Create txt file type:
--type-set
txt=.txt,.rst
# Define i18n file types:
--type-set
i18n=.pot,.po
# More options
--follow
--ignore-case
--nogroup
Important to remember is that ack won't find files if the extension isn't in its configuration. See "ack --help-types" for all the available types.
I also assume you are using omelette so you can grep/ack/find all the related files?
Mark van Lent
2009-06-16 14:43:24
+1
A:
There's also GNU idutils if you want to grep for identifiers in a large source tree very very quickly. It requires building a search database in advance, by running mkid (and tweaking its config file to not ignore .py files). z3c.recipe.tag takes care of that, if you use buildout.
Marius Gedminas
2009-09-17 20:59:42