Parsing a multiline variable-length log file

Question

I want to be able to utilize a 'grep' or 'pcregrep -M' like solution that parses a log file that fits the following parameters:

• Each log entry can be multiple lines in length
• First line of log entry has the key that I want to search for
• Each key appears on more then one line

So in the example below I would want to return every line that has KEY1 on it and all the supporting lines below it until the next log message.

Log file:
01 Feb 2010 - 10:39:01.755, DEBUG - KEY1:randomtext
        blah
        blah2 T
        blah3 T
        blah4 F
        blah5 F
        blah6
        blah7
01 Feb 2010 - 10:39:01.757, DEBUG - KEY1:somethngelse
01 Feb 2010 - 10:39:01.758, DEBUG - KEY2:randomtest
this is a test
01 Feb 2010 - 10:39:01.760, DEBUG - KEY1:more logs here
01 Feb 2010 - 10:39:01.762, DEBUG - KEY1:eve more here
this is another multiline log entry
keeps on going
but not as long as before
01 Feb 2010 - 10:39:01.763, DEBUG - KEY2:testing
test test test
end of key2
01 Feb 2010 - 10:39:01.762, DEBUG - KEY1:but key 1 is still going
and going
and going
and going
and going
and going
and going
and going
and going
and going
and going
and going
and going
okay enough
01 Feb 2010 - 10:39:01.762, DEBUG - KEY3:and so on
and on

Desired output of searching for KEY1:
01 Feb 2010 - 10:39:01.755, DEBUG - KEY1:randomtext
        blah
        blah2 T
        blah3 T
        blah4 F
        blah5 F
        blah6
        blah7
01 Feb 2010 - 10:39:01.757, DEBUG - KEY1:somethngelse

01 Feb 2010 - 10:39:01.760, DEBUG - KEY1:more logs here
01 Feb 2010 - 10:39:01.762, DEBUG - KEY1:eve more here
this is another multiline log entry
keeps on going
but not as long as before
01 Feb 2010 - 10:39:01.762, DEBUG - KEY1:but key 1 is still going
and going
and going
and going
and going
and going
and going
and going
and going
and going
and going
and going
and going
okay enough

I was trying to do something like:
pcregrep -M 'KEY1(.*\n)+' logfile
but definitely doesn't work right.

Answer 1

if you are on *nix, you can use the shell

#!/bin/bash
read -p "Enter key: " key
awk -vkey="$key" '
$0~/DEBUG/ && $0 !~key{f=0}
$0~key{ f=1 }
f{print} ' file

output

$ cat file
01 Feb 2010 - 10:39:01.755, DEBUG - KEY1:randomtext
        blah                                       
        blah2 T                                    
        blah3 T                                    
        blah4 F                                    
        blah5 F                                    
        blah6                                      
        blah7                                      
01 Feb 2010 - 10:39:01.757, DEBUG - KEY1:somethngelse
01 Feb 2010 - 10:39:01.758, DEBUG - KEY2:randomtest  
this is a test                                       
01 Feb 2010 - 10:39:01.760, DEBUG - KEY1:more logs here
01 Feb 2010 - 10:39:01.762, DEBUG - KEY1:eve more here 
this is another multiline log entry                    
keeps on going                                         
but not as long as before                              
01 Feb 2010 - 10:39:01.763, DEBUG - KEY2:testing       
test test test                                         
end of key2                                            
01 Feb 2010 - 10:39:01.762, DEBUG - KEY1:but key 1 is still going
and going                                                        
and going                                                        
and going                                                        
and going                                                        
and going                                                        
and going                                                        
and going                                                        
and going                                                        
and going                                                        
and going
and going
and going
okay enough
01 Feb 2010 - 10:39:01.762, DEBUG - KEY3:and so on
and on

$ ./shell.sh
Enter key: KEY1
01 Feb 2010 - 10:39:01.755, DEBUG - KEY1:randomtext
        blah
        blah2 T
        blah3 T
        blah4 F
        blah5 F
        blah6
        blah7
01 Feb 2010 - 10:39:01.757, DEBUG - KEY1:somethngelse
01 Feb 2010 - 10:39:01.760, DEBUG - KEY1:more logs here
01 Feb 2010 - 10:39:01.762, DEBUG - KEY1:eve more here
this is another multiline log entry
keeps on going
but not as long as before
01 Feb 2010 - 10:39:01.762, DEBUG - KEY1:but key 1 is still going
and going
and going
and going
and going
and going
and going
and going
and going
and going
and going
and going
and going
okay enough

Answer 2

Urgo, I would really recommend using a real programming languages for this. There are many high-level modern programming languages you could use - Python, Perl, Ruby, and so on. Once you learn the required tools, should produce a short and readable program.

If this seems like too much effort now just for this small tasks, you will find the new knowledge very useful in the future as it will allow you to approach problems in a new, powerful way.