Reading java .properties file from bash

Question

I am thinking of using sed for reading .properties file, but was wondering if there is a smarter way to do that from bash script?

Answer 1

In Perl:

while(<STDIN>) {
   ($prop,$val)=split(/[=: ]/, $_, 2);
   # and do stuff for each prop/val
}

Not tested, and should be more tolerant of leading/trailing spaces, comments etc., but you get the idea. Whether you use Perl (or another language) over sed is really dependent upon what you want to do with the properties once you've parsed them out of the file.

Note that (as highlighted in the comments) Java properties files can have multiple forms of delimiters (although I've not seen anything used in practice other than colons). Hence the split uses a choice of characters to split upon.

Ultimately, you may be better off using the Config::Properties module in Perl, which is built to solve this specific problem.

Answer 2

I have some shell scripts that need to look up some .properties and use them as arguments to programs I didn't write. The heart of the script is a line like this:

dbUrlFile=$(grep database.url.file etc/zocalo.conf | sed -e "s/.*: //" -e "s/#.*//")

Effectively, that's grep for the key and filter out the stuff before the colon and after any hash.

Answer 3

If you want to use sed to parse -any- .properties file, you may end up with a quite complex solution, since the format allows line breaks, unquoted strings, unicode, etc: http://en.wikipedia.org/wiki/.properties

One possible workaround would using java itself to preprocess the .properties file into something bash-friendly, then source it. E.g.:

.properties file:

line_a : "ABC"
line_b = Line\
         With\ 
         Breaks!
line_c = I'm unquoted :(

would be turned into:

line_a="ABC"
line_b=`echo -e "Line\nWith\nBreaks!"`
line_c="I'm unquoted :("

Of course, that would yield worse performance, but the implementation would be simpler/clearer.

Answer 4

One option is to write a simple Java program to do it for you - then run the Java program in your script. That might seem silly if you're just reading properties from a single properties file. However, it becomes very useful when you're trying to get a configuration value from something like a Commons Configuration CompositeConfiguration backed by properties files. For a time, we went the route of implementing what we needed in our shell scripts to get the same behavior we were getting from CompositeConfiguration. Then we wisened up and realized we should just let CompositeConfiguration do the work for us! I don't expect this to be a popular answer, but hopefully you find it useful.

Answer 5

if you want to use "shell", the best tool to parse files and have proper programming control is (g)awk. Use sed only simple substitution.

Answer 6

I have sometimes just sourced the properties file into the bash script. This will lead to environment variables being set in the script with the names and contents from the file. Maybe that is enough for you, too. If you have to do some "real" parsing, this is not the way to go, of course.

Answer 7

Hmm, I just run into the same problem today. This is poor man's solution, admittedly more straightforward than clever;)

decl=`ruby -ne 'puts chomp.sub(/=(.*)/,%q{="\1";}).gsub(".","_")' my.properties`
eval $decl

then, a property 'my.java.prop' can be accessed as $my_java_prop.

This can be done with sed or whatever, but I finally went with ruby for its 'irb' which was handy for experimenting. It's quite limited (dots should be replaced only before '=',no comment handling), but could be a starting point.

@Daniel, I tried to source it, but Bash didn't like dots in variable names.

Answer 8

The solutions mentioned above will work for the basics. I don't think they cover multi-line values though. Here is an awk program that will parse Java properties from stdin and produce shell environment variables to stdout:

BEGIN {
    FS="=";
    print "# BEGIN";
    n="";
    v="";
    c=0; # Not a line continuation.
}
/^\#/ { # The line is a comment.  Breaks line continuation.
    c=0;
    next;
}
/\\$/ && (c==0) && (NF>=2) { # Name value pair with a line continuation...
    e=index($0,"=");
    n=substr($0,1,e-1);
    v=substr($0,e+1,length($0) - e - 1);    # Trim off the backslash.
    c=1;                                    # Line continuation mode.
    next;
}
/^[^\\]+\\$/ && (c==1) { # Line continuation.  Accumulate the value.
    v= "" v substr($0,1,length($0)-1);
    next;
}
((c==1) || (NF>=2)) && !/^[^\\]+\\$/ { # End of line continuation, or a single line name/value pair
    if (c==0) {  # Single line name/value pair
        e=index($0,"=");
        n=substr($0,1,e-1);
        v=substr($0,e+1,length($0) - e);
    } else { # Line continuation mode - last line of the value.
        c=0; # Turn off line continuation mode.
        v= "" v $0;
    }
    # Make sure the name is a legal shell variable name
    gsub(/[^A-Za-z0-9_]/,"_",n);
    # Remove newlines from the value.
    gsub(/[\n\r]/,"",v);
    print n "=\"" v "\"";
    n = "";
    v = "";
}
END {
    print "# END";
}

As you can see, multi-line values make things more complex. To see the values of the properties in shell, just source in the output:

cat myproperties.properties | awk -f readproperties.awk > temp.sh
source temp.sh

The variables will have '_' in the place of '.', so the property some.property will be some_property in shell.

If you have ANT properties files that have property interpolation (e.g. '${foo.bar}') then I recommend using Groovy with AntBuilder.