shell-scripting

Shell script for getting the sequence and changing it?

I have a command like echo "abcd0001gfh.DAT" | sed 's/^[^0-9]*\(....\).*$/\1/' | awk '{ print "00"$0 }' This will give me an output of 000001. But I want to run this in a loop where I receive the file name from 0001-9999 and again it becomes 0001. So my output should like below abcd0001gfh.DAT 000001 abcd0002gfh.DAT 000002 . . . a...

Shell script (mac): How to download files from a directory using wget and regular expression?

I'm trying to download images (.jpg) from web folder using wget. I want to download only images, which have a certain sentences in file name. This works fine wget -r -nd -A .jpg http://www.examplewebsite.com/folder/ but I like to include a sentence eg. "john". I tried wget -r -nd -A .jpg '*john*' http://www.examplewebsite.com/folde...

Common Pitfalls in BASH programming

I am not able to find this particular discussion in SO. What are all the common mistakes and pitfalls in the BASH programming / shell scripting? PS: Please close if this has been discussed already. ...

Bash shell scripting - csv parsing

I am trying to parse a CSV containing potentially 100k+ lines. Here is the criteria I have: The index of the identifier The identifier value I would like to retrieve all lines in the CSV that have the given value in the given index (delimited by commas). Any ideas, taking in special consideration for performance? ...

Handle special characters in bash for...in loop

Suppose I've got a list of files file1 "file 1" file2 a for...in loop breaks it up between whitespace, not newlines: for x in $( ls ); do echo $x done results: file 1 file1 file2 I want to execute a command on each file. "file" and "1" above are not actual files. How can I do that if the filenames contains things like spaces ...

using ruby popen wrapped in a shell script

I finished my short file for a homework assignment which uses IO.popen("command").readlines to grab the STDOUT of that command. However, I need to write a shell script to wrap my ruby file in. No problem, but somehow putting it in the shell script makes readlines hang. ruby script.rb foo example > example.out this works script.sh foo...

Compares lines in two files

how could we compare lines in two files using a shell script. I want to compare line in one file with the line in other.diff will give me all the differences in two files at a time.i want a line in the first file to be compared with all the lines in the second file and get the common lines as the output. with the line numbers where the ...

run few commands simultaneously

I am writting a shell script and i want these commands to run at the same time find ./incoming/kontraktor/ -type f -name '*.html' | sort | awk 'NR % 3 == 1' | ./bin/foo.py -m 3 -b 1 | next_command >> log/foo_log.log 2>&1 find ./incoming/kontraktor/ -type f -name '*.html' | sort | awk 'NR % 3 == 2' | ./bin/foo.py -m 3 -b 2 | next_command...

problem in a shell command

Hi, i am trying the following command on the command line ps -u `id | cut -f2 -d"=" | cut -f1 -d"("` -f | grep ppLSN | awk '{print $9}' | awk '{FS="=";print $2}' | grep KLMN | wc -l the value of teh command is returned as 7. but when i am putting the same command inside a script abc_sh like below ps -u `id | cut -f2 -d"=" | cut ...

Problem in running a script

i have unix shell script which is need to be run like below test_sh XYZ=KLMN the content of the script is #!/bin/ksh echo $XYZ for using the value of XYZ i have do set -k before i run the script. is there a way where i can do this without doint set -k before running the script. or is there something that i can do in the script...

problem in shell script

I am trying this option #!/bin/ksh echo $1 awk '{FS="=";print $2}' $1 and on the command line test_sh INSTANCE=VIJAY but awk is failing. Is there any problem here? Basically I need the value VIJAY passed on the command line. ...

Running a java application through shell script in a JSP/Servlet

I am running a shell script through a web application. This shell script looks something like `#! /bin/bash user="" pass="" db_url="" db_instance="" sqlplus -s $user/$pass@$db_url/$db_instance @ ./SqlScripts/foo.sql sqlplus -s $user/$pass@$db_url/$db_instance @ ./SqlScripts/bar.sql CLASS_PATH="./lib/*" java -classpath $CLASS_PATH pack...

How do I use tr to substitute '--' string

I have an output: -- out1 -- out2 -- out3 I want to get the output: out1 out2 out3 I thought of using: tr '--' '' but it doesn't recognize '--' to be the first string I want to substitute. How do I solve this? ...

shell script not executed

I'm trying to set the environment variables in shell script. The command "source .bashrc" is not executed. As long as type the last line in the terminal, everything works fine. What's wrong with my script? thx. echo "export CLASSPATH=.:$HOME/java/lib export JAVA_HOME=$HOME/java export PATH=.:$PATH:$JAVA_HOME/bin" >> .bashrc source .bash...

Bourne shell script to convert a number to telephone format

I want to change a number such as 1234567890 to 456-7890; is there a way to do this in Unix Shell programming? ...

Why don't I see pipe operators in most high-level languages?

In Unix shell programming the pipe operator is an extremely powerful tool. With a small set of core utilities, a systems language (like C) and a scripting language (like Python) you can construct extremely compact and powerful shell scripts, that are automatically parallelized by the operating system. Obviously this is a very powerful ...

Shell Scripting: Using bash with xargs

I'm trying to write a bash command that will delete all files matching a specific pattern - in this case, it's all of the old vmware log files that have built up. I've tried this command: find . -name vmware-*.log | xargs rm However, when I run the command, it chokes up on all of the folders that have spaces in their names. Is there ...

ksh: Iterate through a range

How can I iterate through a simple range of ints using a for loop in ksh? For example, my script currently does this... for i in 1 2 3 4 5 6 7 do #stuff done ...but I'd like to extend the range way above 7. Is there a better syntax? Thanks! ...

How to programmatically determine the current checked out Git branch

In a Unix or GNU scripting environment (e.g. a Linux distro, Cygwin, OSX), what is the best way to determine which Git branch is currently checked out in a working directory? One use of this technique would be automatically labeling a release (like svnversion would do with Subversion). Please also see my related question: How to progra...

How to programmatically determine whether the Git checkout is a tag and if so, what is the tag name

In a Unix or GNU scripting environment (e.g. a Linux distro, Cygwin, OSX), what is the best way to determine whether the current checkout is a Git tag. If it is a tag, how can I determine the tag name? One use of this technique would be automatically labeling a release (like svnversion would do with Subversion). See my related question...