shell-scripting

Linux: Removing files that don't contain all the words specified

Inside a directory, how can I delete files that lack any of the words specified, so that only files that contain ALL the words are left? I tried to write a simple bash shell script using grep and rm commands, but I got lost. I am totally new to Linux, any help would be appreciated ...

Using the read command to parse du -s output in linux shell script

I'm trying to write a shell script to perform actions if a users home directory if over a certain size. Unfortionately when I try to use the read command to split up the du -s output I get 'command not found' since it tries to pass the number through to the shell, not into a variable like I want. Here's what I have so far for the script....

Checking duplicates in terminal?

The following code prints me a long list of files with hashes and file names md5sum *.java I have tried unsuccessfully to list the lines where identical hashes occur, so that I can then remove identical files. How can you filter and delete identical files which have same content? ...

For Loop in Shell Script - add breakline in csv file

Hello all, I wish to introduce a for loop in my shell script and I was hoping I could get some help with this. I am executing a command from a text file. I wish to execute each command 10 times and insert some stats into a csv file. After that command has been done, I want to start the next BUT put a line break in the CSV file after th...

Linux: Which process is causing "device busy" when doing umount?

Linux: Which process is causing "device busy" when doing umount? ...

Filter directory listing with shell script

If I have a BASH variable: Exclude="somefile.txt anotherfile.txt" How can I get the contents of a directory but exclude the above to files in the listing? I want to be able to do something like: Files= #some command here someprogram ${Files} someprogram should be given all of the files in a particular directory, except for those i...

What is the best way to do string manipulation in a shell script?

I have a path as a string in a shell-script, could be absolute or relative: /usr/userName/config.cfg or ../config.cfg I want to extract the file name (part after the last /, so in this case: "config.cfg") I figure the best way to do this is with some simple regex? Is this correct? Should or should I use sed or awk instead? Shell...

How to get all matches from regex?

Hi, I would like to get all occurrences of [0-9A-Z]+? for processing later. I do have if [[ `cat file` =~ '[0-9A-Z]+?' ]]; then echo $BASH_REMATCH; fi Which gives me first match, but how could I process all the matches in the file? Thank you ...

Best way to determine uniqueness and repetition in a text file.

I have a text file of about 20 million lines. Each line is 25 characters long. I estimate that there are probably about 200k-300k unique lines. What I want to find out is exactly how many unique lines there are, and how many occurrences of each line there are (I expect the result to be power-law-esque). I could do this: sort bigfile...

Shell script - Two for loops and changing extension of file

Hello all, I have this Shell script and I've managed to muck it up and I was hoping I could be corrected and put on the right path and hopefully add a few things that I am not competent enough to do myself. I have put what I want do as comments in the Shell script below. #!/bin/bash #Get all files from dir "videos" and send to processL...

Environment variable within variable

I have an environment variable called $TEST which refers to a directory in my bash script I have a variable called $VARTEST which is = $TEST/dir/file now I want to grep the file specified by $VARTEST so I try to do grep somestring $VARTEST but it doesn't translate $TEST into it's directory I've tried different combinations of {}, "" an...

How can I check a file exists and execute a command if not?

I have a daemon I have written using Python. When it is running, it has a PID file located at /tmp/filename.pid. If the daemon isn't running then PID file doesn't exist. On Linux, how can I check to ensure that the PID file exists and if not, execute a command to restart it? The command would be python daemon.py restart which has t...

Parallel execution of shell processes

Is there a tool available to execute several process in parallel in a Windows batch file? I have found some interesting tools for Linux (parallel and PPSS), however, I would need a tool for Windows platforms. Bonus: It would be great if the tool also allowed to distribute processes in an easy way among several machines, running the pro...

Correct Bash and shell script variable capitalization

I run across many shell scripts with variables in all caps, and I've always thought that there is a severe misunderstanding with that. My understanding is that, by convention (and perhaps by necessity long ago), environment variables are in all-caps. But in modern scripting environments like Bash, I have always prefered the convention ...

How do you access ant properties from within a shellscript task?

In ant you can do something like: <property name="version" value="${some.fake.version}" <shellscript shell="bash" dir="${build.dir}"> echo "some shell cmds" df -h ls * svn export http://svn.org/somedir </shellscript> Okay, that shell script doesn't do anything, I know, but how would I the property "version"...

Shell: adding a new line between a given line of text

What this question isn't asking is how to add a new line below or above every line which matches a pattern. What I'm trying to do is add a new line between a pattern that exists on one line. Here is an example. before: Monday:8am-10pm after: Monday: 8am-10pm Thus in this case, insert new line after every 'Monday' patt...

Program to list all windows files in order by size

Does anyone know of a program or script which lists all the files in your hard drive in descending order by size? ...

Adding Shell Scripts to CVS from windows

I am going to add some Linux shell scripts to CVS. Can I download the scripts using SSH secure shell client to windows and then add to CVS using Tortoise CVS client? (Just for ease... We have nice GUI in tortoise otherthan using cvs commands :)) Will it destroy the privileges set from the Linux? ...

documenting shell scripts' parameters

Is there a convention for documenting shell scripts' parameters? For example: #!/usr/bin/env bash # <description> # # Usage: # $ ./myScript param1 [param2] # * param1: <description> # * param2: <description> A few things I don't like about this particular template: the script's file name (myScript) appears within the file itself ...

Extraordinarily Simple Shell Scripting Question: Making sticky changes?

I write shell scripts to do various things in OSX and Linux, but I always have the same problem. The script runs but any changes it makes to the environment (except the disks, of course) do not "stick." When the script terminates the changes revert. How can I make my changes stick? Edit: A lot of the answers have been great, but they h...