unix

NTPD: use an unrestricted port for communication

When querying ntp servers with the command ntpdate, I can use the -u argument to make the source port an unrestricted port (port 1024 and above). With ntpd, which is meant to run in the background, I can't seem to find a way to turn this option on. So the source port is always 123. It's playing around horribly with my firewall configur...

ELF file headers

Hey! A quick question about elf file headers, I can't seem to find anything useful on how to add/change fields in the elf header. I'd like to be able to change the magic numbers and to add a build date to the header, and probably a few other things. As I understand it the linker creates the header information, but I don't see anythin...

What's the best way to sync large amounts of data around the world?

I have a great deal of data to keep synchronized over 4 or 5 sites around the world, around half a terabyte at each site. This changes (either adds or changes) by around 1.4 Gigabytes per day, and the data can change at any of the four sites. A large percentage (30%) of the data is duplicate packages (Perhaps packaged-up JDKs), so the s...

Setting environment variables in linux using bash

In tcsh, I have the following simple script working: #!/bin/tcsh setenv X_ROOT /some/specified/path setenv XDB ${X_ROOT}/db setenv PATH ${X_ROOT}/bin:${PATH} xrun -d xdb1 -i $1 > $2 My question is, what is the equivalent to the tcsh setenv function in bash? Is there a direct analog? The environment variables are for locating ...

UNIX vs Windows memory deallocation

My understanding is that in unix, when memory is freed, the memory doesn't get returned back to the operating system, it stays in the process to be used again for the next call to malloc. On windows, I understand that the memory actually gets returned to the operating system. Is there any big difference between these two ways of doing ...

What are good Linux/Unix books for an advancing user?

Are there any good books for a relatively new but not totally new *nix user to get a bit more in depth knowledge (so no "Linux for dummies")? For the most part, I'm not looking for something to read through from start to finish. Rather, I'd rather have something that I can pick up and read in chunks when I need to know how to do somet...

How to change the format of substitution variables in a template

Hi, I need to to iterate over the files in a directory and perform the following replacement. Before: Hello ${USER_NAME}, you live at ${HOME_ADDRESS}. It is now ${TIME} After: Hello ${userName}, you live at ${homeAddress}. It is now ${time} The number of different tokens that appear within ${} is large, so it's not real...

Removing first line from stdin and redirect to stdout

i need to redirect all of the stdout of a program except the first line into a file. Is there a common unix program that removes lines from stdin and spits the rest out to stdout? ...

How can I validate large numbers of files with search and replace?

I am currently validating a client's HTML Source and I am getting a lot of validation errors for images and input files which do not have the Omittag. I would do it manually but this client literally has thousands of files, with a lot of instances where the is not . This client has validated some img tags (for whatever reason). Just w...

C++/Unix Home Project Ideas

I am a C++, Unix developer, mostly in systems side, product domain. I would like to take up some side project to improve my skills in the following : Object oriented Design Multithreaded Programming, with ample scope for mutexes and semaphores Interprocess Communications in Unix Core C++ stuff (wanna use templates, handle exceptions, u...

List files recursively in linux with path relative to the current directory

This is similar to this question, but I want to include the path relative to the current directory in unix. If can do the following: ls -LR | grep .txt But it doesn't include the full paths. For example, I have the follow dir structure: test1/file.txt test2/file1.txt test2/file2.txt The code above will return: file.txt file1.txt f...

Bash script, match on dates like?

I'm writing a script to remove some build artifacts older than 1 week. The files have names in the form artifact-1.1-200810391018.exe. How do I go about removing only the files that are greater than 1 week old, excluding the time in hours and minutes at the end of the date-time-stamp? Currently it is removing all of the files in the ...

Java memory consumption, "top" and HP-Ux

We ship Java applications that are run on Linux, AIX and HP-Ux (PA-RISC). We seem to struggle to get acceptable levels of performance on HP-Ux from applications that work just fine in the other two environments. This is true of both execution time and memory consumption. Although I'm yet to find a definitive article on "why", I believe ...

How to convert UNIX timestamp to DateTime and vice versa?

As the title says really. There is this example code, but then it starts talking about millisecond / nanosecond problems. http://blogs.msdn.com/brada/archive/2004/03/20/93332.aspx Edit: This is what I've got so far: public Double CreatedEpoch { get { DateTime epoch = new DateTime(1970, 1, 1, 0, 0, 0, 0).ToLoca...

The shell dotfile cookbook

I constantly hear from other people about how much of the stuff they've used to customize their *nix setup they've shamelessly stolen from other people. So in that spirit, I'd like to start a place to share that stuff here on SO. Here are the rules: DON'T POST YOUR ENTIRE DOTFILE. Instead, just show us the cool stuff. One recipe pe...

What are some good resources for learning C beyond K&R

Hi, I'm currently reaching the end of working through the K&R book "The C Programming Language", and I'm looking for things to read next. Any recommendations of: blogs more detailed / advanced books (I've already stuck Advanced Programming in a Unix Environment on my Safari bookshelf) open source code (beginner-friendly and with goo...

Debugging 100% iowait problem in linux

I've been trying to trace down why I have 100% iowait on my box. If I do something like a mysql select query, system goes to 100% iowait (on more than one cpu on my server,) which kills my watchdogs and sometimes kills httpd itself. In vmstat I see that every 8 seconds or so, there's a 5MB disk write. And that causes at least one cpu (...

How to properly handle wildcard expansion in a bash shell script?

#!/bin/bash hello() { SRC=$1 DEST=$2 for IP in `cat /opt/ankit/configs/machine.configs` ; do echo $SRC | grep '*' > /dev/null if test `echo $?` -eq 0 ; then for STAR in $SRC ; do echo -en "$IP" echo -en "\n\t ARG1=$STAR ARG2=$2\n\n" done else ...

Unix question: Automatically recognize path for locally installed libraries

I work on a Fedora Linux box. I have a whole host of binaries and libraries that I've installed locally under my home directory. I'd like to set my system up so installing software there functions the same way (for me) as if the root user installed it without a prefix. I can run binaries installed in ~/local/bin just fine by adding th...

cron jobs change time after DST

My cron job is running an hour later after end of DST. This didn't happen to other user's jobs on the same machine (AIX). What am I doing wrong? ...