What is the best way on Linux platform for the process (C++ application) to check its instance is not already running?
You can use files and file locks to accomplish this, but, beware it isn't perfect and don't copy the infamous Firefox bug where it refuses to start sometimes even if it isn't already running.
The basic logic of it is:
Invariant:
File xxxxx will exist if and only if the program is running, and the
contents of the file will contain the PID of that program.
On startup:
If file xxxxx exists:
If there is a process with the PID contained in the file:
Assume there is some instance of the program, and exit
Else:
Assume that the program terminated abnormally, and
overwrite file xxxx with the PID of this program
Else:
Create file xxxx, and save the current PID to that file.
On termination (typically registered via atexit):
Delete file xxxxx
In addition to the logic above, you should also use a second file that you lock in order to synchronize access to the PID file (i.e. to act as a mutex to make it safe in terms of process-level concurrency).
A related alternative to Michael's solution is to create a directory in a known location (probably under /var/run or /tmp) and use the success/failure of the system call as the mechanism for ensuring mutual exclusion. This is the same mutual-exclusion trick CVS has used for years as directory creation is atomic on most (maybe all) commodity OSes. A PID file is still useful in the case where the directory + PID creating process dies unexpectedly and fails to clean up. Additionally, when checking to see if the existing directory + PID is valid, I'd suggest explicitly checking the /proc/<PID>/exe
symlink to verify that it points to your executable rather than just assuming the PID hasn't been recycled.
You could use a POSIX named semaphore to do this. It is much safer than using a file lock.
For a desktop app, it is probably more feasible to check whether an instance is started for current user, so that two users can have their own instances running.
You could use either some libraries (libunique (GTK+) or QtSingleApplication (Qt)), or do it yourself. In addition to pid-file mentioned earlier, you can open a FIFO or UNIX-domain socket somewhere in user's home directory. This way, you could communicate with running instance, eg. raise window of running instance or tell running instance to open new file/URI/whatever.
The standard way to do this is to create a pidfile somewhere, typically containing the pid of your program.
You don't need to put the pid in there, you could just put an exclusive lock on it. If you open it for reading/writing and flock it with LOCK_EX | LOCK_NB, it will fail if the file is already locked. This is race-condition free, and the lock will be automatically released if the program crashes.
Normally you'd want to do it per-user, so the user's home directory is a good place to put the file.
If it's a daemon, somewhere like /var/run is better.