File/Directory synchronization across network

tags:

network-programming

views:

answers:

+1 Q:

File/Directory synchronization across network

Hi All

I need to synchronize few directories/files within the cluster. Say if a file content changes in one node I need to propagate the change to other nodes so that the file content are same atany point of time.Same applies when some files/directories are deleted. DRBD is not a option so is there any library which can do this for me.

You need to use a distributed filesystem (GlusterFS comes to mind) that can guarantee the synchronization and locking depending on cluster's usage of the files. Otherwise, you may want to consider a centralized storage served via NFS for the simplicity. Beyond that but still centralized would be a SAN filesystem like GFS but be aware that setup requires more to it like fencing.

Rob Olmos 2010-09-01 09:29:18

Have you considered NFS? SMB? If the updates don't have to be immediate you could consider rsync

Robin 2010-09-01 09:39:17

I'd consider using rsync :) A handy tool for syncing between remote hosts.

Marcin Cylke 2010-09-01 09:41:09

I was under impression that it is used to take backups (my bad), can it do mirroring kind of stuff in a cluster?

punith 2010-09-01 09:47:13

It is usually used for backups, but that does not diminish its value as a mirroring tool. I think it would work well enough. It's quite lightweight tool, not comparable (in terms of power) to distributed Filesystems. Syncing more than two machines would be tricky.

Marcin Cylke 2010-09-01 09:51:43

In that case let me give it a try. What problems do you see in the case of more than 2 machines?

punith 2010-09-01 09:58:33

With the addition of a new machine you need to add it to the replication scripts on all the machines: _script maintainance is a problem_, replicating all-to-all seems a waste of time and bandwidth, but also is the easiest way to use rsync. Perhaps you could describe your scenerio? Are you facing multiple machines addition over time?

Marcin Cylke 2010-09-01 10:12:17

well number of machines will be constant and will not change dynamically. But my problem is the directory's content can change in all the nodes so if all of them call rsync then can there be a collision or corruption of data.

punith 2010-09-01 10:41:18

Ok, in this case, rsync won't do you good. It may corrupt your data, unless figuring out some clever way of syncing. But this would be too much of reinventing a wheel. Try some clustered FS.

Marcin Cylke 2010-09-01 11:30:06

ansaurus

tags:

views:

answers:

File/Directory synchronization across network

related questions