Best way to pass data between two servers in C?

views:

130

answers:

+1 Q:

Best way to pass data between two servers in C?

I wrote a program that creates a TCP and UDP socket in C and starts both servers up. The goal of the application is to monitor requests over the TCP socket as to what UDP packets to send it (i.e. monitor for something like "0x01 0x02" and if I see it, then have the UDP server parse the payload, and forward it over to the TCP server for processing). The problem is, the UDP server will be busy keeping another device up, literally sending thousands of packets back and forth with this device. So what is the best way to continuously monitor requests from the TCP server, but send it certain payloads from the UDP server when requested since the UDP server will be busy?

I looked into pthreads with semaphores and/or mutex (not sure all the socket operations are thread safe, though, and if this is the right way to approach it) as well as fork / pipe. Forking the UDP server off as a child process seems easy enough, but I don't see exactly how I would be passing the kind of data I need among both servers (need request data from TCP and payload data from the UDP). Any help would be appreciated as I am new to C. Thanks!

You could use pipes on Unix. See http://tldp.org/LDP/lpg/node11.html

Wernight 2010-10-22 13:11:11

I looked into pipes, but this problem seemed too complex for something forks/pipes alone to accomplish. How easy / difficult would implementing pipes be to solve this? I looked at forks / pipes btw, its the first thing I looked at, but this seemed too "specialized" for that.

Jack 2010-10-22 13:25:49

Well, you certainly picked an interesting introduction to C!

You might try shared memory. What OS?

Charlie Martin 2010-10-22 13:11:42

It will be running in our own embedded Linux environment (running 2.6.34 I believe), but currently I'm testing everything in an Ubuntu 10.10 VM.

Jack 2010-10-22 13:24:57

+1 A:

Firstly, would it make sense to put these two servers into one program? If so, you won't have to communicate between processes, and the whole logic becomes substantially easier. You will have to think about doing asynchronous input and output, and the select() function is designed for just this. There will be many explanations around on how to do this, and a quick look finds this page.

However, if you must have two separate processes, then you will need to choose a mechanism for inter-process communication, of which there are several, and your choice will be affected by your operating system. A pipe, if available, might be suitable, as might a Unix named pipe. Or you could look into third-party message passing frameworks, or just use shared memory and/or semaphores (but be very careful!).

Tim 2010-10-22 13:36:04

Tim, thanks, that helps a lot getting started with this. I have it setup this way basically: toplevel-file.c, toplevel-file.h, udp-program.c udp-program.h tcp-program.c tcp-program.h and finally net.c and net.h (which I put all the UDP / TCP simple socket functions in). I thought this method would make it easier since the UDP server will be busy keeping another device up. If you think it's easier to put it ALL in the top level file (which I normally never do), than I can do that. Select definitely seems easier to implement than a pipe, but having one file (or one process) makes things messy.

Jack 2010-10-22 13:42:22

Don't forget that a single compiled program can come from multiple source files, which can help keep different parts of the program separate. Just make sure that there's only one `main()` function, and check your development system documentation about how to compile and link several source files.

Tim 2010-10-22 13:46:57

Oh sorry Tim, in that case, it is only "one program". I just have multiple source files. The only thing is, I forked the UDP server to run off the side currently so that I could also monitor for incoming TCP requests with the TCP program (so right now, when the user calls start on the main program, it runs and forks the UDP server, and runs the TCP server, only problem I have is obv. the communication between them). This is why I was confused when you said multiple processes. I will look into both solutions: fork/pipes and select but which would be easier to implement for something like this?

Jack 2010-10-22 13:51:13

Got it. I *strongly* suspect that getting rid of the `fork()` and keeping just one process is the best way forward. You probably decided on using `fork()` because you wanted the two servers to run side-by-side. This is a common approach, but much more difficult to control than using a single process with multiple active sockets. What I would do is set up both your sockets, then drop into a main loop which calls `select()` to see which socket needs attention. Use the timeout parameter to carry out housekeeping tasks when there's no traffic. Ask a new question here if you hit difficulties. :-)

Tim 2010-10-22 13:57:05

One reason you might want to go with multiple processes (or multiple threads) is if the operations need to proceed in parallel: for example, a TCP request comes in that may take a long time to process, but you need to keep servicing the UDP socket in the interim. But, if your requests are relatively quick to process, a single-threaded implementation using select() is the most straightforward, particularly at the introductory level.

David Gelhar 2010-10-22 14:17:13

Thanks Tim, your comments here are helping me a LOT, so I appreciate the help. So you would go with one main loop in the top level program and using select? The thing is, the UDP server will never need attention, it needs to keep executing on its own (if it ever stops, the device it is connected to will stop all communication). What I need to monitor for data is the TCP socket. The problem I am having is if the TCP socket does request some data, how exactly do I get the UDP socket to send it over without breaking its own communication with the device?

Jack 2010-10-22 14:27:50

Yes David, the reason I chose forking originally is exactly for that reason. The UDP server will be processing and handling lots of data at once so I wanted to keep it running separately. TCP requests will only come in once in a while. I'll see if I can get it working with a single process / threaded model using select though, as that seems by far the simplest way.

Jack 2010-10-22 14:30:30

OK, you have a main loop with a `select()` call in it. This acts as the main loop for both sockets. You have to think what actions need to be taken, and in response to what. Firstly, what happens when the UDP socket receives a message? Secondly, what happens when the TCP socket receives a message? Lastly, what else do you need to do which is *not* covered by these? Calling `select()` will allow you to find out whether there is a message ready to read on one or both sockets. If there is, you can process them. (continued)

Tim 2010-10-22 14:38:45

If you need to do anything which is not a response to a packet, specify a timeout value for `select()`. This will ensure that you get a chance to do stuff even if no packets are arriving.

Tim 2010-10-22 14:39:29

Also, if this answer is helpful, please consider "accepting" it and/or voting it up. :-) Thanks.

Tim 2010-10-22 14:39:57

Accepted Tim, thanks a lot, that helps. Don't have the rep to vote up or I would, sorry. I am sure I will need some more help as I start coding this part so stay tuned :) This gives me a great start at implementation though. I will read up on select() and also take a look at the libevent lib fuzzy mentioned below.

Jack 2010-10-22 14:44:03

Thanks, and good luck. select is a very powerful weapon when doing network IO, and in my experience is underused because people simply don't know about it. Have fun. :-)

Tim 2010-10-22 14:48:46

I learned a lot about networks from reading Beej's Guide to Network Programming, and he mentions that "in modern times select(), though very portable, is one of the slowest methods for monitoring sockets. One possible alternative is libevent, or something similar, that encapsulates all the system-dependent stuff involved with getting socket notifications." So maybe libevent would be easier, I will look into it. Originally I set both sockets to nonblocking and was polling them which is very inefficient, but even select() may be too slow. I'll keep you updated on what approach I take. Thanks!

Jack 2010-10-22 14:55:28

@Jack: speaking of `select`, you might find [Zed Shaw's](http://sheddingbikes.com/posts/1280829388.html) [articles](http://sheddingbikes.com/posts/1281174543.html) [about epoll](http://sheddingbikes.com/posts/1281765490.html) interesting.

Cristian Ciupitu 2010-10-22 16:03:37

`select` is a terrible choice for this given the alternatives available

fuzzy lollipop 2010-10-22 18:44:04

fuzzy, if select is too complicated / inefficient for what I'm doing, I'll look into libevent for sure. What other libs, suggestions did you have in mind besides that?

Jack 2010-10-22 19:04:33

Tim, after pondering the "select" solution for a while, I don't think it will work. The thing is, the UDP socket will always be busy (it will be sending thousands of packets within milliseconds of each other) so it would not give any chance for TCP requests to come in. Taking time to serve a TCP request also has the potential to kill UDP communication with the device since the device is expecting a UDP packet be sent over about every ms. This is why I wanted to run this process separately.I think perhaps I have to look at callbacks with libevent and see if that is efficient enough.

Jack 2010-10-23 22:27:29

Fair enough. But you've learned about `select()`, which puts you in a much better position than most next time you want to do multiplexed input/output. :-) Good luck with a better solution!

Tim 2010-10-23 22:30:24

Yes I appreciate you putting me on the right track at least. Does my reasoning make sense though? Maybe there is a way to use it and I just don't know the optimal way on how. I saw examples though and it only seems practical to use when only a few requests are coming in. Is this a correct assumption?

Jack 2010-10-23 22:47:57

Well, your concern that the "busyness" of the UDP socket won't allow a TCP request to be handled isn't quite right. `select()` is designed specifically to avoid this sort of problem: each time you call it, you get a list of which sockets have data available on them - when there is stuff on the TCP socket, you will see this listed as well as the UDP one, so your program can determine which to process in which order. So a TCP request could be processed as soon as it arrives. However, your concern about the time taken to process the TCP request is valid. (continued)

Tim 2010-10-24 08:39:02

If it really is the case that the time taken to process the TCP request is sufficiently large that you would not be able to respond to a UDP request in time, and this is would cause sometime serious to happen, then yes, you have a problem using this approach. I would consider several options: 1) Is the hardware I'm using really suitable for this approach, or would a faster system make the problems go away? 2) Could you break the processing for TCP packets into several smaller steps, and do these one at a time within the main loop? (continued)

Tim 2010-10-24 08:42:52

3) Is there some sort of lightweight threads option which would allow a new thread to be created to process the TCP packet, but without the overheads of process creation and communication? 4) Back to what you started with, using separate processes and inter-process communication. Others' comments about `select()` being a bad choice are somewhat valid: `select()` is the traditional, portable approach, but there are sometimes faster, less portable implementations available for particular platforms. (continued)

Tim 2010-10-24 08:54:19

The select_tut manpage http://www.kernel.org/doc/man-pages/online/pages/man2/select_tut.2.html contains some critical advice about how to get the best out of `select()` and also refers to `epoll()`, a Linux-specific function which might give you better performance. Good luck. :-)

Tim 2010-10-24 08:57:02

Again, thank you Tim, I'll look through the man pages :) Your thought process and comments here helps a lot so I appreciate it. The 3 options you gave me are very good to think of while approaching this and I will keep this in mind when I start implementing it. I will start with the select() solution and find the best way to implement it, if it doesn't work, oh well, on to the next idea. I've learned that with coding, you can't really tell if something doesn't work until you've actually implemented it and tested it. Just wanted to focus on one solution at first though.

Jack 2010-10-24 20:39:58

+1 A:

What you should look at is libevent, anything else you are reinventing the wheel writing this low level code yourself. Here is a Tutorial, Google, Krugle

Also you should use some predefined protocol between the servers. There are lots to choose from. Ranging from the extremely simple XDR to Protocol Buffers.

fuzzy lollipop 2010-10-22 13:48:01

Thanks fuzzy, will look into this too. I don't see too many examples on how to use it though, so it may be difficult, but if it easily accomplishes what I need, I will definitely think about using it. I prefer standard libraries, but I like to keep my options open so I appreciate it :)

Jack 2010-10-22 14:21:56

Thanks for the edit, the additional info helps a bit more, I just ran into that tutorial as well, trying to understand exactly how it works. Will hammer down the protocol details as soon as hammer down the best method of implementation.

Jack 2010-10-22 16:27:46

ansaurus

tags:

views:

answers:

Best way to pass data between two servers in C?

related questions