A simple data processor running on Amazon EC2

Question

I'm a newbie to the all cloud computing business.

I'm implementing a data process application running on a Windows server instance on Amazon EC2. The application would listening to a Amazon Simple Queue for new jobs.

should I write a windows server that listens to Simple Queue for new messages, then spawn a process to process data? Keep in mind, I don't know C# at all. I think with windows service, the data processor can be maintained without much effort. If something goes wrong, I can simply reboot the machine, everything will start automatically.

or

should I write a simple application runs in a while(true) loop, the checks for new messages and so on, which means it could be a Java problem, which I'm more familiar with.

To summarize, is it worth the effort and time to learn C# and write a Windows service to poll Amazon Simple Queue?

Answer 1

It would depends on your application whether it really requires C#. If you think the easiest way is to use Java since you are familiar with, go with it.

Create a program that polls SQS and start from that.:)

Answer 2

If you use Java, you could run your app on a Linux instance rather than Windows. Linux is cheaper than Windows on EC2 due to licensing costs.

Answer 3

Just to make sure you're clear: you don't need EC2 to poll SQS. Also, EC2 only makes sense if you are likely to want to change the amount of resources spent processing data in bursts.

If you're only ever going to have one service polling SQS, you may as well get a (cheaper) dedicated server rented by the month and poll SQS remotely. If you are likely to suddenly need to spike up to several machines or more for short periods, EC2 is a likely good fit.

I would advise you write your software in whatever programming language you feel comfortable with. There are excellent libraries for SQS and other AWS services in many languages, so if you don't currently have a preference, go with what you feel is most suitable for your problem domain.

Test it on your own infrastructure at home. Access SQS and debug. Once you're happy to ship, then you can worry about the OS and whether it's EC2 or not.

Just remember that whilst your processing component needs Windows, that doesn't mean you need C# - Java, Python, Ruby and Perl amongst others, all work fine on a Windows machine.