Driving multiple browsers that interact with eachother

Question

Hi,

We have a complex web application which uses a lot of JavaScript along with XMPP and a Rails application. It has come to the point where we need to test the entire system end-to-end. We need to be able to ensure various tasks can be completed without errors, one such task would be to negotiate a 1-on-1 chat session with another party on the system. As all of the tasks on our application envolve user to user interaction we would like to be able to simulate this by driving 2 browsers at the same time which communicate via the web application. I've been thinking of using selenium or watir and their waiting functions to wait for specific things to happen before proceeding. For example:

1. open 2 browsers at our site
2. browser 1 waits for a question to be published
3. browser 2 publishes a question then waits for an offer to be received
4. browser 1 sees the question, sends an offer and waits for it to be accepted
5. browser 2 sees the offer and accepts it then waits for a chat session to start
6. etc...

I can see this kind of testing becoming rather tricky to write and maintain. It would rely on specific elements of the page so if these are changed the tests would need to be altered. Also if we change the flow of the application even slightly the suit would probably need to be modified quite heavily.

So my question is, has anyone done this kind of testing? If so what tools would you suggest using and do you have any tips on writing the tests?

We will need to drive actual browsers, not headless, and it would need to be possible to integrate this with our CI, we currently use Sauce Labs and selenium as part of our CI setup.

Answer 1

To be honest this sounds like it belongs in the "Somethings shouldn't be automated" category.

Foreseeable issues:

• One of the browsers doesn't spin up properly possibly putting the other browser into a deadlock situation until it times out

• If for some reason your test didn't get all the way through because your webserver decided to just be slow you would get another deadlock till timeout situation.

• Maintainability will be extremely hard since if something happens that you need to update your test you need to make sure both browser instances work

I am sure that there are more issues but can't think of them right now.

I would write tests to make sure that you can add new questions to the system. I would have at least one test in the test database that you can always read and answer (Possibly within a transaction that you can roll back).

With the Chat system I would test this manually and this is where dogfooding your app will come into its own. Get everyone in the company(from dev to ops and beyond) to test the chat because I am sure that you are already doing IM of some sort.

Answer 2

We have a similar (actually more complex setup) with iMacros Scripting Edition. The iimInit / iimPlay commands give us full control over the web browser and detailed error codes.

If one of the browsers doesn't spin up properly iimPlay will return an error code and we can act accordingly.

The main drawback of iMacros is that the scripting edition is Windows only (ok for us as we work with ASP.NET). But if that is a problem for you, try WatiR. Whatever you use, good error return codes are the key to success.