Complex software architecture

Question

Hi everyone !

I've got a few questions about the architecture of a software that I'm working on !

So basically, this software allow the user to access to some popular sites :

• Social networks (Facebook, MySpace, ...),
• Common services (RSS, Mails, Twitter...),
• Social bookmarkings (Digg, Delicious...),
• Chats (MSN, AOL...),
• ...

Currently the architecture looks like this : Use of MVC and Observer/Observable design patterns for interaction between the model (TApp_Core) and the user interface.

TApp_Core
 TBookmarkingServices_Core
  TDelicious (implement IBookmarkingServices) 
  TDigg (implement IBookmarkingServices) 
  etc... (implement IBookmarkingServices) 
 TChatServices_Core 
  TMSN (implement IChatServices) 
  TGoogleChat (implement IChatServices) 
  TAOLChat (implement IChatServices) 
  etc... 
 TRSSServices_Core 
 ...

So the software create an instance of TApp_Core and then depending on the user's choice, it creates some instances of other services (ex : App_Core.BookmarkingServices_Core.AddServices(Digg, User, Password);).

Some questions !

• Do you think that the current software design is right ?
• There is currently only one thread for all the software... Is creating a new thread for each request to a service would be better ? (for example the TDigg receive a message from a button, it creates a thread which will create a TidHTTP, generate the request to the server, wait for the response, parse the response, send a message to each observer (callback) and then free the thread.
• It seems very hard to link all View/Controller with each part of the model, is that normal that it requires so much works ? Ex : to send a message for example using with Twitter, it would require to :
  • Attach a controller (button) to a TApp_Core.TMicrobloggingServices_Core.TTwiter object (model)
  • wait the user to click on the button
  • Send a message to TTwiter (the model)
  • Create a thread to send the request to the server
  • Parse the response from the server
  • Execute callbacks to notify that the request has been executed and to give the result
  • Free the thread
• If I use the previous idea, won't it create too much threads and make the computer slow and less responsible ? Except, if I implement a thread pool (but it's quite complex to use it).
• Does SQLite 3 is good enough to store all data on the client ? (data can be mails, rss feed, etc... so it can be quite a lot of data with times).

Thanks and sorry for my bad English !

Answer 1

Looking at your post it is hard to decide if your architecture is "good" or not. Here is what I strive for

• Does it work?
• Do the minimum to make it work.
• Clean up your design as you go.
• Use tools to visualize your design.

I have found it extremely helpful to use tools like NDepend to visualize dependency.

Answer 2

I would advise going with an ORM framework to abstract away from your database. This gives you a meaningful data abstraction layer as well as meaning that you don't have to worry about your choice of database. SQLite is a nice compact database. It may be ok for testing. But it certainly isn't going to cut it in a real multi-user environment (it has file-level locking which will certainly make concurrent CRUD on any meaningful scale impossible). It also uses only a subset of SQL. Use and ORM and you can easily change later.

Answer 3

Do you think that the current software design is right ?

There is probably not an objectively "right" design - there are only better or worse designs. :) It looks like you're on the right track, but I would suggest you keep your GUI and views decoupled from the back-end services. For example:

     View --- Controller --- Service

So the View only knows how to render things (ie. IM messages, web content, or whatever) and passes requests from the user to the Controller. The Controller receives notifications from the Service, and updates the View. The Service knows nothing of the front-end (so it could be scriptable, which is very useful) and just implements the protocol required to talk to the network service. You would have a separate set of these three classes for each distinct back-end system, and an overall controller to manage the app at a global level.

It seems very hard to link all View/Controller with each part of the model, is that normal that it requires so much works ?

Probably, yes - you are attempting a non-trivial application. I assume you're using an MVC-style architecture. It may require more work up front to properly decouple things, but it is definitely worth it. The more complex your application, the more you will benefit from layering and decoupling.

Is creating a new thread for each request to a service would be better ?

Given all the services are network-based, you need to keep the blocking I/O calls (such as socket reads) from blocking your main GUI thread. So you either need to use fully async I/O with callbacks on one thread (not very common), or have a separate thread for each network session (the most common approach). I would not recommend spawning a new thread for every separate request, but for an entire session. You will need to be careful to avoid the usual threading issues, such as race conditions and deadlocks (this topic could fill a book). The thread will need to post a message back to the GUI for notifications, as you cannot update your GUI from the context of another thread.

(for example the TDigg receive a message from a button, it creates a thread which will create a TidHTTP, generate the request to the server, wait for the response, parse the response, send a message to each observer (callback) and then free the thread.

That sounds like you have your GUI coupled together with your controller and protocol handler. I suggest you separate the front-end and back-end, and keep things decoupled.

If I use the previous idea, won't it create too much threads and make the computer slow and less responsible ?

Threads are normally used to make the computer more responsive. Creating threads has a certain overhead, and require a great deal of care to synchronise properly. But if you have one thread per network session, the overhead will not slow things down as you would only have at most a handful.

Does SQLite 3 is good enough to store all data on the client ? (data can be mails, rss feed, etc... so it can be quite a lot of data with times).

SQLite is an excellent database for this purpose. It is used by some of the largest software houses in the world for client-side storage management. It is very fast and efficient, and I'm sure it will be able to keep up with your application.

You seem to be undertaking a complex application. I suggest you buy a book on concurrent programming, and read some more about threading. It is a very complex issue, and can lead to bugs that are incredibly difficult to track down if not programmed with great care. Good luck!