What is the easiest solution for transparent remoting with Delphi?

Question

I have a two tier Delphi for Win32 application with a lot of business logic implemented in a god object I want to outsource into a separate service. This separate service should be accessed by multiple clients via TCP/IP telnet-style protocol.

How do I go about making the transition most simple?

Precisely, I'd like to keep this simplicity: I want to define every function just once. For example, if I want to add a pin code Login functionality to my app, I would just need to define the

 function Login(Username: string; PinCode: integer): boolean;

function in some object on the server, then I can use it from clients without any additional work.

In the worst case, I have to implement three functions instead of one. First, the function body itself on the server, second, the unmarshaller which receives a text line from network, unpacks it and checks for validity:

 procedure HandleCommand(Cmd: string; Params: array of string);
 begin
   ...
   if SameText(Cmd, 'Login') then begin
     CheckParamCount(Params, 2);
     ServerObject.Login(
       Params[0],
       StrToInt(Params[1])
     );
   end;
 end;

Third, the marshaller, which when called by client, packs the params and sends them to server:

function TServerConnection.Login(Username: string; PinCode: integer): boolean;
begin
  Result := StrToBool(ServerCall('Login '+Escape(Username)+' '+IntToStr(PinCode)));
end;

Obviously, I don't want this.

So far I have managed to rid myself of the unmarshaller. Working with Delphi RTTI, I wrote a generic unmarshaller which looks for a published method by name, checks the params and calls it.

So now I can just add a published method to the server object and I'm able to call it from telnet:

 function Login(Username: string; PinCode: integer): boolean;

 > login john_locke
 Missing parameter 2 (PinCode: integer)!

But what do I do about writing a marshaller? I can't dynamically fetch server function list and add functions to the client object. I can probably keep some kind of dynamic pseudo-function collection but that would make my client calls ugly:

ServerConnection.Call('Login', [Username, Password]);

Plus, this defeats type-safety as every parameter is passed as a variant. If possible, I'd like to keep compile-time type-safety.

Maybe client code auto-generation? I can probably write "GetFunctionList()" and "GetFunctionPrototype(Name: string)" in my server:

> GetFunctionList
Login
Logout
IsLoggedIn

> GetFunctionPrototype Login
function Login(Username: string; PinCode: integer): boolean;

So that every time I need to update the client, I just re-query all the function prototypes from the server and automatically generate marshaller code for them. But that would mix compilation with execution: I'd have to first compile the server, then start it and query its functions, then build a client marshaller and recompile client only after that. Complicated!

Another option is to write a generic marshaller function and then call it from all the client prototype functions:

procedure TServerConnection.GenericMarshaller(); assembler;
asm
  //finds the RTTI for the caller function, unwinds stack, pops out caller params,
  //packs them according to RTTI and sends to the server.
  //receives the result, pushes it to stack according to RTTI, quits
  //oh god
end;

function TServerConnection.Login(Username: string; PinCode: integer): boolean; assembler;
asm
  call GenericMarshaller
end;

This saves me writing manual packaging every time (less chance for errors), but still requires me to manually copy server function prototypes into the client object. Plus, writing this generic marshaller is probably going to be a living hell.

Then there's the option of using RPC, but I don't like it because I'd need to re-define all the functions in IDL. And Delphi's IDL editor sucks. And for OLE interfaces Delphi forcefully generates "safecall" functions which suck, too. And there's no way to implement auto-detect-disconnect and auto-restore-connection functionality other than checking EVERY SINGLE function call to the RPC class:

function TServerWrapper.Login(Username: string; PinCode: integer): boolean;
begin
  try
    RealObject.Login(Username, Pincode);
  except
    on E: EOleException do
      if IndicatesDisconnect(E) then
        Disconnect;
      Reconnect;
      RealObject.Login(Username, Pincode);
  end;
end;

We're back to several functions instead of one.

So, what would you guys suggest? What are the other choices I'm missing? Are there common patterns or ready solutions for remoting in Delphi?

Answer 1

You could write a program that used RTTI to build the necessary units for you rather than hand code everything. I have done this in the past with an ORM system that contained thousands of tables...was faster and much easier to write a code generator that spit out the classes and units necessary to work with the system from the database schema.

The other advantage of this approach is that it is easier to test since it can be scaled with predictable behavior.

Answer 2

You should look at one of the frameworks like RemObjects SDK. You use their tool to define an interface, and it does all the work for you. You then just implement the functions, and in the client just call them. I've converted a single app into a client/server app by taking the business logic component and making it the interface. No need to roll your own wire protocol.

Answer 3

You should absolutely be using DataSnap in Delphi 2010 at least, preferably in Delphi XE. What you described is basically what DataSnap is. It is a very powerful RPC system that allows you to create servers that can serve up any Delphi type or class to a client. Once you have the server in place, the client can create proxy code to call the server just like it was a native part of your application. No IDL, no COM, nothing but clean Delphi code. The server can even produce a REST/JSON combination for easy use with non-Delphi clinets.

DataSnap is exactly what you are looking for -- clean, neat, powerful, and easy.

Answer 4

Personally, I don't think you should be architecting a distributed system based around transparent remoting. Procedure-level RPC is simply the wrong level of granularity for robust and performant client-server interaction; making it tolerable will force you to write ill-defined procedures which do whole heaps of things, and take masses of parameters, simply to avoid the performance and reliability hit of network roundtrips of chatty (=> lots of little calls) APIs.

Think more in terms of messages. Consider building up a queue of messages on the client to pass off to your server, perhaps with callbacks associated with each message to handle return values, if any (anonymous methods work great for callbacks!). Then, dispatch all the messages to the server in one go, and receive and parse the results, invoking callbacks etc. as necessary for each message processed.

If your server has some way of encapsulating all state changes while processing a compound request (a queue of messages from a client) into a transaction, it works even better: you get a nice way of handling errors that occur on the nth message - simply discard all work done on the server, and proceed as if the client request had never come in. This normally simplifies the client's understanding of the server's state too.

I've built systems in the past oriented around these kinds of principles, and they work. Making the network transparent, pretending the physical layout of the system doesn't exist, just causes pain in the long term.