How can I update a DataSnap server while clients are still connected?

Question

We use stateful DataSnap servers for some business logic tasks and also to provide clientdataset data.

If we have to update the server to modify a business rule, we copy the new version into a new empty folder and register it (depending on the Delphi version, just by launching or by running the TRegSvr utility).

We can do this even while the old server instance is running. However, after registering the new version, all new client connections will still use the currently running (old) server instance. All clients have to disconnect first, then the new server will be used for the next clients.

Is there a way to direct all new client connections to the new server, immediately after registering?

(I know that new or changed method signatures will also require a change and restart of the clients but this question is about internal modifications which do not affect the interface)

We are using Socket connections, and all clients share the same server application (only one application window is open). In the early days we have used a different configuration of the remote datamodule which resulted in one app window per client. Maybe this could be a solution? (because every new client will launch the currently registered executable)

Update: does Delphi XE offer some support for 'hot deployment' (of updated servers)? We use Delphi 2009 at the moment but would upgrade to XE if it offers easier implementation of 'hot deployment'.

Nanaam

Interesting question. The answer depends on how your server instances are created and managed, and what makes them stateful. Which transport are you using? – Parthenos

Which Delphi/DataSnap version are you using? There are lot of internal changes on datasnap/D2009. A solution for your problem require different perspective on the pre/post 2009 server/clients... on all it will require you to make things happen, because neither provide a out-of-the-box solution for this case. – Abdu

Wondering if you're still interested in this... because you're not answering my previous question. :) – Abdu

Sorry, I forgot to add a comment to notify you about my last edit: we use Delphi 2009 at the moment but would upgrade to XE if it offers easier implementation of 'hot deployment'. – Nanaam

Answer 1

you could separate your appserver into 2 new servers, one being a simple proxy object redirecting all methods (and optionally containing state info if any) to the second one actually implementing your business logic. you also need to implement "silent reconnect" feature within your proxy server in order not to disturb connected clients if you decide to replace business appserver any time you want. never did such design myself before but hope the idea is clear

Inhalator

+1 for the method. I have done this for ISAPI libraries in the past. – Ldopa

However it would introduce another layer of repetetive work, every change of a method signature needs to be done in three places. Programmer's time is very expensive ;) – Nanaam

well, if your tlb has many methods with frequently changing signatures then yes it would require additional cumbersome efforts. my appservers usually only have two additional methods though (Connect and Disconnect) besides standard IAppServer interface, so no big problem here :) – Inhalator

alternatively you could avoid introducing another layer of indirection implemented using Proxy server by placing your business logic within package (bpl) which is loaded by your AppServer object. just set some flag (for example create some external file or signal it any other way you wish) in order to indicate that appserver has to unload currently loaded package and reload newer one and there you go – Inhalator

Perhaps changing method signatures should be expensive. ;-) The original idea of interfaces was that they should be immutable contracts. That means that typically you cannot change method signatures once the interface is published. But you can inherit a new interface and declare your new method there, keeping the old interface for compatibility. – Parthenos

@vavan using a dynamically loaded BPL for the business logic sounds interesting, and could be the best solution for the given environment (low traffic, but users don't like maintenance windows) – Nanaam

there won't be any maintenance window. you create and test newer version, place it under different name and set the signal. from this moment on clients of your appserver will use new package – Inhalator

@TOndrej I agreed that interfaces should be stable once the app is running in production. But the development should not put more burdon than necessary on the developers shoulders. If every signature has to be written three times, this is like writing 'for for for I I I := := := 1 1 1 to to to 10 10 10 ...' and if a change is neccessary, do it it three times again. Can you imagine that this will cause bad dreams :) ? – Nanaam

Answer 2

Have you tried renaming the current server and placing the new in the same location with the correct name (versus changing the registry location). I have done this for COM libraries before with success. I am not sure if it would apply to remote launch rules through as it may look for an existing instance to attach to instead of a completely fresh server.

It may be a bit hackish but you would have the client call a method on the server indicating that a newer version is available. This would allow it to perform any necessary cleanup so it doesn't end up talking to both the existing server instance and new server instance at the same time.

Ldopa

The renaming will not take effect before all clients disconnected, it requires a maintenance window to kick out all users first so that the old server will terminate. Even if this can be automated, users still have to interrupt their work (and maybe they are in the lunch break and dont see the 'please close the application' message). We can also not force-quit the clients, because they know our address :) – Nanaam

Answer 3

There is probably not a simple answer to this question, and I suspect that you will have to modify the client. The simplest solution I can think of is to have a flag (a property or an out parameter on some commonly called method) on the server that the client checks periodically that tells the client to disconnect and reconnect (called something like ImBeingRetired).

It's also possible to write callbacks under certain circumstances for datasnap (although I've never done this). This would allow the server to inform the client that it should restart or reconnect.

The last option I can think of (that hasn't already been mentioned) would be to make the client/server stateless, so that every time the client wants something it connects, gets what it wants then disconnects.

Unfortunately none of these options are the answer you want to your question, but might give you some ideas.

Stenotypy

The callback is a good suggestion, as long as we use stateful connections. And I agree that the stateless approch will be much easier to handle. – Nanaam

Answer 4

1. (optional) set up vmware vSphere, ESX, or find a hosting service that already has one.
2. Store the session variables in db.
3. Prepare 2 web boxes with 2 distinct IP address and deploy your stuff.
4. Set up DNS, firewall, load balancer, or BSD vm so name "example.com" resolves to web box 1.
5. Deploy new version to web box 2.
6. Switch over to web box 2 using whatever routing method you chose.
7. Deploy new version to web box 1 if things look ok.

Using DNS is probably easiest, but it takes time for the mapping to propagate to the client (if the client is outside your LAN) and also two clients may see different results. Some firewalls have IP address mapping feature that you can map public IP address and internal IP address. The ideal way is to use load balancer and configure it to 50:50 and change it to 100:0 when you want to do upgrade, but it costs money. A cheaper alternative is to run software load balancer on BSD vm, but it probably requires some work.

Edit: What I meant to say is session variables, not session. You said the server is stateful. If it contains some business logic that uses session variable, it needs to get stored externally to be preserved across reconnection during switch over. Actual DataSnap session will be lost, so when you shutdown web box 1 during upgrade, the client will get "Session {some-uuid} is not found" error by web box 1, and it will reconnect to web box 2. Also you could use 3 IP addresses (1 public and 2 private) so the client always sees 1 address , which is better method.

Aphrodite

Does the session have to be stored in the db? Once the TCP/IP connection to the DataSnap server is up and running, the client will continue using it even if the server DNS entry changes. – Nanaam

Answer 5

I have done something similar by having a specific table which held my "data version". Each time I would update the server or change a system wide global setting, I would increment this field. When a client starts it always checks this value, and will check again before any transactions/queries. If the value was ever different from when I first started, then I needed to go through my re-initialization logic, which could easily include a re-login to an updated server.

I was using IIS to publish my app servers, so the data that would change would be the path to the app server. I kept the old ones available, to respond to any existing transactions that were in play. Eventually these would be removed once I knew there were no more client connections to that version.

You could easily handle knowing what versions to keep around if you log what server the client last connected too (and therefore would know about).

Hardily

The server app itself however can not be replaced while clients are still connected - so I guess the client will first disconnect and then fall into 'paused' mode for long enough time to replace the server app meanwhile? And if a client connects just after the change in the data version table, it will not see the version change. Maybe a 'maintenance mode until hh.nn' return info from the server would be an option - but then this method needs to be called frequently, or it has to be implemented as a callback / server push method. – Nanaam

Answer 6

For newer versions (Delphi 2010 and up), there is an interesting solution

• for systems using the HTTP transport:

Implementing Failover and Load Balancing in DataSnap 2010 by Andreano Lanusse

• and a related question for the TCP/IP transport:

How to direct DataSnap client connections to various DS Servers?

Nanaam