Pros and Cons of VDSF

The basic idea

The problem: writing powerful and complex server applications often leads to very complex programs with the introduction of multiple threads, the need for synchronization, etc.

The problem compounded: maintaining and updating such an application is often even more complex than developing the original application.

Solutions:

There are multiple solutions to this problem, for example using an existing framework from known vendors (J2EE, for example or .NET). Relational databases have also been used has a solution.

The solution proposed by VDSF is not deemed to be better than the existing solutions, just different... it might indeed be a better choice for some, specially for smaller projects which lack the expertise needed to build a full blown J2EE solution (or something equivalent).

The VDSF solution: instead of having one complex multi-threaded application doing multiple tasks, create a bunch of programs each doing its own single task. All the programs will communicate with each other (as needed) using the shared objects in the Virtual Data Space.

An example:

The project: building software to manage a stock exchange.

Some of the potential tasks:

• Receiving buy and sell orders.
• Processing the orders (including queuing orders for later processing if there is no match between the received order and an existing order).
• Managing alerts (for example if the volume of trades on a stock is 150% more than the average daily volume).
• Backing the data to a RDBMS for safekeeping (and for proper data mining, for example examining trends in trading by industrial sector).
• Alerting customers that their orders have been received and are on hold or are currently being processed.
• Preparing invoices (if invoices are processed on the fly).
• Manage snapshots of stock values (to publish on the web at regular intervals, for example).
• Monitoring of the whole application.
• Being alerted when something goes wrong (or even better... before it goes wrong, for example, if the incoming queues are getting quite large but before the system is overwhelmed).
• Etc.

Pros

• Since each task becomes a single program, the amount of time needed to create complex applications should be greatly reduced.
• The simplicity involved should also reduce the number of overall defects in the application and makes it simpler to test and maintain.
• VDSF takes care of all concurrency issues between processes - this should also reduced the number of defects.
• The modularity of the software will also make it easier to add new features to the application in the future.
• The availability of multi-core CPUs makes a framework like VDSF more attractive for small application servers. Using shared memory as the core technology is also a good option with the relatively cheap price of RAM and the gradual adoption of 64 bits OSes (high-end servers have been using 64 bits for quite a while now).
• Programs can be written in any of the supported languages. This will of course allow projects to better use the resources they currently have in-house.
• As a corollary to the previous point, programs written in Java, in C, in Python, in VB, etc. will be able to exchange data almost transparently (one of the goals is to make the interface to VDSF as simple as possible).
• VDSF is built around the concept of "unit of work" - basically implementing a form of exception handling into the framework.
• The priority of each process can be adjusted independently, allowing greater control on the software (example: setting one of the processes listening for requests at a higher priority than others will insure some minimal QoS for the clients using this process).
• The data being external, programs will be able to display basic status information (number of processed requests, etc.), which is then accessible to any VDSF based monitoring program. Similarly, it will be possible to dynamically change the parameters used by the working programs.
• The VDS can also be used as a cache for some of the data in the backend database.

Cons

• The biggest problem I foresee is one of timing. If this software had been written 10 years ago... but with all the investments made in J2EE and similar technologies, it might be too late.
• VDSF itself must be as bug free as possible. To do this properly requires a lot of time (large number of tests, using a form of Design By Contract, etc.).
• Since the core of the technology uses shared memory, it is always possible for client applications written in C/C++ or derived languages to trash the memory and to crash the whole application (the power and danger of using pointers).
• Similarly, if something bad happen (computer crash, for example), the shared memory might be trashed and some of the data in it might be lost (specially if it was in RAM only, not yet flushed to disk by the operating system). Note that normal applications crashing in a similar environment would loose all their data - VDSF is better that way but ... shared memory has a bad reputation for this and VDSF must do what it can to recover as must as possible from crashes.
• Complex applications requiring a farm of computers to process data and/or requests might not be prime candidates for a shared-memory framework.

Last updated on May 22, 2008.