Scaling

Types of scaling

There are two types of scaling: up/down and out/in.

.

Scaling up and down is probably the easiest to understand and to execute: it is a matter of adding more computing power to a node. For Dime.Scheduler, this usually means bumping up the specs of the web server. This is an easy way to improve the overall performance and is a job that is well understood with network and system administrators, but there are limits as to how much computing power you can add to a server.

Scaling out and in is different altogether. It involves adding nodes and clusters who work together to serve a common purpose. Instead of using one web server, there could be ten web servers spread out geographically to reduce latency and spread the load.

Scaling up

Dime.Scheduler can be scaled in all directions. To scale up and down, nothing needs to happen except ask the IT department to give this virtual machine more virtual processors and memory.

Check out the server requirements here.

Scaling out

In contrast, scaling out is more complex because it involves multiple nodes that somehow need to work together. Dime.Scheduler is connected to other systems as well, so this architecture also becomes increasingly difficult.

Again, network and system administrators have all the tools at their disposal to scale out the Dime.Scheduler web server by adding nodes to the cluster and putting a load balancer in front of the web farm to spread the load of web requests.

.

In this web farm, the installer can be run as usual with one exception: it is only necessary to deploy the database once. This part of the architecture is already centralized and it is almost always hosted on a dedicated SQL server. The remainder of the installation process can be done as usual. Every server has a running instance of the web application, supported by the service bus Windows Service and the connected plugins.

After the installation however, a few configuration item may need to be adjusted to accommodate the web farm dimension.

Service bus message persistence

The service bus is one of the most important hubs of the application. It receives and sends messages to the plugins, which connect to systems like Exchange and Business Central. By default, SQL is used, so we're all good on the web farm front, but it is possible to use other storage mechanisms too.

The configuration is done in the dime.config file. By default this is the configuration:

<endpoint>
  <!-- Transport Type: choose beteen None, Sql, Azure -->
  <!-- Transport Transaction Mode: choose beteen None, TransactionScope, ReceiveOnly, SendsAtomicWithReceive -->
  <transport type="Sql" connection="DB" transactionMode="TransactionScope"></transport>
</endpoint>

The transport type defines how data will be communicated between systems. One can choose between mechanisms such as SQL database, Azure Service Bus, and more. For web farms, this mechanism needs to be a centralized solution.

If you decide to use SQL, your configuration would look like this:

<connectionStrings>
    <add name="SQL" connectionString="MyDimeSchedulerConnectionString">
</connectionStrings>
<endpoint>
  <transport type="Sql" connection="SQL" transactionMode="TransactionScope"></transport>
</endpoint>

If the SQL database instance is an Azure SQL database, make sure the transaction mode is set to SendsAtomicWithReceive.

The XML comments provide the other options, so all you need to do is fill in the values and restart the services in the right order; the service bus service must be running first in order to receive subscription requests from the plugins.

Configure connection tracker persistence

To get real-time updates from NAV and from the other planners, the connection tracker needs a different persistence medium. By default, the server's memory cached is used. Obviously this would lead to incorrect results in a web farm. In other words, this persistence medium needs to be centralized as well. The easiest alternative is the Dime.Scheduler database, which has built-in support for this.

The connection tracker can easily be swapped with some configuration. In the unity.config, you'll find the following configuration by default:

<type type="IConnectionTracker" mapTo="InMemoryConnectionTracker">
  <lifetime type="lifetimeManager" />
</type>
<!--<type type="IConnectionTracker" mapTo="SqlConnectionTracker">
  <lifetime type="lifetimeManager" />
</type>-->
<!--<type type="IConnectionTracker" mapTo="RedisConnectionTracker">
  <lifetime type="lifetimeManager" />
  <constructor>
    <param name="redisManager" type="IRedisClientsManager" />
  </constructor>
</type>-->

To activate the SQL connection tracker, uncomment that section and remove the in-memory connection tracker:

<!--<type type="IConnectionTracker" mapTo="InMemoryConnectionTracker">
          <lifetime type="lifetimeManager" />
</type>-->
<type type="IConnectionTracker" mapTo="SqlConnectionTracker">
  <lifetime type="lifetimeManager" />
</type>
<!--<type type="IConnectionTracker" mapTo="RedisConnectionTracker">
  <lifetime type="lifetimeManager" />
  <constructor>
    <param name="redisManager" type="IRedisClientsManager" />
  </constructor>
</type>-->

The application will automatically pick up the connection string that is configured elsewhere in the web.config file.

Web sockets scale out

Same principle applies for the real-time messaging system. In the dime.config file, you should update:

<!-- SignalR Configuration
    **********************
    1) Specify the backplane of SignalR (None, Azure, Sql, ...)
    2) Specify the connection tracker which will persist the online users
    -->
<signalR mode="Custom" enabled="true">
    <messageBus type="None" connection="AzureServiceBus" topic="dimescheduler:messages" server="localhost" password="" port="6379" />
</signalR>

Depending on the requirements, you may want to modify the message bus type and connection. For example, if you want to use Azure Service Bus as the backplane, you need to set the type to 'Azure' and the connection to a connection string in the connection strings section of the dime.config file:

<dime xsi:noNamespaceSchemaLocation="dimeschema.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <connectionStrings>
    <add name="AzureServiceBus" connectionString="Endpoint=sb://xxx.windows.net/;" />
  </connectionStrings>

  <signalR mode="Custom" enabled="true">
    <messageBus type="Azure" connection="AzureServiceBus" topic="dimescheduler:messages" server="localhost" password="" port="6379" />
  </signalR>

However, this introduces another node to the mix, so we recommend to use Dime.Scheduler's SQL database and use SQL as the scale out plane and the DB connection that's there by default.