Change Tracking and Syncing

Owned by Jacques Marais
Last updated: May 17, 2019


Introduction

Helium features a strategy for change tracking in the app persistence tier. This is provided for all model objects in an app by default. Change tracking provides essential "out of the box" functionality that can be used to construct audit trails for app data.

Change tracking is also used by the Helium Journey Sync Engine to facilitate syncing between Helium and Journey.



Default Behaviour

By default change tracking is enabled for all model objects in a DSL app. More detail is provided here.

The Helium Journey Sync Engine is used to sync data between Helium and Journey apps. The sync engine needs to be configured for each app for which integration is required. The sync engine makes use of the change tracking provided by Helium and by default all objects are synced.



Disabling Syncing from Helium to Journey for an Object

If integration for an app has been configured using the Helium Journey Sync Engine, all objects are synced by default. If an object is to be excluded from syncing from Helium to Journey, the integration can be appropriately configured to indicate this. This is achieved as follows:

  • Log in to the DevOps portal: https://portal.heliumapps.com/#/
  • Select "HeliumJourney Sync"
  • Select the "Edit" row action for your integration
  • Select the "Show Advanced" option
  • Specify the tables to skip under the "Change Log Skip Tables" option.

Note that this will not affect change tracking in Helium. Change tracking will still be enabled and behave as discussed here.

The NotTracked object annotation can also be used to prevent syncing an object from Helium to Journey but this will result in all change tracking being disabled for the object which is, in general, not recommended.



Disabling Syncing from Journey to Helium for an Object

Similarly to how syncing for an object from Helium to Journey can be disabled, syncing for an object from Journey to Helium can also be disabled in the Helium Journey Sync Engine configuration:

  • Log in to the DevOps portal: https://portal.heliumapps.com/#/
  • Select "HeliumJourney Sync"
  • Select the "Edit" row action for your integration
  • Select the "Show Advanced" option
  • Specify the tables to skip under the "Oplog Skip Object Types" option.

Note that this will not affect change tracking in Helium. Change tracking will still be enabled and behave as discussed here.

In addition, if data for an object in the Journey app is only relevant locally on the mobile app, LocalDB can be used in the Journey app. This is described in the Journey documentation that can be found here.



Disabling Change Tracking for an Object

The Helium DSL also provides a mechanism with which change tracking for an app can be disabled on a per object basis. This is done by means of the NotTracked annotation as mentioned here. 

The NotTracked annotation should only be used in cases where audit trails provides no potential use and might result in unnecessary and undesirable duplication.

As an example consider a case where a scheduled function inserts its own "history" records each time it runs. The records being created in this case are always inserted and the inserts are only done by means of a scheduled function being run by Helium. It therefore adds no value to have additional change tracking in this case and might lead to unnecessary resource usage.

Using the NotTracked  annotation will also disable syncing from Helium to Journey if the app is configured for integration on the Helium Journey Sync Engine.



Additional Audit Trails

It's also possible to see the identity of the last user to modify a record in the app schema using the _tstamp_ field on the record itself even if change tracking is disable using the NotTracked annotation. The method described in this section identifies a transaction using a time stamp and although it is possible that more than one transaction occurs at the exact same time, it's highly unlikely. The example below demonstrates this:

select _tstamp_ from systemadmin where _id_ = 'dc494f16-cfbb-47f6-88e6-844d188e6d69';

         _tstamp_          
---------------------------
 2019-05-07 13:13:44.35326
helium-app-1=# select identity_id from __he_sync_transaction_log__ where tx_tstamp = '2019-05-07 13:13:44.35326';

             identity_id              
--------------------------------------
 dc494f16-cfbb-47f6-88e6-844d188e6d69


Summary

  • To disable syncing for an object from Journey to Helium modify the "Oplog Skip Object Types" config value of the Helium Journey Sync Engine configuration. Alternatively use LocalDB on Journey to avoid sending the data to the Journey server if the use case dictates it.
  • To disable syncing for an object from Journey to Helium modify the "Change Log Skip Tables" config value of the Helium Journey Sync Engine configuration.
  • To disable change tracking for an object in Helium use the NotTracked annotation. This will also disable syncing from Helium to Journey for that object.