Importing over high latency network

Ideally, importing to a cloud server would always be done via cloud APIs (i.e. the REST API provided by MoneyWorks Datacentre). However, one of the advantages of MoneyWorks over browser-based cloud solutions is that you can integrate seamlessly with other desktop apps, and this integration typically involves importing via the desktop client, which has been subject to latency issues.

Solution

As of MoneyWorks Datacentre 7.1.8, importing data via the MoneyWorks Gold client by almost any means (Import command, Paste from clipboard, drag and drop, Applescript, COM, Filemaker plugin; tabular or xml formatted data) will proceed on the server if the network connection is high latency (generally more than a couple of milliseconds, so you will also see this behaviour on most wifi networks).

What happens

  • The entire data or datafile to import is first uploaded to the server (it's compressed first), along with the import map supplied (if any).
  • The import is then done by a worker process on the server (using the CLI import command).
  • Any error messages or results are returned by the server.

The result

Importing to a server over a high latency network should generally be much faster, and also use less network bandwidth.

The only other noticeable change is that the first "checking pass" in the manual import/pasting case is not seen by the user. If the data is good to import, it will be imported. The progress bar will also be indeterminate, with any error or success messages not shown until completion.

Controlling serverside importing

Normally the serverside importing mode is selected automatically depending on the detected latency of the network connection, but you may wish to control the mode manually. For this purpose, there is a hidden preference option named remoteImportMode. You can use defaults write or RegEdit to set this.

remoteImportMode = 0 — (default) automatic based on latency
remoteImportMode = 1 — force server side importing even on low latency network
remoteImportMode = 2 — force client side importing (will run as for 7.1.7 and earlier).

If you experience problems with the new importing, please try

defaults write nz.co.cognito.MoneyWorks.Gold remoteImportMode 2

To test your import scripts with server side importing

defaults write nz.co.cognito.MoneyWorks.Gold remoteImportMode 1

To reset to default behaviour

defaults write nz.co.cognito.MoneyWorks.Gold remoteImportMode 0

NOTE: If you're executing the CLI on Mac using the symlink at /usr/bin/moneyworks instead of the full path to the actual executable, substitiute moneyworks for nz.co.cognito.MoneyWorks.Gold in the above commands.

Posted in Uncategorized | Comments Off on Importing over high latency network