The MWScript Debugger

MoneyWorks 9.2.1 and later has a debugger for the MWScript scripting language. The debugger can be used to proactively debug scripts by setting breakpoints and stepping through the script line-by-line, or it can be entered if a runtime error occurs, or a long-running operation needs to be interrupted.

Debugging runtime errors

For a regular user who does not have the Scripting privilege, there is not much they can do about errors in the scripts you deploy, other than reporting the bug to you so that you can fix it. For that reason, every script you deploy should have a constant meta declaration with a string containing your contact details. This string will be shown to the user in error alerts.

For someone who does have the Scripting privilege for the document, a runtime error will come with the option of dropping into the debugger, or opening the scripts window at the point of the error.

Edit selects the line in the script editor. Ignore just aborts execution of the current handler call from MoneyWorks without unloading the script (same as Debugger Stop button).

Infinite loops

The MWScript virtual machine does not know that this is an error, so will happily follow your script's instructions to endlessly perform a loop with no possible exit condition, or — even worse — endlessly put up alert boxes. In the first case, after a few seconds, MoneyWorks should show an indeterminate progress window with a message "A script is running....". Since you have the Scripting privilege, this window will have an enabled Stop button. You can use this to immediately drop into the Debugger. Here you can examine the script to see what is happening, and you can use the Debugger's Stop button to abort execution of the current handler and return control from your script to MoneyWorks. Stop does not deactivate the script, so MoneyWorks will continue to call your handlers as appropriate. If it calls your handler again and you're back in the loop, the bigger hammer is the Deactivate button, which will deactivate the script without calling the Unload handler.

In the case of an infinite loop of alert boxes, there will be no delay long enough for an indeterminate progress window, so in this case, hold the Ctrl+Shift keys down when clicking a button in the alert. Prior to v9.2.1 this would offer the opportunity to disable the script. From v9.2.1, it will enter the debugger.

Breakpoints

While developing or testing your script, you can proactively enter the debugger from any point in your script by setting a breakpoint in the Scripts window. Click in the darker grey breakpoint margin next to the line number of the line(s) that you want to stop at. A white blob will be placed, indicating a breakpoint. When your script reaches that line, the debugger will be invoked.

You must leave the script window open while debugging. Breakpoints are discarded if you close the script window or select a different script in the Scripts window.

When you set a breakpoint on a line that does not correspond exactly to a virtual machine instruction, you will find that the debugger window displays the breakpoint on the line that does correspond to a virtual machine instruction.

The Debugger

The Debugger contains 3 panes:

  • The script with the current line of execution indicated by a ▷ in the margin
  • The local variables for the current handler, along with the script's properties. You can double-click array variables to expand, or double-click on strings and tables to show them in a viewer window. The string viewer will show invisible characters such as newlines and tabs and spaces, as well as other ASCII control characters, along with the length in characters and bytes; it will alternatively display a warning if the string is not valid UTF-8 or if it contains a NUL character (these conditions will prevent a string from behaving properly in most situations)
  • The call stack — the sequence of handler calls that led to where you are now. You can click on the other points in the call stack to navigate to that point in the script and show that handler's local variables (it may be a different script if you have scripts calling each other via public handlers)

You can resize the panes by dragging the separators between them.

Single-stepping

If you entered the debugger via a breakpoint or interrupting the script via Stop button etc, you can continue or single-step through the script using the toolbar icons (or pressing the indicated key).

  • Continue (c) will continue execution; if there are no further breakpoints or fatal errors, the debugger will close
  • Step Over (o) will step to the next line/instruction. Step Over will step over a handler call.
  • Step In (i) will step into handler calls. If there is no handler call on the line, it is effectively the same as Step Over.
     
    You cannot step into MoneyWorks intrinsic functions or calls to public handlers in other scripts (to do that, you need to open that script and put a breakpoint in it).
     
    If you are at a line with multiple handler calls in an expression (for example an expression like:
    let a = Foo(5) + Bar(3)), Step In will step into the first one (Foo); to step into Bar you will need to use Step In when exiting Foo
  • Step Out (u) will execute until the current handler returns to its caller
  • Stop will abort the current handler call chain and return control to MoneyWorks without deactivating the script. The script will also be opened in the Script editor at the current line
  • Deactivate will Stop and deactivate the script, so MoneyWorks will not call your handlers until you reactivate the script. If you close the Scripts window with the script deactivated (thus saving the state to the server), the script will also be deactivated for other users when they next log in.

Changing breakpoints

You can set or clear breakpoints from the debugger by clicking in the breakpoint margin. When you Continue or Step Out, a breakpoint in the intervening code will drop you back into the debugger. Breakpoints set in the debugger will persist until you clear them or until the script is unloaded.

Fatal error

If you entered the debugger due to a fatal runtime error in your script, you cannot Continue execution, but you can examine the call stack and variable values to determine the cause of the error before either Stopping or Deactivating the script. Fatal errors include things like passing the wrong number of parameters to an intrinsic function or a NULL or invalid handle to a function. These are conditions that your script should not allow to happen. If a user without the Scripting privilege gets a fatal error, that will Stop the script for them, but they won't be able to fix the error.

Navigating in the script

As with the Script Editor, you can search the script within the debugger using ⌘-F/Ctrl-F. Highlighted matches can be stepped through with the return key (shift-return to step backwards). Press tab to exit the search field. When the script is active use ⌘-G/Ctrl-G to find the next instance of the highlighted text (Find Selected), or Shift-⌘-G to Find Selected Backwards. Since the text is read-only you can also just use g or shift-G.

Posted in Uncategorized | Comments Off on The MWScript Debugger

Using Sqlite3 with MoneyWorks

MoneyWorks 9.2 introduces bindings to the Sqlite3 API. This means you can:

  • Create and use temporary in-memory sqlite databases in MWScript for data analysis and tabulation (as a more powerful alternative to TableAccumulate etc)
  • Create and/or open local sqlite databases from MWScript on a MoneyWorks client for storing client-local data, or for importing data from a sqlite.db
  • Create and use shared sqlite databases that reside on a MoneyWorks Datacentre server and access the data from all MoneyWorks clients. This allows MWScript to be used to create bespoke databases and user interfaces for them within MoneyWorks

Sqlite3 databases thus created are entirely independent of the MoneyWorks database.

The three sqlite database functions in MWScript are Sqlite3_Open, Sqlite3_Close, and Sqlite3_Exec.

Sqlite3_Open

let dbref = Sqlite3_Open(databaseFileName)

Opens a connection to a new or existing database.

  • Temporary in-memory sqlite database

    You can create a temporary database (that will be discarded when you close it) by using Sqlite3_Open("") or Sqlite3_Open(":memory:"). You can run queries like CREATE TABLE, INSERT INTO, and SELECT. The results of SELECT will be tab and newline delimited records that you can further process with Slice/Dice etc. Sqlite3_Close will discard the database.

  • Client-local sqlite database file access

    Using Sqlite3_Open("filename.db") will open an existing—or create a new—database file in the Automation Files/Sqlite directory of the MoneyWorks client. Using Sqlite3_Open("/full/path/to/filename.db") will open an existing—or create a new—database file on the local client, provided that the path is allowed by the safe scripting paths in the app preferences. You can call Sqlite3_Open on the same file more than once to get additional connections to the database file (this might happen if different scripts access the same database for example). You need to close each handle thus allocated when you are finished with it.

  • Shared sqlite databases

    Using Sqlite3_Open("shared:filename.db") or Sqlite3_Open("shared-public:filename.db") will open an existing—or create a new—database file in the MoneyWorks Custom Plugins/Sqlite directory on the MoneyWorks Datacentre Server (or, if you are not connected to a server, in the MoneyWorks Custom Plugins/Sqlite directory for the MoneyWorks file you have open (this mode will fail if running a MWScript outside of a MoneyWorks document). All MoneyWorks files that share the custom plugins folder can access the same .db file. Using Sqlite3_Open("shared-private:filename.db") will open an existing—or create a new—database file in the MoneyWorks Custom Plugins/Pictures/company-name/Sqlite directory on the MoneyWorks Datacentre Server (or, if you are not connected to a server, in the MoneyWorks Custom Plugins/Pictures/company-name/Sqlite directory for the MoneyWorks file you have open. Only that company file will have access to such .db files. Note that in this case the sqlite .db will be backed up with the Pictures folder (may be a separate backup depending on server prefs).

    A script running on multiple logged-in clients can access the same sqlite database simultaneously.

    For extra robustness, write-ahead logging mode can be used, with:

    Sqlite3_Exec(dbref, "PRAGMA journal_mode=WAL;")
    

    The MoneyWorks Custom Plugins/Sqlite directory will be backed-up as part of the normal Datacentre backup rotation regime.

  • Shared-but-ephemeral in-memory databases in MoneyWorks Datacentre

    It is also possible to share data between clients using an in-memory database on the server, using a filename in the form: "shared:file:MyMemDBName?mode=memory&cache=shared". An ephemeral database like this will exist as long as at least one client has not yet closed its connection.

Sqlite3_Close

Closes the connection to the database. You should always do this when finished with querying the database.

Sqlite3_Close(dbref)

Sqlite3_Exec

Executes a SQL query on the database. Any SQL supported by Sqlite3 may be used (CREATE TABLE, INSERT, SELECT, UPDATE, PRAGMA, etc)

let result = Sqlite3_Exec(dbref, query, [bindargs...])

The Sqlite3_Exec function allows parameter binding, binding the values passed in as additional parameters into the query wherever it contains a ?. This removes the need to deal with escaping special characters in the data to avoid accidental SQL injection from user-supplied data.

e.g.:

let n = "Some_val"
let id = 123
let sub = Sqlite3_Exec(s, "SELECT subject from templates WHERE name = ? and id = ?;", n, id)

Code sample

on Load
    let s = sqlite3_open("") // temporary in-memory  DB
    let r = sqlite3_exec(s, "CREATE TABLE cars ( make TEXT, model TEXT, colour TEXT, registration TEXT, mileage REAL);")
    let sql = "INSERT INTO cars VALUES ( 'Subaru', 'Outback', 'Gold', 'XYZ999', 36000);
               INSERT INTO cars VALUES ( 'Toyota', 'Corolla', 'White', 'ABC123', 46899);
               INSERT INTO cars VALUES ( 'BYD', 'Shark', 'Grey', 'ZZZ000', 1);"
    let r = sqlite3_exec(s, sql)
    let r = sqlite3_exec(s, "SELECT * FROM cars ORDER BY mileage DESC;")
    syslog(r)
    sqlite3_close(s)
end

Output format

By default, the output format is the same as the Sqlite default format: tab separated column values and newline terminated rows. You can modify the output format mode using a .separator directive.

.separator colsep opt_rowsep

Specify the column separator string and optionally the row terminator string. The parameters can just be space separated or you can optionally quote them with single or double quotes. Do not append a semicolon.

Examples:

   Sqlite3_Exec(s, ".separator |") // just set the column separator
   Sqlite3_Exec(s, ".separator '\t' 'END'") // set both separators
   Sqlite3_Exec(s, ".separator + END")  // set both separators
   Sqlite3_Exec(s, ".separator \t \n") // set to defaults
   Sqlite3_Exec(s, ".separator ''") // set column separator to empty string
   Sqlite3_Exec(s, ".separator '' ''") // no separators
   Sqlite3_Exec(s, ".separator '\t' '\n'") // set to defaults
Posted in Uncategorized | Comments Off on Using Sqlite3 with MoneyWorks

Running a scheduled task in MoneyWorks

Sometimes you want to run a process in a MoneyWorks database on a schedule to happen whether people are logged into MoneyWorks or not. Usually this will involve connecting to a remote server and exchanging information one way or the other. You can do this using a combination of MWScript, the MoneyWorks client in command-line mode, and the built-in task scheduling system of your host operating system.

Let's assume you have already written the MWScript script to automate your process. You have a handler called YourHandler in a script named YourScript that does the job, and you want to run this automatically on a schedule.

Step 1: Make your script handler globally accessible

In your script, add the keyword public to your handler declaration

on YourHandler public
    // do stuff
end

This allows the handler to be called from outside of your script by prefixing the handler name with the script name: YourScript:YourHandler(). Importantly, the script can be invoked by MoneyWorks Gold or the moneyworks CLI tool from a terminal command line. Your script should not include any code that relies on a GUI.

Note: Don't use the script's Load handler. Load handlers run for every user every time they log in. Your special handler will only be executed according to your schedule (although you might also call it from a menu command to allow users to invoke it manually).

Activate your script and save the document

This ensure the script will be available to other clients when they log in (including the command line client you are about to employ...).

Step 2: Invoking the script from the command line

You can and should test out invoking your script from the command line in the Terminal app as shown below. Ideally, this would be done on the server using the specialised command line client that is part of the server installation. On Windows, this a a CUI executable, so you will be able to see the stdout output (and the exe can also be used in interactive mode, although you do not need that for the task at hand).

Command line client on the server

Mac:

/usr/local/bin/moneyworks -zve "=YourScript:YourHandler()" "moneyworks://folder:pass@127.0.0.1:6699?doc=user:pass@Document.moneyworks"

(/usr/local/bin/moneyworks is a symbolic link to the moneyworks CLI client which is inside the Datacentre Console app)

Windows Command Prompt:

"C:\Program Files\MoneyWorks Datacentre\moneyworks.exe" -zve "=YourScript:YourHandler()" "moneyworks://folder:pass@127.0.0.1:6699?doc=user:pass@Document.moneyworks"

Windows Powershell:

& "C:\Program Files\MoneyWorks Datacentre\moneyworks.exe" -zve "=YourScript:YourHandler()" "moneyworks://folder:pass@127.0.0.1:6699?doc=user:pass@Document.moneyworks"
  • The -z option tells MoneyWorks to load scripts when it logs into the file (this is not normally done in command-line mode). Note that this will load every script and execute its Load handler, so make sure that you do not have any scripts in the document that present UI in the Load handler (which you should never do anyway). Functions that install menu commands are OK, because they will do nothing when executed in command line mode.
  • The -v (verbose) option is not required, but while testing it will give you more output on the command line about what it going on
  • The -e option invokes the command supplied in the next argument
  • The argument beginning with = is shorthand for the command evaluate expr='...'
  • Finally the URL contains the location and login credentials of the document to be accessed. Use moneyworks://ssl/ if the server is using TLS encryption

This will connect to the document, execute the script handler, disconnect, and exit.

Since the command line client is part of the server installation, it will be updated whenever the server is updated.

Alternatively, if you do not have access to the server to run commands on it (perhaps you are using a MoneyWorks Now server), use the MoneyWorks Gold client on your own computer:

MoneyWorks Gold client

Mac:

/Applications/MoneyWorks\ Gold.app/Contents/MacOS/MoneyWorks\ Gold -ze "=YourScript:YourHandler()" "moneyworks://folder:pass@server:6699?doc=user:pass@Document.moneyworks"

Windows Command Prompt:

"C:\Program Files\MoneyWorks Gold\MoneyWorks Gold.exe" -ze "=YourScript:YourHandler()" "moneyworks://folder:pass@server:6699?doc=user:pass@Document.moneyworks"

Windows Powershell:

& "C:\Program Files\MoneyWorks Gold\MoneyWorks Gold.exe" -ze "=YourScript:YourHandler()" "moneyworks://folder:pass@server:6699?doc=user:pass@Document.moneyworks"

Note: If you have the 32-bit MoneyWorks Gold installed, it will be in Program Files (x86)

The only downside of using MoneyWorks Gold in command line mode like this, is that if the server is updated to a new version, you will need to intervene to ensure that the client is updated. Also note that while the -e option works with the MoneyWorks Gold GUI client on the command line on Windows, you have no access to stdin or stdout (due to limitations in the design of Windows), and therefore cannot use the interactive mode with the -i option.

Once you have verified that you can invoke your script from the commandline, it is time to set up a scheduled job.

Step 3: Running the command on a schedule

This is where the process diverges markedly on Mac vs Windows.

Windows

On Windows, use Task Scheduler to schedule tasks. Do this on a computer that will always be running (the server is ideal, using moneyworks.exe, but if that is not an option, some other computer with MoneyWorks Gold).

  1. Search for and open Task Scheduler
  2. Under Action, choose Create Task...
     

  3. Enter a Name, description, and choose a security option
     

    Since you want the task to run unattended, you will probably want to choose "Run whether user is logged on or not". In this case you will need to enter login credentials.
    If your script accesses a mapped network drive, be sure that it is mapped for the user you have selected to run as.

  4. In the Triggers tab, click New.. and specify the time schedule you want
     
    E.g. Begin the Task: On a schedule ; Daily — specify the start day and time and Recur every 1 days.
    You may want to use the Advanced settings to specify a different time interval such as "Repeat task every: 6 hours for a duration of: Indefinitely".
  5. In the Action tab, click New.. and specify the command
  6.  
    The Action will be "Start a program".
    The Program will be "C:\Program Files\MoneyWorks Datacentre\moneyworks.exe" or "C:\Program Files\MoneyWorks Gold\MoneyWorks Gold.exe" as appropriate.
    The Arguments will be as for the command line you tested above e.g.

    -ze "=YourScript:YourHandler()" "moneyworks://folder:pass@server:6699?doc=user:pass@Document.moneyworks"
  7. Click OK to save the task
     
    The task should show as "Ready" and will run according to your schedule. You can disable it by right-clicking it in the task list and choosing Disable.

Mac

If your script writes to the Mac's filesystem
The location that your script writes to has to be accessible. MoneyWorks restricts scripts to accessing safe paths specified in the application preferences. You can easily set the MoneyWorks scripting paths in MoneyWorks Gold for the case that you use MoneyWorks Gold for your scheduled automation. If you use the moneyworks command line client on the server, then you need to specify the location using defaults write

defaults write nz.co.cognito.moneyworks.cli safeScriptingPaths "/Users/Shared/Files"

In addition, macOS restricts programs from accessing any user's Documents or Downloads folder (or any network drive or externally attached drive) and will usually put up a permissions dialog the first time a program tries to access one of these locations. Difficulty: if you run the moneyworks command line tool from the Terminal, that permission will be applied to the Terminal app, not to the moneyworks command line tool, so granting the permission that way is not going to work when you run moneyworks on a schedule using launchd, and when you attempt it, the permissions dialog might not be shown to grant the privilege. With this in mind, the best advice is: Do not try to automate file writing to any user's Documents or Downloads folder, or any external or network drive. It is not worth the trouble. It is for this reason that the default scripted file creation location is in ~/Library/Application Support/Cognito/MoneyWorks Gold/Automation Files/. This is accessible by default.

Launchd

There is no built-in UI for scheduling tasks on the Mac. The standard method is to use launchd, which uses .plist files to specify the task parameters.

The easiest way to run a launchd task is as a launch agent in ~/Library/LaunchAgents. These will only run while you are logged in. On a server, after a reboot, you might not be logged in unless you set up autologin.

To always run, even with no-one logged into the computer, you can use a launch daemon in /Library/LaunchDaemons/. The property list for a launch daemon needs to be owned by root.

  1. In a text editor, create a property list for launchd that specifies how you want your job to run

This example will run once a day at 5am

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Disabled</key>
    <false/>
    <key>Label</key>
    <string>local.my.moneyworks.job</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/moneyworks</string>
        <string>-ze</string>
        <string>=YourScript:YourHandler()</string>
        <string>moneyworks://folder:pass@server:6699?doc=user:pass@Document.moneyworks</string>
    </array>
    <key>StandardErrorPath</key>
    <string>/tmp/local.my.moneyworks.job.log</string>
    <key>StandardOutPath</key>
    <string>/tmp/local.my.moneyworks.job.log</string>
    <key>StartCalendarInterval</key>
    <dict>
        <key>Hour</key>
        <integer>5</integer>
        <key>Minute</key>
        <integer>0</integer>
    </dict>
    <key>UserName</key>
    <string>your_username</string>
</dict>
</plist>

Note that you must

  • replace moneyworks://folder:pass@server:6699?doc=user:pass@Document.moneyworks with the actual URL to connect to your MoneyWorks document
  • replace YourScript:YourHandler() with the actual name of your handler
  • replace your_username with your username so that the job will run as your Mac user and pick up the preferences that you previously set for the nz.co.cognito.moneyworks.cli domain. This part of the plist is only required for a launch daemon. A launch agent will always run using your username. You can remove the key and its string value if you do not need them

For a launch agent that will run while you are logged into your Mac:

  1. Save the plain text file as (e.g.) local.my.moneyworks.job.plist in ~/Library/LaunchAgents (tip: If you cannot see the Library folder in your home folder, press Shift--. in the save dialog to toggle invisible folders)
  2. To load the job, use launchctl
     
    launchctl load -w ~/Library/LaunchAgents/local.my.moneyworks.job.plist

For a launch daemon that will run even while no-one is logged into the Mac:

  1. Copy the text
  2. Open Terminal and use sudo nano to create the plist file
     
    This is because the file needs to be owned by root
    sudo nano /Library/LaunchDaemons/local.my.moneyworks.job.plist
  3. Paste your xml that you copied
  4. Press ctrl-X to exit
  5. Press Y to save
  6. Press return
     
    Now you have your launchd property list in the required location, and it is owned by root, as required. If you need to edit it again later, go back to step 2
  7. To load the job, use launchctl
     
    sudo launchctl load -w /Library/LaunchDaemons/local.my.moneyworks.job.plist

Other resources

Since there is no UI for task scheduling on the Mac (Shortcuts.app can only do it in the iPhone version(!!)), if you regularly want to set up scheduled tasks, a third party app can make it easier to do. LaunchControl is a good option for this.

Posted in Uncategorized | Comments Off on Running a scheduled task in MoneyWorks

Using sftp or scp from MoneyWorks to transfer files to/from a remote server

SFTP is a nice and easy protocol for secure transfer of files to/from other servers. It's much easier and more secure than FTP. However, the Curl library in MoneyWorks does not include SFTP (which requires the entire OpenSSH library).

However, OpenSSH command line tools are provided by the operating system (standard on Mac; optional install on Windows 10/11)

To use SFTP you can utilise the MoneyWorks External() function to access to the system scp or sftp client.

For security, MWScript is not allowed free access to the command line, so you must first symbolically link any tools you want to access into the MoneyWorks Externals folder.

On Mac

MacOS includes OpenSSH as standard. The scp command is at /usr/bin/scp. To create the symbolic link, in Terminal.app:

mkdir ~/Library/Application\ Support/Cognito/MoneyWorks\ Gold/Externals
ln -s /usr/bin/scp ~/Library/Application\ Support/Cognito/MoneyWorks\ Gold/Externals/

Windows 10/11

OpenSSH is available for Windows, but not installed by default. You can install it from Windows Settings → Add an optional feature. It will install in C:\Windows\System32\OpenSSH. You'll need to link the main client tools into the Externals folder.

At an Administrator command prompt:

mkdir "%HOMEDRIVE%%HOMEPATH%\AppData\Roaming\Cognito\MoneyWorks Gold\Externals"
mklink "%HOMEDRIVE%%HOMEPATH%\AppData\Roaming\Cognito\MoneyWorks Gold\Externals\scp.exe" C:\Windows\System32\OpenSSH\scp.exe
mklink "%HOMEDRIVE%%HOMEPATH%\AppData\Roaming\Cognito\MoneyWorks Gold\Externals\sftp.exe" C:\Windows\System32\OpenSSH\sftp.exe
mklink "%HOMEDRIVE%%HOMEPATH%\AppData\Roaming\Cognito\MoneyWorks Gold\Externals\ssh.exe" C:\Windows\System32\OpenSSH\ssh.exe

Note: Some Windows-hosted non-OpenSSH sftp servers may not work with standard OpenSSH.

Using SCP to upload/download files

You should use SSH keys installed in the user's .ssh folder. See below for a quick tutorial on making SSH keys. These examples assume you have working SSH keys set up, so no credentials parameters are required. Passing passwords to standard scp/sftp is not trivial (requires an additional tool, sshpass). If you really want to use password authenticaiton, you may find it easier to use putty sftp.

In MWScript, to upload the file at sourceFilePath:

External("scp", sourceFilePath, "example.com:~/Documents/")

(use "scp.exe") on Windows.

To download

Be sure to download to a location that you can access from MWScript (e.g. the cache folder)

External(if(Platform = "Mac", "scp" , "scp.exe"), "example.com:~/Documents/remotefile.txt", CacheFolderPath)
let fd = File_Open(CacheFolderPath + "remotefile.txt", "r")
Alert(File_read(fd))
File_Close(fd)

Generating and installing SSH keys

Let's assume your script needs to upload or download files to/from some_server.example.com. You have a login to this server with username Some_user and a password. The script cannot easily use password login (scp/sftp are designed to discourage this). You should instead generate SSH keys to log in with.

  1. Generate the SSH key pair
  2. In terminal, type ssh-keygen (ssh-keygen.exe on Windows) and press return.
    The tool will ask where to save the private key. The default location will be in your ~/.ssh folder.

    % ssh-keygen
    Generating public/private ed25519 key pair.
    Enter file in which to save the key (/Users/username/.ssh/id_ed25519): /Users/username/.ssh/id_ed25519_for_mw_scp
    

    The type of key algorithm you get by default may vary by platform. On Mac ed25519 seems to be the default. On Windows the default is appears to be RSA. If this is the only SSH key you will ever use (and you do not already have a key with that name), you can just accept the default name (id_ed25519 or id_rsa). If you want to use different key pairs for different servers/purposes you should enter a unique name.

    The keygen tool will ask for a passphrase. You can leave it blank. Just hit enter. A passphrase would encrypt the private key and you would need to enter it to use the key.

    Enter passphrase (empty for no passphrase): 
    Enter same passphrase again: 
    

    The private key and public key will be generated and placed in your .ssh folder in your home directory.

    Your identification has been saved in /Users/username/.ssh/id_ed25519_for_mw_scp
    Your public key has been saved in /Users/username/.ssh/id_ed25519_for_mw_scp.pub
    The key fingerprint is:
    SHA256:obqtnDWp2laGnyhxfxO6p0LcnSwY9Wo4qfQ8wGavRd8 username@yourhostname
    The key's randomart image is:
    +--[ED25519 256]--+
    |                 |
    |      .          |
    |     . ..        |
    |    .  ...       |
           ...
    | + XoO=+E.       |
    |  o+O*=.+        |
    |  o=B++= .       |
    +----[SHA256]-----+
    
  3. Install the public key on the remote server
  4. The public key generated above needs to be installed on the server that you will be connecting to. We assume that for the time being you have a password login to the server.

    You can just log into the remote server (using password login) and directly edit the .ssh/authorized_keys file and add a line, pasting in the text of your public key. Alternatively, you can use an ssh tool or a one-liner to do that for you...

    On Mac, you can use ssh-copy-id

    This script simply logs into the server using SSH and appends the public key to the authorized_keys file in your .ssh directory on the server.

    ssh-copy-id -i ~/.ssh/id_ed25519_for_mw_scp.pub Some_user@some_server.example.com
    

    You'll see output as below

    /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/Users/username/.ssh/id_ed25519_for_mw_scp.pub"
    /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
    /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
    Some_user@some_server.example.com's password: 
    

    Enter the SSH login password to log in and complete the upload:

    Number of key(s) added:        1
    
    Now try logging into the machine, with:   "ssh 'Some_user@some_server.example.com'"
    and check to make sure that only the key(s) you wanted were added.
    

    On Windows, use SSH directly because there is no ssh-copy-id

    This one-liner logs into the server using SSH and appends the public key to the authorized_keys file in your .ssh directory on the server (it's formatted on two lines below, but it needs to be entered on one line). This assumes that the server you are connecting to is BSD/GNU-Linux/Mac rather than Windows.

    C:\> type %USERPROFILE%\.ssh\id_ed25519_for_mw_scp.pub | ssh Some_user@some_server.example.com "cat >> .ssh/authorized_keys"
    

    You will need to type your password to complete the login.

    You should then be able to log in with ssh Some_user@some_server.example.com without needing a password.

    Furthermore, your MWScript that runs while you are logged into your computer using MoneyWorks can also automatically log in using your key to SCP files.

Posted in Uncategorized | Comments Off on Using sftp or scp from MoneyWorks to transfer files to/from a remote server

PreviewXMLtoPDF and merging output documents

MoneyWorks Gold 9.1.7 has a new function for converting the "xml" output format (from DoReport and DoForm) to PDF. On its own, this isn't especially useful, since you can just output to PDF in the first place. However, it is possible to use this to merge multiple output files into one PDF, or even insert blank pages by massaging the XML files.

Proof of concept:

constant meta = "XML/PDF merging"

/*

    Outputting forms and reports as "xml" looks like this (this is the format used by the MoneyWorks Preview window) 

    <?xml version="1.0"?>
    <document>
        <paperrect left="-20" top="-20" right="655" bottom="936" />
        <pagerect left="0" top="0" right="635" bottom="889" />
        <meta>
            <title>Balance Sheet</title>
            <generated>26/02/24, 12:20:58 PM</generated>
            <generator>MoneyWorks-macOS-x64</generator>
        </meta>
        <page decimal_sep="46" pageNum="1">
        ...
        </page>
    </document>

    PreviewXMLtoPDF converts one (valid) XML file to PDF
    
    To merge multiple preview XML files, you need to massage the xml a bit to keep the merged XML valid
    In particular, the page numbers must remain sequential
    
    MergePreviewFiles is a q&d function to merge a list of files
    
    Caveat: This will only work if the page sizes, orientation, and scaling of all of the files are the same
    because PreviewXMLtoPDF will not change the PDF page size/scale/orientation halfway through the document

*/

on MergePreviewFiles(a)
    let outfile = File_Open("TMP/merged.xml", "w")
    let numfiles = CountElements(a)
    let pagenum = 0
    foreach k in array a
        let path = a[k]
        let f = File_Open(path)
        let seenHdr = false
        while 1
            let line = File_ReadLine(f)
            if line == NULL or line = ""
                break
            endif
            if TextToNum(k) > 1 and trim(line, true) = "<pagerect@"
                let seenHdr = true
                continue
            endif
            if TextToNum(k) < numfiles and (trim(line, true) = "</document>")
                break
            endif
            if TextToNum(k) > 1 and not seenHdr
                continue
            endif
            if trim(line, true) = "<page@"
                let line = Regex_Replace(line, `pageNum="[0-9]+"`, `pageNum="` + pagenum+`"`)
                let pagenum = pagenum + 1
            endif
            
            File_Write(outfile, line)
        endwhile
        File_Close(f)
    endfor
    let outpath = File_Path(outfile)
    File_Close(outfile)
    return outpath
end

on Load
    let a = CreateArray()
    let a[1] = DoForm("Balance Forward Basic", "xml", "code=`SMITH`") // statement
    let a[2] = DoForm("Product Invoice 1", "xml", "namecode=`SMITH`") // invoices
    //let a[3] = DoReport("Balance Sheet", "xml", "Balance Sheet") // will run into scaling issues
        
    let final = MergePreviewFiles(a)
    let p = PreviewXMLtoPDF(final)
    File_Move(p, "Merged_Forms_"+TimeStamp(TSZ_LOCAL, "%Y-%m-%d_%H%M%S")+".pdf") // move to automation files folder
end
Posted in Esoterica, MWScript | Comments Off on PreviewXMLtoPDF and merging output documents

Web views in the MoneyWorks Navigator

You have long been able to add your own panels to the MoneyWorks Navigator. Custom Navigator panels can be implemented using custom invoice forms or custom reports and are loaded from a folder named Navigator Extras in your custom plugins folder.

In MoneyWorks 9.1.7 and later you can add Navigator panels that are web pages. MoneyWorks will install a Navigator panel for each .html file in your Navigator Extra folder. The .html file can be an actual HTML file, or it can contain a single line with an http or https URL, which will be loaded, or it can contain a single line with a MWScript handler invocation that returns either a URL or html text. It can also be completely empty.

MoneyWorks will call a MWScript Before handler once for each HTML panel the first time the panel is selected/loaded. The specific name of the Before handler should be Before:F_NAVIGATOR:Panel_name where Panel_Name is the name of the panel's HTML file, minus any prepended numbers or symbols, and minus the file extension, and internal spaces changed to underscores (so "1. HTML with ChartJS.html" → HTML_with_ChartJS).

Static page or URL

In the normal case, a local HTML file will be loaded (if the contents are actually html). This file can load other local resources from the same folder. Alternatively you may load an external/hosted web page by its URL by just putting the full URL as a single line in the "html" file.

Loading a page from the Before handler

If your html file is empty (and named Blank.html), you can load a page in a Before handler for the panel:

on Before:F_NAVIGATOR:Blank
    LoadURLInWebView(GetWindowByID("F_NAVIGATOR"), "Blank", "https://cognito.co.nz/developer/")
end

Getting the URL to load from MWScript

If calling a MWScript handler, the handler should be declared public.

Examples for generating a URL, or HTML:

on MakePageURL public
    return "https://xkcd.com/" + Random(2896) + "/"
end

on MakePageHTML public
    let h =  "<html><style>td { padding: 5px; font-family:Verdana;font-size:10pt; }</style><table style=\"padding:10px; margin: 50px;\">"
    foreach a in account CreateSelection("account", "1")
        let h = h + "<tr><td>" + a.code + "</td><td>" + a.description + "</td><td>" + a.type + "</td><td>" + a.category + "</td><td>" + a.taxcode + "</td><td>" + a.group + "</td></tr>"
    endfor
    let h = h + "</table></html>"
    return h
end

Call these by putting MyScript:MakePageURL() or MyScript:MakePageHTML() on one line in the .html file.

The web page can call out to MWScript, and MWScript can inject javascript into the web page. See Communicating with Javascript in a Web View .

To allow the web page to call out to MWScript, you should implement a Before handler that installs a web observer in tothe web view (see below).

To inject Javascript in to the web view, use the WebViewControl function with "evaluate" (also below).

on Before:F_NAVIGATOR:HTML_with_ChartJS(w)
    WebViewControl(w, "HTML_with_ChartJS", "options addObserver='MyHandlerName'")
end

on CompletionHandler
    syslog("completion handler")
end

on AppendData(s, d)
    if s = ""
        return d
    else
        return s + ", " + d
    endif
end

on MyHandlerName(data)
    
    let labels = ""
    let data = ""
    let per = CurrentPeriod()
    foreach cat in general CreateSelection("general", "code = `C@`")
        let labels = AppendData(labels, `"` + cat.code + `"`)
        let data =  AppendData(data, GetBalance ("(type = `EX` or type = `CS`) and Category=`"+cat.code+"`", per))
    endfor

    let javascript = `thechart.data.labels = [`+labels+`]; thechart.data.datasets[0].data = [`+data+`]; thechart.update();`
    
    WebViewControl(GetWindowByID("F_NAVIGATOR"), "HTML_with_ChartJS", "evaluate", javascript, "CompletionHandler")
end

Beware of Windows bug

Unfortunately, on Windows, the Microsoft WebView2 browser component has a serious performance bug, wherein it will force every other window in the app to redraw constantly while the webview is visible, resulting in high CPU usage. Microsoft have thus far declined to fix it even though it has been known for years. To mitigate the bug somewhat, MoneyWorks 9.1.8 and later will hide Webview2 views when they are in inactive windows and replace them with a static image of the last thing the web view contained. You should avoid leaving a WebView2 active if high CPU usage might be a problem.

Posted in Esoterica | Comments Off on Web views in the MoneyWorks Navigator

New web views in MoneyWorks 9.1 and later

WKWebKit on Mac

MoneyWorks 9.1 and later on Mac uses WKWebView instead Webview for embedded web controls. This change should be entirely transparent to existing scripts that use webviews. You get the newer out-of-process javascript engine.

WebView2 on Windows

MoneyWorks 9.1 and later on Windows will use WebView2 for scripted web views instead of IWebBrowser2 if the WebView2 Runtime is installed. WebView2 provides the Chromium-based Edge browser instead of IE11.

If Webview2 is not installed (you can check in Add/Remove Programs — it appears as "Microsoft Edge Webview2 Runtime" — you can get the installer from Microsoft. Use the Evergreen Bootstrapper. It's installed by default in Windows 11, and is highly likely to be installed automatically if you have Office 365 on the machine.

It is worth noting that WebView2 on Windows instantiates asynchronously. If you load a URL, HTML, or javascript immediately when your webview-containing window opens, MoneyWorks will store the request until such time as the Webview2 is instantiated. Do not expect to be able to do multiple loads at instantiation.

Configuring

How to get the old IE11 web view

If for some reason you want the old IE11-based web view on Windows, you can force it to be used by using the following before the window containing the web view is instantiated (note that this usage of WebViewControl will throw an error on earlier versions of MoneyWorks)

WebViewControl(NULL, NULL, "options useWebView2IfAvailable='false')
Finding out if the Edge browser is available

On Windows, you can determine if WebView2 is available with the following:

WebViewControl(NULL, NULL, "query available='WebView2'"))

This will return true if WebView2 is available on the machine. This usage requires MoneyWorks 9.1 or later.

Getting the web inspector

To enable access to the browser's inspector functionality, you can use the following option

WebViewControl(wndhdl, "L_WEB", "options wantContextualMenu='true' enableInspector='true'")

Contextual menus on Windows are disabled by default. This will enable the contextual menu and the web inspector item in that menu. wantContextualMenu is ignored on Mac (the contextual menu is always available).

Posted in Uncategorized | Comments Off on New web views in MoneyWorks 9.1 and later

Communicating with Javascript in a Web View

MoneyWorks 9.1 provides modernised web views on Mac and Windows (see New Web Views in MoneyWorks 9.1). MoneyWorks 9.1 and later also provides new functionality for communicating with the content of web views (requires WebView2 Runtime on Windows).

Calling out to MWScript from Javascript in the web page

You can now call a MWScript handler from javascript in a web page hosted in the Web View, passing it a JSON parameter. You need to set up the callable handler in advance using WebViewControl

MWScript to install an observer handler that can be called from Javascript
on Before:MyWindow(w)
    // L_WEB is the identifier of your web view control
    WebViewControl(w, "L_WEB", "options addObserver='MyHandlerName'")
end

on MyHandlerName(data)
   say(data) // might want to actually parse the JSON using JSON_Parse
end
Javascript to call the observer handler
var message = { // Some JSON to send to the handler
    name: document.getElementById("name").value,
    email: document.getElementById("email").value
    };

// Of course the javascript is different depending on Webkit/Chrome :-(
var ischrome = /Chrome/.test(navigator.userAgent);
if(ischrome)
    window.chrome.webview.postMessage(message); // Windows case
else
    window.webkit.messageHandlers.MyHandlerName.postMessage(message); // Mac case

Note that with WebKit on the Mac, you can install multiple "observer" handlers and call any of them by name using window.webkit.messageHandlers.YOUR_HANDLER_NAME.postMessage. With Chrome on Windows, you can only install one, that will be called by window.chrome.webview.postMessage

Injecting javascript into the web page from MWScript

You can execute some javascript in the web view and be called back with the result

WebViewControl(winhdl, "L_WEB", "evaluate", javascript, "CompletionHandler")

The CompletionHandler is a MWScript handler that will be called when the javascript finishes executing. It will receive as a parameter the last value evaluated by the javascript.

on CompletionHandler(javaScriptResult)
   // ....
end

Injecting one line of javascript to get the value of an input field

WebViewControl(winhdl, "L_WEB", "evaluate", `document.getElementById("name").value;`, "CompletionHandler")

Note that javascript injection should not be attempted until after the html is loaded.

Posted in Uncategorized | Comments Off on Communicating with Javascript in a Web View

Server change notifications

External system notification has been implemented in MoneyWorks Datacentre 9.0.2. This allows another server to be notified whenever a change is made in a MoneyWorks database.

This will remove the need for external systems to poll for changes (a practice which can cause unnecessarily high load on a MoneyWorks server due to constantly opening and closing the document when no changes are being made).

Notifications are generally sent within a minute of changes being made, and you will get a timestamp that predates all of the changes, and a list of changed tables, and/or an indication of whether posting has occurred. You can use the timestamp to search for changed records or transactions posted using LastModifiedTime or TimePosted.

How to enable

Change notifications must be enabled in the document

To enable change notifications in the document, you need to execute the following once:

SetDocumentGlobal("notifyChanges", monitorTablesList)

This will set a global in the document to enable change notifications. You can turn it off by passing an empty string. You will need scripting privileges to execute this.

monitorTablesList is a comma-delimited string listing the tables you want to monitor. It can also contain the meta-table "_posted" to get notifications of posting.

So for notification for posting and changes to jobs and products:

SetDocumentGlobal("notifyChanges", "_posted,job,product")

Change notifications must be enabled on the server

Setting the document notification mode will have no effect until you also enable change notifications on the server.

To do this, modify the folder.conf file for the folder hosting the document. Obviously, to do this, you will need administrative access to the server.

Three notification modes have been implemented:

Simple outgoing POST to url

This mode will do a simple POST, with a payload of doc=docname&time=timestamp&changed=changedtables to a url nominated in the folder.conf

To enable this mode, add the following to the folder.conf for the folder containing the document

databaseModifiedPOST_URL: https://example.com/moneyworks_ping.php

and optionally, a header

databaseModifiedPOST_Header: Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==

Within one minute of qualifying changes being made in the database, the server will POST to the url. The POST will have Content-Type: application/x-www-form-urlencoded and will contain three values: doc will identify the originating document (there may be more than one in the folder).time will identify a time in the server's timezone that predates the changes being notified. You can call back through the DC REST API and search the LastModifiedTime of any table you are interested in to find the changed records whose modified time is >= the given timestamp; changed will have a comma delimited list of table names indicating which tables have changed since the timestamp time. The list may also begin with the word _posted if transactions have been posted. The list will never include the details table. Only tables requested by notifyChanges are included in this list.

It is up to you to host an endpoint that can receive the POST and act upon it. If you are employing third party services with their own authentication or API requirements, then you may need to proxy those through such an endpoint.

This mode is relatively lightweight, in that it does not require an extre process to spin up on the server to send the notification (and does not consume a login on the server while doing so), and relatively secure in that the outgoing URL is known to the server administrator.

Potentially available on MoneyWorks Now

For a MoneyWorks Now server, we may entertain enabling this mode.

Serverside DatabaseModified script handler (sandboxed)

If you need a bit more control over the outgoing post, and want to include an actual data payload to obviate the need for a further REST request back into the server, there is a mode that allows execution of a script on the server.

To enable this mode, add the following to the folder.conf

serversideDatabaseModifiedScriptEnable: 1

You can include a script in the document whose name starts with _SERVER_ and contains a handler named DatabaseModified. Scripts not so named will not be loaded by this process on the server.

Within one minute of changes being made in the database, the server will load the script(s) (logging in as "Admin", which needs to exist) and call the DatabaseModified handler. The timestamp will identify a time in the server's timezone that predates the changes being notified. You can search the LastModifiedTime of any table you are interested in to find the changed records whose modified time is >= the given timestamp.

The tables parameter is a comma delimited list of table names indicating which tables have changed since the timestamp time. The list may also begin with the word _posted if transactions have been posted. The list will never include the details table. Only tables requested by notifyChanges are included in this list.

Executing the script requires spinning up a serverside client, which will consume a concurrent login on the server.

on DatabaseModified(timestamp, tables)
    syslog("DatabaseModifed handler " + DateToText(timestamp, DateFormYYYYMMDDHHMMSS) + " " + tables)
    let posted = false;
    let xml = ""
    foreach t in text tables // e.g. _posted,transaction,ledger,product
        if t = "_posted"
            let posted = true
        else
            let xml = xml + Export(t, "xml-terse", "" , "LastModifiedTime >= '" + DateToText(timestamp, DateFormYYYYMMDDHHMMSS) + "'")
        endif
    endfor

    let msg = ""
    if posted
        let msg = "Transactions were posted.\n\n"
    endif
    let msg = msg + "The following records were updated in the database:\n\n" + Replace(xml, "<?xml version=\"1.0\"?>", "")

    // with serversideDatabaseModifiedScriptEnable = 1 you have access to SandboxedPOST
    // the POST destination and payload size will be logged on the server

    SandboxedPOST("https://server/myscript.php", msg, "Authorization: Bearer blah", "Content-Type: plain/text")

end

The above example uploads all of the changed records. You would not do this in practice.

Potentially available on MoneyWorks Now

Under consideration to be available on MoneyWorks Now. Not sure if the url will need to be restricted—currently it is not. The destination and payload size is logged.

Serverside DatabaseModified script handler (unsandboxed)

To enable this mode, add the following to the folder.conf

serversideDatabaseModifiedScriptEnable: 2

This mode will execute DatabaseModified handlers in scripts with names beginning with _SERVER_, but in this case, the scripts have full access to CURL and File APIs (no sandboxing). This mode would only be enabled on a server you own, where you are fully in control of scripts that will be deployed in documents on the server.

on DatabaseModified(timestamp, tables)  // will execute in CLI on server
    syslog("DatabaseModifed handler " + DateToText(timestamp, DateFormYYYYMMDDHHMMSS) + " " + tables)
    let posted = false;
    let xml = ""
    foreach t in text tables
        if t = "_posted"
            let posted = true
        else
            let xml = xml + Export(t, "xml-terse", "" , "LastModifiedTime >= '" + DateToText(timestamp, DateFormYYYYMMDDHHMMSS) + "'")
        endif
    endfor

    let msg = ""
    if posted
        let msg = "Transactions were posted.\n\n"
    endif
    let msg = msg + "The following records were updated in the database:\n\n" + Replace(xml, "<?xml version=\"1.0\"?>", "")


    //__SendSMTPMail_WithCredentials(to, path, subject, message_text, attachmentName, server, replyTo, authUser, authPass)
    let to = "recip@example.com"
    let server = "mail.example.com"
    let replyto = "me@example.com"
    let user = "bot@example.com"
    let pass = "SMTPp4ss"

    Built_In:__SendSMTPMail_WithCredentials(to, "", "database changed", msg, "", "mx.cognito.co.nz", replyto, user, pass) 
end

This example emails the changes using the built in SMTP script (which in turn uses CURL, for which you need unsandboxed execution).

Not available on MoneyWorks Now

Unsandboxed mode will not be enabled for MoneyWorks Now servers. Allowing arbitrary outbound network connections from our servers seems like a bad idea.

Posted in Uncategorized | Comments Off on Server change notifications

XOR handler

MWScript doesn't have a bitwise XOR function, but it does have an AND with ones-complement function in ClearFlag, which provides a bitwise NOT, which can be used with TestFlags (which is bitwise AND) to do a NAND. Thus you can use these to implement a 32-bit XOR.

on NAND(a, b)
    return ClearFlag(#FFFFFFFF, TestFlags(a, b))
end

on XOR(a, b)
    return NAND(NAND(a, NAND(a, b)), NAND(b, NAND(a, b)))
end


on Load
    let x = xor(#FFFF00FF, #8000FF55)
    syslog(NumToText(x, NumFormHex32))
end
Posted in MWScript, Sample Code | Comments Off on XOR handler