I've managed to find the setting. The wording isn't the best so this didn't stand out initially.
If you set "Event Notifications" under Alarm > Disarming to "Close" it will not alert you while the system is disarmed. However, once you arm the system via DMSS (or the Web UI), it will alert you...