Update. Solved !
Sure enough, my cheeky attempt to try 2020 R2 Smart Client with 2020 R3 server failed because as seems usual with Milestone software, uninstalling a package does not fully uninstall it. 2020 R2 Smart Client thought 2020 R3 Smart Client was still installed.
Anyway, I found that...
I have a very similar problem. I have been running Milestone 2019 R2 for years and MKV exports of around an hour long were under 2Gb. Now with Milestone 2020 R3 (Essential+) I get over 2Gb MKV for around 10 minutes.
In 2020 R3 it takes a long time (much longer than 2019 R2) to export 10 minutes...