The solution is to set "Compress level" to 100. This will force the old RDFT algorithm, which the game is expecting. Tried with one cutscene as proof of concept and got it working: the video plays at correct resolution in game and has working audio.
That said, it does noticably degrade the audio. If you're willing to go an extra mile, it's possible to also download an older version of RAD (webarchive link) and use "Mix in sound" feature to edit only the audio track and get better quality (it can't read modern containers such as MP4, but you can still mess with the audio). Anyway, I'll see if I can sift through the cutscenes and will let you know if anything comes out of it.