Cannot connect to RealVNC server

I'm having problems connecting to a RealVNC server.
Whatever authentication mode I try in the connection profile ("VNC" or "Windows"), I'm getting error "Not supported authentication method".
Any ideas what can be done?

RealVNC server version 5.3.2 x64 on Windows x64
ASG-RD 2016 9.0.5385
Component check says: "The internal VNC component is installed and working properly"