UltraVNC

Thanks Rudi. The scroll lock thing works great for passing through key-commands.

On the system I'm connecting to, there's only a single 4K monitor (laptop screen) connected, as well as 2 display emulators that are running at 1080p (Can easily be removed). I keep the laptop closed, so only the 2 hardware emulated displays are visible. On my viewer system, I've got 4 monitors ...