Update: UltraVNC 1.4.3.6 and UltraVNC SC 1.4.3.6: viewtopic.php?t=37885
Important: Please update to latest version before to create a reply, a topic or an issue: viewtopic.php?t=37864

Join us on social networks and share our announcements:
- Website: https://uvnc.com/
- GitHub: https://github.com/ultravnc
- Mastodon: https://mastodon.social/@ultravnc
- Facebook: https://www.facebook.com/ultravnc1
- X/Twitter: https://twitter.com/ultravnc1
- Reddit community: https://www.reddit.com/r/ultravnc
- OpenHub: https://openhub.net/p/ultravnc

No blue component if with vnc4server 32 bit color depth.

Post Reply
tyan0
8
8
Posts: 21
Joined: 2021-08-10 03:14

No blue component if with vnc4server 32 bit color depth.

Post by tyan0 »

Versions:
UltraVNC viewer: 1.3.3 dev 10, 1.3.2, 1.2.4.0, etc.
vncserver: vnc4server 4.1.1+X4.3.0-37.6 (debian jessie)
Server OS: Debian GNU Linux (jessie)
Client OS: Windows 10 x64 (21H1)

Overview:
If UltraVNC viewer connects to vnc4server (in Linux) with 32 bit color depth, the blue component is lost from entire screen if the server is started with 32 bit color depth.

Steps to reproduce the issue:
1. Install vnc4server in debian jessie or wheezy (very old version).
2. Start vncserver in debian with

Code: Select all

vnc4server -geometry 1024x768 -depth 32
3. Connect to the vncserver using UltraVNC viewer (any version) from Windows10 client machine.

Resulting screenshot with the issue:
Image

Screenshot using TightVNC viewer (without the issue):
Image

The cause:
I looked into this problem and found the cause. The bugs exist in both server and viewer sides. vnc4server offers pixel format bgr101111 if depth is 32. In this case, number of bits for red/green/blue are 11/11/10 bits respectively. In other words, it is necessary to send whole 32bit (4bytes) for each pixel. However, both server and viewer send/receive only 3bytes per pixel due to bugs. The bug comes from overflow in bitshift of redMax, greenMax, blueMax. These values are declared as unsigned 16bit int and signed 32bit int in viewer and server side respectively. Shifting-up blueMax by 22bit causes overflow in both side.

The following patch fixes this issue in UltraVNC viewer side. I built modified version of vnc4server and connected them, then, it was confirmed that the issue is fixed.

The patch which fixes the issue:
https://tyan0.yr32.net/files/vnc4server-patch.zip

However, vnc4server is not maintained anymore, so the bug will not be fixed. In addition, vnc4server is only provided up to debian jessie, which is no longer supported. I never see the other VNC server which uses such pixel format (bgr101111). So it is not worth enough to fix this issue also in UltraVNC side.

This post is just for your information.

Thanks.

P.S.
As for TightVNC (and also RealVNC ) viewer, it does not accept bgr101111 and requests rgb888 connection, therefore the issue does not happen.
tyan0
8
8
Posts: 21
Joined: 2021-08-10 03:14

Re: No blue component if with vnc4server 32 bit color depth.

Post by tyan0 »

Here are the two additional patches for this issue. I confirmed that the connection to vnc4server having the bug can be established without the issue with these patches.
https://tyan0.yr32.net/files/add-comment-vnc4server.zip
https://tyan0.yr32.net/files/revise-vnc ... around.zip
If the server requests a connection with ZRLE encoding in pixel format bgr101111, it will fall back to rgb888. If the encoding is changed to ones other than ZRLE (e.g. Hextile), it will revert to bgr101111.

I noticed that the Ubuntu 18.04 LTS which is still in the support period (till Apr. 2023 and ESM till Apr. 2028) provides vnc4server having the bug. Therefore, I think it should be worth to adopt these patches.

Thanks.
User avatar
Rudi De Vos
Admin & Developer
Admin & Developer
Posts: 6832
Joined: 2004-04-23 10:21
Contact:

Re: No blue component if with vnc4server 32 bit color depth.

Post by Rudi De Vos »

All tyan0 pathes were added in 1.3.4.1
Post Reply