UVNC encodings

[SkyBeam]

I have used UltraVNC viewer 1.2.3.0 and I have to say I like a lot the new main startup screen which seems to be inspired by the Microsoft Remote Desktop Client.

I also love the grouping of encoding variants by CPU and bandwith recommendations. Unfortunately the "Connection Options..." dialog displayed when using the "Gear" icon on an existing connection is not using the same panel design. More weirdly some options seem to be different. For example I can't find a "Video" option in the options dialog on a running VNC session.

Also I have noticed something which looked weird to me. Since ZYWRLE is advertised with small bandwith and high CPU usage this would be my favorite choice in most cases for remote support. But here I noticed a behavior I can't really explain.
When selecting "Full Colors" the encoding seems to use JPEG lossy compression for the whole screen.
When selecting "256 Colors" the encoding seems to use lossless for most parts of the screen (except video canvas?).

Background:
I set zlib compression to 9 and JPEG quality to 1. When selecting "Full Colors" in ZYWRLE mode text becomes very unreadable due to low JPEG quality. But using the same settings with 256 Colors it seems to use some lossless compression where text is displayed very clearly.

For me ZYWRLE with 256 colors provides best UI quality with acceptable video quality but ideally I would prefer an encoding which is using lossless encoding for everything except video canvas also in Full Color mode. Certainly it was unexpected for me that changing color depth changes from lossless to lossy with the same encoding (ZYWRLE) selected.

The only explanation I have for this is that the encoding might decide for each block individually if it's transferred lossy or lossless and in full color mode it always decides for lossy encoding, perhaps due to some treshold value set. Not sure if this is correct by design.

[Rudi De Vos]

Yep, mstsc was used as sample for the layout

The internal options are still using the old panel, no time to convert.
They panel should only contain the "options you use while connected"
-encoders
Any idea what other options you use ?

u2 is a mjpeg with dynamic frame size, giving 25 FPS on LAN
Video was a better name for it.

Jpeg takes RGB as input
To compress 256 colors we first would need to upsample to 24bit and then compress it.
256 color is ideal for RLE + compressors
256 + jpeg -> internal replaced by other encoder

8bit = 2 MB HD frame + RLE + Compression is smaller then
RGB = 9,2 MB with standard jpeg quality.

Classification was done for RGB, could be different for less colors.

[SkyBeam]

They panel should only contain the "options you use while connected"

Not sure if I understand correctly. Though the viewer always shows all encodings and options. At least when connected to an UVNC host.

Any idea what other options you use ?

• Encoding: ZYWRLE
• Colors: Full Colors / 256 Colors
• zlib compression: 9
• Jpeg Quality: 1
• Use CopyRect encoding
• Use Cache Encoding
• Preemtive Updates

u2 is a mjpeg with dynamic frame size, giving 25 FPS on LAN
Video was a better name for it.

So I understand in ultraVNC viewer 1.2.3.0 u2 encoding was simply renamed to "video" but only in initial connection screen, not in connection settings on existing connection where it's still titled "u2".

Jpeg takes RGB as input
To compress 256 colors we first would need to upsample to 24bit and then compress it.
256 color is ideal for RLE + compressors
256 + jpeg -> internal replaced by other encoder

8bit = 2 MB HD frame + RLE + Compression is smaller then
RGB = 9,2 MB with standard jpeg quality.

Well, I did some tests with the settings listed above. Both require roughly the same bandwith (8-12Mbps) when plaiyng a youtube video remotely.

• ZYWRLE, Full Colors, zlib compression: 9 (probably unused in JPEG/Full Colors mode), Jpeg Quality: 1
• ZYWRLE, 256 Colors, zlib compression 9, Jpeg Quality: 1 (probably unused as REL encoded)

The first mode (Full Colors) is better if photos or videos are shared remotely as it's lossy based encoding. Of course also due to higher color depth. But it provides poor quality on high-contrast areas like text and GUI items.
The second mode (256 colors) is better to share remote control and operate GUIs as it provides lossless transfer of text-renderings and GUI elements. But it's of course very limited when it comes to photo and video.

As of my understanding encodings can also work on screen areas like checkerboard encoding also just transfering the areas where changes are detected. Perhaps such an encoding does not exist yet but I was thinking about the possibility to devide the screen into areas and decide for each area individually whether full-color JPEG (perhaps something more modern like BPF/HEIF or WebP) or 8-bit RLE encodig. Perhaps also deciding between WebP lossy and WebP lossless for each area of the screen.
I know this sounds complex but depending on the screen area contents it should be possible to do a cedision what is the most efficient encoding. For example areas with mostly uni-colored areas or high contrasts but limited color range are likely to be GUI elements, text or clipart graphics which is best encoded lossless. While areas with high vibrant color ranges (essentially a large number of different colors) are best encoded in some lossy codec like HEIF.
Automatic detection of areas would allow crystal clear and ultra low bandwith transfer of GUI elements while still allowing high-quality transfer of video canvas and photo preview. Of course at the cost of higher processing effort on server side to detect the best encoding for each screen area. But sometimes CPU power on server side is simply not really an issue while bandwith and image quality is.


So what I am saying is that at the moment you need to chose between to essential transfer types

• Lossless encoding with low color bandwith: Best for GUI remote control and remote assisant; bad to assist photo/video editors
• Lossy encoding with high color bandwith: Best for assisting photo/video editing but bad for GUI elements (high compression causing artifacts)

The problem I see is that none of the encodings provides the "best-for-everything" solution simply as it depends on what is shown on the remote screen. Which might even change over time. You might be assisting someone in word where all the screen can be encoded in 8-bit images perfectly but then this person brings up photoshop and likes you to show some picture manipulation issues so you better switch to some full color encoding. Dynamic detection would in best case adjust dynamically for the best encoding type.

Well, these are just ideas and I guess I am not the first one thinking about this so there might be more difficulties implementing such an encoding than I think about right now...