VNC Server fails to return keyboard/mouse input after VNC Client disconnects

[JonathanS]

Problem Statement: we have encountered a problem when disabling/enabling input (keyboard/mouse) when an operator takes control of a user’s system. Today we have incorporated this into our existing application and when the operator connects (takes control) we successfully disable input however when we disconnect the input (keyboard/mouse) is not returned to the user!

We added some custom code in our version (v1.2.1) so that when a connection is made by an {operator} via vnc client we disable server input and upon disconnect we re-enable it – here is the sequence:

Code: Select all

- Operator takes control via our application with VNC viewer (on the operator PC) send message rfbSetServerInput with a status value of 1  (i.e. disable server input)
- On the server side (user station), the following code sequence is invoke
      o	vncclient.cpp:  handling of rfbSetServerInput.  This calls vncDesktop::SetBlockInputState with a value of true
      o	vncdesktop.cpp: SetBlockInputState
              	if BlankMonitorEnabled is set, blank the monitor by calling vncDesktop::SetBlankMonitor with a value of true
              	Also if BlankMonitorEnabled is set (yes same setting control BlockInput), set m_bIsInputDisabledByClient to true and calls vncDesktop::block_input()
                      •	NOTE that in a later revision of UltraVNC official server code (dev 1.3.5.2 - ~ 2 months ago), the call to block_input was commented out from vncDesktop::SetBlockInputState (in our source code, the call the block_input is executed); was a change/fix done in UltraVNC for this?
                      •	NOTE that the call to vncDesktop::block_input() in SetBlockInputState DOES NOT matter (executed or commented out) as  vncDesktop::block_input() is periodically been called in vncClientThread::run (vncclient.cpp).  As long as bIsInputDisabledByClient is set to true (in SetBlockInputState()), block_input will eventually been called to disable the keyboard and mouse inputs

- When the Operator exit our application the reverse is done.  
- VncViewer sends the message rfbSetServerInput with a status value of 0  (i.e. enable server input)
- On the server:
      o	vncDesktop::SetBlockInputState is called with a value of false
      o	vncdesktop.cpp: SetBlockInputState
              	vncDesktop::SetBlankMonitor is called with a value of false
              	bIsInputDisabledByClient   is set to false
                      •	vncDesktop::block_input() is called directly from SetBlockInputState   (in our code), and from vncClientThread::run to enable the keyboard and mouse
                      •	THIS IS WHERE the problem happen, it seems that block_input DOES NOT always enable the physical keyboard and mouse.  Note that the call SetBlankMinotor was successful as the server screen is no longer blank

[Chris_L]

Issue resolved. The problem was cause by the server (in 1.2.1) not disabling the mouse hook when SetBlockInputState is called with a value of false