SecureVNCPlugin.dsm - viewer reporting error

[swinster]

Hi,

I am recieving a VNCViewer error saying "Insufficient memory to allocate network buffer" and in the VNC Viewer status panel I see the message "Wrong password", even though the password is correct

Just been refreshing a legacy image based on Windows 2000 that runs UltraVNC server that had I had previously configured with MSRC4Plugin.dsm using a shared key. After updating to UVNC 1.0.9.1 I'd thought I'd give the new plug-in a try, however, I have already run into some issues.

Pretty much everything is a standard install on the server.

• I have installed UVNC as a service and set the DSM plug-in to SecureVNCPlugin.dsm via the Admin Properties from the system tray
  
  In the Config of SecureVNCPlugin.dsm, everything is as standard barring a pass-phrase
  
  I have then clicked the "Generate Client Authentication Key" button and saved them in the VNC Program folder

The viewer is running on Windows 7, with again nothing special. A simple install with the DSM plugin set = SecureVNCPlugin.dsm. Nothing is configurable here.


However, the first time I connect I see the password prompt. Oddly, if I try to enter a wrong password, UVNC simply shows the password prompt again. When I enter the correct password, I get the error message above.

In addition, the next two times I try an connect I see a message saying "You have specified and encryption plug-in, however, this connection is unencrypted! Do you want to continue?" If I click No, I refuse the connection, if I click Yes, I am Refused! I have to do this twice and then I get back to the password prompt again, but still no further.


Confused

I'm sure this is something simple I am doing wrong but I can't pick out anything from SecureVNC Plug-in website instructions.

Just to make everyone aware, both server and viewer are running UltraVNC 1.0.9.1 (installed previous version and installed full setup)

SecureVNC Plug-in is at 2.2.4.0

[redge]

did you try the SecureVNC 2.3 before request support with 1.0.9.1 ?
as my knowledge, the 1.0.9.1 require this build about special feature unsupported on previous release or required customized version of vncviewer and winvnc that support all feature of SecureVNC

http://www.adamwalling.com/SecureVNC/

Please use UltraVNC 1.0.9.x Beta or later (eventually) to take advantage of advanced functionality in this plugin. Alternatively you may also use the special builds based off of UltraVNC 1.0.8.2 below. Older versions will use the legacy interface instead; configuration options will be disabled.

[swinster]

did you try the SecureVNC 2.3 before request support with 1.0.9.1 ?
as my knowledge, the 1.0.9.1 require this build about special feature unsupported on previous release or required customized version of vncviewer and winvnc that support all feature of SecureVNC

In this case no. Version 2.2.4 is the one that's included with UVNC 1.0.9.1 download - I didn't realise that this is not the correct version for this build. I guess someone should repack the install files so that the correct builds are distributed.

I will try this tomorrow.

[Rudi De Vos]

It should work with the 1.x 2.x plugins.

If the included plugin doesn't work it need to be something else.
All test where done with the included plugin.

It looks more like there is a problem with the ultravnc.ini file or write permission to that file... verify manual if passwd an encryption is realy in the file.

The error tell that you selected to use encryption on the viewer site, but the signal from the server is unencrypted.

Check the about box of the server if it's realy version 1.0.9.1.

Are you administrator ( local admin) ?

[swinster]

It should work with the 1.x 2.x plugins.

Great.

If the included plugin doesn't work it need to be something else.
All test where done with the included plugin.

It looks more like there is a problem with the ultravnc.ini file or write permission to that file... verify manual if passwd an encryption is realy in the file.

To be sure, I renamed the old ultravnc.ini file and re-ran the UltraVNC Edit Settings program to recreate. After a bit of faffing, I got the main settings to the way I wanted and tested UVNC without any DSM plug-in. This all worked fine.

When you say "verify manual if passwd an encryption is realy in the file", what file to you mean me to check? Is the password stored in an encrypted form?

The error tell that you selected to use encryption on the viewer site, but the signal from the server is unencrypted.

The error which I was getting after a failed connection attempt seems to now have disappeared. However, I still cannot connect with the correct password.

I still get the password prompt and if I enter the correct password, I see the message "Wrong Password" in the VNC Viewer status panel, and the "insufficient memory to allocate network buffer" in the VNCViewer Message Box. as before. If I actually enter the wrong password, I am simply re-presented with the password dialog box.

I know that the password I am being prompted for is coming from the SecureVNC plug-in as I have set the standard VNC and SecureVNC plug-in password differently, with the SecureVNC plug-in password set to 9 characters.

Check the about box of the server if it's really version 1.0.9.1.
Are you administrator ( local admin) ?

Yes, both server and viewer are running 1.0.9.1. The About Box on the server system tray icon says 1.0.9.1 and the dialog title in the VNC Viewer shows the same.

The server is installed as a service (on the Win 2000 PC), and the user that I log on to the box to configure UVNC properties is a member of the administrators group.

On the viewer machine (Win 7), the user I use to run the viewer is part of the local administrators group. In addition, I have tried running the viewer as an Administrator, but this didn't make any difference

[swinster]

The error which I was getting after a failed connection attempt seems to now have disappeared. However, I still cannot connect with the correct password.

I tell a lie. This error is back too. It did appear for the first couple of logon failures.

[swinster]

Just tried version 2.3.0.0 of the SecureVNC plug-in, and re-generated the keys on both client and server but with no luck. I still get the same error messages.

????

[swinster]

Tried connecting from an Win XP box running the viewer and had the same result, so I guess the problem lies on the server.

[redge]

just for check there no mistake, you really use the winvnc and vncviewer of folder w2k of file
UltraVNC_1.0.9.1.bins.zip\w2k

[swinster]

just for check there no mistake, you really use the winvnc and vncviewer of folder w2k of file
UltraVNC_1.0.9.1.bins.zip\w2k

OK, shall do tomorrow. I simply downloaded installed the main FULL package on the Win 2000 machine

[Rudi De Vos]

He is, other bins crash with " function does not exist in kernel error"

[swinster]

He is, other bins crash with " function does not exist in kernel error"

So does this mean I DON'T need to download and try the bins mentioned above? If not, what are the other possibilities?

[swinster]

Just to test, I downloaded the Bins zip of 1.0.9.1 to the W2k machine.

Firstly, I stopped the VNC service and removed, then renamed the "winvnc.exe" and "vncviewer.exe" files.

I then extracted the same exe files from the W2k folder of the bins zip previously downloaded into the UltraVNC program folder. I then Installed and started the service.

Tried to connect to the VNC server and still received the same message saying "Wong password" when typing the correct password and "insufficient memory to allocate network buffer".


Any further ideas?

[Rudi De Vos]

Tested it on w2k in virtualpc.
Ultravnc wasn't installed before.

If you test, you should move the ultravnc.ini first, else old settings stay in it.

I just selected the plugin without configuring it.

Works on my system

[swinster]

Weyhay - fixed, although still some questions...

Uninstalled, yet there were a load of additional files in the program folder so deleted and re-installed.

On reboot, I managed to connect, but although the connection was using the SecureVNC plug-in, there was no encryption!!! (maybe I needed to restart the service to get this functioning properly, which I don't think I did).

So I thought I would need to generate keys on the server by clicking the "Generate Client Authentication Key" button in the SecureVNC plug-in config. However, as soon as I did this (and after restarting the service), I got the same error as before.

I then deleted the generated key files from the program folder and restarted. Now, finally, I can connect and have encryption!

Why is it that generating key files causes this issue? I thought that for PKI to work both sides would need a public and private key, with the public keys being exchanged during handshake.

In addition, when you are using a password configured with the SecureVNC plug-in itself, you get the password box being continually re-presented if you get the password wrong. It might be a good idea to have this configurable so that it only displays say three times before kicking you out of the connection. If you use just the standard VNC password, you get kicked out after you enter the wrong password just the once.

Thanks for the help.

[Rudi De Vos]

If you generate a key on the server, you also need to copy this key to the viewer. Server and viewer need to have the same key or no key...

[swinster]

If you generate a key on the server, you also need to copy this key to the viewer. Server and viewer need to have the same key or no key...

Interesting. I always though that the the keys for PKI encryption where independent for both ends.

Client connects to Server.

Client and server exchange public keys

Client uses server public key to encrypt traffic to server. Server uses client public key to encrypt traffic to client.

Client uses its private key to decrypt traffic sent from server. Sever uses its private key to decrypt traffic sent from client.

In this way, I don't see how or why the keys need to be pre-shared. I thought that was the whole point of PKI in that you don't need to pre-share keys?

Just curious really here.

In any case, I think the error message could be a little more enlightening.

[Rudi De Vos]

The reason is that we need encryption AND authentication.
The rfb protocol has only 8 chars as password size.

Protecting access over a 1024 AES encryption ( auto key exchange) with a 8 char passwd to get access isn't the best practice.

If you don't exchange the key, the key serve as access token. Your vnc access is protected by a 2024bit key and not a 8 char passwd.


The new plugin have many config options.
http://adamwalling.com/SecureVNC/

One of the options is to use key exchange and set a longer password that's used to grand access. ( then you can use a 32chars password)

[swinster]

Thanks, I think, although I'm not sure you answered all my questions.

Are you saying the the only way to get full encryption is to pre-share pre-generated keys?



In addition, why does the password prompt keep coming back when enter am incorrect password when you use the password override in the SecureVNC plug-in?

[Rudi De Vos]

When i'lm correct it work like this.
Adam made the plugins and can answer it better


1) default : key exchange and vnc passwd is used ( 8chars)
2) overwrite passwd: key exchange and vnc passwd isn't used but you use
the password set via the encryption plugin ( 32 chars)
3) You use a key on both sites, then no password is needed.

[swinster]

Ahh, now I get it. Many thnaks

[tonibony]

Yesterday I had exactly the same problem with UltraVNC 1.0.9.5. Luckily, today I found this topic and the solution worked. Anyway, it just didn't feel right. So, I did more tests and carefully read again Adam's "Advanced Setup and Technical Explanation" about SecureVNC Plugin ( http://www.adamwalling.com/SecureVNC/ ). Here is what I think now:

* SecureVNC always requires password to start a session. It can be the short classic VNC password (8 chars) or the long passphrase from the plugin configuration.

* If there is a public client key on the server "*_Server_ClientAuth.pubkey", it will be used, no matter what kind of password is used. The viewer should have the corresponding private key "*_Viewer_ClientAuth.pkey".

* The server should have ONLY ONE public client key "*_Server_ClientAuth.pubkey"! If there are two or more, the server picks ONE of them and there's no way to control WHICH ONE!

* The viewer can have many private client keys "*_Viewer_ClientAuth.pkey". It picks the right private key for each public key on the server by matching the first part of their filenames.

* The inadequate errors "Insufficient memory to allocate network buffer" and "Wrong password" appear if the viewer doesn't have the right private client key for the server. Moreover, after these errors the server disconnects from the repeater and doesn't reconnect again. This should be a bug (I can't tell where the bug is located, though).

* My mistake was the following: I put two different public client keys on the server and only one of the corresponding private keys on the viewer. The server obviously picked the "wrong" key and the connection failed in this strange way.

[ezilg]

Hello, I followed all the steps as described.

1. - I deleted all keys on both sides and was able to connect with the password plus 'use DSMPlugin' being marked.
I suppose this connection is NOT encrypted, is it ?
Even though unmarking 'use DSMPlugin' at the viewer-side didn't give me a connection anymore ?
This puzzles me.

2. - On the server side I decided on a 10-character passphrase
Then I created the keys at the server-side : _Server_ClientAuth.pubkey and _Viewer_ClientAuth.pkey
I copied _Viewer_ClientAuth.pkey on to a USB-stick and copied into the Client (Viewer) folder at the client system.
Trying my usual password as before, I kept on getting the VNC Authentification (which I suppose is correct)
Typing the Passphrase instead of the password, resulted in : Insufficient memory to allocate buffer

Question :
- Where did I go wrong ?
Note :
- winvnc : 1.0.9.6
- VNCViewer : 1.0.9.6
- SecureVNCPlugin.dsm : 2.3.0.0

[ezilg]

I now took out all the keys again on both sides and left my own Passhrase configured on the Server side.
And now I was able to connect only via Passphrase, which I suppose is now correct to get an encrypted connection.
All OK now for me.
Which leaves the question, why it doesn't work with the generated keys.

[YY]

1. - I deleted all keys on both sides and was able to connect with the password plus 'use DSMPlugin' being marked.
I suppose this connection is NOT encrypted, is it ?
Even though unmarking 'use DSMPlugin' at the viewer-side didn't give me a connection anymore ?

Then the server is configured to use SecureVNC.

Having the keys or not is optional. If not exist, the SecureVNCplugin will generate a random key automatically for each session.

I now took out all the keys again on both sides and left my own Passhrase configured on the Server side.
And now I was able to connect only via Passphrase, which I suppose is now correct to get an encrypted connection.

That is right, and the connection is encrypted with a random key.

Which leaves the question, why it doesn't work with the generated keys.

The error you reported implies the keys are not matched pair.

I'm not sure where things went wrong. Since now you have a workable SecureVNC setup, if you are interested, you may try to generate the key, and test it again.

Here is the step:
1. At the server, delete all old keys (if exist.)

2. Goto the SecureVNCplugin GUI, further check it is NOT using pre-shared key.


3. Now click the "Generate Client Authentication Key" button to generate the keys.

4. After generate and save the key, click "Close" to quit the SecureVNCplugin GUI.

5. Goto the SecureVNCplugin GUI again, now you should see pre-shared key is used.


6. Now you know how to check the operating status of SecureVNCplugin. Save your setting, copy the xxxxx_Viewer_ClientAuth.pkey to the viewer, and try if you can make a proper connection with these keys.

Good luck.