Generate RC4 Key using OpenSSL ,encrypt/decrypt using MSRC4

[Phaneendra]

Hi,

I am having an server which needs to generate a unique RC4 key for each session , share the rc4 key with an viewer and both exchange the data.

The server is running on MAC and hence it is using OpenSSL API to generate the RC4 key . The viewer is using Microsoft Crypt API to decrypt.


Following is the code am using to generate 87 bit rc4 key in MSRC4 format..

unsigned char *buffer;

buffer = (unsigned char *)calloc(1024*4, sizeof(unsigned char));
//NSAssert((buffer != NULL), @"Cannot calloc memory for buffer.");

int j = RAND_bytes(buffer, 1024) ;
int k ;
strcat(_RC4Key, "128 bitL ");
for( k = 0; k < 76; k++)
{
_RC4Key[k+11] = buffer[k];
}

//Send the RC4 key to Windows viewer...


I believe the key is stored in reverse order for MSRC4.

Any inputs will be highly useful.

Thanks

[redge]

I believe the key is stored in reverse order for MSRC4.

yes
you can ask adzm, maker of securevnc so have a lot of knowledge about security or krunge knowledge security and alternate OS linux/unix/mac