An interesting test on Repeater Linux version

[star521]

Hi, guys

We all know Linux is faster than Windows, so I presume the Repeater Linux version is faster than the windows one.
What's more interesting is that I noticed the both of them have a connection limitation settings:
For windows, its max connection list is 20, while Linux can be [1...1000].

I'm wandering why there is such a limitation, so I conducted several strength tests.
Before the test, I have changed Ubuntu ulimit -n from 1024 to 20K and also the repeater max session value to 15K in the source code.


I wrote a demo program(A), acting as repeating, which average sending 45kbps to the Repeater, which is running on Ubuntu(Desktop 10.04).
There is also a simple program(B), acting as normal connections, in other words, just do the TCP hand-sharks.
Both A and B are based on socket connections.

I running 20 A instance and keep creating new B instance at the same time.
As the time going, Repeater would stop when it receiving 3000 (range from 3000 to 3200 ) connections from B.
The totle network bandwidth is 0.8Mbps.

Another group test details: average sending 16kbps, the totle network bandwidth 0.3Mbps. Repeater still stop at 3127.

The CPU usage is around 14% to 50% and I still have extra memory.

Is there exist any bottle nack, somewhere in the Ubuntu OS, PC's hardware or the Repeater application?

thanks

[B]

What a great test! It may be that no one's ever really given the repeater that kind of workout before. Thank you for sharing your results.

As to your question, I have a hunch, and a little quick math shows it's probably more than a coincidence. I suspected you reached the limit based on the total number of possible TCP/IP connections.

Well, just take your 3127 and divide it into the maximum of 65,535 (the default maximum TCP receive window size), and you get about 20.9577, which is to say 1 over your "20 A instances".

This would lead to <b>guess</b> that each successful "B" connection is using up about (3126/20 ~ 156) of <i>something</i>; whether that's ports or sockets or handles or stack memory I have no idea.

Anyway, you could try increasing the TCP window scaling and see if that makes any difference.

http://en.wikipedia.org/wiki/TCP_window_scale_option

Thanks again for sharing your results!

[star521]

Thanks B,

Well, I have do some tricks, but still not resolve the problem.
On this URL http://www.jlinux.org/server.html.
The author talked about "File descriptor" and "Process/Thread" limits.
Follow their suggestions, I add/change a lot of settings on my Ubuntu.
This is the screenshot of some settings:

/etc/security/limits.conf

* soft noproc 15000
* hard noproc 15000
* soft nofile 15000
* hard nofile 15000

/etc/sysctl.conf

# Turn off tcp_window_scaling
net.ipv4.tcp_window_scaling = 0
## increase TCP max buffer size setable using setsockopt()
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
## increase Linux autotuning TCP buffer limits
## min, default, and max number of bytes to use
## set max to at least 4MB, or higher if you use very high BDP paths
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
## don't cache ssthresh from previous connection
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_moderate_rcvbuf = 1
## recommended to increase this for 1000 BT or higher
net.core.netdev_max_backlog = 2500
## for 10 GigE, use this, uncomment below
## net.core.netdev_max_backlog = 30000
## Turn off timestamps if you're on a gigabit or very busy network
## Having it off is one less thing the IP stack needs to work on
## net.ipv4.tcp_timestamps = 0
## disable tcp selective acknowledgements.
net.ipv4.tcp_sack = 0

However, they doesn't help. Any idea?
I'm thinking about using the "epoll" instead of the traditional "select" mechanism .

Thank you for your time!

[B]

But you turned Window Scaling <b>off</b>. My understanding was that you need Window Scaling <b>on</b> in order to properly support more than 65535 ports.

Even then, applications (like VNC or the repeater) might not like it.

[star521]

Hello, B

Well, with the window scaling turning on, I got the former testing results.

thanks

[B]

It was on, but whether or not it was actually scaling is open to question. Modern versions of the kernel are supposed to "automatically" tune the window.

One interesting round up is at http://www.psc.edu/networking/projects/tcptune/

Note 1: I know little about this.

Note 2: My assumption earlier (that your problem is related to the 65,535 limit) may be completely wrong.

Let's assume, for the sake of argument, that you've identified a hard limit that's endemic to either the repeater or to VNC (rather than OS tuning).

Would we then surmise that no single repeater on a single computer can every support more than (65535/156 ~ 420) simultaneous connections?

I would expect that virtual machines would be one way to work around that pretty easily. You could try running each of your "A" repeaters in a separate virtual machine. My guess is that you will get about 20x more total connections before locking up.

[addison]

Thank you, Thank you

Anyway, I have to say that the "epoll" is the right PATH.

Best wishes!