Unofficial Benchmark: socat vs Brook for Port Forwarding Performance

socat vs brookport forwarding benchmarkdocker port forwardinghigh concurrency testingnetwork performance comparison

Published·Jan 14, 2021Modified·Jan 14, 2021

Common port forwarding programs include socat, Brook, nginx, rinetd, and iptables. Most have been introduced in previous blog posts; interested readers can refer to the following articles:

Online reviews of port forwarding software are rare. Recently, I packaged both socat and Brook into Docker images. Today, let's compare and test them to see which one is more efficient. (This is an unofficial test; results are for reference only.)

Test Environment

A WebServer (Nginx) has been installed on the target server 163.172.35.xxx. On the relay server 137.175.30.xxx, both socat and Brook are deployed to forward traffic to the target's 80 port. The details are as follows:

Target server: 163.172.35.xxx:80
Relay server: 137.175.30.xxx:32771 (socat)
Relay server: 137.175.30.xxx:32770 (brook)

Initial Environment (Round 1)

Both socat and Brook are deployed via Docker. After deployment, with no active connections, the memory usage is as follows:

socat uses less than 800Kb of memory.

Brook uses 1.8Mb of memory.

HTTP Pressure Test with hey (Round 2)

Next, we use the hey tool to perform HTTP concurrent pressure tests on the two different forwarding ports. The test command is:

./hey -n 10000 -c 1000 -m GET http://xxx/

The command above sends 10,000 requests to the relay server with a client concurrency of 1,000.

socat Test Results:

Maximum Memory Usage: 154Mb
Maximum CPU Usage: 32%
Total Time: 67s
Average Time: 1.5347s
Error Count: 238

Summary:
  Total:        67.5975 secs
  Slowest:      19.9194 secs
  Fastest:      0.3516 secs
  Average:      1.5347 secs
  Requests/sec: 147.9345
  

Response time histogram:
  0.352 [1]     |
  2.308 [8318]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  4.265 [738]   |■■■■
  6.222 [227]   |■
  8.179 [166]   |■
  10.136 [100]  |
  12.092 [48]   |
  14.049 [53]   |
  16.006 [31]   |
  17.963 [39]   |
  19.919 [41]   |


Latency distribution:
  10% in 0.3608 secs
  25% in 0.3721 secs
  50% in 0.7243 secs
  75% in 1.4874 secs
  90% in 3.3814 secs
  95% in 6.0845 secs
  99% in 14.1610 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0225 secs, 0.3516 secs, 19.9194 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0001 secs, 0.0000 secs, 0.0309 secs
  resp wait:    1.4584 secs, 0.3515 secs, 19.9191 secs
  resp read:    0.0002 secs, 0.0001 secs, 0.0270 secs

Status code distribution:
  [200] 9762 responses

Error distribution:
  [238] Get "http://137.175.30.xxx:32771/": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

brook Test Results:

Maximum Memory Usage: 25Mb
Maximum CPU Usage: 64%
Total Time: 44s
Average Time: 1.0304s
Error Count: 67

Summary:
  Total:        44.2130 secs
  Slowest:      19.4560 secs
  Fastest:      0.3522 secs
  Average:      1.0304 secs
  Requests/sec: 226.1777
  

Response time histogram:
  0.352 [1]     |
  2.263 [9106]  |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  4.173 [528]   |■■
  6.083 [158]   |■
  7.994 [69]    |
  9.904 [22]    |
  11.814 [17]   |
  13.725 [7]    |
  15.635 [11]   |
  17.546 [9]    |
  19.456 [5]    |


Latency distribution:
  10% in 0.3605 secs
  25% in 0.3687 secs
  50% in 0.5565 secs
  75% in 1.0805 secs
  90% in 2.0481 secs
  95% in 2.8718 secs
  99% in 7.2565 secs

Details (average, fastest, slowest):
  DNS+dialup:   0.0192 secs, 0.3522 secs, 19.4560 secs
  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:    0.0001 secs, 0.0000 secs, 0.0117 secs
  resp wait:    0.9539 secs, 0.3519 secs, 19.4558 secs
  resp read:    0.0002 secs, 0.0001 secs, 0.0316 secs

Status code distribution:
  [200] 9933 responses

Error distribution:
  [67]  Get "http://137.175.30.xxx:32770/": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

The author, xiaoz, repeated the pressure tests several times, and the data trends were consistent.

Multi-threaded Download Test with axel (Round 3)

Test Command:

time axel -n 32 "http://137.175.30.xxx:32771/test"

The test file is 200MB, and axel is used to open 32 threads for downloading.

socat Test Results:

Time: 48m44s
Maximum Memory Usage: ~5.8Mb

real    48m44.111s
user    0m1.934s
sys     0m10.616s

Brook Test Results:

Time: 36m3.646s
Maximum Memory Usage: ~2.9Mb

real    36m3.646s
user    0m1.730s
sys     0m8.930s

Data Summary

Software/Test Item	socat	brook
Initial Memory	800Kb	1.8Mb
hey Test (Time)	67s	44s
hey Test (Max CPU)	32%	64%
hey Test (Error Count)	238	67
hey Test (Max Memory)	154Mb	25Mb
axel 32-thread Download (Time)	48m44s	36m3s
axel 32-thread Download (Max Memory)	5.8Mb	2.9Mb

Initial memory usage: socat is lower than Brook.
Under high concurrency: socat memory usage is higher than Brook.
Under high concurrency: Brook CPU usage is higher than socat.
Multi-threaded download: Brook efficiency is better than socat.

Conclusion

The above results are based on HTTP forwarding pressure tests. Due to factors such as environment and network conditions, and without deep research into socat parameters, 100% data accuracy is not guaranteed. Interested readers are encouraged to test themselves. Based on the test data, xiaoz offers the following suggestions:

If you are a personal user with low machine configuration and low usage volume, socat is recommended.
If you require higher efficiency under high concurrency, Brook is recommended.

Future tests will compare other port forwarding software. Stay tuned.