Slow wired upload speed vs Linux on same hardware

Dawid Oosthuizen 26 Reputation points
2020-09-09T05:18:31.053+00:00

Intel® Ethernet Controller X550-AT2, 10G network interface on ASRock Rack ROMED8-2T with AMD EPYC 7232P processor.
Windows 10 Pro for Workstations, 2004.
Latest Windows updates and Intel drivers installed as of 09/09/2020.

This machine is a dual boot, the above Windows version, and Ubuntu 20.04.
When doing a speed test, I get good performance from Ubuntu, but very poor uploads from Windows. This is on the same machine, the exact same hardware.

The WAN link is 1000Mbps down, 50Mbps up.

This is the Windows speedtest result:
23320-windows-10-speedtest.png

This is the Ubuntu speedtest result:
23325-ubuntu-2004-speedtest.png

I have tried to tweak the adapter's advanced driver settings in Windows, such as disabling LSO, etc. No luck, performance remains poor.

I've also noticed it on another PC running Windows 10 Pro, and a laptop running Windows 10 Pro for Workstations, both give the same poor upload performance. Whereas my other Ubuntu 20.04 Server machine, and also my phone connected via Wi-Fi, is getting good upload speeds.

I have even taken the Windows laptop and plugged it straight into my incoming WAN connection (bypassing router), and it still gets poor upload speeds.

Incidentally, when the speed test is running, I can see that the upload looks bursty on Windows, like it is only getting chunks of data here and there, while in Linux and on Android it looks the same as the download, the graph is drawn at a consistent high rate and with consistent high values.

Windows for business | Windows Client for IT Pros | Networking | Network connectivity and file sharing

50 answers

Sort by: Oldest
Most helpful Newest Oldest
  1. Gary Nebbett 6,536 Reputation points
    2021-01-04T14:11:21.527+00:00

    Hello @mensa84 ,

    I would guess that the "re-ordering" takes place as the traffic is traversing the 5G part of the connection. Further guesses are that there is nothing that can be done short-term to improve the situation and that the Microsoft developers responsible for the TCP/IP stack are aware of the problem (but it make take a long time before any improvement is made available).

    Here is a detailed explanation of what is happening:

    22:18:14.936343 192.168.1.10.50647 > 84.116.34.253.8080: . 107727:109147(1420) ack 917 win 1022 (DF)
    22:18:14.936344 192.168.1.10.50647 > 84.116.34.253.8080: . 109147:110567(1420) ack 917 win 1022 (DF)
    22:18:14.936344 192.168.1.10.50647 > 84.116.34.253.8080: . 110567:111987(1420) ack 917 win 1022 (DF)
    22:18:14.936344 192.168.1.10.50647 > 84.116.34.253.8080: . 111987:113407(1420) ack 917 win 1022 (DF)
    22:18:14.936344 192.168.1.10.50647 > 84.116.34.253.8080: . 113407:114827(1420) ack 917 win 1022 (DF)
    22:18:14.936345 192.168.1.10.50647 > 84.116.34.253.8080: . 114827:116247(1420) ack 917 win 1022 (DF)
    22:18:14.936345 192.168.1.10.50647 > 84.116.34.253.8080: . 116247:117667(1420) ack 917 win 1022 (DF)
    22:18:14.936345 192.168.1.10.50647 > 84.116.34.253.8080: P 117667:118453(786) ack 917 win 1022 (DF)
    22:18:14.936377 192.168.1.10.50647 > 84.116.34.253.8080: . 118453:119873(1420) ack 917 win 1022 (DF)
    22:18:14.936377 192.168.1.10.50647 > 84.116.34.253.8080: . 119873:121293(1420) ack 917 win 1022 (DF)
    22:18:14.936377 192.168.1.10.50647 > 84.116.34.253.8080: . 121293:122713(1420) ack 917 win 1022 (DF)
    22:18:14.936378 192.168.1.10.50647 > 84.116.34.253.8080: . 122713:124133(1420) ack 917 win 1022 (DF)
    22:18:14.936378 192.168.1.10.50647 > 84.116.34.253.8080: . 124133:125553(1420) ack 917 win 1022 (DF)
    22:18:14.936378 192.168.1.10.50647 > 84.116.34.253.8080: . 125553:126973(1420) ack 917 win 1022 (DF)
    22:18:14.936378 192.168.1.10.50647 > 84.116.34.253.8080: . 126973:128393(1420) ack 917 win 1022 (DF)
    22:18:14.936379 192.168.1.10.50647 > 84.116.34.253.8080: . 128393:129813(1420) ack 917 win 1022 (DF)
    22:18:14.936379 192.168.1.10.50647 > 84.116.34.253.8080: . 129813:131233(1420) ack 917 win 1022 (DF)
    22:18:14.936379 192.168.1.10.50647 > 84.116.34.253.8080: . 131233:132653(1420) ack 917 win 1022 (DF)
    22:18:14.936379 192.168.1.10.50647 > 84.116.34.253.8080: P 132653:133055(402) ack 917 win 1022 (DF)
    22:18:14.936443 192.168.1.10.50647 > 84.116.34.253.8080: . 133055:134475(1420) ack 917 win 1022 (DF)
    22:18:14.936444 192.168.1.10.50647 > 84.116.34.253.8080: . 134475:135895(1420) ack 917 win 1022 (DF)
    22:18:14.936444 192.168.1.10.50647 > 84.116.34.253.8080: . 135895:137315(1420) ack 917 win 1022 (DF)
    22:18:14.936444 192.168.1.10.50647 > 84.116.34.253.8080: . 137315:138735(1420) ack 917 win 1022 (DF)
    22:18:14.936444 192.168.1.10.50647 > 84.116.34.253.8080: . 138735:140155(1420) ack 917 win 1022 (DF)
    22:18:14.936445 192.168.1.10.50647 > 84.116.34.253.8080: . 140155:141575(1420) ack 917 win 1022 (DF)
    22:18:14.936445 192.168.1.10.50647 > 84.116.34.253.8080: . 141575:142995(1420) ack 917 win 1022 (DF)
    22:18:14.936445 192.168.1.10.50647 > 84.116.34.253.8080: . 142995:144415(1420) ack 917 win 1022 (DF)
    22:18:14.936445 192.168.1.10.50647 > 84.116.34.253.8080: . 144415:145835(1420) ack 917 win 1022 (DF)
    22:18:14.936445 192.168.1.10.50647 > 84.116.34.253.8080: . 145835:147255(1420) ack 917 win 1022 (DF)
    22:18:14.936446 192.168.1.10.50647 > 84.116.34.253.8080: . 147255:148675(1420) ack 917 win 1022 (DF)
    22:18:14.936446 192.168.1.10.50647 > 84.116.34.253.8080: P 148675:149461(786) ack 917 win 1022 (DF)
    22:18:14.936474 192.168.1.10.50647 > 84.116.34.253.8080: . 149461:150881(1420) ack 917 win 1022 (DF)
    22:18:14.936475 192.168.1.10.50647 > 84.116.34.253.8080: . 150881:152301(1420) ack 917 win 1022 (DF)
    22:18:14.936475 192.168.1.10.50647 > 84.116.34.253.8080: . 152301:153721(1420) ack 917 win 1022 (DF)
    22:18:14.936475 192.168.1.10.50647 > 84.116.34.253.8080: . 153721:155141(1420) ack 917 win 1022 (DF)
    22:18:14.936475 192.168.1.10.50647 > 84.116.34.253.8080: . 155141:156561(1420) ack 917 win 1022 (DF)
    22:18:14.936475 192.168.1.10.50647 > 84.116.34.253.8080: . 156561:157981(1420) ack 917 win 1022 (DF)
    22:18:14.936476 192.168.1.10.50647 > 84.116.34.253.8080: . 157981:159401(1420) ack 917 win 1022 (DF)
    22:18:14.936476 192.168.1.10.50647 > 84.116.34.253.8080: . 159401:160821(1420) ack 917 win 1022 (DF)
    22:18:14.936476 192.168.1.10.50647 > 84.116.34.253.8080: . 160821:162241(1420) ack 917 win 1022 (DF)
    22:18:14.936476 192.168.1.10.50647 > 84.116.34.253.8080: . 162241:163661(1420) ack 917 win 1022 (DF)
    22:18:14.936477 192.168.1.10.50647 > 84.116.34.253.8080: . 163661:165081(1420) ack 917 win 1022 (DF)
    22:18:14.936477 192.168.1.10.50647 > 84.116.34.253.8080: P 165081:165867(786) ack 917 win 1022 (DF)
    22:18:14.936503 192.168.1.10.50647 > 84.116.34.253.8080: . 165867:167287(1420) ack 917 win 1022 (DF)
    22:18:14.936503 192.168.1.10.50647 > 84.116.34.253.8080: . 167287:168707(1420) ack 917 win 1022 (DF)

    Now some acknowledgements are made available to the TCP stack, all grouped together (as a result of the "Interrupt Moderation" that we discussed earlier):

    22:18:14.962886 84.116.34.253.8080 > 192.168.1.10.50647: . ack 102047 win 525 <nop,nop,sack 117667:118453> (DF)
    22:18:14.967281 84.116.34.253.8080 > 192.168.1.10.50647: . ack 102047 win 525 <nop,nop,sack 104887:106307 117667:118453> (DF)
    22:18:14.967282 84.116.34.253.8080 > 192.168.1.10.50647: . ack 106307 win 548 <nop,nop,sack 117667:118453> (DF)
    22:18:14.967282 84.116.34.253.8080 > 192.168.1.10.50647: . ack 107727 win 570 <nop,nop,sack 117667:118453> (DF)
    22:18:14.967282 84.116.34.253.8080 > 192.168.1.10.50647: . ack 107727 win 570 <nop,nop,sack 109147:110567 117667:118453> (DF)
    22:18:14.967282 84.116.34.253.8080 > 192.168.1.10.50647: . ack 107727 win 570 <nop,nop,sack 111987:113407 109147:110567 117667:118453> (DF)
    22:18:14.967282 84.116.34.253.8080 > 192.168.1.10.50647: . ack 107727 win 570 <nop,nop,sack 109147:113407 117667:118453> (DF)
    22:18:14.967283 84.116.34.253.8080 > 192.168.1.10.50647: . ack 113407 win 592 <nop,nop,sack 117667:118453> (DF)
    22:18:14.967283 84.116.34.253.8080 > 192.168.1.10.50647: . ack 114827 win 614 <nop,nop,sack 117667:118453> (DF)
    22:18:14.967283 84.116.34.253.8080 > 192.168.1.10.50647: . ack 114827 win 614 <nop,nop,sack 116247:118453> (DF)

    The last "ack" acknowledging 107727 also selectively acknowledges 118453 - something that was sent 7 (seven!) segments later than 107727 but which arrived earlier. This causes the TCP stack to conclude that the segment has been lost and to retransmit it (shown below). Because a group of acknowledgements were delivered at once, the trace above shows the segment at 107727 finally being acknowledged before the segment was retransmitted; this is because the stack is still "working through" the group and had not seen/processed the crucial acknowledgement at the time that the decision to retransmit was made.

    22:18:14.967327 192.168.1.10.50647 > 84.116.34.253.8080: . 168707:170127(1420) ack 917 win 1022 (DF)
    22:18:14.967336 192.168.1.10.50647 > 84.116.34.253.8080: . 107727:109147(1420) ack 917 win 1022 (DF)

    Later, the TCP stack is unambiguously informed that the retransmitted segment was received twice via the "D-SACK block" mechanism:

    22:18:14.997584 84.116.34.253.8080 > 192.168.1.10.50647: . ack 168707 win 786 <nop,nop,sack 107727:109147 171547:172967> (DF)

    The Windows TCP stack seems to be able to cope with segments being deliver out-of-order provided the out-of-order "distance" is below 3 or so segments, but 7 segments is well beyond its capabilities.

    The retransmission causes the "congestion window" to be reduced, and it is this that reduces the throughput. This pattern of retransmission occurs many times during the performance test.

    The trace of the Microsoft-Windows-TCPIP provider shows that the receipt of the "D-SACK block" has no impact - it could/should reopen (perhaps just partially) the "congestion window" but it does nothing.

    The trace also records a variable shown as "RackReoWind" which sounds as though it could be used to increase the size of the "re-ordering window", but it is just reported as zero throughout the trace.

    Gary

    Was this answer helpful?

    2 people found this answer helpful.
    0 comments
  2. mensa84 6 Reputation points
    2021-01-04T16:10:06.453+00:00

    The Microsoft developers are aware od that problem? Why do you know that?

    And one other question: Why do you think, that it suddenly stopped working (full speed upload) without installing any updates? How can obe explain that?

    Was this answer helpful?

    0 comments
  3. mensa84 6 Reputation points
    2021-01-04T16:10:20.957+00:00

    The Microsoft developers are aware of that problem? Why do you know that?

    And one other question: Why do you think, that it suddenly stopped working (full speed upload) without installing any updates? How can obe explain that?

    Was this answer helpful?

    0 comments
  4. Gary Nebbett 6,536 Reputation points
    2021-01-04T17:07:13.977+00:00

    Hello @mensa84 ,

    I wrote "I would guess" and "Further guesses are" - I don't "know" that any of my assertions are true.

    Previously, the registry entry "TcpMaxDupAcks" was used to influence the behaviour that I described; see Description of Windows TCP features (key text reproduced below). Windows now seems to be moving to the techniques described in The RACK-TLP loss detection algorithm for TCP, but it doesn't seem to be working perfectly yet - this might be part of the reason why an update resulted in poorer performance - but that is just a guess too.

    [Oops - I read you message incorrectly regarding updates] It may just be the case that new equipment in the network increased the amount of re-ordering that is taking place - pushing it beyond the manageable level. Pure conjecture, but perhaps Huawei equipment in the 5G network was swapped out for Nokia...

    By default, Windows resends a segment if it receives three ACKs for the same sequence number (one ACK and two duplicates) and that sequence number lags the current one. This is controllable with the TcpMaxDupAcks registry parameter.

    The TcpMaxDupAcks value in the following registry key can be edited to control the number of ACKs necessary to start a fast retransmits:

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters

    Gary

    Was this answer helpful?

    1 person found this answer helpful.
    0 comments
  5. Nass86 1 Reputation point
    2021-01-15T13:47:06.933+00:00

    Just wanted to chime in and thank you guys for trying to resolve this. I've been looking for a forum to discuss this specific issue since November.

    I started Live streaming in recent months and bought an old used Lenovo Thinkpad W540 for Live Streaming because it has an Nvidia chip inside that has NVENC capabilities (one of the best ways to stream and take the load off the CPU).

    The interesting part about my perspective is that I bought a 2012 model which was updated with Windows update up to a point then not used for a couple of years so missed some windows updates.

    I did 2 live streams absolutely fine (8mb upload connection, doing a 4000kb/s live stream) and after the Windows update was forced upon me, I'm experiencing the same problems you are.

    The problem was caused by Windows Update as before this it worked perfectly fine.

    I'm now stuttering between 700kb/s and 1300kb/s. It sometimes starts at 2200kb/s for the first split second and then takes a dive from there. Doesn't matter which server I connect to.

    It's not the internet connection because:

    1) Macbook Pro works at full speed
    2) iMac the same
    3) iPhone the same

    Interestingly, if I use the iphone as a personal hotspot for the Windows laptop on 4G LTE, it goes blazingly fast - it is only the Ethernet and Wifi connectivity that is hampered.

    I find it interesting to note that someone mentioned they were connected by Microwave as I am connected in a village to a 5ghz wireless network across 1 or 2 km.

    I was wondering if the users chatting about this might be connected somewhere down the line with a network like mine and that this is to be factored in. That said, my ping is always 90-110 instead of the standard broadband numbers like 20-60 I've often seen and it always was this way when the live stream was working.

    I'm amazed at the depth of technical knowledge in this forum and stumbled on this by googling as this has been baffling me and I've seen one or two other forums opening up discussions but without this level of technical knowledge.

    I'm just hopeful that a following Windows update will fix it as I cannot rewind to 2-3 years ago's Windows version and turn off updates (if I could, I would as I bought this device solely for streaming).

    Any new ideas would be appreciated and I can run tests if anyone wants me to do something to help.

    Nasser, Cyprus

    Was this answer helpful?

    0 comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.