How I Defeated the 3 Mbps Upload Demon

@bge

  • #opnsense
  • #proxmox
  • #networking
  • #xfinity
  • #troubleshooting



"They told me it was a simple bridge mode config. They lied."

๐Ÿง  The Setup

  • ISP: Xfinity (yes, the usual suspect)
  • Modem: Xfinity Gateway, placed into bridge mode
  • Router: OPNSense, virtualized inside Proxmox
  • NICs: One on a dedicated PCIe card (the eventual villain), another on the motherboard

๐Ÿ˜ก The Symptom

  • Download speed: blazing fast ๐Ÿš€
  • Upload speed: 3 Mbps ๐Ÿข
  • Using Xfinity modem in normal router mode? 300+ Mbps upload ๐Ÿ”ฅ
  • Bridge mode + OPNSense VM? Back to turtle town ๐Ÿข

๐Ÿ“ž Contacted Xfinity (Plot Twist)

In a moment of desperation, I reached out to Xfinity for help. Not through their website of course โ€” thatโ€™s all AI now, and unless you're trying to reboot your modem for the 12th time, it's basically useless.

Tip: The real humans hide on Facebook Messenger. Yes, seriously.

To their credit, once I got through, they:

  • Offered a modem upgrade (yay!)
  • Gave me a discount on my plan (double yay!)
  • Sent me a newer modem that looks nicer (triple yay!)

Unfortunately... the upload speed was still garbage ๐Ÿ—‘๏ธ. But hey, cheaper internet and a shinier box.

๐Ÿ” Step-by-Step Troubleshooting (with Interpretive Commentary)

1. Disable Offloading

Offloading is when your NIC says "Hey CPU, Iโ€™ll do the math." Except sometimes, it does it wrong, or like, really wrong.

Run this in OPNSense shell:

1ifconfig vtnet0 -txcsum -rxcsum -tso -lro
2

Then go to Interfaces > Settings in the GUI and uncheck all hardware offload options.

โœ… Result: Nothing changed. But hey, good hygiene.


2. MTU to 1500 (The Goldilocks Zone)

"Jumbo frames? I barely know her."

Set MTU to 1500 everywhere โ€” on Proxmox bridges, OPNsense WAN interface, and virtual NIC settings.

โœ… Result: Still broken. But now with properly-sized packets!


3. Disable the Fancy Stuff in OPNsense

"Traffic shaping? More like traffic choking."

Turned off:

  • Intrusion Detection/Prevention (Suricata)
  • Traffic Shaper
  • NetFlow / Insight
  • Gateway Monitoring

โœ… Result: Still 3 Mbps. At this point, spirits began to break.


4. Looked for Multiqueue Support

Because more queues = more lanes on the highway ๐Ÿ›ฃ๏ธ

In Proxmox, added to VM config:

net0: virtio=XX:XX:XX:XX:XX:XX,bridge=vmbr0,queues=4

And in theory, OPNsense would just go, "Ah yes, more queues, I shall use them." (Spoiler: it didnโ€™t help.)

โœ… Result: Still slow. More queues, same sadness.


5. Attempted to Restore a Known-Good Backup ๐Ÿง 

"Remember that magical time when the internet just worked?"
โ€” Me, nostalgic and desperate

I had a backup of the OPNSense config from a few months ago, when I vaguely remembered everything being fine. So I restored it.

โœ… Result: Upload still slow. Apparently even my memories are unreliable.

But hey, at least I ruled out config drift. The problem was never software. It was that cursed NIC.


6. Swapped NICs: The Surprise Ending ๐ŸŽ‰

"When in doubt, blame the hardware."

I changed the WAN interface from the PCIe NIC โ€” which, I might add, I bought from eBay for $16 โ€” to the onboard motherboard NIC.

โœ… Result: Upload shot up to 300+ Mbps like it had been holding its breath the whole time.

Moral of the story? That PCIe NIC was garbage โ€” or had some driver issue or soul-crushing incompatibility. Either way, it had one job and failed spectacularly.


๐Ÿง  Lessons Learned (So You Donโ€™t Have to Suffer)

  1. Donโ€™t trust your PCIe NIC just because it looks shiny.
  2. Always test alternate interfaces, even if theyโ€™re built-in and sad-looking.
  3. Virtual NICs depend heavily on host NIC quality โ€” a weak link kills everything.
  4. Bridge mode + Proxmox + OPNSense = dark magic unless every layer is tuned right.
  5. Xfinity support = Facebook Messenger or bust.

๐Ÿง™โ€โ™‚๏ธ Final Words

"If your upload sucks, and your config looks fine... maybe itโ€™s not the config thatโ€™s broken."
โ€” Someone wise who definitely uses OPNSense inside Proxmox.

Use this doc to remember what you went through, and share it to prevent others from falling into the same dark abyss of upload throttling.

And if youโ€™re reading this in the future: yes, it really was the NIC.



Back
ยฉ 2025 bowen.ge All Rights Reserved.