fluxcoil wiki

I had a look at MPTCP, collecting here pointers to investigate deeper.

• Mastodon thread
• Hacker news thread
• MPTCP on a phone instructions, uses ShadowSocks to have normal apps open MPTCP sockets
• Tutorial to setup 2 Ubuntu systems with MPTCP, and get TCP apps via mptcpize or a socksproxy over the MPTCP link, much like the article above.
• A paper from 2013, suggesting systems to convert MPTCP ↔ TCP, much like what shadowsocks transparent proxy does now, from what I see. via Olivier Bonaventure
• Besides QUIC (protocol ontop if UDP, includes TLS1.3), there is also TCPLS which includes some MPTCP features. The question is whether rather TCPLS or MPTCP find wide adoption.. or systems converting TCP into MPTCP will spread.
• Seems like quite some providers already deployed hybrid networks.
• A patched OpenSSH can natively open MPTCP connections. I wonder if using OpenSSH's vpn function ontop allows to transparently route normal TCP over that tunnel, effectively using MPTCP.. I basically tried that with OpenVPN, but failed. Maybe because with my method, _all_ connections opened by OpenVPN would become MPTCP.

Depending on the goal, also other protocols besides MPTCP can be considered:

• SMB3 support multiple streams, of could quite pinned down to just file sharing
• QUIC: It's a next level protocol, and the spec includes that it can survive IP changes. So a QUIC client can establish a connection, and then continue that connection sending QUIC from a different src-IP, i.e. when switching to wireless.
  • con: as I see it, combining throughput of multiple media is not possible, just switching
  • con: the involved client/server applications need to implement QUIC

Considering here the several ways for that scenario: I have a laptop from which I access the internet, and I want to freely move that laptop between ethernet and wifi, without applications noticing. So this solution considers a further system (laptop, raspi etc.) on your LAN, as helper. The ways below are so far mostly theoretical and should be tried out one by one (please beat me in trying them before I find the time ;).

The 'real world' scenario from the article linked above has already a part of this setup, but a) it is just terminating the network connections on the other laptop, in real world one wants to connect to plain TCP servers on the internet. Having the MPTCP endpoint doing NAT could help. b) we still need to make all applications on the mobile laptop MPTCP capable, i.e. with 'mptcpize'. Maybe there is a more elegant way than that.

So ideally, below options get researched, and easy to use howto gets created. From the below approaches, the first one is recommended as it enables MPTCP usage in the most transparent way for applications. The bonding approach is also nice and needs no helper system on the internet.

Collection of some approaches. 'pro' means pro-argument, 'con' is a bad/con argument.

1. Use MPTCP, and shadowsocks
   • shadowsocks proxy, has socks5 and transparent proxy mode. Idea from here.
   • pro: Running shadowsocks as transparent proxy enables applications opening normal TCP connections, and shadowproxy then sending the traffic with MPTCP over the subflows.
   • con: One needs 2 helper systems which span up the MPTCP connection.
   • con: shadowsocks is not packaged in many Linux distros, consider to use https://www.openmptcprouter.com/ as distro.
2. Using classic bonding
   • I tried this approach out. You likely won't be able to use bond modes which combine bandwidth, and you are probably pinned down to Linux on both systems.
   • pro: easy to setup, as plain TCP/UDP can run over the bond
   • con: you need control over link layer of both systems, for all of the underlying media (i.e. wireless, ethernet)
   • con: works just with devices speaking the bonding dialect, i.e. 2 Linux devices.
3. Span VPN with Tailscale
   • I tried this approach out. Tailscale is for this purpose a really bad choice, because it's designed to have at all times a working upstream route to the Tailscale servers on the internet.
   • pro: tailscale does failover to other underlying media (wifi) after 5sec
   • con: Tailscale can not combine multiple media for throughput
   • con: a TCP socket over the Tailscale connection stays open/up when the underlying media gets unavailable, but for 4sec no traffic is done, then the remaining link gets used. If original link comes back, it gets used again.
4. Span VPN with Nebula
   • I tried this approach out. Nebula is for this purpose a really bad choice, because it's designed to have at all times a working upstream route to the lighthouse system on the internet, for our scenario that is not always the case. To solve that, maybe you can run the companion system on your LAN as lighthouse.. but then still all the other downsides stay.
   • pro: nebula does failover to other underlying media (wifi) after 15sec
   • con: a TCP socket over the Nebula connection stays open/up when the underlying media gets unavailable, but for 15sec no traffic is done, then the remaining link gets used. If original link comes back, Nebula restart might be required for Nebula to notice that the link is back.
   • con: Nebula is not combining the bandwidth of multiple available underlying media
5. Use MPTCP, and mptcpize an OpenVPN, which by itself is spanning up a VPN where your applications route to
   • I tried this approach out, did not get it to work: OpenVPN can be mptcp'ized (do speaks then MPTCP), but when I run traffic over that VPN, I see only one of the subflows below getting utilized. Also no plain failover happening when the utilized underlying subflow goes down.
   • pro: the applications themself would then just do normal routing into the tunnel
   • con: I did not get multiple subflows utilized, with TCP traffic over the OpenVPN tunnel. Only when I MPTCP'ized the single connections, the subflows got used. But of course, these connections were then terminated on the companion system, they can not be NATed to the internet.
6. Use MPTCP, and use systemtap
   • This is quite similar to wrapping 'mptcpize' around your applications, but uses systemtap to modify TCP requests to mptcp. Details. Not recommend for practicability and debugging, and RHEL9 docs also do not mention this.
   • pro/con: all created TCP sockets will then become MPTCP sockets. Transparent for the ones you want to route to the internet, but also 'ssh localhost' etc. become MPTCP then. It's asking for trouble.. i.e. Xorg using local TCP channels etc.
7. Use MLVPN
   • Thanks to Saulius Krasuckas for the hint
   • 'just' available for Linux & BSD, but seems easy to setup

So with this scenario, we have a home with multiple uplinks to the internet, let's say fiber and 5G. MPTCP or something else would then help to combine throughput of both, and to survive one of the uplinks breaking down. For this scenario, we will need one system on the LAN (might be the router) running i.e. MPTCP, and a counterpart on the internet.