Update on speeds:
I upped the maximum connection number on IDM to 32 i.e. every file is split into 32 parts and downloaded.
Connected via LAN. That pushed the max downloading speed to about 28 MBps.
Started downloading another file, and that maxed the rest of the bandwidth i.e about 27-28 MBps.
Combined, I was hitting 55-58 MBps (440-460 Mbps) of the 500 Mbps connection.
I tried a 3rd file, but the numbers didn't move up. So looks like the loss margin is about 10% i.e. 500 mbps will give about 450 mbps, which is not the best since it's FTTH and LAN.