Once I have finished a videoproject, I would like to archive this on a big storage disk on my server. So what would be the gain if I would create a 10Gbps network at home? It would certainly go faster with copying!
There are not so many options to choose from and although it seems cheap, you might run into incompatibility problems when it comes to fiber.
A MicroTik CRS305-1G-4S+IN 10Gbps switch seems cheap, but you still need SFP+ modules and some fiber cable, or make use of DAC cables which is basically the copper-version with fixed SFP+ copper modules.
If you really take the effort of finding the exact stuff which runs on this MicroTik you might end up investing €300 which is cheaper than the €400 I spent.
I was lucky to find a second hand Netgear ProSafe XS505M for a decent price (€200) which makes things a bit easier. It is a 4 port copper based 10Gbps switch with an optional SPF+ port.
I bought two ASUS XG-C100C 10Gbps network cards for €99 each and made sure all my cables where at least CAT6. Two cables are bought, one was made by myself.
Cables are important!
The one I made myself seemed to be a bit buggy because the link speed wasn’t going higher than 2,5Gbps, so I made a new cable with CAT6E and that did the trick, 10Gbps uplink.
If this already gives a problem, then forget about CAT5E short cables some other folks mention…
Plug and play actually…
Yes, when it is copper, it is easy. Plug and play, but if you are a network guru, you might end up creating vlans to make a separate network to access storage. As the Netgear is an unmanaged switch, you need to make vlans on a second managed switch, do stuff with untagged vlan and… For my case it is getting a bit to complicated and as I already know the end result, I am not going to do the trouble…
MTU
In order to get the best results, you should enable jumbo frames on the network cards. As the Netgear allows a package size up to 9000, I was a bit confused when I had only the option to select 9014 on the ASUS card. But I was told it has something to do with a header, so 9014 it is.
BUT! Altering your MTU on your 10Gbps network makes you run into trouble as the rest of your network probably is using the default package size of 1500.
If you really want to make use of the jumbo frames (and you don’t after my conclusion unless…), you should make this a second network and start configuring a different IP network space. I did, I set both network cards with fixed IP’s 10.20.30.10/24 and 10.20.30.11/24 and now I was able to set jumbo frames without running into conflicts on the normal network and maintaining my internet connection.
What’s the gain?
Well.. Actually. I did some tests copying large files.
Setup #1 A seperate network using fixed IP’s, jumbo frames 9014, MTU 9000
With jumbo frames enabled, I was able to copy at a max speed of 370MB per second. That is, from SSD to SSD. When copying to a normal HDD it dropped down to 140MB per second.
Setup #2 Integrated into normal network, using default MTU 1500
Without the jumbo frames (disabled) I was able to copy at a max speed of 290MB per second. That is, from SSD to SSD.
So setup #1 seems faster but is it worth it? I think not. When accessing \\10.20.30.11\d$ and dropping files in the explorer, Windows first starts to nagg you with a dialogscreen concerning safety of files you’re about to copy.
My conclusion
If you have a storage server with SSD’s in RAID 0 or RAID 10, then you might be better off using setup #1, accepting the naggs of Windows and using fixed IP’s.
Eventually I choose for the second setup, not making use of jumbo frames, no need to connect two network cards on my PC and server, all on the same network.
Copying things to my 2TB SSD on the server is nice at 290MB per second. But copying to my 8TB HDD, the 140MB per second is not much of a gain compared to the 110MB per second it did on my 1Gbps network in the past.
So maybe you can even do it cheaper when buying network cards and a switch that utilize 2,5 or 5Gbps. The 10Gbps at home is actually not a very big gain unless you digg in deep into vlans, SSD in RAID and more like that.
Oh, by the way… My SFP module from a Cisco switch seemed to work on the Netgear once it was already turned on. Yeee! Port 5 active! But when you recycle the power on that switch, it will never boot. Here is an example of compatibility problems 😉
Leave a Reply