Jump to content
XPEnology Community

Recommended hardware for video editing NAS


zzarac

Recommended Posts

I already have one working xpenology setup that has been good to me for the past five years. I use it mostly for home media and recently as an ftp server. I am a freelance video editor so being able to receive large amounts of video footage without physical contact became quite important these days.

 

I am currently in the process of getting a new editing workstation, so naturally I started pondering my storage options. I would be thrilled if I could manage a superfast NAS using xpenology as a working storage for 4K editing.

What I had in mind is a system with 8x 2TB SATA SSD-s in RAID5 connected to my workstation using 2x 10GbE NICs, with one more 1GbE NIC to connect to my home network.

I would prefer a mini-itx setup which could fit inside a case like SilverStone CS280, but I could live with a larger setup.

 

What components would you recommend - CPU, motherboard, memory amount, cache drives, NICs and SATA/SAS controller?

Thank you very much.

Link to comment
Share on other sites

OK, I can see that I may have come here a bit too needy and it could look as if I wanted someone else to do my research instead.

This is not the case, I have been looking into this for a long time. The only thing I am really not sure about is CPU horsepower needed as I don't have any hands-on experience with this kind of setups.

 

I have found several Supermicro motherboards that seem to tick all the boxes (granted they have only up to 6 SATA ports, but I still have a full x16 PCIe for a controller) :

X10SDV-4C+-TLN4F (Xeon D-1518, 4x 2.20GHz, 6MB cache, 35W TDP)

X10SDV-4C-TLN2F (Xeon D-1520, 4x 2.20GHz, 6MB cache, 45W TDP)

X11SDV-4C-TLN2F (Xeon D-2123IT, 4x 2.20GHz, 8MB cache, 60W TDP)

X10SDV-6C+-TLN4F (Xeon D-1528, 6x 1.90GHz, 9MB cache, 35W TDP)

A2SDI-H-TF (Atom C3758, 8x 2.20GHz, 16MB cache, 25W TDP)

A2SDI-TP8F (Atom C3858, 12x 2.00GHz, 12MB cache, 25W TDP)

 

Would any of these CPUs be sufficient for the job at hand? If not, what should I be looking for?

 

EDIT:

I guess I didn't mention one really important bit. This NAS will be used by only one machine. This is for my home workstation, not a studio with multiple computers.

Edited by zzarac
one user info
Link to comment
Share on other sites

If you are not running heavy apps and are mostly seeking file sharing services (if I read your first post correctly), any of these platforms will do.  The Xeon D and Atom platforms have the advantage of on-board 10Gbps.

 

Facing a similar decision years ago, I selected the SuperMicro X11SSH platform for 8 SATA ports and added a dual-port Mellanox Connect X-3 PCIe card for 10Gbps networking.

 

I'm not sure about your reasoning for dual 10Gbps networking?  Do you have two workstations to connect with 10Gbps (that's my use case, dual ports to avoid a 10Gbps switch)?  But binding two ports to one workstation will not be very helpful performance-wise, and the drives you have specified won't consistently saturate a single 10Gbps link.

 

For your use case, I would (and have) selected enterprise SSD's instead of spindles; a 3-drive SATA SSD array maxes out the 10Gbps interface. Surplus market is basically US$90/TB now. Load up with a lot of RAM and forego any SSD cache, it's a waste for your workload.  If you use SSD's for your array, you will want RAID F1 which precludes NVMe cache in any case.

 

 

Link to comment
Share on other sites

Thank you flyride for your input.

 

Yes, you are right. It would serve exclusively as a file server for one connected machine only, heavy usage tho.

 

Concerning the dual 10 Gbe I think you misread my original post. I did state I would use SATA SSDs, not spindles. In this case (8 drives), it would mathematically amount to ~ 1700 MB/s which most certainly saturates a single link. Of course I am not expecting that kind of performance in real life, but it certainly warrants for link aggregation use.

 

All of the mentioned motherboards have dual 10 GbE NICs, dual 1 GbE NICs, and an IPMI port as well, so the only variable is a CPU.

Link to comment
Share on other sites

my thoughts

a nas setup will usually be slower as a local nvme solution, if keeping the actual projects local and use nas for offload/backup might be faster?

 

there is 40G too (qsfp+), maybe thats a option for a all flash setup? better the 2x10G and only slightly more expansive

 

10G RJ-45 has more latency, SFP+ nic and DAC cable is cheap but limit for cheap DAC is 7.5m

 

mini ITX limits the option and makes the base setup much more expansive, micro atx might be a option

i use Gigabyte B365M HD3, slightly more depth then mini itx, and wider because of more pcie slots

can be used with added sas/sata controller and 10G or 40G nic at the same time, wide range of cpu's

if its normal micro atx then there is a wider option of boards

 

i do remember that one user had such a all flash qsfp+ setup

https://xpenology.com/forum/topic/23394-make-configuration-from-fs3017/

Link to comment
Share on other sites

Thanks IG-88.

 

I feel that 40G would be an overkill, as 8 drives in a RAID5 at best can still theoretically achieve less than the maximum bandwidth of a dual 10G combined.

My workstation will have a 2TB local nvme drive (~5000 MB/s), but it will be used for application cache, and there is simply not enough PCIe lanes (and m.2 slots) to accommodate for my needs in terms of total capacity. I am going with Ryzen 3950x and nVidia 2080 Super (still consumer grade workstation - low PCIe lanes count).

Link to comment
Share on other sites

i was thinking if this would be possible

https://www.amazon.com/ASUS-M-2-X16-V2-Threadripper/dp/B07NQBQB6Z/

gpu in 8x slot and 16x for nvme

maybe ASUS Pro WS X570-Ace (90MB11M0-M0EAY0) 16x/8x/8x, so 2x2 nvme in 8x slot each would also possible

i've never used this so you may use google to see if it can hold up to the expectations

 

EDIT. i'm seceptical that you will be able to use 2x10G for single file transfer and get 2000 MB/s

40G is guaranteed to work the way you expect it

Edited by IG-88
Link to comment
Share on other sites

Well, threadrippers are great and I was thinking of getting one, but it's realistically out of my budget. The cheapest 3-series one (24 cores) is around 1400 Euro, compared to 750 for the 3950x (16 cores). Not only the CPU, but I would need a much more expensive motherboard too. For that extra money I would gain only encoding times speeds, but the work experience would remain basically the same.

 

The budget for this NAS is derived from the budget that was going to go into an external RAID disk array connected by fibre. Since solutions like that are quite expensive, I decided to look outside of a box and make a NAS that could do what I need here (not as good as fibre option, of course,) but also theoretically a lot more.

 

I do not expect 2000 MB/s, far from it. Since the theoretical maximum speed for 8 SSDs (at 550 MB/s) in a RAID5 environment is around 1700MB/s, I would be more than happy with 1300-1500 MB/s peak transfer rates. The "heaviest" video footage I regularly use is RED 6K WS, which asks for around 180 MB/s. My experience tells me I need 6x the bitrate of one stream to work comfortably and smoothly with no drops, because I use more than one stream at the time - up to 4. That would be just slightly above 10GbE.

 

Unless there is some kind of overhead that I am not aware of, dual 10G should be almost perfect for my case.

 

Latency-wise, some of the motherboards I mentioned earlier use SFP+ interface (e.g. A2SDi-TP8F has both dual RJ-45 and dual SFP+).

Link to comment
Share on other sites

  • 2 years later...

It seems to me that you should contact a specialist who will help you deal with all the technical issues. I can only recommend a good video editing app for you. I have been a professional video editor for a long time. And recently, I came across the site. I really immediately really liked the app in use. So if you have a powerful computer, it will be very easy to learn how to edit in it. Moreover, if you have editing experience, you will quickly learn how to make cool videos there.

Edited by batistamea
Link to comment
Share on other sites

@batistameaYou do realize this is a 2+ years old thread right? And beyond that, it looks to me you are also doing some advertisement here which is clearly against the rules of the forum. Anymore of this type of posts and that will be the end of it.

 

Quote

6. Advertising or business offers of any kind will be immediately deleted and the account associated banned.

 

Link to comment
Share on other sites

  • 1 month later...
  • 2 weeks later...
В 09.10.2022 в 23:36, batistamea сказал:

It seems to me that you should contact a specialist who will help you deal with all the technical issues. I can only recommend a good video editing app for you. I have been a professional video editor for a long time.  I really immediately really liked the app in use. So if you have a powerful computer, it will be very easy to learn how to edit in it. Moreover, if you have editing experience, you will quickly learn how to make cool videos there.


Editing is not my primary profession, but I still have to do it occasionally, and I’m tired of my old PC. That’s why I want to build a hybrid that would be great for gaming and for video editing at the same time.
I don’t need it to run complicated software because I mostly use online tools like https://www.fastreel.com/add-music-to-videо.html. But I really want it to work smoothly because I’m tired of freezes.

Link to comment
Share on other sites

  • 2 weeks later...
On 12/6/2022 at 4:08 PM, andreiaequal said:


Editing is not my primary profession, but I still have to do it occasionally, and I’m tired of my old PC. That’s why I want to build a hybrid that would be great for gaming and for video editing at the same time.
I don’t need it to run complicated software because I mostly use online tools like https://www.fastreel.com/add-music-to-videо.html. But I really want it to work smoothly because I’m tired of freezes.

 

it depends on how much data you have, how much you would need to push it around or having other people access it (workflow)

if its just for editing solo a 16x pcie card with 4 x nvme ssd as raid0 might be the cheapest solution (the "simple" card like that come without any hardware and need the chipset/bios to support bifurcation (https://blog.donbowman.ca/2017/10/06/pci-e-bifurcation-explained/, there are hardware based cards too but i guess they will cost $250-450 compared to less then $100)

backup or hosting can be done by copy to a 10G connected NAS, even normal disks in a raid set can give 500-700MB/s and when using nvme/ssd cache as write cache its even less impotent what the main raid for data storage is made off

if you want a cost effictive solution its usually kind of tailor cut to your situation - it gets different when money is of no importance and the only thing a pushing the envelop

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...