What’s the Deal with GPU Expansion, Anyway?
What is a GPU expander, why might you want one, and what are the best choices right now?
Eventually, you’ll find yourself wanting more power. Maybe it’s when you click “render” at the end of a project and the estimated time is in days instead of hours. Maybe you’re tired of your project taking 40 minutes to load. Maybe it’s because VR is coming down the pike and you hear Palmer Luckey of Oculus saying Apple users will be left out since our graphics cards aren’t powerful enough. Or maybe you just want to upgrade after all the great GPU releases this year.
Whatever the cause, you want more.
The most popular option for filmmakers, once we have maxed out our memory, is the GPU expander box.
Traditionally, Macs have been harder to upgrade and PCs have been easier. While there has been some movement away from Mac lately, the Mac OS is still pretty dominant in the world of film and media. The older Mac Pro Silver Towers offered some expandability, but the new Mac Pro Tower offers very limited options if you need more horsepower. You can increase RAM, and that's about it. You could switch to PC (many are), or, of course, consider a Hackintosh, but if you want to stay in the world of fully supported hardware and software, and you’ve committed to Mac, you’ve still got some good some options to speed up your workflow.
The most popular option for filmmakers, once we have maxed out our memory, is the GPU expander box. There are two big reasons why. First, what we do is very image-intensive work, which GPUs are well designed to handle, offering a lot of bang for your buck as an upgrade. Resolve, Adobe Creative Cloud, and many other software platforms take advantage of OpenCL to use the graphics card power for faster processing. Even Resolve, which used to rely heavily on the NVIDIA-specific CUDA platform, has worked to integrate OpenCL to such an extent that it's about equal to CUDA in processing speed, meaning you can save money with power from AMD (who acquired the Radeon brand from ATI back in 2006).
One of the nice perks of OpenCL is that more GPUs generally translated directly to faster renders; this means all the power is put to use. With CPUs, more cores can mean more power, but the program needs to be written to take advantage of it, and not all are. OpenCL optimized programs really put all the GPU power you can throw at them to work.
The other big reason that GPUs make a great upgrade is that the video game market has made for very cost-effective power. The filmmaker market is relatively small when compared to the film industry as a whole, which is why filmmaker-specific equipment tends to be more expensive. Since there are not as many of us out here buying, costs like R&D can’t be spread over a large market, and other benefits of mass production don’t come into play. You’ll see this in the cost of most of the Thunderbolt boxes themselves; the market is small, and the price is higher than one might hope.
But there are a lot of people who play video games. Video games require almost precisely the same kind of processing power that we need for video rendering (and also for AI), so there is a huge push for better and cheaper graphics cards to serve the gaming industry. Those factors end up benefiting filmmakers tremendously.
Back in the Silver Tower Mac Pro days, you could buy a PCI CUBIX expander box to stack up a ton of graphics cards all in one machine. And back in 2012, if you took a CUBIX with three GTX cards, a Blackmagic intensity, and a RedRocket, you could do real-time .R3d raws in Resolve.
With the Black Mac Pro (often abbreviated nMP for “new Mac Pro” or called the trashcan or R2-D2), it’s now a closed box with no PCI slots, so you need to use Thunderbolt to add more GPUs. If you want to add the hot new GTX 1080, or the GTX 1060, or the AMD competitor RX480, you’ll need a Thunderbolt expansion box. One benefit of Thunderbolt is that it also works with Thunderbolt-enabled Macs, including the MacBook Pro, or even the MacBook Air, making it easy to built a set-friendly system with the power to handle bigger formats.
For sound-sensitive applications, Thunderbolt offers lengths up to 10 meters (33 feet) over optical cable so you can place the enclosure, with its fans, far away from your workstation.
The two biggest players in the field are Sonnet and Magma, but there are a few other entries in the field worth considering.
With the EchoExpress line, Sonnet offers a few features that make it stand out from the pack. Designed with the RedRocket in mind, it has built-in BNC mounting holes so that the SDI board doesn’t take up another slot in the chassis. It’s also claimed to be upgradeable to Thunderbolt 3, which will offer 40gb/s speeds when it becomes widely available, hopefully in the near future.
Sonnet also offers a rack mount solutions for three single-width cards if you are running a rack-mount set-up, and the xMac Pro Server, which is a 4U Rackmount enclosure; you would mount a Mac Pro inside of this.
Magma has the ExpressBox 1T with a single bay and the ExpressBox 3T and ExpressBox-3T-DB with three bays. Both 3T models offer 3 PCIe slots: two PCIe 2.0 x8 (one available for x16), one PCIe 2.0 x4, 250W total watts of power, and the ability to daisy chain. The main difference with the DB (aside from about $200 more in cost) is 4 2.5” SAS/SATA HDD slots. If you’re looking for a great solution that comes with a lot of room for more cards, and also an external RAID array, this could be a great way to do it.
To solve the RedRocket conundrum (the Rocket takes up a whole slot for just SDI ports), Magma sells a kit to add the SDI ports to the same bracket that holds the Thunderbolt ports for $20.
Netstor offers identical products to the Magma line, at slightly cheaper prices, but reports have been excellent on Magma customer service during the transition from TB1 to TB2. With Thunderbolt 3 looming, I would lean towards paying the slight premium to buy directly from Magma.
mLogic, an early leader with the RedRocket targeting mLink R, has a $399 1/2 length box that might fit a specific need as well. While I'm always one to save on price when possible, Thunderbolt chassis is one area where support really matters, and where I might consider paying slightly more for a company more specifically invested in ensuring the GPUs and other accessories work in film workflows.
Anybody tried any of these, or finally abandoned your Mac for a PC and its easy world of expandability? Tell us about it in the comments.