Announcement

Collapse
No announcement yet.

Building a Mini Render Farm

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Building a Mini Render Farm

    Hey everyone...

    so so here is a quick question that I would like some first hand advice on...

    I am looking to build a mini render farm in my editing suite.
    My biggest question is: which has the greatest impact on render time and rendering "power", CPU or GPU?

    Would I be better off with a half dozen rack mount machines with powerful processors and many cores, but have limited GPU, or, would I be better off with two or three machines with powerful processing but high end graphics cards (6-12gb)? Which makes the greatest difference, processing power and core count or GPU?

    in today's marketplace, it's really easy and affordable to buy multiple rack mount servers with a ton of ram and a lot of processing power/cores. But building machines with high end GPUs is still costly.

    where am I better off -- more machines, which means more cores and more ram, or less machines with high end GPU cards?

    software wise, I use premiere and after effects the most, and occasionally cinema4d and 3dsMax.

    thanks everyone!!!

    #2
    I've briefly looked into this before, and it seemed to me that creating render farms for Premiere at least was simply not possible, if you've found a way i'd gladly hear though!

    Comment


      #3
      More so...Im concerned with After Effects and Cinema4D. As that is were the primary bulk of my intense rendering is done.

      As an added question...does anyone know of any third-party encoding apps that allow multi-core/multiple machine renders? I know Apple Compressor does, but, even though I am a hardcore MAC user, one of my animation machines is PC based, and I was also planning to build a Windows Server based Render Network.

      Thanks!

      Comment


        #4
        I don't think AE can network render either, and not familiar with Cinema4D.

        I do know that Blender can build a render system, and I think they even have it down to rending faster with multiple video cards. So in theory you could build a render system with several computers, each with several video cards. But I haven't really kept up on it so I might be a little off.

        Comment


          #5
          AE does offer network rendering as of recent. It just requires an install of their network rendering engine on each machine. C4D is essentially the same process as well. I'm hoping Adobe Media Encoder will offer that feature soon as well.

          My primary question/concern is.... What do you think makes the greatest Impact on render time/power...CPU or GPU? I'm mainly wondering where I should focus the power on my builds... More processors, power, cores, and threads....or using less machines, but each one maxed out with video cards?

          I know that GPU makes the bigger difference when rendering video in playback and such, but does GPU still make all the difference when rendering across a network?

          thanks!!

          Comment


            #6
            You can use a networked render folder for AE and have it add placeholders and skip existing frames. We did this all the time back on CS5. The trick is that you must open the project on each computer, make sure they can all see the render folder, and manually hit render. Not ideal, but it works and is the poor man's approach. Any plugins must be installed on each machine too.
            sigpic

            Independent Filmmaker
            BMD URSA Mini 4K/Avid Media Composer/NukeX/Blender/Mixcraft/ProTools/Resolve Studio

            Feature Films
            Wulf - 2008 | Leap - 2010 | Leap: Rise of the Beast - 2011 | Surviving The Wild - 2020

            Comment


              #7
              CPU vs. GPU will be up to the render engine, not familiar with your use case so I can't offer suggestions there. IF it has GPU rendering, then adding multiple GPU should be a lot faster than CPU because they are designed to do the type of task you are requesting.

              Comment


                #8
                Thanks Greg!!

                So, as a Mac guy who's looking to build a PC based render network, do you (or anyone) know if you can mix multiple different graphics/video cards in one machine? Such as different brands, VRAM, and other specs?

                I can can occassionally find good deals on various video cards, and would love to be able to keep adding to my "farm" as I go.

                Comment


                  #9
                  I would probably try to keep the same model in each computer, keeps the drivers happy. One thing I read was that connecting a display to one of these cards can make it significantly less powerful for renders. So buy main boards that have on onboard video for a display. Would be nice if you could build it on Linux to keep the price down.

                  On a Mac there used to be a distributed render engine that installed with FCP, but I think they moved it out to the OS, think it was called Xgrid. From what I remember reading, it didn't care what application was using it. If it still exists, this might be your best way forward. I think it was all CPU based so you would want a pile of Mini's.

                  Comment


                    #10
                    Greg,
                    interesting. Thanks for the advice!
                    i do know that FCP provides network rendering through compressor. Obviously, I would love to build a network that is Mac based, but I encountered the following issues:
                    1. Price being most. To really get the power I would want, I would have to use all Mac Pros...especially if I want to add increased GPU power with video card expansion....unless it's possible to buy a pcie expansion chassis. But still, price for power on Mac is pretty high. As a Mac user for years, I'm too familiar with that. I can build two killer PC machines for less than one Mac.
                    2. Some software is Windows only (mainly maya and 3dsmax). I currently use a virtual machine and stability is terrible.
                    3. Since I want to continuously add on, upgrade, and add more power, I think a Windows machine would make that much easier.

                    I do know that some software, specifically 3D applications, do render through GPU if I'm not mistaken? With Mac mini, upgrades are limited.

                    Final question... I obviously plan on building a couple machines with powerful GPUs, but, I also have access to a few greatly priced PC servers. These servers are incredibly powerful CPU wise, but very limited with graphics, and I'm not sure yet if they are capable of adding additional graphics cards. Would it be worth it to add these into the mix?

                    sorry for all the questions...not my primary area of expertise. Thanks!!!

                    Comment


                      #11
                      I would be very surprised if Maya and 3DS didn't have a render solution built in, that's a normal workflow for that type of tool. Might pay off to hit up a specific forum for those applications and check out the available solutions.

                      Most servers will have at least one 16x PCIe slot, but it might only be half height which limits the cards you can buy.mBut it might get you Xeon processors which is the correct choice for number crunching.

                      Comment


                        #12
                        How many nodes are you talking about getting? How many cpus? You may want to look into render management software which automates the process for most popular vfx software. I've used deadline mostly but there are a lot of other options out there. That said, for a small studio I'd think in some ways just investing in one or two very high powered workstations might be a simpler alternative. Render farms primary benefit is that they are scalable to hundreds and thousands of cpus but if you are considering <64 cpus maybe a couple 16 or 32 core workstations with nice gpu that can do double duty would be more cost effective and simpler to manage unless you have someone in house dedicated to managing all your farm nodes and your shared storage.

                        As for gpu support it depends so much on software support.
                        Noah Yuan-Vogel
                        http://www.noahyv.com

                        Comment


                          #13
                          First off...thank you guys both for the input and response!
                          As far as render solutions for Maya and 3DsMax, I have heard there are network solutions available, including third-party? I honestly haven't dug too deep into my research on that particular set of programs yet, as I have been working on Mac exclusively for the past decade+. That said, as I mentioned before, the virtual machine operation on Mac has always been problematic. I had actually written it off for years until I took a new animation certification course a year or so ago, and upgraded my software, and was forced to use it some more. Now, for some projects that are coming up this summer, I intend on doing a good amount of work within those programs. As I mentioned, I know C4D (which had been my program of choice for years) offers network rendering solutions.

                          So, as far as weighing the decision on the size, scale, and specifics of my network build, I have been weighing a lot of factors.
                          I apologize in advance for my ignorance on certain aspects of this. Having worked independently for most of the past 7 or 8 years, and being a jaded Mac user, I have kind of fallen out of touch with a lot of this subject.

                          This summer, I will be working on both an episodic television program, as well as a feature length film project. Because of budget, I will be splitting my duties between directing and visual effects. The last time I did this duty on a project, my workstation was tied down for 26 hours once while rendering a sequence out of C4D (on a 12 core Mac Pro with a rather good quality graphics card). Considering I will be working on two projects, back and forth throughout the summer (not to mention if I pick up any smaller projects here and there), I can't risk having my actual workstations tied down with rendering. My goal is to start building a render network that I can offload my renders to, while I move on to other tasks on my workstations.

                          My first thought was to build 2 or 3 (preferably rack mounted) machines that have both powerful processing (and high core count) and powerful GPU processing. Which was the purpose of my initial question, wanting to know where I should try and focus the most power in my builds. As neither of these projects have allotted an actual budget for doing this, I'm paying for this out of pocket....so I'm trying to build smart, and get the biggest bang for my buck.
                          During my research, I found several refurb single space rack servers for an incredible price. They each have 8 cores of Xeon processing, a fair amount of ram, but obviously very limited on board graphics. If they do have the ability to add one PCIE card (which I will find out today), it would at least give me the option to add some GPU power.
                          Do you think it would be practical/make sense to add these machines into the mix?
                          Regardless of if I add the aforementioned servers, my plan is to still build at least two or three powerful workstations. I'm just curious if it would be worth it to add these servers into the chain (simply to add more processors), or if it would end up being a speed bump in my rendering (because of their limited GPU capability).

                          As I said, as time and needs progress, I plan to continue to build and expand this network....I have a great amount of rack space available, which is why I'm planning on making these builds rack able.

                          Im sorry for the length of my posts. I'm kind of a detail freak lol
                          But I will look into more of the third-party network rendering solutions today.

                          Thanks again for everything!!

                          Comment


                            #14
                            Networking is going to be a second issue for you. Getting something that will perform fast enough is not going to be cheap, the network is a bottleneck for all renders until you scale up so far that it no longer matters. But you are going to need a very good switch to get any real benefit out of some of the common network rendering workflows. Look at Cisco 2960mk2 switches at about $2000+ depending on the number of ports. Extreme Networks has the C5 series that will be nice. HP and Dell both have something but I don't know specific products. All in all you are looking at many thousands of dollars for a decent switch.

                            I guess what you need to decide is if you want the fastest render so you can use the material immediately, or if you just need to free up the workstations to move on to another task. Two completely different workflows and requirements. Maya has a way to do network renders, but your workstation needs to be master. Didn't look up 3DS.

                            All in all, depending on needs, I might chose a Blender workflow because they offer the second way of rendering to free up the workstations. This is somewhat detailed in their workflow summaries. Not sure if you could do the artwork on the software you know, then pass it all off to a Blenderfarm that can be either CPU or CPU/GPU based.

                            There is an alternative... There are many cloud based services you can rent to render out your work. These might be cheaper than trying to roll your own and potentially faster too.

                            Comment


                              #15
                              Hey Greg,
                              Thanks again for the input!
                              You're right about the networking issue, as that is something I have been aware of.
                              Primarily, my primary concern is in offloading the workload to other machines so I can move on to other tasks. I've already developed a workflow that factors and compensates for my current slower than desired render times. But having the ability to "farm out" my renders to other machines is something that is important to me and my needs.
                              That said, I do have access to some high end network switches pulled out of another facility (along with some storage servers and some other gears) for an incredibly great price! But regardless, my biggest concern is just moving the renders off of my workstations. Obviously, anything I can do to speed up those renders would be great.

                              I have used a couple of those cloud based render services, which worked out alright for some one off work here and there, project to project. But, with what I have coming up this summer, it makes more sense to be able to keep things in house. I'm sure there will come a time (mainly with this series), where changes will have to be made last minute. It might get a little impractical/costly to have to keep subbing out revisions to a third party service.
                              That said, however, it sure would make sense to farm out big renders (which I know are final) to a third party service when time is a major factor.
                              But I still need to increase my rendering power in house.
                              Honestly, After Effects is going to be its primary function.
                              Of course my 3D work and animations will be important, however, there won't be nearly as many as with After Effects.
                              A lot of times, I will edit a complete sequence in my NLE, and then bring the entire sequence into AE for CC, Compositing, and final render.
                              Sometimes, that workflow is reversed (depending on the projects needs).
                              If my need to composite and such is minimal, I will just render my composited sequences, import into an NLE and do my CC through there (or a third party app) and then render the master through something like AMC.

                              Comment

                              Working...
                              X