View
232
Download
0
Category
Preview:
Citation preview
8/12/2019 Building Home Linux Render Cluster
1/22
This is the story of Helmer. A linux
cluster in a IKEA Helmer cabinet.
2010-02-20
Helmer 2 year!!!!!! and still going!!
Helmer was built to last just a month or two and complete a crazy episode in mylife, but have performed faar beyond the original plan. Uptime is crazy, it's full of
dust and dirt. Have probably sucked throu many-many cubic meters of air...
Helmer have a new pal, a i5 FreeNas server with 16TB of storage. I have plan for
new renderfarm and I have also already gotten the new hardware, BUT I have not
time to build it. Helmer is doing such a good job I have noo need to change
anything at the moment.
Last job just completed a few min ago for a TV-show "Vetenskapsmagasinet" for
Swedish TV. Talked about the first fish that got legs and walked up on land.
http://helmer.sfe.se/http://helmer.sfe.se/http://helmer.sfe.se/http://helmer.sfe.se/8/12/2019 Building Home Linux Render Cluster
2/22
A dusty Helmer with Bull the new file server. Notice the "on" buttons is the wires
on side. To start, just choirt the wires :)
Last job, just done a few min ago. "fish with legs" (model, rigg, animation, camera
track and render by me)
8/12/2019 Building Home Linux Render Cluster
3/22
My workplace.
8/12/2019 Building Home Linux Render Cluster
4/22
The fish. A poly-proxy mode by me.
------------------
2009-02-23
Helmer is one year old now. Been working fantastic good, and rendered more the a
half million frames. Going to rebuild the network from a normal switched to a
direct bus. Now network is througha a normal consumer gigabit swithc that handle
about 25 MB/s. Now I'll use a Gigabyte EX58-UD5 motherboard for the new Intel
i7 cpu as the backbone. I insert 3x2 gigabit network cards so all motherboards in
helmer get a separate gigabit cable direct to the board. The new drives are 4x1TB
enterprise WD in a raid-0 just for speed. Speed rocks. Safety is for wimps :) This
all will in theory increase network bandwith from 25MB/s to 750MB/s!!! Now I
can do video/large data processing and not just small hard crunshing stuff.
Here is some pic for the last job I did "Time Capsule"
http://dalaro.sfe.se
I post some more here in a few days when I get going on the rebuild.
Cheers
http://dalaro.sfe.se/http://dalaro.sfe.se/http://dalaro.sfe.se/8/12/2019 Building Home Linux Render Cluster
5/22
Janne
--------2008--------
Yahhh, The TV-show that I'm rendering with Helmer have started airing. Mondays
Swedish tv "Odens Rike" SVT1 10:40.
Vert scary, because I have 3 more shows to do!!!
Hehe, You can buy the DVD, alreadyhereKinda funny, because it's not done yet
:)
-------------
3D computer rendering are very CPU intensive and the best way so speed up slow
render problems, are usually to distribute them on to more computers. Renderfarms are usually very large, expensive and run using ALLOT of energy. I wanted
to build something that could be put in my home, not make too much noise and run
using very little energy... and be dirt cheep, big problem? :) no computer stuff cost
almost nothing these days, it just a matter of finding fun stuff to play with.
I wanted to use Intel Quad core 65 nm or better. I surfed the web and found 6 of
then at a good price. Then I looked after the cheapest motherboard that could run
these. The result of my investigation was the Gigabyte S-series GA-G33M-
DS2R/S2 card.
http://www.sli.se/prodinfo.asp?sid=0&db=0&a=UR147457http://www.sli.se/prodinfo.asp?sid=0&db=0&a=UR147457http://www.sli.se/prodinfo.asp?sid=0&db=0&a=UR147457http://www.sli.se/prodinfo.asp?sid=0&db=0&a=UR1474578/12/2019 Building Home Linux Render Cluster
6/22
..and my cat approve.
8/12/2019 Building Home Linux Render Cluster
7/22
Next, finding memory. The motherboard could hold up to 8GB of memory each,
and I wanted to maximize all I could. Here are the 12x4GB =48GBmemory
modules.
8/12/2019 Building Home Linux Render Cluster
8/22
8/12/2019 Building Home Linux Render Cluster
9/22
Mounting CPU coolers.
Next problem was to find a good computer case that could hold these
motherboards. But this was a big problem. 6x cases cost almost as much as themotherboards and CPU's! So here was some room for improvement. I found the
IKEA helmer cabinet. Perfect! Who knew IKEA made there stuff ATX
compatible. :)
8/12/2019 Building Home Linux Render Cluster
10/22
Some extra support for backside of box.
8/12/2019 Building Home Linux Render Cluster
11/22
Mounting power supplies and fans in zig zag pattern to maximize airflow.
8/12/2019 Building Home Linux Render Cluster
12/22
8/12/2019 Building Home Linux Render Cluster
13/22
Motherboard H1, H2 and H3 mounted. Fans on wrong side. I mounted 3 mm
plexiglas under the motherboard so it would not short circuit when I installed
them.
8/12/2019 Building Home Linux Render Cluster
14/22
Big mess on floor.
8/12/2019 Building Home Linux Render Cluster
15/22
8/12/2019 Building Home Linux Render Cluster
16/22
Disk cloning in progress. I used Fedora 8, and the command "dd" to clone the
discs from the bootableSystem rescue CD disc.
> dd if=/dev/sda of=/dev/sdb
http://www.sysresccd.org/http://www.sysresccd.org/http://www.sysresccd.org/http://www.sysresccd.org/8/12/2019 Building Home Linux Render Cluster
17/22
Done! I use the softwareDr Queueas batch job dispatcher. All connected to a 8
port 3 com gigabit switch. Common storage using nfs to aFreeNASserver running
on a Via C7 machine (that just need 20 W power .. another story :) Instead of
power switch I used a simple cable to the "pwr" jumper on motherboard. (Wires
tejped on right side)
Instructions of how I did the DrQueue setup is here:
DrQueueHOW-TO.pdf
http://www.drqueue.org/http://www.drqueue.org/http://www.drqueue.org/http://www.freenas.org/http://www.freenas.org/http://www.freenas.org/http://helmer.sfe.se/DrQueueHOW-TO.pdfhttp://helmer.sfe.se/DrQueueHOW-TO.pdfhttp://helmer.sfe.se/DrQueueHOW-TO.pdfhttp://www.freenas.org/http://www.drqueue.org/8/12/2019 Building Home Linux Render Cluster
18/22
The most amazing is that this machine just cost as a better standard PC, but has 24
cores that run each at 2.4 Ghz, a total of 48GB ram, and just need 400W of
power!! This means that it hardly gets warm, and make less noise then my desktop
pc.
Render jobs that took all night, now gets done in 10-12 min.
..next one will be in red..
/Jannewww.sfe.se
Update 1:
It has been crazy last days. I have got allot of questions regarding details,
construction and software setup. I'll do a better page as soon... or I build the Red
Helmer II :) Would be fun to use the AMD 16 core Bulldozer cpu x 6 = 96 core in
a box :)))
If a hardware manufacturer/pr-guy happened to read this. I'm very happy to receive
hardware play around with ;)
Update 2:
It has happened! I filled the ram, and all boards begun to swap. During this the
power consumption was up to 800W. Running now a while, I noticed that H1
board (bottom one) gets hotter than the other boards. I think this is because the
http://www.sfe.se/http://www.sfe.se/http://www.sfe.se/8/12/2019 Building Home Linux Render Cluster
19/22
other boards are cooled from the fan below also.
Update 3:
Well, genral planning and webpage have begun on Red Helmer II - thank you
Google :)
Update 4:
First alternative hardware for Red Helmer is selected, and will do about 12Tflops.
(64x times faster then Helmer 1) and cost about the same :) Stay tuned for more!
Update 5:
A hardware god may help me, and then Helmer II will do about 50 TFLOPS!!!!
please please, may the force be with me :)
Some design test of Helmer II here
Helmer Data Sheet
More Helmer pics here
Get the Helmer desktop image!
http://helmer2.sfe.se/http://helmer2.sfe.se/http://helmer.sfe.se/HelmerDataSheet.pdfhttp://helmer.sfe.se/HelmerDataSheet.pdfhttp://helmore.sfe.se/http://helmore.sfe.se/http://helmer.sfe.se/helmer1-1024x768.pnghttp://helmer.sfe.se/helmer1-1024x768.pnghttp://helmer.sfe.se/helmer1-1024x768.pnghttp://helmer.sfe.se/helmer1-1024x768.pnghttp://helmore.sfe.se/http://helmer.sfe.se/HelmerDataSheet.pdfhttp://helmer2.sfe.se/8/12/2019 Building Home Linux Render Cluster
20/22
Some render tests result here:The storming sea scenefile 24 frames in resolution 4000x4000 pixels.
- On a MacPro 2 x DualCore Xenon 2.66 Ghz with 4 GB ramtook 552minutes (9.2 h)
- Helmer did same 24 frames in 4k format in64min.
Some approximate numbers give Helmer a floating point capacity of 186 Gflops
Here is the scale of Helmer (about 2,5 foot high:)
http://helmer.sfe.se/HelmerRendertest1.jpghttp://helmer.sfe.se/HelmerRendertest1.jpghttp://helmer.sfe.se/HelmerRendertest1.jpg8/12/2019 Building Home Linux Render Cluster
21/22
My other life-hacks are here:
Building a wind sensor from a PS2-mouse
Building a small linux robot
Building a underwater camera house
Building a solar powered UAV
A better spelled version of ths page is here (by Dan - Thanks:)
Some example of rendered frames here.
http://nslu2windsensor.sfe.se/http://nslu2windsensor.sfe.se/http://robot.sfe.se/http://robot.sfe.se/http://underwaterhouse.sfe.se/http://underwaterhouse.sfe.se/http://dementor.sfe.se/http://dementor.sfe.se/http://helmer.sfe.se/thedanpage.htmlhttp://helmer.sfe.se/thedanpage.htmlhttp://render.sfe.se/http://render.sfe.se/http://render.sfe.se/http://helmer.sfe.se/thedanpage.htmlhttp://dementor.sfe.se/http://underwaterhouse.sfe.se/http://robot.sfe.se/http://nslu2windsensor.sfe.se/8/12/2019 Building Home Linux Render Cluster
22/22
Recommended