It’s now no secret that I’ve been planning to expand my at-home storage and computing resources. In my last post, I outlined some criteria that I used to help me consider what hardware to purchase. I thought it’d be a good idea to expand on that list of criteria in case anyone else wanted to use it to help them work through finding some new hardware. This is going to be a long one, so settle in!
Keep in mind, this list is more geared towards creating a virtualization server. There are some different things to keep in mind if you’re looking to put together a gaming rig, NAS, as-cheap-as-possible workstation, media center machine, or other machines designated for other tasks.
For easier reference, here are each of the criteria:
CPU(s)
RAM
Storage
Network Connections (NICs)
Reliability
Power Efficiency
Extensibilty
Noise and Heat
Size and Location
Educational Opportunities
At the “core” (ha!) of any computer is the processor. Unless you’re only checking Facebook and typing emails, your processor can make a huge difference in how well your machine will perform and how much power it will use. We’re looking to run a bunch of computers’ worth of stuff in one box, not check emails, so the processor will get a lot of attention. You might want to ask yourself some questions, though keep in mind, your needs may be different than mine.
How many processor cores do I need? This can get complicated, but on the surface, more processor cores tend to be better for virtualization than having fewer, faster processors. This is because you’re splitting resources up between a bunch of operating systems. Depending on your hypervisor, you can also sometimes get away with splitting up processor resources by using virtual processors, as can be done with some versions of Microsoft’s Hyper-V. If you’re going with consumer-grade parts, get a processor with as many cores as you can afford. You’ll thank me later.
How fast does my CPU need to be? This goes hand in hand with how many cores you need, at least a little. If you have a bunch of processor cores (4, 6, 8, or more), then you can usually get away with having slower processors. For a lot of homelab (or even some business/enterprise) activities, you typically don’t need screaming fast processors as many of your VMs will probably be sitting idle much of the time. Processing big sets of data or encoding videos, on the other hand, sometimes don’d lend themselves to using multiple processors at once and instead perform faster on faster processors. Think about the intended use of your machine here. Many processors (consumer and enterprise alike) will have a letter in their name. Intel processors designated with an “X” will be the highest-performing and most power-sucking CPUs, whereas “E” or “L” models in a given series will typically be clocked a little slower but save a lot of power and generate a lot less heat.
Do I need multiple CPUs? Probably not, unless you’re getting a really good deal, are hosting a lot of machines, or have a heck of a lot of data to process. Almost all consumer-grade motherboards only have one CPU socket, meaning you can’t install more than one processor anyways. If you’re entering the realm of enterprise-grade hardware, having multiple processors becomes a possibility. Depending on the server you choose, you can probably start with one and add more later. A lot of older refurbished enterprise servers tend to come with motherboards that support two processors, though newer server motherboards can sometimes have more than that.
This can be a real bottleneck in terms of what you can do with your machine. Like I talked about in the last post, almost all consumer-grade motherboards are limited to 64GB of RAM and usually 4 slots, making it expensive to really get the most out of your motherboard. Without having enough RAM, you will at best get a lot of slowness as you use your drives to cache data.
How much do I reasonably need? Plan out the projects and programs you want to run on your machine first, and take a look at their system requirements. If you’re building a simple file server, then you probably won’t need much at all, but if you’re planning on running something like Windows Server and a few desktop operating systems, you’ll want to pack in as much RAM as possible.
What size/type do I need? There are a TON of types of RAM out today, each with different features. The most important thing to remember is to check your machine’s motherboard documentation and make sure the RAM you’re going to add is compatible. As tempting as buying used server memory might be, it might not be compatible with that old desktop you dug out of the closet. If you’re building a sandbox machine from consumer-grade parts, you probably won’t need to pay the premium for fast DDR4 ECC (error-correcting-code) memory with super low latency; slower, non-ECC, DDR3 RAM will almost certainly fit your needs. On the enterprise side, there are a few additional options, which I’ll talk about in a later post, because that information definitely deserves its own post.
Will this affect my choice in motherboards? Probably not. If you’re building a server with consumer-grade parts on a small or reasonable budget, you’re going to realistically be limited to 32GB of memory. If you’ve got some 16GB sticks of RAM already at your disposal, then go ahead and get a motherboard that’ll support it. If you’re looking to buy four 16GB sticks of RAM to hit the limit of most consumer-grade processors, then you might as well spend your money on a used/refurbished enterprise server.
Should data be stored on this machine, or should it be stored separately (NAS, etc.) and accessed over the network? Before looking into motherboards or computers with tons of on-board ports for your hard drive, first consider if you’ll actually be housing your data on this machine, or if you’ll be offloading it on a NAS (Network Attached Storage) box, some external drives, or other computers. This will help you figure out how much internal storage you’ll need to buy.
How big should my drives be? Take a look at what you’ll be storing. Virtual hard drives don’t take up a ton of space, but a movie library or system backups for you and your family/friends probably do. As you push into the multiple-terabyte territory, your cost per GB tends to go down, as does the space required to store each drive and (negligible) power costs. Physically, high-capacity 3.5″ hard drives are almost guaranteed to be cheaper, so unless you are going SSD-only or only have room for some 2.5″ or smaller drives, stick with 3.5″ sized drives; The slight increase in storage density isn’t worth it. One important note is to check to see if your motherboard/RAID controller will be able to see drives larger than 2TB. Some older hardware can’t, and at best your massive 6TB drive will be seen as 2TB by your machine.
What kind of drives/interfaces do I need? You’ll almost certainly want to shop for SATA hard disks. There’s no reason to shell out an extra $20-$50 per drive to get SAS connections. If you choose to go with enterprise equipment, know that almost all SAS connections will work with SATA drives. As far as SSDs (solid-state drives) go, you can usually get away with a tiny (30GB+) drive to use as the installation drive for most hypervisors or NAS OSs. Even then, you may not see improved performance with a SSD. Unless money is no object, you can definitely get away with using only spinning-disk drives.
How fast should my drives be? Just like with RAM, see what transfer speeds your motherboard will accept. There’s no reason to buy 10,000 RPM, 6Gb/s SATA drives when your motherboard will only use them as 3Gb/s drives. As far as hard disks’ RPM stats go, 5400 RPM is okay for archiving data or for storing data where speed won’t be a big concern. 7200 RPM is a pretty good sweet spot, giving decent performance for how much power they consume and how much they cost; use these for most virtual harddisks and other files. Anything above 7200 RPM would be a nice-to-have, but there are a few cons, including noise and price.
How many physical network adapters do I need? Probably one, unless you’re going to be transferring a lot of data across your network or want to isolate traffic for certain programs/VMs, in which case having at least two would be advisable. You can get PCI-based cards with 4 or more gigibit ports for a pretty good price, so you can almost certainly add more NICs later if you need to.
Will gigabit suffice, or do I really need to shell out the cash for 10Gb/s NICs? Ha! It’s my opinion that 10-gigabit network connections (right now) are really only needed for businesses. If you’ve got a TON of users who will need to access data across the network and want to spend the cash on upgrading practically every networking component in your environment, then 10Gb may be for you. In most cases, with or without the need to serve lots of users quickly and spend lots of cash, you’ll see bottlenecks in your drives’ read/write speed well before you can harness the bandwidth of 10Gb network connections.
Do I need enterprise-grade equipment for its reliability, or will consumer parts be good enough? If you’re building up something for home use, educational purposes, or for non-critical tasks, then you can almost certainly count on consumer parts being more than good enough. If you’re going to be archiving important data, make sure its in multiple physical locations as on-site disaster (storms, fire, theft) are more probable than manufacturer-caused drive failures. Your other needs (CPU sockets, RAM slots, case size, etc.) will be way better indicators of the kind of hardware you need to get.
Can I use some “gently used” or refurbished parts, or do I need to buy everything new? For many components, you can usually get away with used or refurbished parts instead of buying them new. Cases/chassis, some PSUs, optical drives, some RAM, and even some processors and motherboards can definitely be purchased second-hand, especially if your vendor or manufacturer offers a warranty on used components. I tend to feel a little uneasy about buying used/refurbished hard drives (SSDs and HDDs) and used (not refurbished) power supplies, but that’s a personal preference. Processors can be a little up in the air. The most important takeaway here is if it looks too good to be true, it probably is.
Do I want to risk buying bottom-of-the-barrel parts and hope for the best? Unless you want to make a hobby out of flipping parts or absolutely need that old Xeon processor for $6 shipped, don’t. The risk/reward ratio almost always isn’t worth it.
Will replacement parts be available in case of hardware failure? Thinking in the future a little, think about how you’ll replace something when it fails. Buying a used small-form-factor desktop for a quick and easy Plex server sounds like a good idea, but a few years from now you probably won’t be able to get a new power supply that fits in the case. Also keep in mind what (if any) proprietary connectors are used for a given component. A lot of manufacturers (looking at you, Dell) of prebuilt desktops have started using proprietary power connections for tons of internals to keep costs and prices low. They’re great if you’re installing 100 of them a year and have surplus parts around your shop, but not so much if you’re a few years in and find out you have to shell out tons of cash for “vintage” parts on eBay.
How much power will this machine use? Like most parts, this can vary A LOT. Like I ranted about in the last post, running a gaming desktop 24/7 will eat up a lot more space than a power-conscious NAS with Plex server installed. If you get free-to-you electricity (I lived in a dorm a few years back and have dreams of solar panels on my future roof), then this might be less of a concern. Otherwise, think about how much electricity you’ll be using. Ancient (or even some modern) desktops and retired servers can be HUGE power drains, so do a little research on your top contenders. As touched on in the processor section, look for “E” or “L” model processors, as they’ll be cooler and less power hungry.
Running something 24/7 will come with a slight cost, but do I need to spend that much more for efficiency? For the most part, you probably won’t need to spend a lot more just to get a little boost in efficiency. Like I talked about above, some old servers and desktops can eat up a lot of power and put out a lot of heat for what you get, so it may be wise to spend a little more. If you’re building from desktop parts, at least get an 80+ certified PSU. In a later post, I’ll talk about the benefits of choosing more expensive 80+ rated power supplies for desktop use.
How much will I be able to put in this machine before I “max it out”? Like I mentioned in the CPU and RAM sections earlier, a lot of desktop motherboards will feasibly max out at 32-64GB of memory. Replacing a CPU is usually pretty expensive and often times won’t offer that much of a boost in performance, even if the new CPU will fit in your current motherboard’s CPU socket(s). If you see your needs growing quickly, look for used enterprise hardware like refurbished servers; those will let you start relatively cheaply while still giving you lots of room to grow.
Will I be able to expand feasibly and at a reasonable cost? Again, here’s the downside with using desktop parts. Adding additional RAM is cheap when you’re going from 4 to 16GB, but beyond that you’re going to run into significant costs if it’ll even be possible to upgrade the soon-to-be-yours hardware, possibly resulting in the need to buy an additional machine to compensate.
Will I be hurting if I don’t “max out” this machine from the get go? This might be more specific to my current situation, but the answer is maybe. If you’re aready on a gaming-level desktop or high-end workstation and still need more resources, getting back that 1-3GB of RAM you were reservign for the host OS won’t make buying another machine worth it if you’re buying or building a virtualization server from desktop parts. If you’re buying a used/refurbished server, on the other hand, then you can almost certainly wait to pick up more sticks of RAM or that second CPU until you need it. Storeage is less of a concern here as many middle-of-the-road desktop motherboards offer 4 or more SATA ports and PCI-* slots to add more SATA ports if absolutely necessary. On top of that, you also have the more obvious choice of external storage via USB or NAS boxes.
How loud will the machine be? Desktops, especially those in big cases with large fans installed, tend to be pretty darn quiet, even if you’re pushing them to their limits. Servers, on the other hand, can be, well, LOUD. Take a tour of a datacenter’s server room some time, and you’ll get an idea for how loud rackmount servers can be. This can be mitigated a little by looking for larger servers (e.g. a 2U server will probably be much quieter than a 1U) since they can take advantage of larger, slower-spinning fans. If you’re going to keep the machine in a spare bedroom or the basement, then you probably won’t have to give this a second thought.
How much heat will the machine produce? As computers operate, they use electricity, but they’re not perfectly efficient, and excess power used is dissipated in the form of heat. Newer machines will tend to run much cooler than old ones, and desktops will usually produce less heat than most rack-mount servers. If you’re putting the server in another room (like the basement or a spare bedroom), make sure it’ll still get some AC. Think twice about putting it in a tight space or in a room that is usually warm already.
How important is having a compact-sized case? Something like a NAS or business-use PC might be small either way, but if you’re looking for some real horsepower, you’re probably going to need to find some space to stash your machine. I prefer larger cases and am fortunate to have space to store them, but if you don’t then you may need to spend a few more dollars or lower your expectations when it comes to what you’ll be able to add to your case. Rack-mount servers will usually require an expensive rack to hold them, though you can usually fit quite a few servers to a rack. An important note here is that many rack-mount server chassis can be installed in Ikea side tables! Look up the “Lack Rack” for more info and pictures.
Will size limit my performance? Not necessarily, though you’ll be hard pressed to install six 3.5″ hard drives in a small form factor desktop. If you only need a drive or two, then you can probably look on the small side of things.
Where will I keep this machine? Are you building a NAS to keep by your router/wireless access point combo in the living room, or are you looking for a rack-mount server capable of handling everything you can throw at it and more? Refer back to the noise and heat section to get an idea of how loud and hot your machine will be. As cool as it would look, that server rack that sounds like a fleet of prop planes taking off mght keep you up at night and make it hard to watch anything in the living room.
How much can I learn from what I’m spending? This speaks to the core of most people’s homelab-ing, myself included. I want to be able to learn more about networking, server administration, desktop management, different virtualization options, and lots more, and spending a few bucks on some hardware will definitely let me further my self-paced education. If you’re looking to pursue some networking certifications, then you probably won’t need to buy a full rack-mount server and could instead be better server by looking into the specific brand and models of networking gear for the tests you’re taking.
Can I gain some knowledge, skills, abilities, or at least familiarity with something new? If you’re looking to broaden your horizons and push yourself, then do a little independent research and get something you’re not as familiar with if it fits in your budget. Never opened your computer case before? Pick up a cheap, used desktop from Craigslist and slap in a few more sticks of RAM and a new hard drive. If you’re not too familiar with the ins and outs of hardware, then it might be time to build a desktop instead of paying someone (or a company) to do it for you. In my situation, I’m heavily considering buying a refurbished rack-mount server so I can get a taste for that kind of hardware (there’s a learning curve) while also taking advantage of the power it’ll give me to run the software I need.