A basic guide to securing your datacenter- part 1
This blog post is intended only as a basic guide to securing your datacenter and it’s a repost with some new stuff added into it. I also wrote a couple of follow ups to it that I will repost later.
It is not intended to be the gospel of security since, well.. let’s be honest, there is no such thing as absolute security.
As an example, in Sweden there was a case where a computer was locked in a vault, with no network access whatsoever and only a few people had access to it and still, it leaked information to foreign countries. Yes . a computer controlled by the military.
Absolute computer security is a myth and a beautiful dream. With that said, system administrators can still do a lot to make more difficult to access data and prevent attacks and sabotage to their systems.
Let’s start off with physical aspects .
The actual georgraphic location
Where is your server actually located and who has physical access to it?
Resetting administrator passwords or root passwords
Once you gain physical access to a sever, there are numerous way of accessing the data. The root password or Active Directory Domain Administrator password or NDS Admin password can be easily reset (yes, the sentence “easily reset” is used loosely) with a USB stick and booting the server. Have look a at for instance HIren’s Boot CD, Burglar for NDS or just play around with Google and search for terms such as “Reset administrator password”, “decrypt password” and so on .
You’ll be amazed when you realize how easy it actually is. Personally, I always carry with me some USB stick that enables me to boot up any system and “have my way with it”
So, we need to secure the server from outside access. The data center must be protected with card keys, cameras and access control. We need to who and why they are in the data center in the first place.For instance, a janitor or cleaning staff tends to have complete access due to what they do. Many companies hire outside help to get these kinds of jobs done.
Sabotage and disrupted data services
If I wanted to gain access to server, I would try to infiltrate one of those subcontractors to get a job as a let’s say janitor, and try to gain access to the data center somehow and from there, I can basically do what ever I wanted with the servers.
Maybe my goal wouldn’t be to steal data but to kill the data operations by corrupting the filesystems on the servers (for instance just “piping” data into various files from command prompt or hding files behind other files ), randomly switch disks on RAID systems or just put small holes in network cables to cause network errors, trigger an EMP in the data center and so on .
There would be numerous ways to disrupt the data operations, once i gained access to the data center or servers room.
If I’m out to steal data, I would probably not target the servers themselves but instead I’d start looking for backup tapes or backup disks. Far too many companies have their backups in the same location as the servers themselves and since the backups usually are not encrypted , I’d go for stealing a complete backup set, go home and start doing a scan of the tapes to figure out what backup software was used and then do a complete restore of it all.
The backups are most likely to contain the data I’m after, although probably a maximum of 24 hours old but from them I can gain access to all kinds of information about the operations and crack administrative passwords for the server systems and so on . In the comfort of my own data center. or my couch.
This way would of require some skill of Disaster Recovery scenarios and how to get data back from backups but I’m fairly sure I’m not he only one in the world who has the expertise in those matters.
Backups are always a weak point from several aspects.
You need to know who has access to them , at all times.
If you are company that for instance have your backups shipped to another location on a daily basis or weekly, you need to know that people handling them haven’t been compromised. It wouldn’t take that long for anyone with the proper skills do clone your tapes or disks en route to their destination and you would have no way of telling they’d been cloned. In that scenario, all of your critical data would be in the wild, without you actually knowing it.
If you are using any kind of online backup service, remember to choose your encryption password wisely and be extremely restrictive with who has the password. A lot of backup software do not let you change the encryption password without doing the first backup all over again, thus doubling your usage of space and your costs.
Still , I highly recommend you use an encryption password for several reasons.
If you don’t use it, it’s just pure laziness and fear of administrative hassle and that’s just not an excuse. The risk of people at your online backup provider being able to access your data is of course also an obvious risk. Do you know these people and how could you be absolutely certain that they aren’t poking around in your data and maybe giving it to your competitors?
If you are thinking about online backups there are a few very essential questions you should ask yourself but we’ll get back to that later in another post.
Do not use the same encryption password as you have for Root / Administrator / Admin . That´s just crazy talk. Use a password generator to create a unique password. This is also very much valid for tape backups or backups on NAS / disk. Always encrypt your backups with password and be very, very restrictive with who has the password.
If an employee quits your company or an outside consultant quits his project and he / she has had knowledge of the password, change it.
If you’re doing a planned Disaster Recovery test for instance, change the administrative passwords right after the DR test , thus not enabling anyone to reuse what they’ve found out during the test
During the actual DR test, you should be there and be the one that actually types in the encryption password for the backups, not any outside technicians or consultants , not even the online backup service provider or your Disaster Recovery Service Provider. You , and you alone should be the one that has those passwords.
I’ll will return to backup questions and stuff regarding DR plans and so further down the line in some upcoming blog post
So, the conclusion is, always know who and why people are in proximity of your servers and NEVER let anyone be there alone without supervision and here’s a few more pointers.
A short cheatsheet for datacenters
- Always have you data center locked and secured from unauthorized access, If you have the means, also have it secured against an EMP attack from the outside. Of course, I haven’t even touched the subjects but be sure your data center has all the necessary fire prevention/extinction equipment in place, UPS backups and , if possible, also an outside source for generating current in case the UPS or battery runs out of current. There should also be a system in place for protecting you servers against spikes in current.
Be sure to know where water pipes are running in the building so you don’t place your server directly underneath one.
- Don’t keep cardboard or any other kind of flammable materials in the data center. Be sure to take them with you when you’ve set up a new server or switched disks. Don’t be lazy. The cost of laziness can be extreme.
- In the data center, always have your servers locked in cabinets that requires keys and access card to gain physical access to keyboards and stuff. Also remember to protect the cabling and the back of the servers! Never have a server logged on the console. Be sure to have all cabling to the and from the firewall and the internet access secured.
- If you’re “expecting company” i.e. external consultant and so on, be sure to think abut NOT having different kinds of network maps, administrative passwords and different kinds of information in plain sight for anyone to see. I’ve seen it hundreds of times, the IT department having their entire IT operations and information of their systems on white boards or on print on documents next to their workstations. It’s very quick and easy to take a picture of it with a cell phone and once you understand the infrastructure you can also start exploring weak points in it.
Sure , it makes their lives easier but as one might gather, it also makes the life of the attacker easier. Knowledge is a powerful weapon, especially when it comes to data protection
- Don’t have software laying around in the data center with software keys and stuff on them . All it takes is a mobile phone for someone to coy your license keys thus possibly putting you in an awkard situation having to explain to Microsoft for instance how come that your Volume License keys are a bit too easy to find on Piratebay, thus putting you n the risk of your licenses becoming invalid and bringing your server operations to a halt , or at least a short disruption.
- Know where your backups are, at all times. Have them encrypted. If using online backup services, be sure to use an encryption key and , if possible, be sure to have restrictions on the online backup service providers end on to and from where backups and restores are allowed
- Do not allow mobile phones in the data center due to the risk of people photographing your equipment or software license keys or using the mobile phones to copy data via USB cables and stuff.
- If possible, disable any USB ports on your servers. This can be done in the BIOS (and of course also have a system passwords for accessing the BIOS, a unique one for each server ) or you can physically kill it by putting glue or something in there (I haven’t tried that myself so do try at your own risk .. )
- If you are sharing a data center with others, there is no need for you to have your company logo or anything revealing the servers are yours. Keep it as anonymous as possible. There is also no need for you to tell anyone where your servers are physically located (although it can be fairly easy for anyone to fin out using traceroute commands and so on ),
- Be sure to have a Disaster Recover Plan (DRP ) / Business Continuity Plan (BCP) if your site is compromised or an accident should occur. Also in this case, treat the secondary DR location as mission critical data. Far too often, the secondary site is forgotten and poorly updated.
- Once again, do not underestimate the powers of social engineering. Although it’s not hacking in the usual sense, it’s merely good acting but it can still be as harmful as I’m trying to point out here
So, there’s a few tips anyway and that’s just the start really. It’s not a complete recipe for securing your physical environment and I’m sure I’ve missed out loads of stuff but it’s a start anyways.
I hope you you liked this post and I’d love to hear you thoughts on it and if you want me to write a few others on the matters of securing your server operations, I was thinking in the lines of brute force protection, change management, 0day attacks, certificate management , password policies, protecting web servers and so on , You get the picture 🙂
// Juha Jurvanen
Senior consultant in backup, IT security, server operations and cloud