This is the fifth and last article in this series of posts. The first two articles (Article I., Article II.) in the series focused on hardening possibilities on a project level with Google Cloud Platform. The third post highlighted the topic of built-in GCE Security. In the fourth article in the series, we went through several network-level protection tools available for your Google Compute Engine instances. This article is about the possible security-related settings on the instances’ operating systems themselves. This is not closely related to either Google Cloud Platform (GCP) or Google Compute Engine (GCE). Protecting the OS of your instances is also very important. You cannot ignore these security controls if you use GCE. Read this article to learn know more about GCE OS Security.
GCE OS Security is a huge topic with many articles, documents, and papers available. My focus will be on Linux instances and will only mention a small subset of OS protection techniques.
Only Use Encrypted Protocols for Any Traffic
It is a long-standing truth that you should not trust a random Internet provider of your client from the other side of the world. Especially, you should not trust cafés and their Wi-Fi networks. So any traffic leaving your instances toward the Internet should use an encrypted channel like VPN tunnels or HTTPS. It guarantees two benefits. One is that the data will not be read by anyone between your service and the client. The other very important reason is that the data cannot be modified during transit. Thus, the client will really see your content and not a modified version by a malicious network operator.
Even if you stay inside your private network on GCE, it is important to use encryption when possible. In general, GCP is very well secured, but you should apply any additional security measures if that does not incur a crippling overhead or an insurmountable amount of work for you. Fortunately, encryption is readily available today, with a minimal additional configuration in most software. You should use those options to make sure that your data in transit is secure, even if an attacker somehow captures traffic on your private network.
For example, it is advisable to use databases over SSL by changing their configuration to accept encrypted connections. It is also good practice for your services if they talk to each other’s APIs over HTTPS. The computational overhead of these encryptions is around 1‒2%, so it should not incur high costs on your monthly bill. Here is an interesting story of Gmail going full HTTPS in 2010(!) and doing so without any additional hardware or network equipment.
As a part of its mission, the US nonprofit organization called Center for Internet Security has to “identify, develop, validate, promote, and sustain best practice solutions for cyber defense”. It has a set of documentation for a lot of operating systems and for other software containing the known best practices and configuration options to improve security. These are the CIS benchmarks. They are available for all the major Linux distributions. The benchmarks are detailed and constantly updated, and are a very good source if you want to make your OSs more secure. Check out these freely available benchmarks here.
Install the Stackdriver Logging Agent
Stackdriver logging is a good service for more convenient operations. It is easy to filter or search logs. It also aggregates all your logs in a single place, so you do not have to individually access each machine to see them. Looking at it from a security perspective, the tool is very valuable. By design, it is “append only” which means that if a log entry goes into it, then it cannot be deleted before hitting the log retention time.
In practice, it cannot prevent an attack. However, if your instances constantly log audit- and security-related events to Stackdriver, at least the attackers cannot hide their trail after compromising an instance. This is because you can check out their actions later in the non-alterable Stackdriver log stream.
(Automatic) Security Updates on the Operating System
New security issues are discovered in many software programs almost daily. Most of those problems are fixed very quickly by either the open source community or by the vendor. This is a good thing but it puts the burden of updating your OS software on you. This is something which should be done very often, possibly automatically.
It is a good idea to set up a system where you update your OSs automatically, stage by stage. The usual setup is to update your test or staging machines as the first stage and then run tests on them. Even if everything is okay, wait for 1‒2 days before doing the next stage, which is usually a small number of the production machines. And then, again after waiting for a specified time, do the next stage with a few more production machines and so on. This way, you can find usually find problems in time, before they could cause an outage. Updates happen automatically rather than leaving the issue in the hands of human forgetfulness.
For example, one Ubuntu tool for this job is called Landscape and another CentOS tool is called Spacewalk.