Posted on 14 Sep 2015 by Ray Heffer
Following the theme for ELS (Essential Linux Skills) with CentOS 7 (see part 1), today I want to share what I consider to the the most important topic of the lot. Firewalls. Securing your Linux host is, in my opinion, the first thing you should be doing before hosting any web services. In my last post, you learned all about systemd and hopefully are now comfortable with the switch from SysV init.
If you are responsible for building Linux hosts for web applications then this will be an especially important topic for you. The same applies if you want to master security with Linux. This might get a little technical, but hang in there.
RHEL (RedHat Enterprise Linux) and CentOS 7 introduces firewalld which is now installed by default instead of iptables. Another newcomer, but not yet loaded by default with CentOS 7 is nftables. What’s the difference? Well firewalld is new to the user-space, but it doesn’t replace iptables. Nftables will eventually replace iptables.
Confused? I don’t blame you, so let me explain the iptables architecture. It’s important to understand how iptables works in order to understand the changes that firewalld and what nftables brings to the table (pun intended).
We’ll start with this basic architecture diagram for netfilter:
Type | Description |
---|---|
User-space | Iptables resides in what we call the user-space, this is your interface to the firewall for setting up your firewall rules. The same applies to ip6tables, arptables, ebtables and so on. As a firewall administrator, this is where you get stuff done! |
Kernel | Next we have netfilter, the framework which iptables is based on. Netfilter implements a series of ‘hooks’ that inspect packets in the protocol stack, such as IPv4. These hooks allow for kernel modules to interact with them. Iptables has a huge list of kernel modules used for its firewalling capabilities. We have everything from TCP and UDP to pkttype (Packet Type). In fact if you want to see a list of iptables kernel modules, type: cat /proc/net/ip_tables_matches. |
Hardware | This is our network adapters; eth0, eth1, and so on. We’re talking about the data link layer (OSI layer 2). Netfilter uses prerouting and postrouting to and from the network stack to inspect packets sent and received on each interface. |
So the packet inspection is done at the kernel layer with the netfilter, and all the firewall rules and tools to manage the firewall reside in the user-space. Got it.
As I mentioned earlier, firewalld is now the default with RHEL and CentOS 7. The main difference is that firewalld gives us dynamic rule management (in place changes) as opposed to iptables which has a static ruleset. It also introduces network ‘zones’ and the configuration is stored in various XML files.
If I want to make a rule change with iptables then I have to flush out all the existing rules (iptables -F) then save the new rules to /etc/sysconfig/iptables (iptables-save). I don’t have to do that with firewalld.
Both firewalld and iptables use the iptables service to talk to the netfilter. The change is at the user-space layer (firewall-cmd).
Whether or not to use firewalld or to switch back to iptables is up to you. One thing is for certain, and that is firewalld is NOT a replacement for iptables.
Personally I like to use iptables, so that is what I will use here. If you already use Linux as your desktop OS then firewalld will be easier to manage with the GUI (firewall-config) and dynamic configuration. I don’t believe there is any security benefit to firewalld. Think about it. Both use netfilter, and both use iptables tools to function.
So what about the new kid on the block, nftables?
There have been some interesting developments since the 3.13 kernel was released in 2014. A new firewall, nftables, looks set to replace iptables in the long run. It’s not just a simple change to the user-space (now nft), it’s also a completely new packet filtering framework (kernel). You can read more about nftables here.
Nftables hasn’t replaced iptables yet, and you won’t find it as easy as a yum install either. Give it another year or two from now then it might just be the new firewall for RHEL and CentOS. Getting Started with iptables
Let’s get started with some iptables basics. First we need to replace firewalld with iptables. To do that, we’ll stop and disable the firewalld service (see my previous blog post about systemd), install iptables-services and enable iptables.
Update: As it has been pointed out in the comments below, it is worth masking the service to avoid it being started manually or by another process. Disabling the service will remove it from what we used to know at the runlevel, with systemd this is our .wants file (e.g. basic.target.wants which is where firewalld starts). When using the mask option, it creates a symbolic link to /dev/null so it cannot be started.
# systemctl stop firewalld
# systemctl mask firewalld
# yum -y install iptables-services
# systemctl enable iptables
# systemctl enable ip6tables
# systemctl start iptables
# systemctl start ip6tables
Note: Remember, the options for iptables are case-sensitive.
At this stage you have no firewall rules. To check your existing rules we use:
# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
There are heaps of tutorials out there for iptables, but I want to give you enough information to understand the basics and adopt some security best practices. Before we go any further I should explain what chains are all about.
Note: Using verbose output (iptables -L -v) is useful to show stats on each rule. If there are no matches on a given rule then you can mark it for deletion.
A chain is a set of rules, checked one by one until it is matched. There are 3 chains: INPUT, FORWARD and OUTPUT (they are processed in that order).
Notice in the example above that the default policy for each chain is set to ACCEPT. In other words it will expect you to create your rules on which traffic to deny, leaving everything else to get through. It’s far more secure to do the opposite, so we’ll set the default policies to DROP.
Wait! If we do this now then we’ll lose connectivity to our Linux host, assuming of course you are connected via SSH. We’ll do that in a later step.
Order of Chains
Remember that iptables contains a static configuration, unlike firewalld, so if you implement a set of rules and want to make a change you’ll need to flush the ruleset, save and restart. Let’s look at this now.
First stop the iptables service:
# systemctl stop iptables
Flush our rules (start over)
# iptables -F
We would normally start with connections we want to drop, but I’ll discuss that in the Limited Other Attack Vectors section.
So let’s create our first basic ruleset (set of chains) to allow incoming SSH connections.
iptables -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --sport 22 -m state --state ESTABLISHED -j ACCEPT
Don’t panic yet, I’ll break this down into chunks that are easier to understand. Look at the first line:
iptables -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW,ESTABLISHED -j ACCEPT
Command | Description |
---|---|
iptables -A INPUT | Append the rule to the end of the selected chain, in this case INPUT. |
-i eth0 | We specify our interface, in this case it’s eth0 |
-p tcp | The protocol is TCP |
–dport 22 | Our destination port is 22 (default for SSH) |
-m state | Specifies what we want to match, in this case we want to check a state |
–state | These are our states. Typically we would use NEW and/or ESTABLISHED, but other options are RELATED or INVALID. In this case we want to match NEW and ESTABLISHED |
-j ACCEPT | Since we’re going to drop everything else, we want to accept this chain (rule) to allow SSH |
iptables -A OUTPUT -o eth0 -p tcp --sport 22 -m state --state ESTABLISHED -j ACCEPT
Later in this guide we will set the default policy to deny all other traffic for all 3 chains, so we will also need to allow SSH traffic back out of eth0 (notice the use of -o for out interface). We only use the ESTABLISHED state here as we’re dealing with the response back to the SSH client (OUTPUT) for a connection that has already been established.
I find it really useful to have a log for our INPUT and OUTPUT chains for anything that is dropped. Remember that iptables processes each rule sequentially, so once we’ve finished adding our rules to ACCEPT a connection, we will add our logging rules.
iptables -N LOGINPUT
iptables -N LOGOUTPUT
iptables -A INPUT -j LOGINPUT
iptables -A OUTPUT -j LOGOUTPUT
iptables -A LOGINPUT -m limit --limit 4/min -j LOG --log-prefix "DROP INPUT: " --log-level 4
iptables -A LOGOUTPUT -m limit --limit 4/min -j LOG --log-prefix "DROP OUTPUT: " --log-level 4
Let’s break this down.
iptables -N LOGINPUT
iptables -N LOGOUTPUT
Here we create new chains for logging. We’ll call it; LOGINPUT and LOGOUTPUT.
iptables -A INPUT -j LOGINPUT
iptables -A OUTPUT -j LOGOUTPUT
We will then use iptables -A to append a new rule for our INPUT and OUTPUT chain and send it to a target with -j. In this case our target is our new chain LOGINPUT or LOGOUTPUT.
iptables -A LOGINPUT -m limit --limit 4/min -j LOG --log-prefix "DROP INPUT: " --log-level 4
iptables -A LOGOUTPUT -m limit --limit 4/min -j LOG --log-prefix "DROP OUTPUT: " --log-level 4
Using the limit matching module (-m limit) we can ensure that we log no more than 4 per minute, this will stop our logs filling up too fast. The logs will be sent to /var/log/messages (–log) and classified as warning entries (–log-level 4). I’ll cover monitoring for attacks later in the article. Set policy to drop all other traffic
Our final three lines change our policy (-P) for the built in chains (INPUT, FORWARD and OUTPUT) to DROP. We could leave the last OUTPUT chain to ACCEPT, given this is for traffic coming from the host (we which trust), but I personally think it’s good practice to be as secure as possible. It also means you specify exactly what you let in, and out.
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT DROP
Note: If you do decide to only DROP INPUT and FORWARD then you can simplify your firewall rules as you won’t need to specify anything for the OUTPUT chain (iptables -A OUTPUT). What we have so far
So there you have your first basic iptables firewall rules. This is what we have learned so far:
Here is our first basic firewall ruleset with iptables, go ahead and copy the following into your console (remember to flush your configuration first).
iptables -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --sport 22 -m state --state ESTABLISHED -j ACCEPT
iptables -N LOGINPUT
iptables -N LOGOUTPUT
iptables -A INPUT -j LOGINPUT
iptables -A OUTPUT -j LOGOUTPUT
iptables -A LOGINPUT -m limit --limit 4/min -j LOG --log-prefix "DROP INPUT: " --log-level 4
iptables -A LOGOUTPUT -m limit --limit 4/min -j LOG --log-prefix "DROP OUTPUT: " --log-level 4
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT DROP
Now you’ve copied your first few iptables rules into the console, you need to save it and restart iptables.
# iptables-save > /etc/sysconfig/iptables
# systemctl restart iptables
Troubleshooting Tip: If you lose access to your SSH console at this point then you’ve blocked your own connection. Make sure the interface name is correct (-i eth0) as you may have something different. Once you’ve found the issue, stop the iptables service, flush the rules and add your rules again. Advanced SSH Security
Within minutes of booting a Linux virtual machine with my hosting provider, my host was under a brute force attack for root. Great, let’s have some fun. Don’t worry, we’ll dive back into iptables again after this :)
It’s strongly recommended to disable root login with SSH. By default, your SSH configuration will allow root to login. There are some more tricks we can deploy. Let’s change the maximum number of SSH authentication attempts, change our SSH listening port from 22 to something else (e.g. 9292), and tweak our firewall rules to rate limit the number of SSH connections within a given period of time.
# vi /etc/ssh/sshd_config
Change:
#PermitRootLogin yes
To:
PermitRootLogin no
Restart SSH:
# systemctl restart sshd
This alone won’t stop an attacker trying to brute force your host using SSH, but it will prevent the misfortune of cracking your root password. I’d rather know that the attacker is just barking up the wrong tree.
Change:
#MaxAuthTries 6
To:
MaxAuthTries 3
Changing the SSH Port, change:
#Port 22
To:
Port 9922
Note: If you change the listening port then remember to change your firewall rules accordingly! I just picked 9922, but this could be something else if you wish.
Finally, my favorite trick is rate limiting, to drop more than 3 connections per minute from an IP address. In fact you can adjust the time frame accordingly to be more aggressive than that. In my example, which works best for me, I’ve set rate limiting to no more than 3 connections every 60 seconds. Notice here that I’ve changed the incoming and outgoing rules slightly, only accepting ESTABLISHED connections, while new connections are rate limited.
iptables -A INPUT -i eth0 -p tcp --dport 9922 -m state --state NEW -m recent --set --name SSH
iptables -A INPUT -i eth0 -p tcp --dport 9922 -m state --state NEW -m recent --update --seconds 60 --hitcount 4 --rttl --name SSH -j DROP
iptables -A INPUT -i eth0 -p tcp --dport 9922 -m state --state NEW, ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --sport 9922 -m state --state ESTABLISHED -j ACCEPT
You will see the last two lines are exactly the same as before, but now I’ve changed the port to 9922 as I changed the listening port in the last section. We have two new rules that use the –recent extension. This creates a list of IP addresses that match a certain criteria. Let’s take a closer look:
iptables -A INPUT -i eth0 -p tcp --dport 9922 -m state --state NEW -m recent --set --name SSH
Command | Description |
---|---|
iptables -A INPUT -i eth0 -p tcp –dport 9922 -m state –state NEW | Same as before, appending (-A) the INPUT chain on interface eth0 on TCP destination port 9922, and matching the NEW state. |
-m recent | We want to match our rule using the recent extension. This allows us to create our list of IP addresses. |
–set | This will add the source address of the packet to the list |
–name SSH | This is the name for the recent list. Call this what you want |
iptables -A INPUT -i eth0 -p tcp --dport 9922 -m state --state NEW -m recent --update --seconds 60 --hitcount 4 --rttl --name SSH -j DROP
The second line then then checks if the connection was last seen in the last 60 seconds and it’s received at least 4 connections (hitcount => 4).
Command | Description |
---|---|
–update | Update the match when used with –seconds and –hitcount |
–seconds | Used with –update, will match if last seen in the specified number of seconds |
–hitcount | Also used with –update, will match if the hit equals or is higher than the hitcount |
It stores the recent list in /proc/net/xt_recent/
src=43.229.53.27 ttl: 55 last_seen: 4342973913 oldest_pkt: 20 4342965086, 4342967640, 4342970007, 4342970007, 4342970306, 4342970306, 4342970907, 4342970907, 4342971509, 4342971509, 4342971808, 4342971808, 4342972410, 4342972410, 4342973013, 4342973013, 4342973313, 4342973313, 4342973913, 4342973913, 4342747400, 4342747700, 4342747700, 4342748301, 4342748301, 4342748903, 4342748903, 4342749203, 4342749203, 4342749804, 4342749804, 4342962767
Christmas trees, floods and empty packages. Sounds like someone is having a really bad time! Let’s make sure it’s not us :)
We can use iptables to help protect against these attacks. A Christmas tree packet has all options set (SYN, FIN, URG and PSH). If you are receiving a large number of these then someone may be trying to conduct a denial-of-service attack.
Null packets usually indicate that your host is being scanned, as a single packet with no flags set. This is never legitimate, so we should drop it.
Syn-floods are also a possible threat from denial-of-service attacks, as the connection is started with SYN (part of the three-way handshake) but then it doesn’t respond with an ACK (acknowledge). A bit like ringing someone’s doorbell, waiting for them to open the door and then running away. (lol)
iptables -A INPUT -p tcp --tcp-flags ALL ALL -j DROP
iptables -A INPUT -p tcp --tcp-flags ALL NONE -j DROP
iptables -A INPUT -p tcp ! --syn -m state --state NEW -j DROP
This has nothing to do with iptables, but worth a mention. There are two host access files (/etc/hosts.allow and /etc/hosts.deny), that are part of the TCP_WRAPPER package. This makes it possible to allow or deny access to certain services based on the IP.
It has been around since the late 1990s, and it provides an additional layer of security. Not all services support TCP wrappers, but sshd does support it. You can check whether a service supports TCP wrappers by getting the path to the service (E.g whereis sshd) and then searching for the libwrap.so library.
# whereis sshd
# ldd /usr/sbin/sshd | grep libwrap.so
ldd /usr/sbin/sshd | grep libwrap.so
libwrap.so.0 => /lib64/libwrap.so.0 (0x00017ffceg8f1100)
If you try it with httpd (which is typically in /usr/sbin/httpd) you’ll see it produces no result, as httpd doesn’t support TCP wrappers.
Here are some examples of its usage:
# vi /etc/hosts.allow
sshd:<IP ADDRESS>
# vi /etc/hosts.deny
ALL:ALL
Using this example, we are allowing just a single IP address to SSH (sshd) in hosts.allow. Then we deny everything else in hosts.deny. Be careful with this though, if your IP address changes then you will not be able to access your server without console access.
The
So you may be wondering why would I need to do this if iptables can achieve the same thing? Well don’t choose one over the other. Always use iptables. But think about using both, as you are adding additional security which is always a good thing. Blocking a IP Addresses with iptables
This is a simple example of how to block a specific IP address.
iptables -I INPUT -i eth0 -s x.x.x.x -j DROP
If you want to block an entire subnet, just use the / notation for the subnet mask (E.g. A class C subnet of 255.255.255.0 would be /24).
iptables -I INPUT -i eth0 -s x.x.x.0/24 -j DROP
If you want to run a web application then there is no better place for it than a Linux host, providing you have taken these measures so far. Because our approach is specify which services are allowed then deny everything else, we need to create a minimal rule set for our host. Here is what we have done so far, and what we require for a typical web server:
You may want more rules depending on what server applications you are running, but this is the minimum for most web hosts.
iptables -A INPUT -i eth0 -p tcp --dport 80 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -i eth0 -p tcp --dport 443 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --sport 80 -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --sport 443 -m state --state ESTABLISHED -j ACCEPT
Perform DNS Queries
iptables -A INPUT -i eth0 -p udp --sport 53 -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p udp --dport 53 -m state --state NEW,ESTABLISHED -j ACCEPT
Allow Inbound/Outbound to Localhost
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
Web Browsing
iptables -A INPUT -i eth0 -p tcp --sport 80 -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -i eth0 -p tcp --sport 443 -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --dport 80 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --dport 443 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
Allow SMTP outbound (sendmail)
iptables -A INPUT -i eth0 -p tcp --sport 25 -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --dport 25 -m state --state NEW,ESTABLISHED -j ACCEPT
# Drop NULL packets
iptables -A INPUT -p tcp --tcp-flags ALL NONE -j DROP
# Block syn flood attack
iptables -A INPUT -p tcp ! --syn -m state --state NEW -j DROP
# Block XMAS packets
iptables -A INPUT -p tcp --tcp-flags ALL ALL -j DROP
# Web server rules to allow incoming HTTP and HTTPS requests
iptables -A INPUT -i eth0 -p tcp --dport 80 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -i eth0 -p tcp --dport 443 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --sport 80 -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --sport 443 -m state --state ESTABLISHED -j ACCEPT
# Allow incoming SSH connections and drop more than 3 attempts in 1 hour
iptables -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW -m recent --set --name SSH
iptables -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW -m recent --update --seconds 60 --hitcount 4 --rttl --name SSH -j DROP
iptables -A INPUT -i eth0 -p tcp --dport 22 -m state --state NEW, ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --sport 22 -m state --state ESTABLISHED -j ACCEPT
# Allow DNS queries
iptables -A INPUT -i eth0 -p udp --sport 53 -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p udp --dport 53 -m state --state NEW,ESTABLISHED -j ACCEPT
# Allow LOCALHOST
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
# Allow connections outbound on HTTP and HTTPS (browsing)
iptables -A INPUT -i eth0 -p tcp --sport 80 -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -i eth0 -p tcp --sport 443 -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --dport 80 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --dport 443 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
# Allow outbound SMTP
iptables -A INPUT -i eth0 -p tcp --sport 25 -m state --state ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp --dport 25 -m state --state NEW,ESTABLISHED -j ACCEPT
# Log all dropped packets
iptables -N LOGINPUT
iptables -N LOGOUTPUT
iptables -A INPUT -j LOGINPUT
iptables -A OUTPUT -j LOGOUTPUT
iptables -A LOGINPUT -m limit --limit 4/min -j LOG --log-prefix "DROP INPUT: " --log-level 4
iptables -A LOGOUTPUT -m limit --limit 4/min -j LOG --log-prefix "DROP OUTPUT: " --log-level 4
# Set policies to drop everything else
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT DROP
You’ll find your logs in /var/log/, and the two you should be looking at are messages and secure. I like to use the tail command to follow data being appended to a log file as it comes in. This is particularly useful to monitor what is being denied (see Logging) and also SSH connections.
Monitor all security related logs, such as authentication failures, SSH logins, failed login attempts, and more:
# tail -f /var/log/secure
Monitor the system log (we send our iptables logging here)
# tail -f /var/log/messages
Obviously there are lots of other log files, and I won’t detail them here but do take a look. Specifically the tail -f command allows you to watch them as entries are added!
Remember your SSH rate limiting rules? This creates a file in /proc/net/xt_recent/ (E.g. /proc/net/xt_recent/SSH). Check the contents of the file to see which IP addresses have been caught exceeding the rate limit.
Finally, as I mentioned earlier don’t forget to use iptables -L -v (verbose). This is a trick I used back in the days when I was managing Cisco PIX firewalls. Why? If you see a firewall rule has no hits after some time, then the likelihood is that it’s not required. Be careful though!
When I woke up on Sunday morning and decided to write this blog, I thought it would be a case of sharing a few words. There is so much to iptables, so if you have made it this far then give yourself a pat on the back, make a coffee, go to the gym… whatever you do, you have mastered Linux firewalls!
Hopefully this gives you enough to secure your Linux web server and you have now learned the basics of iptables so you can create your own rules. You can do so much with iptables, in fact I could write a book on it!
Something I haven’t mentioned is Fail2ban. It’s an intrusion prevention framework that searches your logs for patterns. We can use this to rate-limit SSH connections, but do so much more. It isn’t fail-safe, you can never be 100% secure.
Comments are closed for this post.