SELinux enforcing with containers
Part 4 of the series that starts with Four servers, three distributions: why I consolidated.
Every one of my hosts runs SELinux in enforcing mode. Every container runs in the confined type container_t, with no exceptions. This post shows the few tools that make this practical: volume labels, file contexts, booleans and small policy modules. It also shows how to debug a denial.
Why bother
A container that breaks out of its namespaces lands in a process on the host. With rootless Podman that process runs as an unprivileged user. With SELinux it is also confined to container_t. That type may read and write only files labelled container_file_t, and each container gets its own MCS category pair (for example s0:c123,c456). So one container cannot read another container’s files, even when both belong to the same user.
The cost is small once you know the patterns. Most of the time, the right volume flag is all you need.
Volume labels: :Z and :z
When you mount a host directory into a container, its files usually have a type such as user_home_t. A confined container may not read that. Podman can relabel the files at start:
| Flag | Label | Use for |
|---|---|---|
:Z | container_file_t with this container’s private categories | data of one container |
:z | container_file_t without categories (shared) | files that several containers mount |
# Only this container uses it
Volume=%h/.config/ntfy/server.yml:/etc/ntfy/server.yml:ro,Z
Volume=ntfy-data:/var/lib/ntfy
# Three website containers mount the same file
Volume=%h/.config/nginx/realip.conf:/etc/nginx/conf.d/realip.conf:ro,z
Named Podman volumes get the right label when Podman creates them. The :Z on a volume does no harm and makes the intent clear.
Two warnings:
- Never use
:Zon a system directory such as/etcor/home. Podman relabels everything below it, and the host then cannot use those files any more. - If two containers mount the same file and both use
:Z, the second start relabels it for itself. The first container then gets “permission denied”. Use:zfor shared files.
Directories outside the home: file contexts
Some files are not mounted from a home directory. For example, node_exporter reads text files with metrics from /var/lib/node_exporter/textfile, and my backup script writes them. Instead of relabelling the directory at every start, make the label permanent with a file context rule:
sudo semanage fcontext -a -t container_file_t '/var/lib/node_exporter(/.*)?'
sudo restorecon -Rv /var/lib/node_exporter
restorecon applies the rule now. The rule also survives a full relabel of the filesystem. With Ansible this becomes two tasks:
- name: SELinux label for the textfile directory
community.general.sefcontext:
target: /var/lib/node_exporter(/.*)?
setype: container_file_t
state: present
register: textfile_fcontext
- name: Relabel the textfile directory
ansible.builtin.command: restorecon -R /var/lib/node_exporter
when: textfile_fcontext.changed and not ansible_check_mode
changed_when: true
Booleans
The policy has switches for common cases. List them with getsebool -a | grep container. One example: container_use_devices lets containers use host devices. I keep the booleans per host in the inventory:
# host_vars/<host>.yml
selinux_booleans:
container_use_devices: true
- name: SELinux booleans
ansible.posix.seboolean:
name: "{{ item.key }}"
state: "{{ item.value }}"
persistent: true
loop: "{{ selinux_booleans | dict2items }}"
Prefer a boolean over a policy module when one exists. Somebody else maintains it, and it is easy to see what it allows.
Small policy modules for the rest
Two of my cases have no boolean.
Traefik and the Podman socket
Traefik reads container labels through the rootless Podman socket. The socket file can be relabelled with :z. But connecting to the socket means talking to the process behind it, and that process runs as container_runtime_t. container_t may not do that. The denial in the audit log looks like this:
type=AVC msg=audit(...): avc: denied { connectto } for pid=... comm="traefik"
path="/run/user/1001/podman/podman.sock" scontext=system_u:system_r:container_t:s0:c12,c345
tcontext=unconfined_u:unconfined_r:container_runtime_t:s0 tclass=unix_stream_socket permissive=0
The fix is one rule:
module traefik_podman_socket 1.0;
require {
type container_t;
type container_runtime_t;
class unix_stream_socket connectto;
}
allow container_t container_runtime_t:unix_stream_socket connectto;
Build and load it:
checkmodule -M -m -o traefik_podman_socket.mod traefik_podman_socket.te
semodule_package -o traefik_podman_socket.pp -m traefik_podman_socket.mod
sudo semodule -i traefik_podman_socket.pp
This rule applies to every container_t process, not only to Traefik. A container still needs the socket mounted to use it, and only Traefik and my Podman exporter get that mount. Think about this trade-off before you load such a rule. Access to the Podman socket is close to access to the user’s account.
A VPN container (gluetun) needs the same kind of module for /dev/net/tun:
allow container_t tun_tap_device_t:chr_file { getattr ioctl open read write };
node_exporter: a type of its own
node_exporter reads the host’s mount table and asks systemd about unit states. Most guides run it with SecurityLabelDisable=true (unconfined), and I did that too at first. I replaced it with a confined type of its own, written in CIL. The base is the container template from udica (it ships with container-selinux), plus the few extra rules that node_exporter needs:
(block node_exporter
(blockinherit container)
(blockinherit net_container)
; filesystem collector: init's mount table, statfs of every mount point
(allow process init_t (dir (search getattr)))
(allow process init_t (file (open read getattr)))
(allow process init_t (lnk_file (read)))
(allow process filesystem_type (filesystem (getattr)))
(allow process mountpoint (dir (search getattr)))
; selinux collector: current mode
(allow process security_t (dir (search getattr)))
(allow process security_t (file (open read getattr)))
; systemd collector: read-only queries to systemd over the system bus
(allow process system_dbusd_var_run_t (dir (search)))
(allow process system_dbusd_var_run_t (sock_file (write getattr open)))
(allow process system_dbusd_t (unix_stream_socket (connectto)))
(allow process system_dbusd_t (dbus (send_msg)))
(allow system_dbusd_t process (dbus (send_msg)))
(allow process init_t (dbus (send_msg)))
(allow init_t process (dbus (send_msg)))
(allow process init_t (system (status)))
(allow process systemd_unit_file_type (service (status)))
)
t=/usr/share/udica/templates
sudo semodule -i node_exporter.cil $t/base_container.cil $t/net_container.cil
The quadlet then uses the new type:
[Container]
Image=quay.io/prometheus/node-exporter:latest
Network=host
PodmanArgs=--pid=host
Volume=/:/host:ro,rslave
Volume=/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket:ro
SecurityLabelType=node_exporter.process
Debugging a denial
When a container fails with “permission denied” and the file permissions look fine, look at the audit log:
sudo ausearch -m avc -ts recent # the last 10 minutes
sudo ausearch -m avc -ts boot | audit2why # with an explanation
Some denials are hidden by dontaudit rules. If ausearch shows nothing but the problem stays, switch those rules off for a moment:
sudo semodule -DB # disable dontaudit, rebuild
# reproduce the problem, run ausearch again
sudo semodule -B # enable dontaudit again
audit2allow turns denials into rules:
sudo ausearch -m avc -ts recent | audit2allow -m mymodule
Read its output, but do not load it as it is. It allows exactly what was denied, for the whole source type. A container that tried to read /etc/shadow gets a rule that allows container_t to read shadow_t, for every container on the host. Often the right fix is something else: a :Z on the mount, a file context or a boolean. Write a module only when none of those fit. Keep it to the rules you understand.
Make it code
I keep the module sources in the repository (selinux/*.te, selinux/*.cil), never just the compiled .pp. If you have an old module and lost its source, you can extract it:
sudo semodule -c -E gluetun_policy # writes gluetun_policy.cil
My base Ansible role copies each source to /usr/local/share/selinux/. It builds and loads a module when the source changed, or when the module is missing:
- name: SELinux module sources
ansible.builtin.copy:
src: "{{ playbook_dir }}/../selinux/{{ item }}"
dest: /usr/local/share/selinux/{{ item }}
mode: "0644"
loop: "{{ selinux_modules }}"
register: semodule_src
- name: Installed SELinux modules
ansible.builtin.command: semodule -l
register: semodules
changed_when: false
check_mode: false
- name: Build and load SELinux modules
ansible.builtin.shell: |
set -e
cd /usr/local/share/selinux
name={{ item.item | regex_replace('\.(te|cil)$', '') }}
case {{ item.item }} in
*.te)
checkmodule -M -m -o "$name.mod" {{ item.item }}
semodule_package -o "$name.pp" -m "$name.mod"
semodule -i "$name.pp" ;;
*.cil)
t=/usr/share/udica/templates
semodule -i {{ item.item }} $t/base_container.cil $t/net_container.cil ;;
esac
loop: "{{ semodule_src.results }}"
loop_control:
label: "{{ item.item }}"
when: item.changed or (item.item | regex_replace('\.(te|cil)$', '')) not in semodules.stdout_lines
changed_when: true
Guard rails
Two checks keep me honest.
The base role stops the play if a host is not enforcing, and also if somebody made a domain permissive with semanage permissive -a:
- name: SELinux is enforcing (running and configured)
ansible.builtin.assert:
that:
- ansible_facts.selinux.status == 'enabled'
- ansible_facts.selinux.mode == 'enforcing'
- ansible_facts.selinux.config_mode == 'enforcing'
fail_msg: "SELinux on {{ inventory_hostname }}: {{ ansible_facts.selinux }}"
# semanage permissive -a stores a local permissive domain as module permissive_<type>
- name: No local permissive domains
ansible.builtin.command: semodule -l
register: semodule_list
changed_when: false
check_mode: false
failed_when: semodule_list.stdout_lines | select('match', '^permissive_') | list | length > 0
The role does not switch enforcing on by itself. A host that is not enforcing needs a look first (a relabel, old denials), so the play stops.
The app role refuses quadlets that disable confinement:
- name: "{{ app_name }}: containers stay confined"
ansible.builtin.assert:
that: lookup('ansible.builtin.file', item.src) is not search(unconfined)
fail_msg: "{{ item.path }} disables SELinux confinement; use a policy module in selinux/ instead"
loop: "{{ quadlet_files }}"
vars:
unconfined: '(?mi)^\s*(SecurityLabelDisable\s*=\s*true|SecurityLabelType\s*=\s*(spc_t|unconfined_t)|PodmanArgs\s*=.*(--privileged|label[=:]disable))'
With this check, “just disable SELinux for this one container” is no longer an option.
Check
getenforce # Enforcing
ps -eZ | grep -E 'container_t|node_exporter' # every container process confined
sudo ausearch -m avc -ts boot # ideally: <no matches>
Things to watch
- Restores.
resticrestores the SELinux label as an extended attribute when it was in the backup. A backup from a host without SELinux has no label, andresticreports errors when it tries to set one. Restore with--exclude-xattr security.selinux, then runrestoreconor let:Zrelabel at start. - Docker. My one Docker host (Mailcow) runs with SELinux too. Fedora’s Docker package already starts
dockerdwith--selinux-enabled, so do not add it todaemon.jsonas well. The daemon refuses to start when an option comes from both places.
Next: Ansible groundwork: inventory, bootstrap and a base role.
Further reading
- Getting started with SELinux (Fedora Quick Docs): the basics, if this post went too fast.
- container-selinux: the policy behind
container_tandcontainer_file_t, and its booleans. - podman-run(1): the
:zand:Zvolume options and the label settings of--security-opt. - udica: generates SELinux policies for containers. The source of the templates in this post.
- The SELinux Notebook: the deep reference: types, CIL, modules and how the kernel decides.