Linux Lunatix

SELinux enforcing with containers

Stefan Marx
| 8 minutes

Part 4 of the series that starts with Four servers, three distributions: why I consolidated.

Every one of my hosts runs SELinux in enforcing mode. Every container runs in the confined type container_t, with no exceptions. This post shows the few tools that make this practical: volume labels, file contexts, booleans and small policy modules. It also shows how to debug a denial.

Why bother

A container that breaks out of its namespaces lands in a process on the host. With rootless Podman that process runs as an unprivileged user. With SELinux it is also confined to container_t. That type may read and write only files labelled container_file_t, and each container gets its own MCS category pair (for example s0:c123,c456). So one container cannot read another container’s files, even when both belong to the same user.

The cost is small once you know the patterns. Most of the time, the right volume flag is all you need.

Volume labels: :Z and :z

When you mount a host directory into a container, its files usually have a type such as user_home_t. A confined container may not read that. Podman can relabel the files at start:

FlagLabelUse for
:Zcontainer_file_t with this container’s private categoriesdata of one container
:zcontainer_file_t without categories (shared)files that several containers mount
# Only this container uses it
Volume=%h/.config/ntfy/server.yml:/etc/ntfy/server.yml:ro,Z
Volume=ntfy-data:/var/lib/ntfy
# Three website containers mount the same file
Volume=%h/.config/nginx/realip.conf:/etc/nginx/conf.d/realip.conf:ro,z

Named Podman volumes get the right label when Podman creates them. The :Z on a volume does no harm and makes the intent clear.

Two warnings:

Directories outside the home: file contexts

Some files are not mounted from a home directory. For example, node_exporter reads text files with metrics from /var/lib/node_exporter/textfile, and my backup script writes them. Instead of relabelling the directory at every start, make the label permanent with a file context rule:

sudo semanage fcontext -a -t container_file_t '/var/lib/node_exporter(/.*)?'
sudo restorecon -Rv /var/lib/node_exporter

restorecon applies the rule now. The rule also survives a full relabel of the filesystem. With Ansible this becomes two tasks:

- name: SELinux label for the textfile directory
  community.general.sefcontext:
    target: /var/lib/node_exporter(/.*)?
    setype: container_file_t
    state: present
  register: textfile_fcontext

- name: Relabel the textfile directory
  ansible.builtin.command: restorecon -R /var/lib/node_exporter
  when: textfile_fcontext.changed and not ansible_check_mode
  changed_when: true

Booleans

The policy has switches for common cases. List them with getsebool -a | grep container. One example: container_use_devices lets containers use host devices. I keep the booleans per host in the inventory:

# host_vars/<host>.yml
selinux_booleans:
  container_use_devices: true
- name: SELinux booleans
  ansible.posix.seboolean:
    name: "{{ item.key }}"
    state: "{{ item.value }}"
    persistent: true
  loop: "{{ selinux_booleans | dict2items }}"

Prefer a boolean over a policy module when one exists. Somebody else maintains it, and it is easy to see what it allows.

Small policy modules for the rest

Two of my cases have no boolean.

Traefik and the Podman socket

Traefik reads container labels through the rootless Podman socket. The socket file can be relabelled with :z. But connecting to the socket means talking to the process behind it, and that process runs as container_runtime_t. container_t may not do that. The denial in the audit log looks like this:

type=AVC msg=audit(...): avc:  denied  { connectto } for  pid=... comm="traefik"
  path="/run/user/1001/podman/podman.sock" scontext=system_u:system_r:container_t:s0:c12,c345
  tcontext=unconfined_u:unconfined_r:container_runtime_t:s0 tclass=unix_stream_socket permissive=0

The fix is one rule:

module traefik_podman_socket 1.0;

require {
        type container_t;
        type container_runtime_t;
        class unix_stream_socket connectto;
}

allow container_t container_runtime_t:unix_stream_socket connectto;

Build and load it:

checkmodule -M -m -o traefik_podman_socket.mod traefik_podman_socket.te
semodule_package -o traefik_podman_socket.pp -m traefik_podman_socket.mod
sudo semodule -i traefik_podman_socket.pp

This rule applies to every container_t process, not only to Traefik. A container still needs the socket mounted to use it, and only Traefik and my Podman exporter get that mount. Think about this trade-off before you load such a rule. Access to the Podman socket is close to access to the user’s account.

A VPN container (gluetun) needs the same kind of module for /dev/net/tun:

allow container_t tun_tap_device_t:chr_file { getattr ioctl open read write };

node_exporter: a type of its own

node_exporter reads the host’s mount table and asks systemd about unit states. Most guides run it with SecurityLabelDisable=true (unconfined), and I did that too at first. I replaced it with a confined type of its own, written in CIL. The base is the container template from udica (it ships with container-selinux), plus the few extra rules that node_exporter needs:

(block node_exporter
        (blockinherit container)
        (blockinherit net_container)

        ; filesystem collector: init's mount table, statfs of every mount point
        (allow process init_t (dir (search getattr)))
        (allow process init_t (file (open read getattr)))
        (allow process init_t (lnk_file (read)))
        (allow process filesystem_type (filesystem (getattr)))
        (allow process mountpoint (dir (search getattr)))

        ; selinux collector: current mode
        (allow process security_t (dir (search getattr)))
        (allow process security_t (file (open read getattr)))

        ; systemd collector: read-only queries to systemd over the system bus
        (allow process system_dbusd_var_run_t (dir (search)))
        (allow process system_dbusd_var_run_t (sock_file (write getattr open)))
        (allow process system_dbusd_t (unix_stream_socket (connectto)))
        (allow process system_dbusd_t (dbus (send_msg)))
        (allow system_dbusd_t process (dbus (send_msg)))
        (allow process init_t (dbus (send_msg)))
        (allow init_t process (dbus (send_msg)))
        (allow process init_t (system (status)))
        (allow process systemd_unit_file_type (service (status)))
)
t=/usr/share/udica/templates
sudo semodule -i node_exporter.cil $t/base_container.cil $t/net_container.cil

The quadlet then uses the new type:

[Container]
Image=quay.io/prometheus/node-exporter:latest
Network=host
PodmanArgs=--pid=host
Volume=/:/host:ro,rslave
Volume=/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket:ro
SecurityLabelType=node_exporter.process

Debugging a denial

When a container fails with “permission denied” and the file permissions look fine, look at the audit log:

sudo ausearch -m avc -ts recent           # the last 10 minutes
sudo ausearch -m avc -ts boot | audit2why # with an explanation

Some denials are hidden by dontaudit rules. If ausearch shows nothing but the problem stays, switch those rules off for a moment:

sudo semodule -DB      # disable dontaudit, rebuild
# reproduce the problem, run ausearch again
sudo semodule -B       # enable dontaudit again

audit2allow turns denials into rules:

sudo ausearch -m avc -ts recent | audit2allow -m mymodule

Read its output, but do not load it as it is. It allows exactly what was denied, for the whole source type. A container that tried to read /etc/shadow gets a rule that allows container_t to read shadow_t, for every container on the host. Often the right fix is something else: a :Z on the mount, a file context or a boolean. Write a module only when none of those fit. Keep it to the rules you understand.

Make it code

I keep the module sources in the repository (selinux/*.te, selinux/*.cil), never just the compiled .pp. If you have an old module and lost its source, you can extract it:

sudo semodule -c -E gluetun_policy     # writes gluetun_policy.cil

My base Ansible role copies each source to /usr/local/share/selinux/. It builds and loads a module when the source changed, or when the module is missing:

- name: SELinux module sources
  ansible.builtin.copy:
    src: "{{ playbook_dir }}/../selinux/{{ item }}"
    dest: /usr/local/share/selinux/{{ item }}
    mode: "0644"
  loop: "{{ selinux_modules }}"
  register: semodule_src

- name: Installed SELinux modules
  ansible.builtin.command: semodule -l
  register: semodules
  changed_when: false
  check_mode: false

- name: Build and load SELinux modules
  ansible.builtin.shell: |
    set -e
    cd /usr/local/share/selinux
    name={{ item.item | regex_replace('\.(te|cil)$', '') }}
    case {{ item.item }} in
      *.te)
        checkmodule -M -m -o "$name.mod" {{ item.item }}
        semodule_package -o "$name.pp" -m "$name.mod"
        semodule -i "$name.pp" ;;
      *.cil)
        t=/usr/share/udica/templates
        semodule -i {{ item.item }} $t/base_container.cil $t/net_container.cil ;;
    esac
  loop: "{{ semodule_src.results }}"
  loop_control:
    label: "{{ item.item }}"
  when: item.changed or (item.item | regex_replace('\.(te|cil)$', '')) not in semodules.stdout_lines
  changed_when: true

Guard rails

Two checks keep me honest.

The base role stops the play if a host is not enforcing, and also if somebody made a domain permissive with semanage permissive -a:

- name: SELinux is enforcing (running and configured)
  ansible.builtin.assert:
    that:
      - ansible_facts.selinux.status == 'enabled'
      - ansible_facts.selinux.mode == 'enforcing'
      - ansible_facts.selinux.config_mode == 'enforcing'
    fail_msg: "SELinux on {{ inventory_hostname }}: {{ ansible_facts.selinux }}"

# semanage permissive -a stores a local permissive domain as module permissive_<type>
- name: No local permissive domains
  ansible.builtin.command: semodule -l
  register: semodule_list
  changed_when: false
  check_mode: false
  failed_when: semodule_list.stdout_lines | select('match', '^permissive_') | list | length > 0

The role does not switch enforcing on by itself. A host that is not enforcing needs a look first (a relabel, old denials), so the play stops.

The app role refuses quadlets that disable confinement:

- name: "{{ app_name }}: containers stay confined"
  ansible.builtin.assert:
    that: lookup('ansible.builtin.file', item.src) is not search(unconfined)
    fail_msg: "{{ item.path }} disables SELinux confinement; use a policy module in selinux/ instead"
  loop: "{{ quadlet_files }}"
  vars:
    unconfined: '(?mi)^\s*(SecurityLabelDisable\s*=\s*true|SecurityLabelType\s*=\s*(spc_t|unconfined_t)|PodmanArgs\s*=.*(--privileged|label[=:]disable))'

With this check, “just disable SELinux for this one container” is no longer an option.

Check

getenforce                                   # Enforcing
ps -eZ | grep -E 'container_t|node_exporter' # every container process confined
sudo ausearch -m avc -ts boot                # ideally: <no matches>

Things to watch

Next: Ansible groundwork: inventory, bootstrap and a base role.

Further reading

Previous: Traefik in front of rootless containers Next: Ansible groundwork: inventory, bootstrap and a base role

Related content

One generic Ansible role for every app
Traefik in front of rootless containers
From Docker Compose to rootless Podman quadlets