Skip to main content

Security & Protection: The Fortress in the Kernel

In the modern interconnected world, the OS is the final line of defense. Security is not just a feature; it is the fundamental property that ensures isolation between users, protects the kernel from applications, and safeguards data integrity.

This chapter explores the mechanisms used to enforce access control, the reality of hardware-level vulnerabilities, and the modern tools used to build secure, sandboxed environments.

1. The Multi-Layered Security Model​

1.1 Authentication vs. Authorization​

  • Authentication: Verifying who the user is (Passwords, SSH keys, Biometrics).
  • Authorization: Verifying what the user is allowed to do (Permissions, ACLs).

1.2 The Principle of Least Privilege​

The foundational rule of security: A process should have only the minimum set of permissions required to perform its task, and only for the minimum duration needed.

2. Access Control Mechanisms​

2.1 DAC (Discretionary Access Control)​

The traditional Unix model (UID, GID).

  • The Owner: The user who owns the file decides who can read, write, or execute it.
  • Weakness: If a user's account is compromised, the attacker has full control over all that user's files. It cannot prevent "accidental" misuse by the owner.

2.2 MAC (Mandatory Access Control)​

A centralized authority (the OS Security Policy) determines access. Users cannot override these rules.

  • SELinux (Security-Enhanced Linux): Uses "Labels" for every file, process, and socket. Policies are based on Type Enforcement. Even the root user can be restricted.
  • AppArmor: Uses "Profiles" based on file paths. It is easier to configure but less granular than SELinux.

2.3 RBAC (Role-Based Access Control)​

Permissions are assigned to Roles, and users are assigned to roles. This simplifies management in large organizations.

3. Sandboxing and Isolation Technologies​

How do we run untrusted code safely?

3.1 Linux Capabilities​

Traditionally, a process was either root (UID 0) or user. This was too "all-or-nothing."

  • Capabilities: Linux breaks down root privileges into 40+ discrete units (e.g., CAP_NET_BIND_SERVICE allows a process to bind to port 80 without full root access).

3.2 seccomp-bpf (Secure Computing)​

The most powerful tool for reducing the kernel's attack surface.

  • The Mechanism: A process tells the kernel "I will only use these 10 system calls. If I ever use anything else, kill me."
  • Use Case: Chrome/Firefox use seccomp to sandbox their renderer processes.

3.3 Landlock​

A new Linux LSM (Linux Security Module) that allows unprivileged processes to sandbox themselves by creating private "file system rules" (e.g., "I can only read from /tmp").

4. Hardware Security and Side-Channels​

4.1 Protection Rings and ASLR​

  • Rings: Hardware-enforced privilege levels (Ring 0 vs. Ring 3).
  • ASLR (Address Space Layout Randomization): Randomizes the memory locations of the stack, heap, and libraries every time an app starts, making "Buffer Overflow" and "Return-to-libc" attacks much harder.

4.2 Side-Channel Attacks: Meltdown and Spectre​

These attacks exploit the Performance Optimizations of the CPU rather than software bugs.

  • Meltdown: Exploits Out-of-Order execution to read kernel memory. Mitigated by KPTI (splitting the user and kernel page tables).
  • Spectre: Exploits Speculative Execution and Branch Prediction to leak data between processes. Mitigated by compiler changes (Retpolines) and microcode updates.

5. Cryptography in the Kernel​

The kernel provides high-performance cryptographic primitives to applications and itself.

  • Kernel Crypto API: A library of ciphers (AES, SHA) that can use hardware acceleration (like Intel AES-NI).
  • dm-crypt / LUKS: Provides Full Disk Encryption. Data is encrypted/decrypted on the fly at the block layer.
  • Kernel Key Retention Service: A secure "vault" inside the kernel to store passwords, keys, and tokens, preventing them from being leaked to user-space memory.

6. The Modern Threat Landscape​

6.1 Privilege Escalation​

An attacker starts as a low-privilege user and finds a bug in a SetUID binary or a kernel driver to gain root access.

  • Defense: Minimize SetUID binaries and use MAC (SELinux).

6.2 Supply Chain Attacks​

Malicious code is injected into a dependency (e.g., a NPM package or a C library).

  • Defense: Static Analysis, Software Bill of Materials (SBOM), and strict Sandboxing.

7. Security Auditing and Tools​

ToolFocusKey Concept
auditdLoggingTracks every system call, file access, and login
sealertSELinuxExplains why an action was blocked and how to fix it
getcap / setcapCapabilitiesInspects/Sets discrete process privileges
lynisAuditingA security scanner for Linux hardening
faillogAuthInspects failed login attempts

8. Summary Checklist​

  • Explain the difference between DAC and MAC.
  • Why is seccomp essential for modern browser security?
  • How does ASLR mitigate buffer overflow attacks?
  • What is a "Capability," and how does it improve security?
  • Explain how Meltdown allows an unprivileged user to read kernel memory.

End of Chapter 08. Continue to Chapter 09: Virtualization & Containers.