Unix Underworld: Tales from the Dark Side of z/OS

Overview: Chad Rikansrud (Broadcom) and Philip Young (Soldier of Fortran, NetSPI) analyze the Unix subsystem within IBM mainframe operating systems, known as z/OS Unix System Services (USS). Bridging penetration testing and mainframe architecture, the speakers demystify z/OS internals for security practitioners. The talk covers native enumeration methods, filesystem permissions, and the Authorized Program Facility (APF), showing how attackers leverage standard Unix access to escalate privileges to supervisor state, Memory Key 0, and full RACF ESM compromise.

Key Topics: - Architecture of z/OS Unix System Services (USS) and external security managers (RACF, ACF2, Top Secret) - CPU operating modes (Problem vs. Supervisor State) and storage keys (Key 0 vs. Key 8) - Native enumeration tooling using Rexx, C shell, and memory control block extraction - Privilege escalation via APF authorization (extattr +a) and ACEE control block hijacking

Technical Details: Modern z/OS environments integrate Unix System Services (USS), a POSIX-compliant subsystem featuring hierarchical filesystems and SSH. Access control is governed by an External Security Manager (ESM)—predominantly IBM RACF, Broadcom ACF2, or Top Secret. At the CPU level, IBM z/Architecture executes in either problem state (unprivileged) or supervisor state (privileged). Memory isolation is enforced via 16 storage protection keys; standard workloads run in Key 8, whereas Key 0 grants unrestricted memory access across all address spaces.

For stealthy reconnaissance, the researchers created open-source tools: enum.rex (a Rexx script querying memory control blocks for RACF database locations and crypto settings without audit logs), OMVS_enum (a C shell enumeration script based on LinEnum), and zos_hog (a credential scanner). The authors highlight operational hurdles like ASCII-to-EBCDIC translation during SCP transfers and batch execution via JCL. Attackers frequently find world-writable (777) files like inetd.conf and weak legacy DES/MD5 password hashes crackable at billions of guesses per second compared to KDFAES.

The authors demonstrate privilege escalation starting from an unprivileged Unix shell. The operating system uses the Authorized Program Facility (APF) to restrict transitions into supervisor state, represented in USS by an extended attribute toggled via extattr +a. If a user holds READ access to RACF facility profile BPX.FILEATTR.APF, they can assign APF authorization to any compiled binary. When executed, an APF-authorized program issues MODESET KEY=ZERO,MODE=SUP (unauthorized binaries terminate with abend 047). In Key 0, the exploit issues the RACROUTE REQUEST=VERIFY macro for an account with RACF SPECIAL or OPERATIONS attributes, creating an elevated Accessor Environment Element (ACEE). Overwriting the active task's ACEE pointer in the ASXBSENV control block fully impersonates the administrator.

Key Takeaways: Mainframe security cannot rely on obscurity; z/OS USS exposes POSIX attack vectors interfacing directly with core mainframe subsystems. Organizations must rigorously protect RACF profiles—especially BPX.FILEATTR.APF—to prevent unauthorized APF binary creation. Security teams must audit Unix filesystem permissions to eliminate world-writable system files and plaintext credentials. Finally, mainframe telemetry must detect anomalous supervisor-state memory modifications and unauthorized ACEE pointer alterations.

(AI summary — Gemini Flash)