Skip to content

BlueHat IL 2026 - Alon Livne, Matan Mansheroff - Two Tales of hacking 90s MS-DOS Software

Alon LivneMatan Mansheroff

CommunityMicrosoft Israel R&D Center43 minAdvancedOpen on YouTube ↗

Analysts investigate attack surfaces and defenses concerning BlueHat IL 2026 - Alon Livne, Matan Mansheroff - Two Tales of hacking 90s MS-DOS Software.

Summary

### Overview At BlueHat IL, Alon Livne (CTO) and Matan Mansheroff (software expert) of Botanica Software Labs presented two real-world case studies in "software archaeology" involving 1990s MS-DOS software. Commissioned by paying clients, both engagements required understanding vintage computing architectures, memory models, runtimes, and packers to solve practical issues without over-engineering.

### Key Topics - **16-bit 8086 MS-DOS Architecture**: Operating in real mode with a flat address space, no memory protections (DEP, ASLR), and a 20-bit segmented memory bus (segment:offset) yielding a 640 KB user-accessible limit. - **Case Study 1: Qtext Hebrew Word Processor**: Reversing an encrypted document format in an early Hebrew word processor to recover access to forgotten password-protected files. - **Case Study 2: Saga Case Management**: Debugging a 1999 DOS-based legal software package impacted by a year rollover bug interpreting two-digit years (such as 26) as 1926 instead of 2026. - **Vintage Tooling & Archaeology**: Using older analysis tools (IDA Free 5.0, DOSBox-X debugger, vintage unpackers) and modern signatures to dissect legacy executables.

### Technical Details

#### Qtext Document Password Cracking - **Packing and Overlays**: The Qtext executable was packed as a self-extracting binary. The presenters recovered a vintage unpacker called `unp_lite` via web archives. Once unpacked, IDA prompted for Pascal overlays (`.OVR` file). Turbo Pascal bypassed the 640 KB memory limit via runtime swapping using `INT 3Fh` stubs, which fetched code overlays from disk and patched them into memory via far jumps. - **Debugging and Function Discovery**: Running under DOSBox-X's built-in debugger, the researchers hooked DOS system calls (`INT 21h`) on file open and read operations. Stepping through execution after file reading revealed a `decrypt_document` function. - **Cryptanalysis**: The document header contained a 16-byte variable key derived from a 4-character uppercase alphanumeric passcode. The key derivation function took 4 bytes, performed in-place sum mixing, concatenated the 4 bytes into 16 bytes, and mixed again. Rather than brute-forcing or analyzing all $3^{16}$ candidates under modulo 256 arithmetic (where division by 2 yields multiple candidates), the team exploited the internal 4-byte repeating structure and printable character constraints. This reduced the candidate space down to $2^7$ branches, allowing near-instant demixing on paper or via a simple script.

#### Saga Case Management Year 2026 Bug - **Binary Architecture**: The main 1.6 MB executable (`CTMENU`) was wrapped with `warp_wrap` from the Causeway 386 DOS Extender. Causeway launched from DOS real mode and transitioned into 32-bit protected mode using a proprietary `3P` executable format, enabling large memory support via VCPI/DPMI. - **Bypassing Tooling Overhead**: Reversing the custom 3P format in IDA Free 5.0 would have required building dedicated loader infrastructure. Instead, pressing `Alt+M` inside the running application exposed an internal debug/status window showing call stacks and settings for **Clipper** (a dBase III replacement). - **The Fix**: Clipper uses a `SET EPOCH` setting to define the pivot year for 2-digit dates. Searching for `1926` in little-endian within the binary located the hardcoded epoch value. Changing the epoch to `1990` via a single-nibble patch (`A` to `C`) resolved the bug, causing `1/1/26` to properly evaluate as a Thursday in 2026 instead of a Friday in 1926.

### Takeaways - Vintage 16-bit and DOS extender environments require understanding archaic paging (Turbo Pascal overlays) and protected-mode loaders (Causeway, DPMI). - Heavyweight reverse engineering is not always required: analyzing internal structural constraints in toy cryptography can drastically prune search spaces, and inspecting framework configurations or debug hooks can turn an arduous disassembly task into a single-nibble patch.

Related talks