Why OS/2 is Hard to Virtualize

For a number of reasons, OS/2 is surprisingly difficult to virtualize, much more so than DOS, Windows 3.x, Windows NT, Windows 95, or many variants of UNIX. Most of these reasons have to do with OS/2’s architecture and history.

To begin with, old OS/2 versions (especially the versions older than Warp 4) are difficult to run even on physical Pentium III class hardware. There are various software timing loops which simply do not work on fast processors and cause division overflows and similar errors. This problem especially badly affects almost all versions of OS/2 1.x. Support for large IDE disks tends to be problematic, although that is not something to blame on OS/2 directly; the IDE interface was a moving target. Here virtualization in fact helps, because virtualizing, say, a 120 MB IDE disk is much easier than finding a real functioning disk with such a tiny capacity.

Then there are numerous problems specific to virtualization. Let’s start with the oldest OS/2 versions. IBM OS/2 1.0 and 1.1 uses 286-style switching from protected to real mode. This method resets the CPU and relies on the BIOS POST code to resume execution at a specified point; the reset is usually accomplished by triple faulting the CPU. Neither VMware, Virtual PC, nor VirtualBox support this method and treat the triple fault as a fatal error. Interestingly, the corresponding Microsoft OS/2 versions detect a 386 CPU and use the much more efficient method of modifying the CR0 control register.

OS/2 1.x versions also have an mildly unusual floppy driver which determines media capacity based on the disk controller data rate. 3½” 720 KB floppy media use 250 KB/sec while 1.44 MB media use 500 KB/sec data rate. Virtualization products usually do not bother emulating this minor detail and allow OS/2 to read from a virtual 1.44 MB diskette when the floppy controller is programmed for 250 KB/sec data rate. That confuses OS/2 into thinking it’s working with a 720 KB floppy, which results in interesting errors and inability to boot from said floppy. Needless to say, this makes OS/2 1.x difficult to install in a virtual machine.

With OS/2 2.x, things are a little different. The floppy driver is less tricky, switching from protected to real mode always uses 386 specific code, but overall the situation isn’t much better. OS/2 2.x still has problems with timing and may randomly lock up, especially very early in the boot. Because of the way OS/2 2.x mixes 16-bit and 32-bit protected mode, it is very sensitive to correct emulation of rarely-exercised x86 semantics. Software emulation often has subtle bugs which prevent OS/2 from functioning. Unassisted virtualization (e.g. raw mode in VirtualBox) typically isn’t possible because OS/2 utilizes much of the LDT and GDT, leaving a hypervisor no room to “hide” the part which exists in the guest’s address space.

Another problem is old style (DOS compatible) FPU exception handling. In non-SMP versions, OS/2 uses the 8086-compatible method of generating the FERR# signal and converting it to IRQ 13, rather than using #MF exceptions like all operating systems do nowadays. This mechanism needs to be correctly emulated for OS/2 to work.

Only the last versions of OS/2 (the Convenience Packs, MCP and ACP 1 and 2) are available as supported guests in some hypervisors (Virtual PC and VirtualBox). By that time, IBM sorted out the software timing loop problems which showed up on “too fast” Pentium 4 processors, and the hardware support is suitable for the devices emulated by virtualization products.

Especially with hardware-assisted virtualization (VT-x and AMD-V), OS/2 MCP and ACP run well as guest operating systems. Both Virtual PC and VirtualBox also offer guest additions for OS/2, further improving the virtualization experience. As always, it is a question of the money and effort thrown at a problem: almost anything can be solved if there is sufficient demand.