How the PC Industry Screws Things Up

Posted on April 7, 2017 by Michal Necasek

I was recently involved in investigating a problem that turns out to be a complete SNAFU which nicely illustrates the chaos that is the PC platform. It’s about the NX/XD bit. Let’s start with a bit of history.

It turns out to be useful, and in fact required for improved security, to separate code and data. On the 8086, no such separation existed. On the 80286, protected-mode segmentation clearly distinguished between code and data segments. Writing to a code segment required creating a data-segment alias. The same segmentation rules carried over to 32-bit protected mode on the 80386, but lost any meaning in flat-model operating systems (OS/2 2.0, Windows NT, 32-bit UNIX). Because the data (DS) and code (CS) segments mapped the entire memory, any data page could be written to and executed. In the age of Internet, that turned out to be a terrible idea because any buffer overflow on the stack could be immediately exploited to inject and execute code in a vulnerable process.

In 2003, AMD introduced the NX (No-eXecute) bit as part of the AMD64 architecture. When enabled in the EFER MSR, 64-bit and PAE page tables support an additional bit which makes a page non-executable. As a result, it is again possible to separate between code and data, and make data pages non-executable. That makes exploits harder (far from impossible!) because there is no longer a ready supply of writable and executable pages and most importantly, the stack is no longer executable.

Microsoft saw NX as a Good Thing and added support for it in a Windows XP SP2 (2004), calling the feature DEP, or Data Execution Prevention. This was initially an opt-in feature because certain classes of legitimate software did not bother distinguishing between data and code and crashed with DEP enabled (typically software which generated code on the fly in one manner or another).

Microsoft presumably made it known to Intel that NX was a desirable feature. Intel had to support NX in all AMD64-compatible CPUs, but also added NX support to late-model 32-bit Prescott Pentium 4 CPUs, and even the later Pentium M processors.

Intel of course couldn’t leave well enough alone and called the bit XD (eXecute Disable), and more importantly, also added a way to completely disable the NX through the IA32_MISC_ENABLE (1A0h) MSR. That’s where things started going south. Continue reading →

Posted in AMD, Intel, Microsoft | 37 Comments

OS/2 2.0, Summer ’91 Edition

Posted on April 5, 2017 by Michal Necasek

In a fascinating example of poor timing, disk images of OS/2 2.0 pre-release level 6.605 from July/September 1991 were missing for over 25 years, only to show up literally one day after after the 25th anniversary of the OS/2 2.0 release (big thanks to a very helpful reader!).

Install disk of OS/2 2.0 6.605

Let’s go back to the odd numbering for a moment. The build “levels” of OS/2 2.0 started apparently at 6.00 and went all the way up to 6.307 for the GA release (though the 6.2xx range may have been skipped entirely). The 6.605 pre-release from September 1991 became available between 6.149 (July) and 6.167 (October). It is not known what possessed IBM to assign it a completely out-of-sequence number. It is known that 6.605 is very close to 6.149 but contains additional fixes to work better with LAN Server (LS) and Extended Services (ES) pre-releases.

For reference, OS/2 1.0 used internal revisions 3.xx, version 1.1 used internal revisions 4.xx, and OS/2 1.2 used 12.xx. The fact that OS/2 2.0 used the 6.xx range probably reflects the fact that its development started after OS/2 1.1 but before 1.2.

Even though the 6.605 disks are clearly labeled 7/91 (July), they were just as clearly finalized in September 1991. It is possible that 6.605 was somewhat unplanned because the next level (6.167, or OS/2 2.0 LA) was the first to contain the Workplace Shell (WPS), and putting that together may easily have taken longer than anticipated.

The upshot is that 6.605 was in fact the last OS/2 2.0 pre-release which didn’t use the WPS and instead utilized the same old Desktop Manager as OS/2 1.2/1.3, which makes it the closest surviving relative of the MS OS/2 2.0 SDK:

Desktop in OS/2 level 6.605

The look and feel was not quite the same as OS/2 1.3 and in fact was a cross between OS/2 1.3 and Windows 3.1.

Continue reading →
Posted in 386, IBM, OS/2, PC history | 37 Comments

OS/2 2.0 at 25

Posted on March 31, 2017 by Michal Necasek

Twenty-five years ago, on March 31st, 1992, IBM released OS/2 2.0, the first mass-market 32-bit PC operating system. The road to OS/2 2.0 was quite long and winding, and the OS was a proud member of the vaporware club (just like, say, Windows NT or Windows 95).

Talk of 32-bit OS/2 goes almost as far back as OS/2 itself. In 1987, Gordon Letwin wrote about OS/2-386 in Inside OS/2 and it is clear that future 32-bit extension was a consideration when OS/2 1.0 was on the drawing board. And we now know that in late 1986, Microsoft already started experimenting with using the 80386’s V86 mode to run Virtual DOS Machines (VDMs) on top of OS/2.

1991 OS/2 2.0 6.167 (left) and 1992 GA (right)

Microsoft launched actual development of OS/2 2.0 probably in 1988, the same year development of NT OS/2 (later known as Windows NT) started. The goal was to provide a PC operating system which is upwardly compatible with OS/2 1.x, supports native 32-bit applications, and supports VDMs; the latter was especially important in light of the fact that compatibility with existing DOS applications was a weak point of OS/2 1.x.

The entire history of OS/2 2.0 development is one of missed deadlines and broken promises. In July 1988, OS/2 2.0 was supposed to be released in early 1989. In February 1989, an OS/2 2.0 SDK was supposed to be released by the end of the year and the finished product in 1990. Continue reading →

Posted in 386, IBM, Microsoft, OS/2 | 34 Comments

Something New Every Day…

Posted on March 26, 2017 by Michal Necasek

Apparently in 1990, Intel still sold complete systems, and they were PC compatible at that point. And perhaps not surprisingly, Intel also OEMed MS OS/2 1.21:

Intel OEM OS/2 1.21 Boot

And there was this cute ASCII-art logo:

Intel OEM OS/2 1.21 Installer Logo

There very little customization that Intel did, the biggest difference from standard MS OS/2 1.21 was that Intel shipped additional DISK01.SYS driver which was in fact standard Adaptec ASW-1420 driver for 154x HBAs (actually 1x4x, presumably also covering EISA if not MCA). That supported the Adaptec 1542 SCSI controller used in the Intel 303 (386/33) tower system that these MS OS/2 disks came with.

Big thanks to a helpful reader.

Posted in Intel, OS/2 | 14 Comments

There’s More to the 286 XENIX Story

Posted on March 22, 2017 by Michal Necasek

It turns out that there is a rather interesting story behind the 286 XENIX incompatibility with 386 and later processors. Here’s roughly what happened in chronological order.

In 1982, Intel released the iAPX 286 processor, later known as the 80286 or simply 286. This was a highly complex CPU for the time, with an intricate system of software privilege and security enforcement known as protected mode. The 286 used descriptor tables with 48 bits (6 bytes) of information, but in order to keep the design simple, full 64 bits were taken up for each descriptor; the last 16 bits were not used by the CPU and were documented as reserved.

Later in 1982, work was also underway on the 386, Intel’s 32-bit successor to the 286.

In late 1982 or in 1983, Microsoft ported XENIX (a variant of UNIX System III licensed from AT&T) to the Intel 286 processor. Intel and IBM were among companies which licensed the 286 XENIX port from Microsoft. Although Intel had the XENIX source code, the porting was done entirely by Microsoft. Developers at Microsoft decided to store bookkeeping information in the “reserved” 16 bits of a descriptor table entry not used by the 286. This approach reliably worked on 286 processors. Continue reading →

Posted in 286, 386, Intel, Microsoft | 43 Comments

PCI Corner Case

Posted on March 20, 2017 by Michal Necasek

Last weekend I had the pleasure of debugging a curious case of older PCI configuration code (circa 2005) failing on newer (post-2010) hardware. The code was well tested on many 1990s and 2000s PCs (and some non-PCs) with PCI/AGP/PCIe and never showed problems. But at least on some newer machines (in my case a Sandy Bridge laptop) it just hung.

I quickly found out that code meant to walk over a PCI capability list got stuck in a loop. But the reason why it got stuck was unexpected. For reasons lost to the mists of time, the code reads the next capability pointer and the capability type (total of 16 bits of information) as two bytes, not one word. That was not the problem though. The problem was that the code always read a full DWORD from the config space and discarded the top bits. On the problematic system, that worked as long as the read was DWORD-aligned, but not otherwise. The code logic may seem unusual but it worked perfectly well on so many older systems, so why not anymore? And who violated the PCI specification, the code or (unlikely) the hardware? Continue reading →

Posted in Bugs, Intel, PCI | 11 Comments

IBM XENIX: Two Steps Forward

Posted on March 16, 2017 by Michal Necasek

There are reasons to revisit an old topic. Very old, considering that IBM Xenix 1.0 was released in 1984, well over 30 years ago. To recap, this version of Xenix is unique in that it runs only on 286 processors. It requires a 286 but is incompatible with 386 and later processors due to the use of a reserved word in GDT/LDT descriptors. Good times. And there is a story related to the Xenix 386 incompatibility, but that’s something for another post.

Starting the IBM XENIX installer

Last time the OS/2 Museum attacked IBM Xenix, the incompatibility with any 32-bit x86 processors was a big hurdle. In the meantime, VirtualBox learned to emulate a 286 processor (version 5.1.16 or later). The emulation is far from perfect but it is good enough to run IBM Xenix 1.0 with only slight patching. The required modifications concern incompatibility of IBM Xenix 1.0 with post-PC/AT hardware, namely EGA/VGA graphics and hard disks bigger than 20 or 30 MB. Both issues had been explored 30+ years ago by contemporary Xenix users. Continue reading →

Posted in 286, IBM, VirtualBox, Xenix | 22 Comments

Tales From the Xenix Crypt

Posted on March 13, 2017 by Michal Necasek

What does Xenix have to do with the Enigma machine? Perhaps surprisingly, there is a clear connection…

When reconstructing 386 Xenix 2.2.3, the libmdep.a library proved to be a particularly tough nut to crack. That is because it’s one of the three files in a standard Xenix install (/xenix, /etc/getty, /usr/sys/mdep/libmdep.a) which need to be “branded”. Let’s back up a bit and describe the SCO Xenix copy protection.

The Xenix disks did not use any kind of physical copy protection. The media could be backed up and copied without any technical difficulties. The copy protection consisted of a serial number and an activation key which needed to be used in order to “brand” a Xenix installation before it could be used. The branding process wrote a scrambled copy of the serial number into the files, and for files other than the kernel, also decrypted the data which was shipped encrypted on the installation disks.

The installation disks were no good without the serial number and activation key, and if files installed on hard disk were copied, they would contain the serial number and could thus be traced to the source. The mechanism was just complex enough that it was not trivial to defeat, although from a modern perspective the cryptography used was laughably weak.

The Xenix kernel was special in that it was branded but not encrypted, but without a valid serial number severely restricted the number of concurrent processes. It was just good enough to install the OS from floppies but not enough to run a multi-user system.

Now back to libmdep.a. On the restored disks, one sector near the beginning of the library was missing. It turned out that reconstructing the data was not that hard: The missing 512 bytes covered part of the symbol directory and part of the OMF header of the first object file. The OMF header was recreated by copying it from another file and tweaking slightly, and ar had no trouble recreating the symbol directory. So we had a working libmdep.a but we needed an encrypted library which the installation process can decrypt.

Now, Xenix comes with /etc/brand which does what the name suggests. It writes the serial number into files and decrypts files which are shipped encrypted. But when renamed to debrand, the exact same binary turns around and encrypts files (as it happens, the encryption is symmetric). So obviously the debrand utility can be used to encrypt the reconstructed library for placement on the installation media. But just as obviously, debrand needs some kind of key or password, and that is not the serial number or activation key. Rather, it is a separate 3-character code. Which we don’t have. Continue reading →

Posted in SCO, Software Hacks, Xenix | 17 Comments

What a Coincidence

Posted on March 11, 2017 by Michal Necasek

Timeline:

  • Mid-1988: SCO 386 Xenix 2.2.3 is released
  • 1996 (probably): Someone dumps the 386 Xenix 2.2.3 disks, but enough sectors are missing that the OS is impossible to install
  • January 3rd, 2013: OS/2 Museum posts about the damaged Xenix dump
  • February 19th, 2017: Michael Casadevall asks about restoring 386 Xenix 2.2.3
  • Late February 2017: Michael and Michal successfully restore 386 Xenix 2.2.3
  • March 3rd, 2017: Someone posts on Reddit about rediscovered 386 Xenix 2.2 disks
  • March 5th, 2017: The same person (almost certainly) uploads the disks on archive.org
  • March 6th, 2017: Michael publishes the first of a series of articles about the restoration
  • March 9th: Michal publishes an article which mentions the restoration
  • March 10th: John Elliott points out the fresh disk dump on archive.org.

Almost 30 years after Xenix 2.2.3 was released, it’s really hard to believe this is a coincidence. But it is, because until March 10th Michael & Michal knew nothing of the dump on archive.org, and the person uploading the dump couldn’t have known about the restoration project because it was first publicly mentioned a day after the new disk images were uploaded to archive.org.

To be clear, the archive.org dump is of Xenix 386PS, that is Xenix for PS/2 systems. The Xenix release being restored is 386AT, that is Xenix for PC/AT compatible 386 systems. So the disks aren’t identical, but they are really close. Before embarking on the restoration project, we of course checked what other Xenix releases were available. 386 Xenix 2.2.x was not… until a few days ago. Incredible.

Posted in PC history, Xenix | 1 Comment

Oldest Surviving 386 PC OS?

Posted on March 9, 2017 by Michal Necasek

Four years ago, the Xenix 2.2.3 mystery cropped up (twice). The issue has been revisited and thanks to Michael Casadevall, an enthusiastic reader, came to a happy conclusion:

386 Xenix 2.2.3c booting up

386 Xenix version 2.2.3 was finalized in late June 1988 (the newest files are timestamped June 27, 1988) and is thus one of the oldest surviving 386 PC operating systems.To put it in perspective, it was released at about the same time as IBM DOS 4.0 and several months before OS/2 1.1.

It is not the oldest since 386 Xenix 2.2.2 was released sometime in late 1987 (more or less coinciding with the release of 16-bit OS/2 1.0!), and there were possibly 386 Unixes from Microport and/or ISC available in mid to late 1987, though evidence is lacking.

At any rate, the search is for the oldest 386 PC OS, that is an operating system which runs on 386 PC compatibles, has at least substantial amount of 32-bit code in its core, is a self-contained system (i.e. no DOS extenders), can run 32-bit applications, and ideally uses paging. 386 Xenix checks all those boxes. Continue reading →

Posted in 386, Microsoft, SCO, UNIX, Xenix | 24 Comments