Another Trip to Drive Geometry Hell

Recently I took another close look at the IDE.DSK driver in NetWare 3.12. Among other things, I wanted to know how it differs from ISADISK.DSK. On some systems, the two drivers are interchangeable and either will work. But there are also systems that only one or the other driver can handle.

The ISADISK.DSK driver should really be called ATDISK.DSK because it’s written to the PC/AT fixed disk programming interface. As such, it will work with ST506 style MFM/RLL drives attached to an AT-compatible controller. It will also work with ESDI drives attached to an AT-compatible controller (such as the WD1005A or WD1007V). And it will also work with IDE drives.

However, ISADISK.DSK will only work with up to two drives supported by the system’s BIOS. ISADISK.DSK looks at the drive type information in the CMOS and relies on the FDPT (Fixed Disk Parameter Table pointed by interrupt vectors 41h/46h) to query geometry information. Said geometry is used internally and also fed to the INITIALIZE DRIVE PARAMETERS command.

For that reason, ISADISK.DSK will work with IDE drives as long as the FDPT geometry is compatible with IDE, i.e. does not have more than 16 heads; the BIOS limit on sectors per track, 63, is lower than IDE’s 255, so that won’t be a problem.

The IDE.DSK driver only works with IDE disks, more or less. More specifically, it needs a working IDENTIFY DRIVE command, which was also available on some ESDI controllers.

Unlike ISADISK.DSK, IDE.DSK does not directly rely on BIOS support at all, and does not look at the CMOS or FDPT information. For that reason, IDE.DSK can happily use drives on secondary, tertiary, or quaternary IDE interfaces (and has been able to do so since January 1992, if not earlier).

IDE.DSK (at least in NetWare 3.12) does not use LBA at all. At least for disks managed purely by NetWare, it is limited to about 8 GB because that is the highest capacity that ATA drives are allowed to report through the “compatible” geometry words (i.e. words 1, 3, and 6) of IDENTIFY DRIVE data (16383/16/63 is the largest compatible CHS geometry).

Note that especially newer IDE.DSK versions may be able to use larger disks, but that is not guaranteed. The trouble is that ATA drives can theoretically support up to about 128 GB through CHS addressing, but don’t have to. In fact ATA drives larger than circa 8 GB need not support CHS addressing at all.

The IDE.DSK NLM also has a /B switch which turns on multi-sector reads and writes on drives which support those commands. I could not find any documentation for the /B switch. I don’t believe the IDE.DSK version in NetWare 3.12 will ever default to multi-sector transfers, although it’s possible that some other version does.

So where’s the problem? The problem is, of course, booting from a disk managed by IDE.DSK. The BIOS must by definition be able to read from the disk. To do that, the disk must be partitioned and the partition table has disk geometry baked in. Notably, the geometry used by the BIOS is often not the same as the geometry reported by IDENTIFY DRIVE, even on drives that do not require translation.

IDE.DSK looks at each drive’s MBR and attempts to derive drive geometry from it. Specifically it uses the ending head and sector of the first non-empty partition. Now, there are two problems with this approach.

Customs and Traditions

NetWare’s IDE.DSK assumes that partitions end on a cylinder boundary. If they do, this approach will reliably discover drive geometry because the last head and sector of a partition will exactly correspond to the drive’s CHS geometry.

There is of course no guarantee that the partition really ends on a cylinder boundary. This was one of those things that traditionally disk partitioning software used to do, but there was never any real specification, let alone enforcement of this rule. Over time, the rule stopped being applied entirely, especially as disk cylinders grew to several megabytes in size.

IDE.DSK doesn’t even try, but it would have no reliable way to validate whether partitions end on cylinder boundaries or not.

This issue probably wasn’t a huge problem in practice because NetWare servers very rarely dual-booted to another OS. The partition tables would almost always be created by software shipped by Novell or by an old version of DOS, and those did create cylinder-aligned partitions.

Big Disks Go Boom

The other problem is related to disks where BIOS geometry does not match IDE geometry. NetWare was one of the big reasons why mid-1990s BIOSes usually had a way to configure the disk geometry translation type.

At least in NetWare 3.12, IDE.DSK would break down spectacularly if the geometry obtained from the MBR had more than 16 heads, as it almost always would with larger drives and translating BIOSes.

Again, the way to get around it was to make sure that the translation used by the BIOS didn’t use more than 16 heads.

But WHY?!

Why the heck, do you ask, did Novell ever do this? The answer is obvious… because they had to. In the early 1990s, Novell was in a situation where translating BIOSes did not yet exist, there was no EDD specification or anything like that, but there were big IDE disks on the market. Novell customers would have been obvious buyers of such drives, and Novell needed to make those drives work.

For secondary IDE drives that only NetWare needed to work with, there was never any problem. NetWare could manage the geometry any way it liked and it didn’t clash with anything else.

The problem was always sharing a drive between NetWare and the BIOS. Before BIOSes learned how to translate, there was not much of a problem either—the BIOS could not access the full drive capacity (it was cut off at 1,024 cylinders) but it typically only needed to access a small DOS partition at the beginning of the drive. NetWare could work with well over 10,000 cylinders and as long as the number of heads and sectors per track matched, everything was happy.

The trouble only really started with translating BIOSes because then it was more or less guaranteed that the boot partition used a geometry that was not compatible with IDE. It should be noted that NetWare 3.x was able to boot from floppy, which would have worked around problems with clashing disk geometries, but floppies had their own drawbacks.

Early on, high capacity disks were almost exclusively SCSI. That started changing in the early 1990s, and even non-huge drives started running into problems. The Conner CP-3544 (Summit) is an example of an early 544 MB IDE drive that was available in 1991 and already bumped against geometry limitations. It reported a geometry with 16 heads and 63 sectors per track (the maximum usable for both BIOS and ATA) and 1,053 cylinders. NetWare was able to get at the thirty or so cylinders (almost 15 MB) that BIOS could not access. By 1993, IDE disks with well over 500 MB capacity were readily available.

It is unclear when exactly IDE.DSK first shipped but it was no later than January 1992. At that time, translating BIOSes did not exist, and certainly NetWare customers didn’t have them, so Novell had to forge its own path. That path made sense for NetWare, but it wasn’t what the rest of the industry chose to do later on, when drives larger than 500 MB were a reality but they still had to work with DOS, Windows 3.x, or Windows 95 without losing capacity.

The result was a mess where certain operating system required one type of translation, while other systems required a different one. And system administrators spent countless hours cursing the problems this caused.