As most everyone knows, the AT Attachment standard (informally known as IDE) started by literally bolting the previously standalone AT disk controller onto a MFM drive with a ST506 interface and connecting the assembly to the host system with a 40-pin ribbon cable.
As it often happens, the standardization process was far behind the technology and the first ATA standard officially became ANSI standard X3.221-1994 in (obviously) 1994. The standard had been in the works for several years by that time, but even so the first ATA standard drafts appeared in mid-1990, when at least half a dozen vendors have been shipping ATA drives for some time.
From a software perspective, an IDE drive is almost indistinguishable from an ST506 drive plus an AT controller—in fact that was the whole point of IDE. One crucial difference is the IDENTIFY DRIVE command.
The IDENTIFY DRIVE command (soon renamed to IDENTIFY DEVICE, in order to reflect the broadened scope of ATA) makes it possible to build self-configuring systems, also known as Plug and Play, even though apparently no one thought of calling it ATA PnP until about 1995. But where did it come from and why?
Hardware
We know that IDENTIFY DRIVE was specified in the early ATA drafts in summer 1990. But we also know that at that point, there were numerous IDE drives implementing the command, for example Seagate ST-157A from early 1989, the Quantum ProDrive 40 from mid-1988, the Miniscribe 8051A of similar vintage, or the even older Conner CP-342 (in all cases confirmed by examining actual drives).
The very fact that the early ATA drafts already included the IDENTIFY DRIVE command (with command code ECh) indicates that it was widespread; at the same time, those drafts (and the final ATA-1 standard) mark IDENTIFY DRIVE as optional, which suggests there were IDE drives with no IDENTIFY DRIVE support.
Document Drought
Researching the history of the IDENTIFY DRIVE command prior to 1990s turned out to be remarkably difficult. Wikipedia claims the first ATA draft was “Common Access Method AT Bus Attachment, Rev 1, April 1, 1989”, document CAM/89-002, created by the CAM Committee, which is plausible but there’s no sourcing.
For whatever reason, the ATA standardization effort started in the CAM Committee, which was part of the X3T9.2 working group. The primary focus of X3T9.2 was SCSI, although it also standardized ESDI. CAM (or Common Access Method) was an ultimately unsuccessful attempt to create a standardized software interface to SCSI hardware, and its draft documents do not appear to have survived. In 1990, the ATA Working Group was folded back directly into X3T9.2, and several ATA drafts from that period did survive.
The next source of information would be technical references or OEM manuals for pre-1990 IDE drives. But… there’s nothing. By the end of 1989, Conner, Seagate, Quantum, IBM, Micropolis, CDC, Western Digital, and several others had IDE drives on the market. But the manuals are not available online anywhere, even though there is plenty of archived documentation for SCSI and ESDI drives from the same period.
Part of the problem is no doubt that for the vast majority of users, including technical users, those manuals weren’t all that interesting. For a basic IDE drive, there was very little they wouldn’t already know from the PC/AT Technical Reference—again, the compatibility was the point. The one exception would have been the IDENTIFY DRIVE command.
Software
Drawing a blank on the hardware side, I thought I’d perhaps have more a bit more luck tracking down which software used the command. What I found surprised me a little: Even though the IDENTIFY DRIVE command had been around since well before 1990, major operating systems do not appear to have used it until 1992-1993:
- The WDCTRL.386 disk driver in Window 3.1 (1992) did not use IDENTIFY DRIVE
- Windows NT 3.1 used IDENTIFY DRIVE in the atdisk.sys driver in the final July 1993 release, but not in the March 1993 beta
- Novell introduced the IDE.DSK driver for NetWare in 1993
- OS/2 2.0 did use IDENTIFY DRIVE in its poorly named IBM1S506.ADD driver in March 1992
- Novell used IDENTIFY DRIVE in IDE.OBJ from August 1991 (IDE286.ZIP package for NetWare 2.1x/2.2)
- Microsoft OS/2 1.3 used IDENTIFY DRIVE in ESDI-506.BID on installation diskette A but not in the AT disk driver on diskette B (1991)
- Microsoft OS/2 1.21 used IDENTIFY DRIVE in BASEDD01.SYS (June 1990)
OS/2 clearly used IDENTIFY DRIVE relatively early on, but the details were complicated. While Microsoft’s OS/2 1.2 and 1.3 used IDENTIFY DRIVE, IBM’s did not. What’s more, the betas of OS/2 2.0 did not use IDENTIFY DRIVE until 1992, most likely because the disk driver was forked from the OS/2 1.x code base before the support was added.
IBM used the DISK01.SYS driver in OS/2 2.0 pre-releases throughout 1991, but the final beta in early 1992 switched to IBM’s brand new modular storage stack with ADD and DMD drivers.
Reviewing the 1992 source code of the IBM OS/2 2.0 storage stack (which fortunately did survive) it’s clear that the information about IDENTIFY DRIVE command must have come from the IBM drive division.
The OS/2 2.0 driver uses IDENTIFY DRIVE to query the default disk geometry, which might seem like a good idea but probably isn’t, because it may not match the geometry the BIOS setup programmed into the drive. The IBM1S506.ADD driver also uses IDENTIFY DRIVE to query and (if present) enable multi-sector transfer support, a feature which may have brought a noticeable performance boost when available.
I don’t have the source code for the MS OS/2 1.21 BASEDD01.SYS driver, but at this point June 1990 is the earliest known usage of IDENTIFY DRIVE by an operating system. Similar to IBM’s OS/2 2.0, the Microsoft driver uses IDENTIFY DRIVE to detect whether READ/WRITE MULTIPLE commands are supported, and if so, enables multi-sector transfers.
Okay then, operating systems probably didn’t really use IDENTIFY DRIVE before 1990. But surely someone must have used it, right? Why else would the command be there in the first place? Drives definitely supported the IDENTIFY command well before 1990.
A BIOS could have, in fact should have used it… but I couldn’t find any evidence of common BIOSes using IDENTIFY DRIVE before 1993 or 1992.
Then again, the most likely user of IDENTIFY DRIVE would have been Compaq, since Compaq (together with Western Digital) was the driving force behind IDE and Compaq machines used IDE drives before anyone else.
Yet checking a 1988 DeskPro 386 BIOS revealed no trace of IDENTIFY DRIVE usage. Still, if anyone used IDENTIFY DRIVE in the 1980s, it should have been Compaq.
Deskpro 386 Tech Ref to the Rescue
Jeff Parsons pointed me to old Technical References for the Compaq DeskPro machines. That turned out to be extremely helpful.
Volume 2 of the DeskPro 386 Technical Reference from 1986 (the original DeskPro, the first 386 PC) shows that the 40 MB drive option did in fact use a 40-wire ribbon cable nearly identical to “modern” parallel ATA. The 40 MB drive was a half-height 5.25″ CDC Wren with a custom Western Digital PCB; the PCB included the equivalent of the PC/AT hard disk controller (conveniently also a WD product) and was connected to a multi-I/O board inside the DeskPro 386 with the above mentioned 40-wire ribbon cable.
As an aside: The Wren HH with a WD adapter board is often claimed to be the first IDE drive, but that’s only half of the story—Compaq used a 3.5″ Miniscribe drive with a separate WD adapter board and 40-wire ribbon cable in the Portable II a few months before the DeskPro 386 came out. There are even photos (scroll down a bit). The WD adapter board was model WD1003-IWH, pictured near the beginning of this article. It is likely that the Compaq-only Wren was the first actually integrated IDE drive, but the DeskPro 386 where the drive appeared was not Compaq’s first machine using IDE cabling.
Okay, so the DeskPro 386 definitely had an IDE drive of some sort of another. Even better, the DeskPro 386 Tech Ref also documents a rudimentary “Identify” command with code ECh. That means the command already existed in 1986! Here’s what it looked like:
Needless to say, this is only a small subset of the IDENTIFY DRIVE command specified by the original ATA standard, but it is entirely compatible with it.
But wait! A note clarifies that the Identify command (plus two others) is only available on the 130 MB drive, not on the 40 MB IDE drive! What was the 130 MB drive then? It was an ESDI drive, and the controller was almost certainly a Western Digital WD1005-WAH. So the IDENTIFY DRIVE command was implemented by an ESDI controller before it showed up in IDE drives! That must be why the DeskPro Tech Ref labels the contents of IDENTIFY command results as “ESDI Parameter Words”.
Back to Hardware
What’s ESDI, Anyway?
Unfortunately I could not find detailed documentation of the WD1005 or WD1007 ESDI controllers. There are detailed datasheets available for the WD1002 and WD1003 series of controllers, but not for the ESDI ones.
The only solid information (thanks to Compaq) is that the WD1005-WAH implemented the Identify command, but what does it have to do with ESDI drives? And how did the WD1005-WAH work?
From extant documentation it is apparent that the Western Digital WD1005-WAH was a 16-bit ISA adapter similar to the PC/AT WD1003-WAH, but instead of a ST506 style drive it could control an ESDI drive (or two). At the same time, it provided a register and command interface fully compatible with the WD1003-WAH, and thus did not require its own BIOS.
In other words, an ESDI drive connected to a host through the WD1005-WAH controller looked to the host system just like a standard PC/AT drive. There were possible issues with geometry translations because the higher-end ESDI drivers were already running into the BIOS limits, but aside from that, a WD1005-WAH with an ESDI drive was, as far as software was concerned, exactly the same as a PC/AT fixed disk controller with a ST506 style drive.
Except, again, for that IDENTIFY DRIVE command. (And possibly the READ/WRITE BUFFER commands.)
But the ESDI connection provided the vital clue that let me piece together the answer to the question where the IDENTIFY DRIVE command came from, or at least form a very informed guess.
ESDI Drive Interface
ESDI drives were an outgrowth of ST506 drives commonly used for higher-end hard disks. ESDI was faster—while ST506 was limited to 5 Mbps, ESDI could handle 10 Mbps and over time, more than 20 Mbps. ESDI drives had a data separator on the drive, and not on the controller; while ST506 controllers moved a digital but noisy signal over the data lines, ESDI uses NRZ encoding synchronized with a clock signal produced by the drive. ESDI drives could also handle larger capacity, upwards of 500 MB and theoretically into many gigabytes.
ESDI drives also accept a small set of commands that do things like select a head, seek, or return drive configuration.
The people designing ESDI were presumably aware that the complete lack of ST506 drive intelligence had one nasty consequence: Requiring users to tell the controller what the drive geometry is. The controller had no way to tell but had to know, and the only means available was the least reliable one: Humans. ESDI drives know what geometry they have and a controller can ask them.
The ESDI command interface is very simple: Write a 16-bit command word to the drive, get a 16-bit result back. Here’s what ESDI drives could tell the controller about themselves, from an ESDI draft specification published by CDC in 1984:
Surely it is no coincidence that ESDI drives return 16-bit words in response to the REQUEST CONFIGURATION commands, and that the first nine words line up exactly with the Compaq Tech Reference. Ever wondered why the third word of IDENTIFY DRIVE is reserved in ATA? Because it reported the number of cylinders for removable-media drives, something ATA did not support.
Here’s what the first word (general configuration) looked in the 1984 ESDI spec:
For comparison, here’s the same table in the final ATA draft from late 1993:
General configuration bit-significant information: 15 0 reserved for non-magnetic drives 14 1=format speed tolerance gap required 13 1=track offset option available 12 1=data strobe offset option available 11 1=rotational speed tolerance is > 0,5% 10 1=disk transfer rate > 10 Mbs 9 1=disk transfer rate > 5Mbs but <= 10Mbs 8 1=disk transfer rate <= 5Mbs 7 1=removable cartridge drive 6 1=fixed drive 5 1=spindle motor control option implemented 4 1=head switch time > 15 usec 3 1=not MFM encoded 2 1=soft sectored 1 1=hard sectored 0 0=reserved
It’s very obviously the same table… and again, the ESDI origins explain why half the bits don’t even make any real sense for IDE. Who cares if an IDE drive is hard- or soft-sectored, or if the internal transfer rate is 5, 10, or 15 Mbps? Certainly not the software using it. But an ESDI controller would very much care about such parameters when driving an ESDI disk.
In fact does ECh, the command code of IDENTIFY DRIVE, stand for ‘ESDI Configuration’? It very well might.
As an aside, I was not able to confirm if my Western Digital WD1007V-SE2 controller supports the IDENTIFY DRIVE command because I have no working ESDI drive (help, anyone?). I am fairly sure that the controller’s BIOS does not use IDENTIFY DRIVE, but it uses a different command, E0h. That command clashes with STANDBY IMMEDIATE specified in the ATA standard, but the WD1007V clearly uses it to send a command word to the drive and read back the 16-bit reply. The WD1007V BIOS uses the E0h command to query the same information that IDENTIFY DRIVE would return, but piecemeal. One possible reason for that could be the difficulty of safely allocating a buffer for the IDENTIFY DRIVE results in the controller POST routine.
Deskpro 286 Confusion
We can also examine the Compaq DeskPro 286 Technical Reference from December 1987, roughly a year newer than the original DeskPro 386 Tech Ref. It’s similar, but different, and it’s different in odd ways. Here’s what it says about the IDENTIFY command:
Note that while the older 1986 Tech Ref defined words 7 and 8, those are gone from the late 1987 Tech Ref. And the missing two also happen to be words which are somewhat specific to ESDI and are not defined in ATA (left as vendor-specific). Most IDE drives do not report any values there, although some—notably old Conner drives—do. Also note that the newer Tech Ref removed references to ESDI, presumably because the command was no longer specific to ESDI drives.
Which is why it gets rather confusing when the Tech Ref states that “The 20-MB and 40-MB integrated drives do not support the IDENTIFY, READ SECTOR BUFFER, or WRITE SECTOR BUFFER commands.” The Dec ’87 DeskPro 286 Tech Ref covers IDE (aka “COMPAQ 16”), ST506, and ESDI drives. If the statement were true, it would imply that both ESDI and ST506 drives/controllers support the IDENTIFY command, and IDE drives do not, which is extremely implausible. It is much more likely that the Tech Ref is wrong and it is the ST506 drives which do not support those commands, whereas ESDI and IDE drives do.
Whodunnit?
So… the IDENTIFY DRIVE command (ECh) existed in 1986 and must have been implemented by the Western Digital WD1005 ESDI controller. It was, at least initially, obviously intended as a way to conveniently return all of an ESDI drive’s configuration information to software.
The question is, what software? As we’ve seen above, OS/2 started using IDENTIFY DRIVE in 1990, but many other operating systems waited another couple of years. It’s hard to say if OS/2 was first, but it may well have been, at least when considering generic drivers.
At the same time, the command had already existed for at least four years, and by 1990 it was supported by numerous IDE drives and at least some ESDI controllers.
On the ESDI side, WD was not the only vendor implementing IDENTIFY DRIVE in their controllers. Documentation is scarce, but at least the OMTI 8640 documentation from 1989 describes a “Read Parameters” (ECh) command, as well as an “Initiate ESDI” (E0h) command that looks extremely similar to what the BIOS of my WD1007V is internally using.
On the IDE side, it’s difficult to say who supported what without examining the actual drives, but the IDENTIFY DRIVE command was certainly out there. I have been able to confirm that a Conner CP-341i from 1988 supports IDENTIFY DRIVE and not only that, it even supports the READ/WRITE MULTIPLE commands.
I have also examined a somewhat odd Conner CP-342 drive and found that it also supports IDENTIFY DRIVE, though not READ/WRITE MULTIPLE; the CP-342 was likely the first IDE drive available to OEMs other than Compaq. Why is my CP-342 odd? The drive was made in 1989, but the PCB is full of components made in 1987. Yet the sticker on the ROM clearly says “CP 1989” and the revision on the sticker (5.85) matches what IDENTIFY DRIVE reports. The upshot is that while Conner’s 1989 firmware provably supported IDENTIFY DRIVE, that is not exactly new information and I can’t conclusively say that the 1987 firmware did too, although it is extremely likely. Curiously, the CP-342 calls itself CP-341 in the IDENTIFY DRIVE output (didn’t expect that!), the CP-341 being an earlier Compaq-only model.
After most of this article was written, Tom Gardner kindly provided another valuable piece of information. The Conner CP3022 Product Specification from February 1988 does document the IDENTIFY DRIVE command; it is the fourth revision of the document and there is no reason to suspect that the IDENTIFY DRIVE command was new at that point; Conner almost certainly implemented it in 1987, possibly earlier. The Conner CP3022 spec is notable for calling the command IDENTIFY DRIVE, rather than just IDENTIFY as it was named in the Compaq technical references.
Here is what the Conner documentation shows for IDENTIFY DRIVE:
It is notable that Conner documents words 7 and 8 which Compaq un-documented. It is likewise notable that Conner’s IDENTIFY DRIVE definition is significantly extended compared to the Compaq Tech Refs, and it is in fact a fairly large subset of what later appeared in the first ATA standard (it is actually almost the same as 1990 ATA standard drafts).
At this point it is impossible to tell whether Conner extended the IDENTIFY DRIVE information on its own initiative or if someone else (Compaq?) asked for it. At any rate, Conner supplied the model number, firmware revision, and serial number. There is already a provision to support the READ/WRITE MULTIPLE commands—not all Conner drives did that, but Conner IDE drives probably implemented this capability before anyone else. There is also the mysterious “double word transfer flag”, as well as information about the drive’s buffer.
And there is information about the number of ECC bytes passed on READ/WRITE LONG commands. That may well have been important to Compaq’s diagnostics: ST506 and early IDE drives used 4 bytes, while ESDI drives used 7 bytes of ECC.
It is apparent that the IDENTIFY DRIVE command was somewhat widespread before the first ATA draft showed up, even though it does not appear to have been widely (if at all) used by operating systems.
A retired WD firmware engineer confirmed my suspicion that the driving force behind this command were PC OEMs, notably Compaq, but likely also Dell and others. OEMs had a vested interest in drives returning identifying information in a compatible manner, and had enough clout to force drive manufacturers to make it so.
Still, that leaves the question what those OEMs actually used the IDENTIFY DRIVE command for, and it is almost a given that they must have—otherwise why ask for it?
Who Needs It?
The most obvious user of IDENTIFY DRIVE would be Compaq’s setup and/or diagnostic utilities. I have not been able to find any use of IDENTIFY DRIVE in Compaq’s utilities from 1987 or earlier, even though Compaq documented IDENTIFY DRIVE as far back as 1986. Compaq diagnostics version 5.02 from May 1987 do not appear to use IDENTIFY DRIVE.
Finally I struck gold while examining the Compaq Portable II diagnostic floppy from this page. This is Compaq diagnostics 5.08, with files dated January 29, 1988. They don’t look like much:
But despite their humdrum appearance, both the Compaq diagnostics (TEST.COM) and the setup utility (SETUP.EXE) issue an IDENTIFY DRIVE command when starting up.
I’m not sure what exactly the utilities do with it and the executables are obfuscated in a silly way that makes analysis harder, but IDENTIFY DRIVE is there.
This narrows the software timeline considerably: Compaq presumably added IDENTIFY DRIVE to their diagnostic floppies sometime between May 1987 and January 1988. Probably not coincidentally, this was the time period when Compaq started moving away from ST506 drives to IDE drives in their then-current systems.
Remaining Questions
We’ve clearly established that the IDENTIFY DRIVE command (however it was originally named) was first implemented in Western Digital ESDI controllers (not drives), and was documented by Compaq as early as 1986. Not much later, IDENTIFY DRIVE showed up in IDE drives, and was documented by Conner no later than February 1988, almost certainly earlier. The command was used in Microsoft OS/2 in 1990 and several years later in numerous other PC operating systems.
Some of the remaining questions are:
- Did all of Conner’s IDE drives support IDENTIFY DRIVE? If not, which was the earliest Conner drive with IDENTIFY DRIVE support?
- When exactly did Compaq’s utilities start using IDENTIFY DRIVE?
- Did any operating system use IDENTIFY DRIVE before MS OS/2 1.21 did?
- When did BIOSes start using IDENTIFY DRIVE to implement automatic drive detection and which one was first?
If any reader has an IDE drive manufactured in 1988 or earlier, it would be great to get a dump of the IDENTIFY DRIVE data. Seagate’s FIND-ATA is one option, but there are others.
Another source of information would be old IDE drive documentation; there must be more out there than the Conner CP3022 product spec. Finding more Compaq Technical References would be great too, and possibly references from other vendors if they mention IDENTIFY DRIVE.
Updates
- The following was found in AT&T SVR4 UNIX:
#define AWC_READPARMS 0xec /* WD1005-WAH Read Parameters command */
This confirms what the DeskPro 386 Tech Ref strongly hinted at, i.e. that the IDENTIFY/ECh command came from Western Digital’s WD1005 ESDI controller. - 386BSD added support for IDENTIFY DRIVE between version 0.0 (March 1992) and 0.1 (July 1992); the command macro was WDCC_READP.
- Plus Development Corporation (Quantum subsidiary) adapter ROM used IDENTIFY DRIVE in 1989 in order to query drive geometry and set up BIOS disk parameter tables; this is the earliest known use of IDENTIFY DRIVE to implement PnP.
You are leaving out one important piece of software that would be an early adopter of the Identify command: Ontrack Disk Manager. According to Ontrack BBS thread* regarding Q&A for IDE drives, version 4.2 will automate the translation which presumably requires finding out the information on the drive from the drive. That would suggest active usage of the Identify command by mid-1990 since the thread compilation is dated October 1990.
* There is a copy online but at a very dubious website which I will not link to. Nothing that triggered warnings but certainly does not seem like a planned archive site.
Your WD1007V-SE2 almost certainly supports IDENTIFY DRIVE. One of the two ROMs on the board is the firmware for the 80C188, which is responsible for interpreting commands from the host. A while ago I disassembled the firmware from a WD1007V-SE1 and it supports the following commands:
0x10-0x23 (recalibrate and various flavors of read)
0x30-0x33 (various flavors of write)
0x40-0x41 (read verify)
0x50 (format track)
0x70-0x7F (seek)
0x90-0x91 (diagnostics and set parameters)
0xA0 (read defect list, I think)
0xAD (I haven’t figured out what this one is)
0xC4-0xC6 (read/write/set multiple)
0xE0 (raw ESDI commands)
0xE4 (read buffer)
0xE8 (write buffer)
0xEC
0xEF (cache control)
It also reports the current settings of the W1 jumper block in the IDENTIFY DRIVE data. I think it was in the last byte, but I’d have to dig up the rest of my notes to say for sure.
The OMTI 8640 technical reference manual calls 0xEC “read parameters”, which would make it a nice counterpart to 0x91 “set parameters”. (But it’s an ESDI controller, not an IDE drive.)
So I assume that “constant” 0A5Ah that Conner drives return according to the documentation above still fills out the generic configuration bits?
If so, that makes Conner drives: Have a high rotational speed tolerance (>0.5%), a transfer rate between 5MHz and 10MHz, be fixed drive (duh), have a slow head switch time (>15uS), RLL encoded instead of MFM, and hard sectored.
But given that Conner’s documentation calls it a constant instead of configuration, I wonder, was it really never meant to change? I’m especially curious about the high rotational speed tolerance, transfer rate, and slow head switch time bits being set. It seems like future models at least could well be different there. Again an IDE host would likely never care about those bits, but then why have them set in the first place? Did they set them once to what was true at the time, and then actively “forget” given their irrelevance?
The other possibility that this “constant” only represents the configuration for a few specific models does not fit well with the generic description of all the other fields.
I forgot to add: It does not help that the constant is a very “marker” looking bit pattern (with the first nibble being 0 instead of 5 to aid with endianness maybe), although given how well it resolves to actual generic configuration bits I think it’s more likely that that’s coincidence.
Heh, I did the same thing. And came up with the same results. Yes, it definitely supports IDENTIFY, it just won’t work without a functioning drive. I have a very good idea what IDENTIFY DRIVE would return with the WD1007V but can’t actually test it.
I just put up what I’d written down as the next post.
The old Conner drives were standard RLL designs, so the transfer rate between 5 and 10 Mbps was right. They were certainly hard-sectored in that you couldn’t reformat them with different sector size or anything. I think they basically filled it out as best they could and then forgot about it.
The reason why the configuration bits were there is clear, that came straight from the ESDI spec. Whether software ever needed that information I’m not sure. An ESDI controller/firmware yes, but ESDI host software? Probably not. IDE host software, definitely not (and the onboard controller didn’t need to be told what the drive is). The value would be purely informational, but the fields soon stopped making sense for IDE drives. Which is no doubt why ATA-2 already obsoleted almost all the bits, only leaving bits 6 and 7 with any useful function.
Thanks for the suggestion. I simply don’t know what software there might have been that used IDENTIFY DRIVE, and finding out is somewhat difficult in that I have to actually have the software and either run it or disassemble it.
Mid-1990 was actually after the first ATA draft were already public. Then again, as far as I know a number of vendors shipped Ontrack software with their drives, so Ontrack could very easily have had non-public information before then.
Ontrack would have had one extremely good reason to use IDENTIFY DRIVE as soon as they could… it was a good way to bolt an OEM-specific release to that OEM’s drives.
This must be the document. Interesting reading.
InfoWorld Jun 29, 1987 printed Conner CP-342 announcement, 2MB/s transfer, $785. Right next to it is an ad for some bargain peripheral dealer offering 40MB MFM drives at $699 with a controller. Cheapest 40MB drives advertised in that issue were $585-599 Seagates. September 1988 40MB AT drives are already cheaper than MFM+controller combos, $350 vs $375. In that one year ram prices jumped 4x :-).
Yes, that June ’87 announcement is the oldest mention of an IDE drive available on the open market that I know of. As far as I can tell, the CP-342 was essentially identical to the Compaq-only CP-341. At that point, Conner was the only source of IDE drives, and they probably supplied the vast majority of them to OEMs. By about 1990, IDE drives had taken over the upgrade market, and a year or two later ST506 (and ESDI) drives were effectively gone.
A lot of this coincided with the move toward 3.5″ drives, but that might be coincidental.
BTW that “2MB/s” was of course a lie… the CP-342 was a standard RLL drive, so the theoretical max sustained rate was under 1 MB/s. In practice you were probably lucky to get 300 KB/s out of it.
Some low hanging fruit, you probably also found those:
“History (1987): Conner CP341 HDD”:
>Article was written by Tom Burniece with contributions from Tom Gardner and I. Dal Allan. Version 27 was reviewed and approved by the Computer History Museum (CHM)’s Storage SIG on July 21, 2011, posted on WikiFoundry
but going to WikiFoundry we learn “We decommissioned the main platform on June 1, 2021” 🙂
I found https://d1yx3ys82bpsa0.cloudfront.net/groups/compaq-conner-cp341-ide-ata-drive.pdf
https://web.archive.org/web/20190830041308/http://chmhdd.wikifoundry.com/page/Main+Timeline+of+Significant+Events+and+Products
https://web.archive.org/web/20190821051230/http://chmhdd.wikifoundry.com/page/Conner+CP340+Family
https://web.archive.org/web/20210224022517/http://chmss.wikifoundry.com/page/Conner+CP341+Drive+(ATA/IDE)
https://web.archive.org/web/20081123063209/http://ata-atapi.com/histcam.html
Yes, found those. The 1988 Conner product specification excerpt is from Tom Gardner who co-authored the Conner articles. Those histories are very handy but not quite detailed enough for my purposes.
> There is also the mysterious “double word transfer flag”
Some ESDI controllers on EISA, VLB or nonstandard 32-bit bus?
Like later translating PCI ATA HBAs with 32-bit PIO support.
ESDI on EISA is my guess. The doubleword flag was there no later than 1988, long before VLB. Yes, PCI (and VLB) IDE controllers can do this, but again, the flag was there long before PCI, and moreover it’s part the information that the drive returns, not controller. For an ESDI controller on an EISA bus the flag might actually make some sense, and that was a thing in 1988.
A related question mark is what all those fancy DMA transfer modes are for in old IDE drives. I’ve never seen an old (pre-VLB/PCI) motherboard or IDE adapter capable of DMA transfers to/from IDE drives, yet it’s there. Compaq had some kind of IDE RAID solution back in 1989 or 1990, but so far I’ve not been able to find any technical information about it. So maybe there was some Compaq custom thing capable of IDE DMA back in 1990 or so, or it was again something that ESDI controllers could do and it ended up in IDE more or less by accident (because if nothing else, it made sense to keep the IDENTIFY DRIVE format compatible across devices).
Another mystery is why the heck ASCII strings are byte-swapped in the IDENTIFY DRIVE data. I suspect it’s a bug turned into a standard feature, but that’s only a guess.
> I’ve never seen an old (pre-VLB/PCI) motherboard or IDE adapter capable of DMA transfers to/from IDE drives, yet it’s there.
AFAIK the only thing you have to do is to connect DMAREQ/DMAACK from the drive to one of the three appropriate ISA 16 bit DMA pin pairs. I read descriptions (some drivers for DOS with DMA code) that should work for old drives. This physical connection is rare on pure ISA IDE HBAs but easy to do yourself.
Yes, it should have worked with the 8237 DMA controller, but if it were so great surely it would have been common? Most likely the performance wasn’t good (actually it’s almost a given that it was worse than PIO), quite apart from the added complexity. But again, on EISA machines the situation was different, there was no 24-bit DMA addressing limitation, and in fact I see “EISA type B DMA” explicitly mentioned in some drive manuals. Again the finger points at Compaq but… where are the details?
As an aside, looking at the command code table in an old WD Caviar manual it strikes me that there are two distinct groups of commands added after the classic PC/AT controllers: commands in the Cxh range and commands in the Exh range. The Exh ones almost certainly started out on ESDI controllers and it’s hard to believe the ‘E’ is a coincidence. I strongly suspect that the Cxh commands start with ‘C’ because of Compaq and/or Conner.
Can you really just wire IDE to 8237 DMAREQ/DMAACK lines, program 8237 and get Single Word DMA 0 for free? would that be in any way attractive? Sure, it would be slower than even PIO 0, but maybe it would have some benefits in Windows where multitasking is expected?
ATA-1 specification seems to have been defined in 1994. 1994 Promise EIDE2300 already supported 16MB/s MWDMA 2 straight from upcoming ATA-2? Benchmarks on Vogons show >10MB/s reads.
>Note: DMA transfer is between disk and PDC20630 only, for transfer from PDC20630 to computer standard ins/outs are used.
linux/blob/master/drivers/ata/pata_legacy.c
>The 20620 DMA support is weird being DMA to controller and PIO’d to the host and not supported.
since there is no ram on Promise card PDC20630 needs to incorporate internal FIFO. So at least in this particular case MWDMA is a halfway hack using DMA invisibly to the system BUS on a non busmastering VLB card, fascinating.
Intel ICH4 IDE controller datasheet:
>8237 style DMA: DMA protocol that resembles the DMA on the ISA bus, although it does not use the 8237 in the ICH4. This protocol off loads the processor from moving data. This allows higher transfer rate of up to 16 MB/s.
this is clear, PCI busmaster.
No, ATA-1 was not defined in 1994, it was approved as an ANSI standard in 1994. It was defined largely in the 1990-1992 period, but you have to look at the surviving drafts to see what was added when. DMA transfers, as it happens, were there in the August 1990 draft (revision 2.2), the oldest draft I’ve seen. Unfortunately the ATA standards are not too informative here because they (logically) focus strictly on what the drive does. Even so, ATA-1 talks about a “slave-DMA channel”, not any kind of bus-mastering DMA, which presumably reflects the early IDE DMA transfer usage (since the drive does not know or care how exactly the DMA is done on the host side).
My guess is that in AT class machines, using DMA was in every way imaginable worse than PIO. There were all the silly limitations with 24-bit addressing and (much worse at the time) inability to cross 64K boundaries. All the trouble with EMM386 UMBs or otherwise remapped memory. DMA was slower than PIO, but even if it were equally fast, the DMA controller would just hold the bus and the CPU couldn’t do much anyway. Add the overhead of setting up the DMA controller and arbitrating the bus, and everyone just runs away screaming.
And yet the DMA capability was there, and presumably the people who added it weren’t idiots so… what did they know that we don’t? I can only guess they were working with EISA systems where DMA is much less limited and actually fast. Type B DMA should be at least as fast as PIO, and possibly faster if the IDE adapter were smart enough to run 32-bit cycles on the host bus. Even so, I suspect this was added for the benefit of some multi-tasking OS, most likely OS/2 or NetWare.
I have a circa 1993 VLB board (486) with an Adaptec IDE controller and it is likewise quite fast, capable of around 10 MB/s transfers if the drive can do it. Though I think it just uses fast PIO modes, not DMA.
FYI, that August 1990 ATA draft already defines PIO and DMA Mode 2, both with 240 ns cycle time, which ought to be 8.33 MB/s. Again, that’s mid-1990, with the value presumably chosen to match the top ISA/EISA speeds.
> EISA systems where DMA is much less limited and actually fast
wiki.osdev.org/ISA_DMA states
“it will be artificially slowed down to 4.77 MHz when transferring each ISA DMA byte/word. This includes EISA and PS/2 32-bit controllers”
so even with EISA you would need bus mastering controller? On the other hand
http://www.ece.ufrgs.br/~fetter/eng04476/datasheets/eisabook.pdf
“Using burst bus cycles, a 32-bit DMA device can transfer data at speeds up to 33 MB/second.
EISA DMA channels may be programmed to use one of four DMA bus cycle types when transferring data between the I/O device and memory. The default DMA bus cycle type, ISA-compatible, delivers a higher data transfer rate than ISA-compatible computers. The improvement is the result of EISA’s faster bus arbitration and requires no hardware or software modifications to ISA-compatible DMA devices. Type A and B cycles are EISA modes that permit some ISA-compatible DMA devices to achieve higher performance. The burst DMA (Type C) bus cycle type is the highest-performance DMA bus cycle and is only available to DMA devices designed specifically for EISA burst.”
dont know what to believe anymore 🙂
According to osdev 8237 doesnt merely steal bus cycles here and there, it can only perform 210ns ones so DMA block/demand transfers pause cpu completely? This bit finally clears 8237 suckiness for me. I always somehow assumed it only took 4.7mt out of available cpu bus cycles, slaps forehead.
vcfed “Was there ever any ISA IDE controllers that supported DMA transfers?” identified GSI Model 4C – storage controller – ATA-2 – ISA by Pacific Digital Corp. Picture https://www.vcfed.org/forum/forum/genres/later-pcs/42471-looking-for-information-on-large-at-xt-cases/page2#post516921
Unidentified 32-pin tqfp Microchip. They made CPLD chip back in the day, but I cant find anything in this pin configuration. 24pin DIP is also something you dont see every day.
There was at least one EISA IDE controller out there, the DTC-2290. Its pretty damned rare, but I’d love to get a hold of one to see how much faster it really was and what transfer modes it used. Most of the VLB cards did PIO mode 2, not DMA. I have a caching VLB IDE controller that takes 30-pin SIMMs, but its onboard ROM doesn’t support large drives and LBA. I haven’t done much testing with it as a result.
>Yes, it should have worked with the 8237 DMA controller, but if it were so great surely it would have been common?
Because it has zero added value for single-tasking environment (i.e. DOS over BIOS).
As i remember even first PCI ATA HBAs do not support DMA and there is no DMA code in Retail Windows 95 ESDI_506.PDR.
>actually it’s almost a given that it was worse than PIO
Yes. Harder to program and slower.
I would not trust osdev.org too much. As far as I know they’re right about ISA DMA, but with EISA it was a lot more complicated and EISA definitely had fast DMA. The book about EISA is much more likely to be correct 🙂 For bus mastering, EISA can do 32-bit transfers at 8.33 MHz so 33 MB/s theoretical bandwidth.
And yes, the bus hold is why 8237 DMA sucks on AT compatibles. The transfers are slow but they either completely lock out any other bus traffic (demand/block transfers) or have a high overhead (single mode transfers). For something like a Sound Blaster or a floppy drive with relatively slow data rates the overhead is manageable and DMA is a good fit for the real-time demands of the device. For anything with a buffer, 8237 DMA is a losing proposition.
Bus-master ISA DMA is not speed limited since the 8237 isn’t clocking the transfers, but it has the same issue with holding the bus. Pretty much all bus-master DMA devices have some kind of settings for how long they can own the bus because if they hold it for too long, even memory refresh may be unable to run (and serial comms and all that).
Both Compaq and Dell had IDE RAID controllers around 1990, but technical information is sparse. There’s some information on the Compaq one in this patent but there’s not quite as much as I’d like in there.
That GSI Model 4C is interesting, I wonder if it used 8237 DMA or bus-master DMA. Hard to tell from just jumper settings, that would look the same in either case. In any case it would have needed custom drivers of some sort.
I suspect that there was no DMA support in ESDI_506.PDR because IDE DMA simply wasn’t a thing — yet. My recollection is that ISA (bus-master) DMA became somewhat common only after the Intel Triton/PIIX came out, and that was 1995. Until then, only custom vendor-specific drivers might do DMA with IDE drives.
With PCI IDE of course bus-master DMA was the only option, 8237 style DMA was next to impossible. At any rate there would have been very little out there before 1994.
Hah, look at this. The EISA spec is there as an appendix, “CONFIDENTIAL INFORMATION OF BCPR SERVICES”. Over 400 pages, not a great scan but legible.
Wow! Thank you for the EISA specification, I’ve been looking for a complete copy. Bitsavers only has chapter 4. I never would have thought to look in patents.
I didn’t think of it either, but I was trying to figure out what the heck the Compaq Intelligent Disk Array (IDA) was, and the patents are about the only source of information. It just so happens that Compaq filed the EISA spec as an appendix, and there it was…
Incidentally, the Compaq IDA was a bus-mastering IDE RAID controller using (at least two) Conner IDE drives. It could act like an IDE disk, but also had an alternative fancy bus-mastering interface. There is information about the IDA to host interface (including driver source code), but no real information about how exactly the controller interfaced with drives and what IDE commands it used. I couldn’t even figure out which drives exactly it used; I suspect it was Conner CP-3204F and/or CP-3201G, but that’s mostly a guess. In any case it seems to have been a “proper” RAID controller in that the host only saw a logical drive and did not access the underlying physical drives directly.
Perhaps it was with an eye to alternate architectures? The 68k series was well-established at the time, with multiple beefy RISC architectures either on the market or on the horizon. Perhaps they envisaged it eating some of SCSI’s lunch, or that these architectures would eventually produce budget models (this actually happened with Macintoshes of the era, IIRC) where they’d throw away the expensive SCSI drives for cheap (but still decently performant for single user/single tasking workloads) IDE drives.
Or perhaps I’m just attributing an intelligence or forward-thinking to drive manufacturers that simply isn’t there. 🙂
Not sure about alternate architectures, if anything SCSI would have been the way to go. But this was a RAID controller designed circa 1989, when high-end drives were ESDI or increasingly SCSI… but also almost exclusively 5.25″ form factor. There’s only so many giant 5.25″ drives you can pack inside a server chassis. Conner was apparently able to offer decently performing IDE drives, obviously in 3.5″ form factor. I believe it was possible to install two Compaq IDA controllers with four drives each inside a SystemPro case. That was just not going to happen with 5.25″ drives.
Obviously 5.25″ drives could have had bigger capacity, but the point of RAID was mirroring and that needs more drives, not fewer, and the goal was not to maximize the storage capacity.
For their next generation RAID, Compaq switched to SCSI drives, quite possibly because there were more 3.5″ drives available.
There were DMA capable PCI IDE controllers that predated Triton and the PIIX, namely the infamous CMD-640 and RZ1000.
Absolutely. I hope I didn’t imply that the PIIX was the first bus-mastering PCI IDE controller, it definitely wasn’t. But the CMD-640 and RZ1000 were nowhere near as widespread as the PIIX and its successors, and ended up being more famous for data-corrupting bugs than anything else.
I know Intel used at least the RZ1000 on some of its boards; I wonder if that gave Intel an extra incentive to replace the early controllers with something less problematic. These controllers were definitely very important because with the demise of VL Bus, they were the way to get IDE drives beyond ISA bus speeds.
A possible use case for identify could had been software used during PC manufacturing.
I.E. hook up a new drive to a special “preparation station” and let software identify the drive type, fill the drive with suitable MBR, FAT, DOS and whichever other software that were supposed to be delivered with a computer. Would had been way faster than the regular way of booting of floppy disks, partitioning the disk, formatting the disk and installing/copying files.
Also during production identify could had been used by some software at the end of the production chain that could had emitted configuration data to the serial port and that information could had been gathered together with the computers serial number and entered in a database, or perhaps rather the configuration could had been compared with what was already stored in a database containing information on what was ordered.
P.S. another relatively early adoption of the identify command would had been in the Commodore Amiga 600, the first Amiga model with built in IDE controller. Although the partitioning tool (called HdToolBox) could use manually entered parameters, the regular way to use it was to tell it to read the parameters from the drive (and that was a feature available in the tool since before IDE as it originally were used with SCSI disks).
Yes, I believe manufacturing/testing was the initial purpose of IDENTIFY DRIVE. Software “Plug and Play” support came years later.
Bought bunch of new old stock Compaq spares in late 90s.
Among those were sealed in box Compaq IDA with docs, EISA config disk, cables etc. (one with KITT leds, not the newer IDA-2). Never was able to get it working as I think cables were odd they had split or twist if I remember correctly – and lacked proper hard disks as well. Since IDA is oldest card that NetBSD cac raid driver supports I think it is close relative of Compaq SmartArray SCSI controllers.
Unfortunately my storage was burglared 10 years ago and among useless junk they took that IDA, rare Cirrus Logic VGA cards for Microchannel, 5.25″ HH CDC ide disks, 5.25″ FH IBM ESDI drives (branded CMC) etc…
Pingback: IDENTIFY ESDI DRIVE | OS/2 Museum
I do realize that this thread has been dead for a long time, but it turned out to be very helpful while working on an IDE driver for vintage computers (for an OS called TLVC, a fork of ELKS).
What brought me here was a search for what IDE commends were actually supported that early. Thanks for the pointers.
FWIW – here’s the IDE ID dump from a 40MB Conner drive shipped with the Compaq Portable III and Portable/386 (and probably others).
425A 0326 0000 0004 4113 0269 001A 2020 2020 2020 2020 2020 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
Thanks for the report! The dump data looks really strange, are there some bytes missing? At least I cannot properly interpret it. The first 7 words make good sense but then there are five words with ASCII spaces in them, and those should really be at a later position. And there should be more ASCII characters, 10 words instead of just 5.
Also, what Conner model is it exactly, do you know? Looks like CP342 or so? The geometry at least matches my CP342. And do you know when the drive was made?