This is from the “learn something new every day” category. I’ve been using CompactFlash cards together with IDE adapters for several years now. It’s a terrific way to manage storage for vintage PCs. CF cards are cheap, fast, (relatively) capacious, and widely supported. It’s trivial to plug a CF card into a modern PC (typically using a USB adapter), copy files, and plug it into an old 486 or Pentium.
I’ve been using CF cards with DOS, Windows 3.x, and Windows 9x and never noticed much difference. Just recently I also tried using CF cards with Windows XP. The results were interesting.
Industrial (left) and consumer (right) CompactFlash cards.
The first card I tried was a weird 16GB Unigen Enterprise CF card. The card works, but it has very undesirable performance characteristics. Basically the card is very fast… except for writing small blocks. In practice it means that performance kind of sucks, but XP had no trouble using it.
The second card was SanDisk Ultra 32GB. This one subjectively performs much better, no doubt because writing small blocks isn’t penalized nearly as much. But the card had very very strange problems with XP. Straight after installation, XP kept complaining that it didn’t have a swap file, and it was impossible to create one.
After some head scratching, I realized that XP thought it was installed on a removable drive, and refused to create a swap file. OK, I understand the logic of not wanting to have removable-media swap files, but then again the boot drive isn’t really going to be removed, is it. Anyway, clearly the two CF cards were behaving differently—XP considered the SanDisk a removable drive, but the Unigen was fixed. But why?
Let’s not discuss whether it makes sense to report a CF card in IDE mode as a removable device; the fact is that some do it and some don’t. There are two questions: How to fix it, and why it is happening at all.
The first question has numerous answers all over the web. I landed on this page which actually deals with USB devices and Windows 7, but it is just as applicable to CF cards in IDE mode and Windows XP. The short story that Hitachi ran into the same problem with their Microdrives, and developed a filter driver which removes the removable bit (so to speak) in order to placate Windows.
Installing the filter driver involves editing the INF file to include the exact signature of the drive, but it does work. With the filter driver, XP is able to create a swap file and no longer complains. Good.
So why is this happening, and how to tell which CF card is which? Someone clearly thought it was a good idea. For CF cards used with, say, cameras, it makes sense to report them as removable. For CF cards intended as hard disk replacements, it makes sense to report them as fixed.
The practical problem is of course that when purchasing a CF card, this is the kind of information that is impossible to find. CF cards billed as “industrial” are likely to not report themselves as removable, but that is a very rough guide. In some cases, the type can be changed by updating the card’s firmware, but that is far from a generic solution.
ATA Bits
The “removable” bit is reported in bit 7 of word 0 of the IDENTIFY DEVICE command response. This is an old interface, and it was already defined in the first ATA standard (early 1990s). It was intended for removable-media devices such as ZIP drives.
Note that there is a significant difference between removable-media IDE drives and ATAPI devices such as CD-ROMs. Removable-media drives are regular IDE hard disks with a few additional commands to address media lock/unlock and media change tracking. ATAPI devices use a completely different command set (SCSI, basically) and typically support a whole host of other capabilities beyond simply reading and writing sectors.
It makes sense to report CF drives as removable when using the PCMCIA interface. It’s questionable how much sense it makes when using the IDE interface, because there is normally no hot-plugging supported. But the reality is that many CF cards report themselves as removable, and it makes a difference to operating systems.
There is a good chance that the removable bit has an impact on ThinkPads, both when booting off of a PCMCIA drive, and specifically for the ThinkPad 701 when booting off of IDE hard disk. To recap, unlike other ThinkPads, the 701 refuses to boot from most CF cards using an IDE adapter. I now strongly suspect that the 701 requires the CF card to not report itself as removable.
SD vs. CF
I have no first-hand experience with using SD cards as IDE storage. What I do know is that the situation there is different and much more straightforward. CF cards are also IDE devices and CF-to-IDE adapters have no logic in them; therefore, the behavior depends strictly on the CF card.
SD cards are not IDE compatible at all. What that means is that SD to IDE adapters must contain protocol translation logic, and that logic will decide whether the IDE drive seen by the host reports itself as removable or not. In other words, the behavior depends on the adapter and not the card.
TODOs
It would be nice to have some quick way of identifying whether CF cards report themselves as removable or not. That may be doable when using a CF to USB adapter, but then again maybe not. Suggestions?
I’m quite puzzled that you were able to install Windows XP to CF card with removable bit at all. I remember that installing Windows XP on for example USB had to be performed on normal HDD.
I’m also almost certain, that one cannot install Windows 2000 on CF card with removable bit set – after text part of setup I was getting 0x7b STOP (inaccessible boot device). This was a kind of stopgap for installing Windows 2000 on Seagate-branded CF microdrive plugged into Eizo 15″ thin client (or Compaq Clipper, for that matter). Removable bit even caused SMBR (Smart Boot Manager) not to recognize partitions, though this was easy to fix (it is open source software after all). In case of Windows 2000, I’ve changed INF files provided on installation CD, to load Hitachi filter driver along with standard disk drivers, though I don’t remember details now (I remember some editing of hives though, I need to revisit it some day).
I ran into the same problem, as I use CF cards for the exact same purpose. Turns out, you can change those bits and make the CF card corretly report as a fixed disk. During my searches, I discovered a tool called “ATCFWCHG.COM”. It can be used to set a card to fixed disk and back to normal. You can get it here: http://www.floutsch.com/pub/download/ATCFWCHG.COM
To answer your second question: I own a SD2IDE adapter (and also an SD2SCSI – but different story) and can say, thay do work, but I feel more comfortable with an CF2IDE for no particular reason.
Cheers,
Florian
Some companies released tools to “reflash” their CF cards to non-removable mode. The problem is they tend to only give out the tool by request or only to OEMs. Some have leaked out into the wild though. Windows XP is a pain with CF cards with the removable bit set, it simply won’t install if it isn’t on the first primary partition. MS clearly changed something with XP because Windows 2000 and older don’t care at all what kind of drive they are installed on.
In the future I might switch to a SD to IDE adapter. CF cards are getting expensive and even though its another layer of abstraction, its likely easier to tell the SD card adapters that they are non-removable!
Does the ATCFWCHG.COM utility work with any CF card? Or only with specific firmware? I’m not aware of there being any standardized way to control the removable bit.
Actually I’m severely bugged by the fact that even Windows 7 refuses to see anything but the primary partition on my CF card. Ironically I think it’s the card with no removable bit, but is seen as removable due to the CF to USB adapter in between.
I had no trouble installing XP SP3 on the card. It was a clean card with one manufacturer-provided partition. Only after the installation did XP start complaining. Could be a service pack difference or it might depend on the exact CF card and possibly also system BIOS.
I cannot answer if it’s working with any brand. But maybe another hint: Transcend cards are somehow smart enough and flip their fixed disk bit on their own.
Cheers,
Florian
One solution for this problem is using the TrueIDE CF-to-IDE-adapter, which can be purchased e.g. here:
http://www.vesalia.de/d_trueide.htm (german product description)
http://www.vesalia.de/e_trueide.htm (english product description)
This adapter is “active” and disables the “removable” flag 🙂
I have no doubt it works… but unfortunately it kills the big CF advantage which is dirt cheap adapters. If I were buying a relatively expensive adapter, I’d go for SD to IDE because SD cards are quite a bit cheaper than CF nowadays.
CF cards have two modes, PCMCIA and IDE. It is entirely plausible for the different modes to return different data. I need to see what Transcend CF cards I have lying around 🙂
To add some detail, I read somewhere that XP will install if there is a bootable FAT32 primary partition on the device (fairly common with CF cards I think). It supposedly refuses to install on a clean removable drive.
AFAIK the Sandisk removable bit twiddling utility doesn’t work on all cards as it uses a non-standard command to tell the CF card controller to flip it. Some CF cards are forced to one particular mode and you can’t do anything about it. Same applies if it uses a controller that doesn’t understand the Sandisk tool commands.
Regarding the IDE or PCMCIA mode. Quite a few years ago, people were buying MP3 players and small portable HDs purely to gut them for the standard CF Microdrive inside so they could be put into DSLR cameras. The retail cost of a similar CF Microdrive was significantly more than what the MP3 player/portable HD was selling for. Once they noticed people were doing this, they altered the configuration on these embedded Microdrives to force them into IDE mode only as the DSLRs used PCMCIA mode.
In the original specification of Compact Flash there are only removables. In this point it’s similar to the PC-Card ATA specification. So everything should be right. Harddisks and other stuff was introduced later with CF+ specification – but perhaps I’m not up-to-date or mongolian microtechnology will not care about.
Pingback: More on CF Cards | OS/2 Museum
Regarding Windows not accessing anything but the first partition: this is another problem of Windows treating devices that report themselves as removable specially. I also encountered problems formatting such devices when multiple partitions were present.
One workaround I used with CF cards was to use a CF-IDE adapter connected to IDE-USB adapter – that’ll pass the card’s actual type to Windows.
As I’ve stumbled into this very problem these days again (I’ve been one of the guys suggesting the ATCFWCHG.COM in the original article), let me add this. I’ll cite from here: https://kb-de.sandisk.com/app/answers/detail/a_id/51/~/enabling-fixed-disk-mode-on-sandisk-compactflash-cards
“SanDisk does not support its implementation with our consumer or professional-level cards, as it is outside of the scope of their intended use. Thus, all of our consumer and professional cards are shipped, by default, in Removable mode.”
Checked today.
Also, it’s not only Windows XP that does not like removable media, but also BeOS (Installs, runs, but complains about swap) and Solaris 8 and 9 (haven’t tried 10, but have no faith) wich plain ignores the disk. Also, back then I read that Transcend CFs flip their bit automatically. Well, my 32GB Transcend for sure does not. This one is also from 2017. So consider this an update on everything I said earlier. It’s BS now.
Cheers,
Florian
One addendum to the last one… According to Transcend, they sell at least their industrial cards as “fixed”: https://de.transcend-info.com/Support/FAQ-399
“Transcend Commercial CF cards are Removable Disk, Transcend Industrial CF cards are Fixed Disk”
VOGONS recommended Lexar BootIt to work with more CFs than ATCFWCHG.COM, but I have not yet given it a try.
Sorry for necro-ing this old post. Maybe this info is helpful to someone.
Cheers,
Florian
It is useful. Thanks!
OK, tested BootIt both on a Transcend card and an SanDisk card. Both purchased fairly recently. It didn’t work on either one.
To sum things up: the only way is to use either industrial CFs or SD2IDE adapters.
Cheers,
Florian