Sometimes the quest for backwards compatibility has unintended consequences. In some cases, the presumably beneficial backwards compatibility turns into a source of problems. The costs end up far outweighing the benefits, yet the “feature” may be difficult to get rid of. If only it hadn’t been implemented in the first place…
If you’re on a Windows system, try a small experiment. Take an existing text file, say foo.txt, and try renaming it to aux.txt. Oops, it doesn’t work:
Try renaming to aux.doc, or aux.with.extension. It doesn’t work either. That surely looks like a bug, doesn’t it? Why would the operating system arbitrarily refuse to rename a perfectly good file? In fact Windows does that to be backwards compatible with an operating system that was obsolete before many current PC users were even born.
Reserved names in DOS 1.0
In DOS 1.0, character devices existed in the same namespace as ordinary files. It is questionable whether that was a good idea even in 1981, but it was done for compatibility with CP/M. What it meant was that DOS reserved several “special” names for built-in devices. Part of the problem was that there was in fact nothing special about the names themselves, it was just a somewhat arbitrary list which included (looking at IBMDOS.COM shipped with PC DOS 1.0): PRN (printer), LPT1 (parallel port), AUX (auxiliary device), COM1 (serial port), CON (console), and NUL (the null device). Additional names like COM3 or LPT7 were also reserved (for numbers in the 1-9 range).
In DOS 1.x, there were no installable device drivers yet, so the list was predictable. When DOS 2.0 introduced installable drivers, the scheme applied to all devices, not just the ones built into DOS. By convention, installable character drivers included a dollar sign in the device name, such as CLOCK$ or POWER$. In some cases, the name was merely chosen to be “unusual”, such as XMSXXXX0 (for HIMEM.SYS). This was done to reduce the probability of clashes with actual file names.
However, the real bug-formerly-feature was elsewhere. In DOS 1.x, like CP/M before it, there were no directories. Files were not organized in a hierarchy, they simply existed in a flat name space on a (floppy) disk. In the era of 160K or 320K floppies, there were only so many files that would fit on a disk, therefore the lack of hierarchy didn’t pose major difficulties.
Reserved names in DOS 2.0
When DOS 2.0 introduced support for fixed disks (typically 5-10MB in size), a flat namespace was no longer an option. Microsoft chose to implement a UNIX-style hierarchical directory structure (along with numerous other UNIX-like features). And this is where backwards compatibility comes in.
Applications written for DOS 1.x knew nothing of directories. Yet it was desirable to run them under DOS 2.0, and it was desirable to run them from subdirectories. When run in a subdirectory, a DOS 1.x application would think that the directory is the entire disk. There was just a small problem with the built-in devices. Since DOS 1.x applications expected the CON, AUX, PRN, etc. devices to exist alongside regular files, a decision was made to replicate the special names throughout the entire directory hierarchy. Because of that decision, probably made sometime in 1982, you are now unable to create a file called aux.txt on a 64-bit Windows system 30 years later.
This was markedly different from UNIX where devices were typically accessed through the /dev directory. But in UNIX, even though it’s much older than DOS, hierarchical directories were designed in rather than merely bolted on later.
Back in 1982-83, Microsoft and IBM could have decided to create a similar special device directory, or they could have decided to only propagate device names throughout the directory hierarchy for legacy DOS 1.x applications. But they did neither. The problem was compounded by the fact that for device names, DOS does did consider extensions at all. That’s why any file name whose base portion is e.g. PRN is reserved.
As a result, applications (and batch files, in many cases) continued to refer to CON or PRN and expected them to exist “everywhere”. As time went on, the amount of software referring to those special files only grew and getting rid of them was harder and harder.
Reserved names in Windows NT
When Windows NT was designed, use of these special names was so widespread that it had to be replicated in the new operating system. In NT the DOS-compatible names aren’t hardcoded, but they do exist by default. Interestingly, in the name of consistency and backwards compatibility, these device names are propagated even on network shares hosted by foreign operating systems.
That provides for amusing (or annoying) situations where e.g. a Samba share on a Linux or OS X system contains a file named, say, com5.txt. Windows will see this file in directory searches, but will be unable to open it. It is very difficult to argue that this is a useful feature.
The situation in Windows NT is a little more complicated. The core OS does not subscribe to the notion of reserved filenames. The backwards compatibility is implemented in the Win32 layer, which is admittedly 99.99% of what comprises Windows NT today. However, it does mean that for example POSIX applications can create and access files with “reserved” names unimpeded. Microsoft in fact recommends the POSIX rm utility for getting rid of undesirable files with reserved filenames which cannot be easily removed through the Windows user interface.
Clearly this feature has outlived its usefulness, but removing it would no doubt lead to subtle and not so subtle breakage in unexpected places. For example, batch files which redirect output of programs to NUL (in order to throw it away) would suddenly start creating files called NUL all over the place. And that’s probably one of the more benign examples.
The moral of the story is simple: When considering backwards compatibility, consider the future first; it’s often much longer than the past! And in general, try not to end up in an inconvenient situation where the best explanation is “it seemed like a good idea at the time”.
This is the most nice description from the history and language perspective I’ve ever read for the problem: “What’s wrong with aux.txt file?”. Thanks Michal.
Win32 applications can prefix paths with “\\?\” in calls to CreateFile to bypass most of the Win32 logic.
In DOS 1.0, character devices existed in the same namespace as ordinary files. It is questionable whether that was a goodidea ?even in 1981, but it was done for compatibility with CP/M.
Everything is a file. What wrong with this?
This was markedly different from UNIX where devices were typically accessed through the /dev directory.
On *nix device can be created in any directory or, more modern approach, devfs can be mounted to any directory. ‘DEVICE’ file attribute would be a most rational method. Btw, MSDOS versions 5-6 have virtual \dev\ directory, e.g. echo lalala >\dev\nul worked.
How does does that help an ordinary user manipulate a file called aux.txt from the Explorer?
That everything is a file isn’t wrong. What’s wrong is that (on DOS) an unpredictable set of reserved file names exists in every directory in the filesystem. In UNIX, devices can be certainly created anywhere in the system, but any UNIX which tried to create device nodes everywhere would be laughed out.
Sure, in new DOS versions there’s a \dev directory, that’s not a problem. The problem is that those versions still pollute the filesystem namespace with driver names, so the problem remains. If devices were accessible only through the \dev directory, there would be no name clashes.
“Back in 1982-83, Microsoft and IBM could have decided to create a similar special device directory, or they could have decided to only propagate device names throughout the directory hierarchy for legacy DOS 1.x applications. ”
That would do nothing for batch files. For example “type con >config.sys”.
Would it have been so hard to implement compatibility behavior for batch files? Perhaps a new extension for “modern” batches like was later done with .CMD files on OS/2 and NT? Don’t tell me that Microsoft did not have people smart enough to figure out a better solution than the horrible hack they ended up implementing…
Sorry for the delay, I didn’t receive notification on email.
How does does that help an ordinary user manipulate a file called aux.txt from the Explorer?
It does not help. it just thoughts about proper design devices as files. In two words: copy from unix and simplify.
Sure, in new DOS versions there’s a \dev directory, that’s not a problem. The problem is that those versions still pollute the filesystem namespace with driver names, so the problem remains. If devices were accessible only through the \dev directory, there would be no name clashes.
It will breaks compatibility with myriads of bat files.
Solving this problem without breaking compatibility is hard.
I’m kinda interested, can you create aux.txt from explorer in new x64 windows (Vista, 7) ? They not even have dosvdm anymore.
On other hand, cmd backward compatible with command.com
The screenshot is actually from 64-bit Windows 7. So yes, the same behavior is still there even though the NTVDM component is gone.
A backwards compatible solution could have been requiring names in the format aux: or con: (with a trailing colon). Those would have worked on DOS 1.x AFAIK but were reserved names in DOS 2.0 anyway.
A backwards compatible solution could have been requiring names in the format aux: or con: (with a trailing colon). Those would have worked on DOS 1.x AFAIK but were reserved names in DOS 2.0 anyway.
Yes, it would have worked, if were implemented in DOS 2. In times of DOS 3.x the compatibility with tricks like this:
if exist dir\nul.ext echo do-something
was much more important.
Still, after dos support removal, issue with explorer can be fixed without losing compatibility by moving code handling the reserved names from win api layer to cmd.
If the special name functionality were moved into CMD.EXE, that would break an unpredictable number of (Win32) applications which open PRN, CON, etc. There are some, but who knows how many.
And yes, by the time DOS 3.0 was out it was a bit late to start fixing the problem. It should have been done right when DOS 2.0 was released to minimize the effort. This is the kind of issue that is harder and harder to fix as time goes on.
DOS had the virtual \DEV directory since 2.0. DOS 2.x and 3.x even had an AVAILDEV option in CONFIG.SYS to make it mandatory.
The fun thing is that this may have been done for CP/M compatibility, but CP/M doesn’t have device names at OS level; they’re implemented by programs, and always followed by a colon so they don’t share the same namespace as normal files.
Not requiring the colon (which was allowed but optional) seems to have been the fatal flaw. Once there was no way to distinguish between a file and a device based on its pathname, there was just no good way out.
BTW I don’t think the AVAILDEV keyword was there in DOS 3.x. The internal functionality was apparently still there, but not the CONFIG.SYS switch. The fact that AVAILDEV wasn’t documented didn’t help much either…