For a long time, I couldn’t quite grasp what the DOS APPEND command could possibly be good for. Until I came across a situation which APPEND was made for.
When I worked on organizing and building the DOS 2.11 source code, I tried to place the source files in a tree structure similar to that used by DOS 3.x (this is known from DOS 3.x OAKs):
C:.
└───src
├───bios
├───cmd
│ ├───chkdsk
│ ├───command
│ ├───debug
│ ├───diskcopy
│ ├───edlin
│ ├───exe2bin
│ ├───fc
│ ├───find
│ ├───format
│ ├───more
│ ├───recover
│ ├───sort
│ └───sys
├───dos
├───inc
└───msdos
The inc subdirectory unsurprisingly contains shared include files such as DOSSYM.ASM, which are included just about from everywhere. No problem, right?
Except… to get output that most closely matches existing DOS 2.x binaries, it is necessary to use an old version of MASM (version 1.25 seems to do the trick). But MASM 1.25 is designed to run on top of DOS 1.x, and knows nothing whatsoever about directories.
It is possible that back in the day, DOS 2.x was built from a single huge directory on a hard disk. In fact it is known that DOS 2.0 could not be built on PCs at all, and was built on DEC mainframes. Yet DOS 2.11 was also clearly modified such that it could be build on PCs using Microsoft’s development tools.
However it was done back in 1983, lumping 150+ assembler source files into a single directory, and then adding hundreds of object and executable files, did not sound at all appealing. Cloning DOSSYM.ASM to every directory where it was needed seemed even worse.
That’s when I somehow remembered that APPEND exists, and realized that it’s the perfect solution to the problem. Before building, one can run
APPEND ..\..\INC;..\INC
and the inc directory becomes accessible from all of its sibling subdirectories and from subdirectories one level deeper. It would have been possible to use an absolute path as well, but this way the build batch file does not need to know where it lives.
With APPEND in place, the old MASM 1.25 which uses FCB I/O will find the centrally located include files, and the source code can be organized into a neat hierarchical structure that’s far easier to work with than one giant blob.
What is APPEND?
APPEND is a “DOS extension”, in fact it is a TSR which intercepts INT 21h and adds special handling for several subfunctions. These are primarily:
- 0Fh FCB File Open
- 3Dh Handle File Open
- 23h Get File Size
If these subfunctions fail to find a file in the current directory, APPEND will retry them using the list of paths it manages.
When building DOS 2.11, MASM 1.25 will try to open DOSSYM.ASM using INT 21h/0Fh (FCB File Open). Because the file does not exist in the current directory, the initial attempt will fail. APPEND will then try opening ..\INC\DOSSYM.ASM and, if that is unsuccessful, also ..\..\INC\DOSSYM.ASM. Old MASM is thus magically upgraded to handle multiple directories, without actually knowing anything about them.
The working principle of APPEND is not complicated. It primarily serves as a bridge between old DOS applications which have no or poor support for directories, and users who really, really want to organize files and programs in multiple directories and possibly across multiple drive letters. Of course the actual APPEND implementation is anything but straightforward.
APPEND Evolution and Implementation
The first DOS version which came with APPEND was DOS 3.3 (1987), not coincidentally the first DOS version developed by IBM.
But APPEND is older than that—it first appeared in the IBM PC Network Program 1.0 in 1985. It is hard to speculate why it was shipped with the PC Network Program (later the PC LAN Program) because APPEND does not really have anything to do with networking. It is plausible that it was especially useful with networking, when users were motivated to store applications on a central network server and data files on their own machines. And that’s a problem for applications which cannot handle directories well.
Now that we can see the source code for APPEND, some things are clearer. The original PC Network Program version of APPEND was written by someone with initials G. G. A., and in 1986 it was adapted for shipping with DOS by B. A. F., no doubt Barry A. Feigenbaum (best known for developing the SMB protocol).
APPEND by default manages the path list in its own internal storage. But it also has a /E option which instead causes APPEND to look for an eponymous environment variable. This mechanism has a disadvantage in that the APPEND= variable needs space in every newly created environment. On the other hand, it also allows different DOS processes to have different APPEND paths.
It should be noted that OS/2 implements a mechanism analogous to APPEND /E through the DPATH environment variable. Only on OS/2 it’s built in, with no need to load TSRs.
Another APPEND addition was the /X switch, which causes APPEND to hook further DOS subfunctions, most notably Find First and Exec. This effectively allows APPEND to supplant the PATH environment variable.
APPEND is listed in the PC DOS 3.3 reference as both internal and external command. At first glance that doesn’t make any sense, but it’s actually true.
The first time APPEND is run, it is an external command. But when it installs itself as a TSR, it hooks INT 2Fh/AEh. COMMAND.COM, in turn, calls INT 2Fh/AEh when it is asked to execute a command that COMMAND.COM does not know about. This mechanism allows APPEND to function as an internal command once it is installed.
That is, the first time APPEND is run, it must be loaded and executed from disk. But any subsequent attempt to run APPEND through COMMAND.COM, either from the command line or a batch file, will take a shortcut directly to the already installed TSR, effectively turning APPEND into an internal command. The INT 2Fh/AEh interface between COMMAND.COM and a TSR was added in DOS 3.3, quite likely for the benefit of APPEND.
APPEND also has its own programming interface, accessed through INT 2Fh/B7h. This allows programs to control APPEND behavior and query the current APPEND path. How widely this is used isn’t entirely clear.
Summary
APPEND is one of the things that are completely irrelevant 99.99% of the time… yet can be extremely useful when the need arises. It is a TSR which allows applications to find files in a directory other than the current one.
The first appearance in APPEND was in the IBM PC Network Program (1985), but since version 3.3 (1987) it was integrated into DOS, with an interesting link to COMMAND.COM which allows APPEND to become an internal command once it is installed.
I have this vague memory of playing DOS game, protected from copying by hardcoded CD DATA path, using DOS command to redirect it to C: “backup”. At first I thought I was using APPEND, but it was another another redirecting utility SUBST
There’s also JOIN. And that’s when I realized, that when all three
programs are brought together, it will open Shub-N…
Sorry, wrong story :X Lovecraftian horrors abound.
Is there an implication in this article that Microsoft were using an internal version of APPEND to organize the DOS 2.11 source code and compile it with MASM 1.25 as well? And that neither they nor IBM really saw that it could be useful to release it as part of DOS until 3.3?
No. I see no evidence that Microsoft did that.
I also have no idea if Microsoft had anything like APPEND, but my guess is that if they did, IBM wouldn’t write their own.
I strongly suspect that Microsoft primarily built the DOS code on DEC mainframes in the 2.x days. It is possible that near the end of the DOS 2.x cycle, they used a newer MASM version with directory support. It’s also possible that individual developers built on PCs using a newer MASM, yet the official OAKs were built on DECs as before, using an older MASM version.
And it’s also possible that developers using PCs just shoved everything into a single hard disk directory. It’s difficult to say.
There is an APPEND.COM in the MSDOS 3.2 install over at PCJS. I find it strange that the MSKB notes on APPEND don’t include version 3.2 in the list of applicable versions.
@Richard. Confirmed. I checked my printed DOS 3.2 manual and APPEND is documented there.
What do you mean by “DEC mainframes”?
I believe early Microsoft (1975 onwards) did stuff on PDP-10s running TOPS-10, but then when they got Xenix (1978 onwards) they started to run stuff under Xenix on PDP-11s. PDP-10s are often called “mainframes” (although I know some people dispute that), while PDP-11s are usually called minicomputers.
I don’t know what they developed DOS under, but by the time DOS 2.0 was being developed I think a PDP-11 minicomputer running Xenix would have been more likely than a PDP-10 mainframe. But I don’t really know.
I used append (or maybe it was join) to allow applications too large to run on a single floppy disk to share my A and B drives simultaneously, thus giving 2+MB of storage. Think of it as very dumb RAID0.
In my mind this is what the tool was really meant for, but I guess you make your own purpose.
I have some explanation for that. APPEND was not included with PC DOS 3.20 (files dated Dec ’85) or 3.21 (Feb ’86). Which makes sense because according to the APPEND source code, conversion to DOS was done in June 1986.
There is a MS-DOS 3.21 OAK which definitely includes APPEND, but that was released in May 1987.
There are also disk images of MS-DOS 3.20 floating around, with files dated 07/07/1986. That also includes APPEND, significantly different from the one that shipped with PC DOS 3.3, and it’s APPEND.COM rather than APPEND.EXE.
PC DOS or MS-DOS?
The only thing that is 100% certain is that several files in the MS-DOS 2.11 source code contain the following comment: “The DOST: prefix is a DEC TOPS/20 directory prefix. Remove it for assembly in MS-DOS assembly environments using MASM.”
So whatever ran TOPS-20. Microsoft literature (The MS-DOS Encyclopedia) also says that when they ported 86-DOS to the IBM PC, a “DEC-2020” system was used, presumably a DECSYSTEM-2020. That would have been 1980/1981.
A DECSYSTEM-2020 running TOPS-20 then seems like a reasonable inference.
I know that Microsoft’s development tools for PC XENIX could compile for DOS. I do not know if Microsoft had that before they wrote their own compiler (1984). Of course there’s no telling what unreleased internal tools Microsoft may have had.
APPEND, JOIN, and SUBST were oddities which pointed the way to features like symlinks, but DOS never quite got there. (I still use SUBST to this day to map letter drives on a Windows server so that the drive letters can be the same as client PCs mapping those shares.) OS/2’s DPATH was another abortive attempt.
It’s surprising no true symbolic link (or even hard link) support existed until the NT era; you can create hard links on a FAT filesystem, but they will be destroyed the next time you run CHKDSK.
One person’s weird hack is another person’s union mount.
Windows NT had nothing like symlinks before Windows 2000 AFAIK, and the NTFS support was barely exposed to users before Vista.
It does look like a scenario for symlinks. I tend to do my DOS development in DOSEMU with drives other than C: mapped to UNIX directories, so I probably would have used links to make files appear in more than one directory.
That’s certainly one way to do it. I wanted something that can be used on real DOS, so symlinks were out of the question.
Side track:
The things that Microsoft ran on “DEC mainframes” and Xenix, for internal use, is IMHO a missing important piece of computing history.
Is anything of this preserved at all?
@MiaM: Yeah, good question. Me’s also wondering if anything of the
native NT (!windoze) environment survived.
OS/2 had the Toronto Virtual File System (TVFS). Looking at the copyright, 1994, 95, 96, 97, I thought it was older.
To quote the Introduction from TVFS.INF,
…
The Toronto Virtual File System (TVFS) is an OS/2 Installable File System that combines VM and UNIX (**) file system concepts. The specific VM file system concept employed is the “CMS search path”. The specific UNIX file system concept employed is the “symbolic link”. Each of these concepts is described below.
…
It was very handy for symlinking multiple drives together as well any other use that calls for symlinks. You could also symlink multiple directories to one TVFS directory with a search order that could be customized, guess this was the CMS search path originally meant for mounting a bunch of minidisks in one directory. Permissions were simple, r w and rw
The two things / categories I probably would find the most interesting is the development tools but also the server side things that they used with various DOS/Windows clients. Like how a BCC field found it’s way onto the screenshots of Microsoft Mail which is said to had been a part of the Xenix mail backend/server that they used internally but never sold.
It would also be interesting to know what, if any, file services and whatnot they ran on Xenix that weren’t available to the general public.
It would also be interesting to know what hardware they ran Xenix on. PDP-11 is mentioned here, but at some point in time I would guess that they moved to PCs.
Another piece of history that would be interesting to know is what connectivity/network they had at different points in time. Did they use serial lines / terminal ports for everything at early stages? If so that would probably explain why all their development tools were of the kind that appeal to people who let things take some time to simmer. Like those who write pages after pages of code, let it compile while they go on a coffee break, and somehow they only made three syntax errors and after correcting them the code just works. I.E. the opposite of the rapid cycle integrated development tools like Borland’s Turbo products.
Find the article titled “The Sun Never Sets on Microsoft’s Network” in the August 22, 1988 issue of InfoWorld. It might answer one or two of those questions.
@MiaM: on your latter point: me finds both approaches to be helpful–
messing about in a rapid-prototyping environment[0] in order to test
new ideas, and only then taking a good while to re-write it in code
that will last decades.
That zeroth step can be done in one’s head though, to a surprising
degree, though there’s really no substitute for a “whack it and see
if it sticks” facility.
[0] There’s this old UNIX dream of combining the shell with an actual
C interpreter; one abortive such experiment let to the thus-misnamed
csh(1).
TVFS was a great piece of technology (I used it back in the late 1990s to build Unix-based applications whose build process otherwise expected a lot of symlinking to be going on). One of many, many EWS packages that really shoud have been incorporated into the GA OS/2 releases.
To add to the redirection mess: re-reading[0] reminded me that there is
also ASSIGN. Clearly, the desperate need for symlinks, in all forms, was
there quite early.
[0] https://os2museum.com/wp/dos/dos-2-0-and-2-1/
Perhaps it was only balanced by the desperate need to avoid the horrible mess that symlinks inevitably create.
@zeurkous: There were earlier efforts to simplify locating files on disk. Having worked with DEC OSes, I can point to a couple of examples. The DCL Assign command could do what ASSIGN and SUBST did with DOS. RSTS had baked in support to let programs access special system directories in addition to the user directory. RSTS did require a character prefix to the filename to get at the special directories but anyone who had problems with automatic path searches would appreciate the ability to specify when not to launch files in other directories.
Unix did produce a generalized solution for all such uses with symlink but that was years later on systems with much more memory.
@Necasek: from an elegance POV, symlinks are a horrible kludge, yeah.
They’re also a life-saver, one that me wouldn’t to do w/o.
@Richard Wells: that sounds more like a generalization of $PATH to find
arbitrary files — me’s actually (partially) implemented something like
that on this machine, w/ a var named PREFICES that contanes a list of
hierarchies in the way PATH contains a list of directories w/ binaries.
(In theory, PATH and ../ could be used, but that’s just *kludgy*…)
But yeah, that’s all easy to do now, with the abundant (cough)
resources at our disposal… It’s just that having 4 diff overlapping
(yet incompatible) kludges to do what’s essentially the same job *is*
pushing it a little.
I use SUBST in my everyday work to this day on my local drive.
I do a lot of work with Visual Studio and I’ve found my paths to some of the projects can get nested fairly deep. I considered setting the long paths in Windows itself, but that would make the next persons life miserable.
I have been using SUBST to set my project to a short path and everyone is happy. Git and VS work with the relatives paths very well. It is a feature that has lived up to the task.
Seems like I suck at finding things online. Any hints on where to find August 22, 1988 issue of InfoWorld?
Re the general problem that assign, join and subst, and also symlinks and environment variables tries to solve:
I think that Assigns on the Amiga, and the similar Logicals on VMS, is an elegant solution. In both cases the OS uses names for each disk/partition, like DOS and CP/M, but unlike DOS and CP/M the names are multi letter and arbitrary rather than a fixed series of single letter names. In addition to the disks having names, assigns/logicals allows additional names to point to arbitrary subdirectories on any disk, and they are used with the same syntax as if you were accessing a disk. On the Amiga, C: is where simple commands are stored, usually in the c directory in the root on your boot disk. Since version 2 of AmigaOS it’s possible to point one assign name to multiple locations, so you can for example have the OS default commands in one directory, and other commands you add in another directory, and refer to both as C:. Although there is a path, I would say that the path mechanism was implemented due to AmigaOS before version 2 couldn’t do these multi assigns. Also with assigns you don’t need environment variables for things like where you have your include and library files for software development. The compiler would just look in the current directory (or specified path) for a quotation mark include file name, and look in include: for less-than/greater-than enclosed include file names. Similarly the linker would look for libraries using an assign. The elegant thing is that each program using this only has to throw the assign name and file name to the file system open API, without parsing environment variables (and/or configuration files). Sure, for something large like a compiler or linker this doesn’t matter much, but for something simple this removes a noticeable burden when writing an application.
Environment variables exist in AmigaOS, but they exist as files on the (dynamic) ram disk (and there is also an “archive” that is usually stored on your boot disk, that gets copied to the ram disk at boot). Since they are files, they can contain arbitrary binary data, in particular configuration settings for the OS and applications.
@MiaM: Try Google Books and look for the issue list. Click on 1985. It should show as a very long horizontal scroll and just keep going until hitting the issue.
I just modified all the source to point to the files within the tree so I could just use other tools..
INCLUDE ..\..\inc\DOSSYM.ASM
INCLUDE ..\..\inc\DEVSYM.ASM
etc etc.
https://github.com/neozeed/dos211/blob/main/cmdms/command/command.asm#L52C1-L53C37
It just seemed easier.
I can’t imagine not using something like Xenix, or later MS-DOS 4.00 or OS/2 to build stuff, but apparently, they did not.
Also Merry Christmas!
Sure it’s easier — but doesn’t help if you want to get the same binaries and MASM 1.x must be used.
As for later DOS versions, that is an interesting question. For example with MS-DOS 4.0, there are several custom build tools that aren’t built from source and the binaries are DOS only. Yeah it could probably be built on OS/2… in a DOS box. Even though the compiler/assembler/linker were available as native OS/2 apps.
It is plausible that individual developers could have worked on XENIX or OS/2 or perhaps even NT later on, but it looks like the official builds were done on DOS probably since the days of DOS 3.0 or so.
Or worst case try this: https://books.google.com/books?id=YToEAAAAMBAJ&lpg=PA1&hl=en&pg=PA38-IA4#v=onepage&q&f=false
Thank you very much for laying the mystery to rest. I started out exploring DOS under Windows 98, never managed to figure out what APPEND was for.
Pingback: The Other Three | OS/2 Museum
I must be mis-remembering but I was sure that there were tools for hard and soft file links, and junctions for directories in windows nt 3.51 SDK/DDK, and also that `ln` and `ln -s` was available for Services for Unix.
@James Wright
Hardlinks have always been available on NTFS since the beginning, going back to NT 3.1. However, reparse points (such as directory junctions) rely on the Reparse Manager, which didn’t exist Windows 2000. NTFS symbolic links didn’t come along until Vista (it’s another type of reparse point), but some clever Japanese fellow implemented them on 2000 and XP using a filter driver – https://emk.name/symlink.html
TL;DR: SFU must have been doing something VERY tricky to implement symlinks on pre-5.0 NT.
This KB article https://www.betaarchive.com/wiki/index.php?title=Microsoft_KB_Archive/322733 makes it sound like symlinks were new in SFU in 2001 or so. Which happens to match when NTFS implemented them.
I use SUBST on our local server to map
O:toC:\User\username\OneDrive - Business Name.On other client PCs, there is a logon script (nowadays I think we call that a GPO)
NET USEofO:to a share ofC:\User\username\OneDrive - BusinessName. (We do this so that every client PC isn’t repeatedly downloading the same files from OneDrive over and over, which is a bit of a problem when we use OneDrive to store 1GB-sized image scans, and yes, the resident OS/2 machine maps that drive too.)But the server itself can’t use that, and it’s handy to be able to hardcode O: various places. So I just run a
SUBST O: C:\User\username\OneDrive - BusinessName on the server… well… sort of. That had an annoyance that each logon user (when accessing server via Remote Desktop) had to have that in their logon script, and that administrative or elevated sessions wouldn’t see the drive unless they ran their own SUBST command; nowadays we add this to the registry:[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\DOS Devices]
“O:”=”\\??\\C:\\Users\\username\\OneDrive – BusinessName”
… except that left the Recycle Bin in an odd state where it wouldn’t function for 32-bit apps, but instead files would be instantly deleted. (And OneDrive provides special Recycle Bin function I don’t want to bypass.) The GUID below can be whatever you want; generate a new one for each drive.
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\FolderDescriptions\{9147E464-33A6-48E2-A3C9-361EFD417DEF}]
“RelativePath”=”O:\\”
“Category”=dword:00000004
“Name”=”O_Mapped_OneDrive”
[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\BitBucket\KnownFolder\{9147E464-33A6-48E2-A3C9-361EFD417DEF}]
“MaxCapacity”=dword:0000c7eb
“NukeOnDelete”=dword:00000000
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Explorer\FolderDescriptions\{9147E464-33A6-48E2-A3C9-361EFD417DEF}]
“RelativePath”=”O:\\”
“Category”=dword:00000004
“Name”=”O_Mapped_OneDrive”
[HKEY_CURRENT_USER\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Explorer\BitBucket\KnownFolder\{9147E464-33A6-48E2-A3C9-361EFD417DEF}]
“MaxCapacity”=dword:0000c7eb
“NukeOnDelete”=dword:00000000
SUBST lives, and in a particularly bizarre version accessed via a registry key called “DOSDevices”, with yet-more registry keys needed to make 32-bit and 64-bit executables work the same way.
@Josh Rodd:
At least on NT4 it was possible to map drives to your local computers shared files.
Many many years ago I used that as a method to hide files, sort of, on a computer. A shared directory (with a dollar name) deep enough, and the directory above it set so that no user had permissions to access that directory, made it impossible to traverse it as part of the local file system. However it was easy to access it as a network share, locally.
Don’t know if you could use that on your server? Would likely decrease the performance as compared to SUBST, but still.
As far as I know, this is generally possible on any machine with IBM/Microsoft file sharing. You can do ‘net use’ with the drives shared by the local system, not just remote systems.
I have no idea how that compares performance-wise with SUBST. It might be slower, but it shouldn’t be slow.
I didn’t want the machine to rely on network functioning, and I also wanted an elevated user or Administrator to be able to see the O: drive too.
This stuff all still works on Windows 2025 Arm, too…
DOS 2.x was not built on PCs, nor was Xenix or the C Compiler.
OpenNET from Intel linked PCs to VMS/VAXen. I have searched long and hard for this.. but I found the diagram in Intel’s Documentation while looking for Intel’s mid level developer engine called CompileEngine. ( a 286 with two hard disks, and Intel OpenNET hardware and software)
The diagram was from a product being used in large software houses in 1987, “Intel Development Tools Handbook, Pg 18 or xiv ( pg 14 ), having surpassed the need for a VAX when a dedicated 386 could do the job, slower, and a lot cheaper.
The PC Xenix C compiler had its own problems:
“The documented history of Xenix-11, Xenix-1, Xenix-3, and SCO System V, Xenix 386, 286 and 8086.”
Do you have some evidence that OpenNET existed in 1983, when DOS 2.x was built? I would be quite surprised if you could find any.
From the released source code it is obvious that DOS 2.0 was built on a DEC machine, but by the time DOS 2.11 was out, the source code could be built on PCs. DOS 2.0 was definitely not built on PCs, but DOS 2.11 might have been.