Learn Something Old Every Day, Part XIII: InDOS Is Not Enough

The other day I spent a while trying to understand the purpose of a rather strange looking piece of code inside Borland’s THELP.COM utility shipped with Turbo Pascal 6.0 (THELP.COM was misbehaving under emulated DOS).

The THELP utility performs the following actions:

• Use INT 21h/34h to get the address of the InDOS flag
• Starting from the beginning of the InDOS segment, search for word 803Eh using the SCASW instruction
• If found, check if the location in memory five bytes past the 803Eh word holds the value BCh
• If so, store the word just past 803Eh for later use

This logic is applied to DOS version 4 and below (effectively 2.0 to 4.x), not newer versions. But what could it possibly be good for?

To find out, I applied the THELP.COM search logic to a running copy of DOS 3.3. I quickly discovered that it’s looking for a specific piece of code within DOS.

The 803Eh word (or rather the byte sequence 3Eh 80h) is the opcode of ‘cmp byte ptr [mem], imm’. And BCh is the opcode of ‘mov sp, imm’. There is almost certainly only one instance of this particular instruction sequence within any DOS 2.0-4.x.

Cross referencing the instructions with the DOS 4.0 source code, I quickly enough found the following sequence inside the INT 21h function dispatcher:

    CMP     [ERRORMODE],0
    JNZ     DISPCALL
    MOV     SP,OFFSET DOSGROUP:IOSTACK

What THELP.COM is after is the offset of the ERRORMODE variable in the DOS data segment (which will be the same segment the InDOS variable is in).

OK, but why is this necessary? Trying to find documentation about the long-undocumented INT 21h/34h function provided most of the explanation.

The InDOS flag is normally set when DOS INT 21h is entered and cleared when leaving. It is apparent that a TSR like THELP.COM must check the InDOS flag before trying to do things like load something from a file.

The catch is that if DOS is within the critical error handler, then the InDOS flag is clear yet it’s not in fact safe to call into DOS.

The PRINT.COM utility has exactly the same problem, but because it comes with DOS, it solves it in a vastly more elegant fashion than THELP.COM. The code can be seen here and this is the relevant section:

   lds  si,[INDOS]      ;Check for making DOS calls

                ;---------------------------------------
                ;
                ; WARNING!!! Due to INT 24 clearing the
                ; INDOS flag, we must test both INDOS
                ; and ERRORMODE at once!
                ;
                ; These must be contiguous in MSDATA.
                ;
                ;---------------------------------------
   cmp  WORD PTR [SI-1],0

PRINT.COM fully agrees that checking INDOS is not enough. But rather than fishing for the critical error flag (ERRORMODE) in memory and then checking both separately, PRINT.COM knows that the ERRORMODE flag is stored right below the INDOS flag in memory, and therefore both flags can be checked with a single word comparison.

In DOS 2.x, the ERRORMODE flag was stored right after INDOS, again making it possible to test both in one go. The deal breaker was apparently Compaq DOS 3.0 which stored the ERRORMODE flag in a completely different location. The code in THELP.COM should be able to find it and work with such a “strange” OEM DOS version. Chances are that Compaq DOS 3.0 was not the only odd one.

It is worth noting that the THELP.COM version I looked at was released in 1990. Being conservative, it only searches known DOS versions, up to 4.x; the search logic is not applied to DOS 5.0 and later. Which is just as well because it might not work when the DOS code is separate from DOS data.

For a utility like THELP.COM, not being able to check the critical error flag is unlikely to be a fatal problem in practice. Users who try to invoke the TSR in the middle of a critical error handler only have themselves to blame.

PRINT.COM does not have that luxury because it is invoked from the timer interrupt, and might try to run at any time when interrupts are not disabled, critical errors or not. It is a given that for PRINT.COM, attempting to run during a critical error handler was a real problem that needed to be solved.