An important fragment of PC history was unearthed a few days ago: An image of a Compaq Deskpro 386 supplemental disk from August 1986, containing among other things CEMM.EXE, Compaq’s original expanded memory emulator shipped with the Deskpro 386. The image is available on pcjs.org (in the 3.10 directory).
Why is CEMM important? It was essentially by definition the first publicly available utility using the 386’s new Virtual 8086 (V86) mode, and it was the first 386 memory manager. Eventually CEMM morphed into EMM386 shipped with DOS and was the “standard” counterpart to memory managers such as Quarterdeck’s QEMM, Qualitas’ 386MAX, or Helix’s Netroom. EMM386 née CEMM was itself a very important piece of software.
This early CEMM release is dated August 8, 1986 and reports version 3.20, presumably to match the then-current EMS (Expanded Memory Specification) version. It’s highly likely that this was the disk shipped with the first ground-breaking Compaq Deskpro 386 systems in September 1986.
This version of CEMM is strictly an EMS emulator with no support for UMBs. It uses V86 mode and paging to turn memory beyond the 1MB address line into expanded memory and maps it into and out of the EMS page frame (typically a 64K window located between 640KB and 1MB). When EMS is not in use, CEMM can “disappear” and run the processor in real mode. When EMS is used, the CPU must run in protected/V86 mode.
There is no support for VCPI or any similar interface in this CEMM release, but since there were no 386-specific applications yet when it was released, it would have been redundant.
Unfortunately, this CEMM version does not run on a generic 386 AT compatible. However, it can be convinced to run with a bit of hacking:
Perhaps one day pcjs will emulate the original Compaq Deskpro 386, especially now that it’s open and anyone can contribute to the project (it’s made great strides recently in emulating an IBM PC/AT). Then this version of CEMM and other Deskpro 386 utilities would be easy to run in any modern browser.
The exact history of CEMM is unclear. It is obvious that Intel must have been involved, but it’s uncertain whether Microsoft also took part in the initial development or only later on. Compaq and Microsoft did collaborate on the development of Windows/386, which shared a few similarities with CEMM/EMM386. Microsoft in turn slightly influenced Intel’s 386 design, and was one of the early 386 adopters (386 XENIX, Windows/386).
At any rate, this version of CEMM is a museum piece in every sense of the word—an important piece of the PC history puzzle, nearly 30 years old. And perhaps other versions of CEMM from 1987-1988 will surface in due time, too.
Very interesting blog post, as always.
Have you tried running CEMM in pcem? How about MESS? I think it could actually work in pcem (not so sure about MESS). Maybe I’ll try it on the weekend, as a fun side project 🙂
I don’t think either pcem or MESS will work, unless you have a Deskpro 386 ROM (if you do, I want it 🙂 ). As you can see from the screenshot, I was able to run CEMM, but it requires hacking either the CEMM executable or the BIOS. It’s not that hard to figure out what/how.
Compaq’s patent application (4926322) underlying CEMM is interesting reading including what looked to be a major design flaw. Many of the important protected mode data structures like the GDT are placed in Segment E (E0000-EFFFF). Non-Compaq systems might be a bit more likely to use that for hardware ROM. See the last paragraph of the patent filing before the Appendix.
EMS 3.2 was 1985; EMS 4.0 was 1987; and VCPI was released in 1989 (along with Lotus 123 R3). VCPI was included in several 16-bit DOS extenders. Before VCPI got invented, 16-bit DOS extended programs would not work if CEMM was loaded.
Robert Moote has sections in Extending DOS devoted to VCPI and multitasking challenges for protected mode. The diagram on page 385 of how VCPI memory gets mapped into emulated EMS which in turn is mapped to extended memory is delightfully baroque.
With the benefit of having the actual software at hand, I can say that the patent application does not describe the released CEMM 3.20 from 08/08/86. This one puts the IDT/GDT/TSS in low memory, next to the protected-mode code and data. It is theoretically possible that the behavior might be different on an actual Deskpro 386, then again the same patent application claims that CEMM tries to place the EMS page frame at segment E000h, so it is internally inconsistent.
VCPI was the usual after-the-fact design… problems don’t get solved until they’re big problems, and then they are all the harder to solve 🙂
Oh, and this is a very minor nit, but there was a difference between “CEMM loaded” and “CEMM active”. Just having CEMM loaded didn’t necessarily mean the CPU was in protected mode.
The Extending DOS books are excellent (both editions).
Were there ever any EMS 3.0 drivers (presumably for an EMS hardware board) or was there too little time between EMS 3.0 and 3.2 ?
EMS 3.0 was announced in May 1985 at the Spring 1985 COMDEX
EMS 3.2 spec is dated September 1985
EMS 4.0 spec is dated October 1987
Interestingly Compaq had the EMS 3.2 spec a month before it was publicly released though Compaq was a major player so this isn’t too surprising.
Something I noticed: besides the check for the Compaq BIOS ID at F000h:FFE8h, CEMM also checks the DOS version. Apparently it is doing this to handle VDISK.SYS and its VDISK control block:
DOS 3.0 VDISK is VDISK V1.0 (not supported by CEMM 3.2)
DOS 3.1 VDISK is VDISK V2.0 (supported)
DOS 3.2 VDISK is VDISK V3.2 (supported)
DOS 3.3 VDISK is VDISK V3.3 (didn’t exist at the time so CEMM will incorrectly use VDISK V3.2)
DOS 4.0 VDISK is VDISK V4.0 (didn’t exist at the time, not supported)
@dosfan: Intel AboveBoard was shipping at the time of the EMS 3.0 announcement. I don’t have the disks so can’t confirm the contents. Dave Williams’ DOSREF lists EMS 3.1 so there may have been a number of minor updates before EMS 3.2 solidified.
I have never seen any official mention of an “EMS 3.1” or any other mention for that matter so that list doesn’t count without corroborating evidence. The only expanded memory specs that I’ve ever seen mentioned are EMS 3.0, EMS 3.2, EEMS (by AST and Quadram) and EMS 4.0.
I’ve never seen an actual EMS 3.0 driver or the official EMS 3.0 spec but supposedly it supported INT 67h functions 49h and 4Ah which were dropped in EMS 3.2 and it didn’t support INT 67h function 4Eh which was added in EMS 3.2.
I found that “DOSREF” list and it is filled with errors, misstatements and hyperbole. That is definitely NOT a legitimate source of info like say Ralf Brown’s Interrupt list. Unless I see real evidence to the contrary (actual spec or an old EMS driver) I’m going to say that either there was never an “EMS 3.1” or it just wasn’t released publicly.
I know you’ve been seeking this for quite some time, glad you finally found it!
It’s kind of confusing with the version numbers, but I guess 1.x stuff was written on simulators or beta chips? And is it given that Microsoft emm386 is an offshoot of cemm386?
Having seen the EMM386 source code I can say that EMM386 is definitely an evolution of CEMM.
EMS 3.0 is the first version of the spec. It is lost to time as to why it was called 3.0 but I would guess to match the then current DOS version.
Tall Tree System had their memory paging method for DOS at version 2 when Lotus/Intel made their announcement. Tall Tree created an EMS compatible driver for their cards by June 1985. Jumping version numbers past the competition isn’t exactly unknown.
The only online reference to the pre-EMS banked memory designs is in the April 2, 1985 issue of PC Magazine which mentions the programming interface for Tall Tree JRAM and a competing design from Datatron.
IBM had their own proposed real mode banking system with the XMA cards. Topview 1.1 supported the banking method. IBM was so late that other software instead used EMS emulation software (either IBM’s internally developed variants or a copy of Above Disc that IBM sold with some cards). There were several other programs that managed to have EMS interfaces into what was officially extended memory. Compaq approached the problem from a slightly different angle and came up with what proved a better alternative.
Since (excellent) Extending DOS books have been mentioned I would like to ask how much different is first from second edition. I only have second edition and this is the one I’ve read back in ’90. It seems that there was around a year difference between publishing each edition so I wonder how much content has been added. I guess the DPMI chapter has been extended/rewritten. Are there any sections that got removed from the 2nd edition?
The interesting thing that I’ve realized later it that Extending DOS and Dos and Windows Protected Mode: Programming with DOS Extenders in C by Al Williams (who also authored DOS 5: A developers guide) books are in fact a collection of articles published earlier in Dr Dobbs. Here is one example: http://www.drdobbs.com/windows/programming-with-phar-laps-286dos-extend/184408713
Let’s see… the books are © 1990 and 1992, respectively, so they’re a bit further apart. The first edition only has a brief chapter on DPMI for that reason. The first edition covers DOS/16M and OS/286 in the 16-bit extender chapter. I don’t think PharLap’s 286 extender existed yet and that’s not covered at all. There’s less about Windows in the first edition. Overall the structure is quite similar with chapters on EMS, XMS, VCPI, DPMI, Windows, DesqView etc.
I don’t think even the second edition covers DOS/4G which is ironic, as that was probably by far the most widespread DOS extender. It is however understandable because it was a relative latecomer (1991-1992).
Thanks for a summary Michal. I can’t remember when DOS/4G became available, earlier they were selling DOS/16M extender but it was DOS/4G(W) that made the company and it’s product “famous”. There were also producing Instant-C – very interesting product based on their DOS Extender technology. It is described in Appendix E of Extending DOS 2nd edition. I’ve checked and DOS/4G is being mentioned on page 199 in this book edition along Phar Lap 386|Dos Extender (predecessor of TNT extender) however Dos and Windows Protected Mode book provides more detailed description of both extenders. On the other hand both books are build around Phar Lap products as far as example code goes. In case of Extending DOS this shouldn’t be a big surprise since Andrew Schulman used to work for Phar Lap (he even mentions it in the book if I remember correctly) at that time.
The one interesting fact is that many extenders came from Massachusetts state.
One more detail: the Tenberry site (authors of DOS/16M, DOS/4G(W) and Instant-C) is still available after all those years although I couldn’t find any history files so it is. The earliest DOS/4G version I could find in my archive comes from 1992 and is 1.40. The last one I’m aware of is 2.61 from 1998.
I think I have a DOS/4G beta somewhere… will have to put all those versions together some day.
DOS/16M goes back to 1987. I believe some version of Instant-C essentially included a DOS extender and Rational spun that off as a separate (and quite successful) product. Several major packages (Lotus 1-2-3 R3, dBase IV) used DOS/16M. It indirectly forced the creation of DPMI because these products somehow had to coexist with Windows, whether Microsoft wanted that or not.
DOS/4G came out in 1992 I believe. Of course the killer was bundling it (as DOS/4GW) with 32-bit Watcom C/C++, now I can’t remember if 9.0 or 9.5 was the first version. There were probably hundreds of DOS games that used DOS/4G in some form, not to mention all the other software. And yeah, Extending DOS is understandably Phar Lap biased.
The Massachusetts connection is interesting (it’s memory managers, too) but probably was a logical byproduct of MIT and Harvard.
When you find the DOS/4G beta make a post about please 😉 I used to have Instant-C somewhere but can’t find it right now.
One notable game example that wasn’t using DOS Extenders but instead unreal mode was Novalogic Comanche. Great game BTW. Due to Windows popularity Novalogic had to switch to different memory management option later.
Ultima VII was another famous game with a home-brew DOS extender. I don’t actually remember if Origin used unreal mode or something else, but it was definitely not compatible with anything, starting with EMM386. Given how memory hungry the game was (required over 600K conventional memory IIRC), the only possible reaction was “WTF were they smoking”.
Will look for my DOS/4G archive.
That was the Voodoo Memory Manager – http://wiki.ultimacodex.com/wiki/Voodoo_Memory_Manager
Apparently Ultima VII did several things which made it hard to run – http://nerdlypleasures.blogspot.com/2013/06/ultima-vii-on-real-hardware.html
Origin must have gotten LOTS of support calls for that game. It’s incredible they used unreal mode considering that many people used a v86-mode EMM. It’s even more incredible that even with using unreal mode the game still needed over 600KB conventional memory.
This was in 1992 so it wasn’t too hard to get over 600KB free if DOS was loaded in the HMA. Note I’ve determined the theoretical maximum free conventional memory with DOS 5 or later is ~631KB assuming everything possible is loaded in upper memory and you are running in real mode using hardware UMBs obtained from RAM normally used for ROM shadowing. PC DOS 7 gives you the most with 631.4KB free conventional memory possible.
Yes, Voodoo is what it was. Most appropriate name. Getting over 600K free with a basic configuration was no problem, but the game was unusable without a disk cache. And that’s where things got a lot more interesting. If you needed some kind of memory resident drivers for a sound card, that was another problem. Also the BDA can steal a few KB in some systems. And you already needed mouse and possibly CD-ROM drivers. Like I said… what were they thinking?
The issue with Ultima VII thrashing the hard disk is more troubling than the memory usage. What the heck was this game doing ? Granted using unreal mode in a commercial product was a bad idea considering there were four EMMs commonly in use at the time (EMM386, QEMM, 386MAX and Netroom) though I’m guessing that Origin didn’t want to pay to license a proper DOS extender and setting up unreal mode was significantly easier than writing a DOS extender. Worse is the fact that Origin wasn’t alone in doing this as Alex mentioned that Novalogic Comanche also used unreal mode.
That brings up an interesting question: how many programs actually used unreal mode ? Obviously HIMEM.SYS used it to implement the XMS move block function.
About EMS: has anyone ever come across the original EMS 3.0 specification ? I have the EMS 3.2 and 4.0 specifications but could never find the original 3.0 specification and I don’t recall ever seeing it back in the day. My guess is that the time period between 3.0 and 3.2 was so short (~4 months) perhaps only a few major players of the time got it.
Lots of games were just amazingly badly written, usually by inexperienced programmers. Developers always used a disk cache so they wrote dumb code that probably didn’t do any buffering whatsoever and just kept hitting the disk all the time… and no one ever told them that they were doing it wrong.
I remember CEMM first coming out with the original deskpro/386, and looking into how it enabled V86 mode
Came out at the same time, and possibly ‘inspired’ the creation/release of Digital Research Concurrent DOS/386.
There was already a Concurrent Dos /XM which used the more advanced EMS boards (neede one that could page ’ems like’ memory into low memory regios, as opposed to the mainstream board that only paged in the EMS top of 1MB region.
So the use of V86 mode allowed the same idea to work on a new 386 system and not need add on dedicated memory cards
dosfan: I read about the announcement of EMS 3.0, but when a customer ordered a board ( The intel board was late so they ordered a TallTree JRam board ), it came with EMS 3.2, literally dated two days before the board arrived. Looks like They had EMS 3.0, and updated it quickly. Funny no one mentions the LIM spec, Lotus/Intel/Mircosoft spec, with Intel providing the chips, Microsoft providing the OS Support, and Lotus providing the Application. ( EXPANDED MEMORY SPECIFICATION, Version 4.0, October, 1987 ), Which would make the time frame about a year after the Compaq hit the market, and people were still holding on to they 286s. ( I was given a 20Mhz 286, after someone upgraded to a 386… and it ran a tiny bit slower… but just for dos ).
Thanks guys. It’s good hearing from people who were there, vs stupid wikipedians.
@StephenC:
I wonder about this thread:
http://www.vcfed.org/forum/showthread.php?59157-Concurrent-DOS-GEM-and-KnowledgeSet
There are claims that GEM already had problems with 1024×768 for example.