The other day I started wondering why certain old versions of 286 and 386 XENIX look a bit weird in emulation:
The characters are cut off, because XENIX sets up an EGA text mode with 8×14 character matrix but uses 8×16 VGA fonts.
Investigation showed that the XENIX console driver rather closely follows the inner workings of the EGA BIOS, no doubt because finding out exactly what IBM’s EGA BIOS did was not difficult.
Specifically, text modes normally access memory in EGA/VGA plane 0 where character codes and attributes are stored. Fonts are stored in plane 2 and the video hardware needs to be reprogrammed to let the host CPU access that plane. For whatever reason, the EGA BIOS uses a rather heavy-handed but very straightforward method: It defines two “fake” modes in its internal mode table, namely mode entry 0Bh for color and 0Ch for mono modes. The EGA BIOS simply internally sets mode 0Bh or 0Ch, uploads the font bitmaps, and then sets the real desired mode.
XENIX 2.2 uses the exact same method. If the BIOS does not fill out mode table entries 0Bh/0Ch, XENIX will be unable to program the fonts and the original VGA 8×16 font will remain in place, but there won’t be other ill effects.
So obviously the BIOS could just supply the EGA compatible mode table entries 0Bh/0Ch. And indeed that helps with some XENIX versions, such as 2.2.3c, which now gets the right 8×14 font:
But oops! With other XENIX versions, such as 2.2.3b, the result is much, much worse:
Why would that be? Finding out requires a little more digging…
First off, the problem was documented by SCO. Specifically XENIX 2.2.3 was documented to work with Compaq’s VGA, but not with IBM (ISA) or Olivetti VGAs. What was so special about Compaq VGA cards? As it turns out, nothing special per se, but it had to do with exactly how XENIX programmed text mode fonts. The logic is is not particularly difficult to analyze because the console driver is supplied in object form as cnscreen.o, included in /usr/sys/io/libio.a.
The BIOS mode tables mentioned above are easy for XENIX to find. All it needs is the pointer at 40:A8 in the BIOS data area which addresses the video parameter/save pointer table on EGA and VGA cards. The first entry in the pointer table points right at the mode parameter table, and the format of that table remained consistent across EGA and VGA BIOSes. Reading the table can be easily done from protected mode because real-mode far pointers are trivial to translate to physical addresses.
A much bigger problem XENIX had was finding the font bitmaps supplied by the BIOS. There is no convenient pointer that XENIX could read once it’s in protected mode. In retrospect, XENIX should have either called INT 10h/11h, AL=30h to figure out the font pointers while it was still in real mode, or it should have supplied its own fonts.
Instead, when XENIX was extended with EGA support, it was written to assume IBM EGA BIOS behavior: When a text mode is set, interrupt vector 43h points to the 8×8 font. Additionally, the fonts in the EGA BIOS are laid out such that the 8×14 font is first, immediately followed by the 9×14 extension, and the 8×8 font is last. When the XENIX console driver is initialized, it looks at the INT 43h vector, subtracts 0F30h bytes (the size of 8×14 font plus 9×14 extension), and assumes that that’s the 8×14 font.
On more or less any VGA, this logic breaks down, for two reasons: The INT 43h vector may not point to the 8×8 font, and the fonts are rarely laid out the same way as on an EGA.
There are some versions of XENIX (this is known to include 2.2.3b) which have added support for those above mentioned Compaq VGAs. The code looks strange at first glance but fortunately I’ve been able to cross reference it with a dump of the old Compaq VGA BIOS available on bitsavers.
The fonts in the Compaq VGA BIOS are laid out like this (the relative offsets are what XENIX cares about):
| Font | Rel offset (hex) | Abs offset (hex) |
| 8×16 | 0 | 2DC0 |
| 9×16 | 1000 | 3DC0 |
| 8×14 | 1144 | 3F04 |
| 9×14 | 1F44 | 4D04 |
| 8×8 | 2072 | 4E32 |
That is exactly what XENIX 2.2.3 expects. If the system has a VGA, INT 43h will (XENIX hopes) point at the 8×16 font, the 8×14 font is at relative offset 1144h, and 8×8 font at relative offset 2072.
Sadly, that’s not the case on, for example, IBM’s ISA VGA (aka PS/2 Display Adapter). In IBM’s VGA BIOS, the fonts are laid out as follows:
| Font | Rel offset (hex) | Abs offset (hex) |
| 8×8 | 0 | 378D |
| 8×14 | 800 | 3F8D |
| 9×14 | 1600 | 4D8D |
| 8×16 | 172D | 4DBA |
| 9×16 | 272D | 5EBA |
The fonts are ordered completely differently, and the assumptions XENIX makes about the Compaq VGA BIOS spectacularly break down.
Better Solution
In XENIX 2.2.3c (and presumably also in supplements xnx084/xnx086), SCO implemented a rather interesting solution which turned out to work reasonably well. After surveying the VGA landscape, SCO presumably concluded that their then-current approach of assuming certain layout of the font tables in VGA BIOSes was a lost cause. The fonts can be pretty much anywhere in a VGA BIOS, in any order, and don’t necessarily even have to be grouped together.
Someone at SCO then said down and thought, OK, we have no chance of guessing where the fonts are. But we can find them. Even though the character glyphs don’t have to (and in practice, don’t) all look the same across BIOS implementations, there are a couple of graphical characters that do, or at least really should, look the same.
XENIX 2.2.3c (namely function cnfindfont) expects a BIOS to provide fonts conforming to IBM PC code page 437, and looks for the “full block” character at code point DBh. This character should be represented as a sequence of FFh bytes. The length of the sequence in bytes equals the height of the font. The code looking for fonts does a few more “spot checks” for specific bytes in a couple of nearby locations. If the heuristic is satisfied, cnfindfont calculates the starting address of the font and returns that to the caller.
The heuristic would be easy to defeat, and perhaps was defeated by BIOSes with fonts not using code page 437. In practice the font search worked quite well on most systems.
It’s worth mentioning that if cnfindfont is unsuccessful, it returns a NULL pointer. There is no attempt at error checking, and in such case, the kernel will probably crash during initialization, quite possibly with a blank screen. That was, again, documented by SCO.
XENIX 2.3
With XENIX 2.3 (tested with XENIX 386 2.3.1a), SCO reworked the console driver to add full VGA support and did what they should have arguably done from the beginning—fonts were now included. I believe XENIX 2.3 still tries to find BIOS fonts, but has a fallback in case BIOS fonts can’t be found for whatever reason.
XENIX 2.3 also introduced the vidi command which allows the user to change the console mode, including CGA/EGA/VGA text modes. Support for dual color/mono adapters was also reworked relative to XENIX 2.2.
All in all, the XENIX 2.2 approach shows the limits of compatibility. The logic used for EGA worked, but relied on a high degree of IBM compatibility. The Compaq VGA was not compatible enough and XENIX broke down. SCO then added support for the Compaq VGA, but without validating that the system does in fact have a VGA card made by Compaq. SCO relied on internal implementation details (exact layout of fonts and INT 43h contents on entry) that were different on other VGA systems. SCO then added a heuristic to find fonts in a video BIOS, but even that probably wasn’t 100% reliable. It was clearly a learning process.
Addendum: XENIX 2.1
XENIX 2.1 may end up looking exactly the same as XENIX 2.2, but fonts won’t get corrupted. XENIX 2.1 has somewhat limited EGA support, in that it can deal with all the different EGA configurations (CGA monitor, MDA monitor, or hi-res EGA monitor) but does not even attempt to reprogram fonts.
The assumption with XENIX 2.1 seems to be that XENIX will set the same text mode the BIOS did, and therefore the BIOS will have uploaded the right font into video memory.
I believe that logic fell apart when SCO started adding support for graphics on XENIX (CGI libraries and more); the uploaded fonts were likely to be overwritten in graphics modes, and XENIX needed some way to correctly program them again. It’s just that the implementation in XENIX 2.2 was a bit too optimistic.
Reminds me of that issue where VGA cards were dropping support for the EGA ROM font (usually to free up space for VESA BIOS routines), and instead pointing to the VGA font. Some software like Soleau Software’s games had corrupt looking fonts as a result of this.
I wonder if EGA support was written when IBM was still doing Xenix.
I believe the VESA VBE 3.0 spec suggested dropping the 8×14 font to free up space.
The original IBM XENIX from 1984 did have limited EGA support… very limited in that it IIRC assumed a CGA monitor, and didn’t work at all with a proper EGA monitor. I haven’t compared the code in detail to get a sense of how much SCO did or didn’t change.
It did. I’ll quote the relevant text because it’s kind of interesting:
VESA strongly recommends that removal of the 8×14 VGA font become a standard way of freeing up space for VBE 3.0 implementations. The removal of this font leaves 3.5K bytes of ROM space for new functions, and is probably the least painful way to free up such a large amount of space while preserving as much backwards compatibility as possible. The 8×14 font is normally used for VGA Modes 0*, 3* and Mode 10h, which are 350-line or EGA compatible modes. When these modes are selected the 8×16 font may be chopped and used instead. When chopping a 16 point font to replace the 14 point, there are several characters (ones with descenders) that should be special cased.
Some applications which use the 8×14 font obtain a pointer to the font table through the standard VGA functions and then use this table directly. In such cases, no workaround using the 8×16 font is possible and a TSR with the 8×14 font is unavoidable. Some OEMs may find this situation unacceptable because of the potential for an inexperienced user to encounter “garbage” on the screen if the TSR is not present. However, OEMs may also find eventually that demand for VBE 3.0 services is great enough to justify the inconvenience associated with an 8×14 font TSR. To date, no compatibility problems are known to be caused by the use of such a TSR. VESA will make available a TSR that replaces the 8×14 font, please contact VESA for more information.
Another option with the fonts in Turn-Key systems (such as Laptops, Notebooks etc.) is to move the fonts to another location in the System ROM. In fact VBE functions could even be relocated. This however is not an acceptable solution for most desktop systems, where they are expandable.
I wonder why Xenix 2.2 did not add a real mode INT 10h/AX=1130h call to get the correct pointers.
That is indeed the mystery. My best guess is that XENIX didn’t have an architected way of passing information from the real-mode boot loader to the protected-mode kernel. Otherwise calling the BIOS would have been the most straightforward and reliable solution.
Especially as VGA did exist in 1987.
Yes, but — SCO had support for MCA machines and their MCA specific code knew how to fish out the right font out of IBM’s PS/2 BIOS. It probably took a while before VGA popped up on ISA machines in any kind of numbers, and it probably took a bit longer for SCO to realize that the ISA VGAs are all a bit different.
I know Compaq was selling Xenix at the time of their VGA introduction and I think IBM paid SCO to get Xenix over to the MCA machines. I believe the third VGA card SCO supported would be whatever was shipped with the Tandy 4000, another company paying for the privilege of selling Xenix. Note that Xenix did not support IBM’s ISA VGA card.
Xenix seems to have been placed in maintenance mode at the time with SCO’s focus being on SCO Unix instead.
How would it be possible to get the boot screen into 43-line mode? Was it a change of mode and a change of font or just a change of font?
The change of font is a change of mode, sort of. It’s not enough to point the character generator at different bitmaps, you must also adjust the CRTC character height and such. But it is not a “true” mode change in that all the basic CRTC parameters (width, height, blanking periods, etc.) remain unchanged, and other settings like text/graphics and memory access setup are also unchanged.
Anyway, which “boot screen” do you mean? A boot screen in general (used at the point where a boot sector takes over), or specifically the Xenix console screen? A boot sector might change the font/lines early on, but who knows how much software expects 80×25 to be the mode the system comes up in.
Tandy 4000 was a MCA machine, and I think that because they licensed MCA from IBM, they would have used an IBM chip, vs ATI, Tseng, Paradise, but I may be wrong.
The text mode I was looking for was the EGA 80×43, like DOS Mode command -> Con: Lines=43. The lab at school got a IBM AT as a server, and I ran TPascal on it, and I had read the DOS 3.1 manual nearby, and saw that video mode command. ( It had EGA Card, EGA monitor ). FrameWork worked in 43 line mode, but Lotus 1-2-3 required their own driver.
Xenix 2.3 supported it with the vidi command:
“e80x43 80 43 8×14 EGA (VGA) ”
but, the font is not 8×14, it needs an 8×8 cell.
“80×43 with 8×8 pixel font (effective resolution of 640×344)”
The text was in color, and had all the EGA dimming modes.
You would think that something this old, and available, after a few years would be supported, ( it was supported by Linux from the v2 kernel onward. )”
and you would also think that the documentation in Feb ’91 would have been QA,
It was iffy in Windows 98, but worked in every version of NT on a VGA monitor. It currently works flawlessly in Windows XP, 7, 10.
By boot screen, I mean the diagnostic screen where it shows the copyright.
I understand that this all happens in real mode, and that when it switches the processor to protected mode, with protected mode having no way to know what video mode the video card was in, except by leaving it alone and querying the hardware…
one slight extra cool feature… I wonder what resolutions the programmers were working at?
There was a vi clone on the Amiga, which would reset its lines and column settings every time you resized or moved the window, but that was on a graphics screen.
Actually *nix console programs can also adjust their text rows/columns at runtime if they’re written to support it. See SIGWINCH signal.
I never dug deep into it but XENIX may have had some way to adjust the initial kernel boot screen. It’s something that evolved and proper VGA support was added relatively late in XENIX’s life. At first glance I can’t find anything in the XENIX /etc/default/boot that would provide control over the video mode, it’s only possible to switch the console to a serial port.