OS/2 TCPBEUI Name Resolution

Sometimes I have the following problem to deal with: An OS/2 system uses NetBIOS over TCP/IP (aka TCPBEUI) and should communicate with a SMB server (likewise using TCPBEUI) on a different subnet. This does not work on OS/2 out of the box without a little bit of help.

History and Technology

NETBIOS (originally literally the ROM BIOS on the 1984 IBM PC Network adapter) was designed to work on a LAN, specifically a single LAN segment. There is no need for centralized infrastructure, workstations can come and go. This makes using ad hoc networks very easy and does not require additional dedicated infrastructure and administration.

When NETBIOS (or NetBIOS) moved to Ethernet, there were initially many different ways of encapsulating it. Eventually the world settled on NetBIOS Frames aka NBF.

But in the 1980s, there was also a parallel effort to move NetBIOS on top of TCP/IP, eventually standardized as RFC 1001 and RFC 1002 (both dated March 1987). This effort was originally driven by non-PC platforms, but soon enough DOS-based (e.g. HP ARPA Services circa 1990, PC/FTP likely earlier) and OS/2-based (MS LAN Manager 2.1 in 1991) implementations of NetBIOS over TCP/IP became available.

As mentioned above, classic NETBEUI (whether using the original IBM PC Network Adapter, Token Ring NETBEUI, the NBF protocol, or some other variant of NetBIOS over Ethernet) always resolves names using broadcasts. When a workstation (i.e. a NetBIOS application running on that workstation) looks for a NetBIOS “name”, it uses a broadcast to find out the network address of the machine which owns that name; this is not unlike ARP.

Note that NetBIOS names are labels for various resources or “objects”. A workstation or a server has a NetBIOS name, a logged-on user has a NetBIOS name, a workgroup or a domain has a NetBIOS name.

When a client machine wants to use the resources of a server, it needs to resolve the NetBIOS name. With NetBIOS over TCP/IP, broadcasts can also be used. A so-called B-Node (Broadcast Node) is a machine with an IP address which resolves NetBIOS names purely using broadcasts. Just like the original NetBIOS, such a network is limited to a single LAN, or more specifically a single IP subnet.

Another kind of node defined in RFC 1001 is a M-Node, or Mixed Node, which can use broadcasts to communicate with NetBIOS applications on the same subnet, but also use NetBIOS Name Servers (NBNS) and NetBIOS Datagram Distribution Servers (NBDD) to extend the reach across multiple subnets. This obviously requires additional infrastructure in the form of NBNS and NBDD servers.

The NBNS architecture solves much the same problem as DNS. An NBNS implementation can be centrally controlled, but it can also be loose and allow NetBIOS names to be registered and unregistered freely.

There are also P-Nodes (Point-to-point Nodes) designed for dial-up connections; these are no longer relevant.

As a side note, RFC 1001 also defines the concept of a NetBIOS scope. This is not often used, but a NetBIOS scope allows separate logical NetBIOS networks to exist on a single TCP/IP network. Only machines with the same NetBIOS scope identifier can “see” each other. This is handled on the NetBIOS over TCP/IP protocol level and is transparent to NetBIOS applications.

While NBNS allows NetBIOS over TCP/IP to function across large networks, it doesn’t work with B-Nodes, requires additional infrastructure, and therefore users may wish to avoid it. More about that later.

From NETBEUI to TCPBEUI

In the PC space, Microsoft drove the switch from the NetBIOS Frames protocol to NetBIOS over TCP/IP. In Windows XP, NBF support took extra effort to install and in Windows Vista it was dropped altogether.

On the OS/2 side, NetBIOS for TCP/IP was initially available with Microsoft’s LAN Manager. When IBM took over OS/2 networking, things were initially rather complicated. IBM’s LAN Server and the LAN Server clients only supported the NBF protocol out of the box. However, IBM’s separate TCP/IP product offered an add-on NetBIOS kit. With the kit in place, a LAN Server client could use the classic NETBEUI, or NetBIOS over TCP/IP, or both.

LAN Server 4.0 integrated NetBIOS over TCP/IP support. Instead of requiring a separate IBM TCP/IP product with an add-on NetBIOS kit, a new implementation called TCPBEUI was now part of the Multi-Protocol Network Support (MPTS), alongside the classic NETBEUI.

Starting with OS/2 Warp Connect, the LAN Server client, TCP/IP, and TCPBEUI support were all shipped with the OS. When using the advanced install path of Warp Connect, users were confronted with the following dialog:

Note that the radio buttons are misleading because selecting NetBIOS over TCP/IP results in both NETBEUI and TCPBEUI installed.

The IBM OS/2 TCPBEUI support consists of two core components, TCPBEUI.OS2 and NBTCP.EXE. TCPBEUI.OS2 is an NDIS driver which plugs into the NDIS Protocol Manager (PROTMAN.OS2) and handles most of the NetBIOS protocol. It is configured through the PROTOCOL.INI file, which is typically located in the C:\IBMCOM directory.

The NBTCP.EXE component is a daemon which implements the actual NetBIOS over TCP/IP protocol and handles the TCP/IP configuration and name resolution. Both are required for TCPBEUI to function.

Helping TCPBEUI with Name Resolution

From the earliest days, there have always been some methods to assist the NetBIOS over TCP/IP name resolution. If all you need is connecting to one or two machines on a different TCP/IP subnet, then setting up NBNS is probably an overkill.

Note that the following screenshots are from OS/2 Warp Connect. It is assumed that the Warp Connect system has TCPBEUI installed and that TCP/IP is functional.

The IBM TCPBEUI offers a way to define NetBIOS name to TCP/IP address mapping. There is a GUI method accessible through the MPTS/LAPS configuration interface. To change the configuration, the user needs to select the IBM OS/2 NETBIOS OVER TCP/IP protocol and hit the Edit button:

This leads to a fork in the road:

The Names list interface allows adding the actual names:

The NetBIOS names may need to be entered in all caps to be safe. Each NetBIOS name must correspond to either an IP address or a host name resolvable through DNS.

Note: As the dialog hints, the name entered by the user is a NetBIOS name prefix. This is because NetBIOS names that need resolving often have a one-byte binary suffix. The prefix is meant to act as a kind of wildcard and match NetBIOS names regardless of the suffix. This avoids the need to enter all the required permutations of a NetBIOS name, which are far from obvious.

On the TCP/IP side of things, the name may be resolvable through a DNS server, but it can also be configured locally in the HOSTS file (usually in C:\MPTN\ETC\HOSTS). Or, as noted above, an IP address can be used directly instead of a host name.

The NetBIOS name to host machine mappings are stored in a file called RFCNAMES.LST, normally stored in the C:\IBMCOM directory. Note that RFCNAMES.LST is a standard text file and can be edited with a text editor. If the file is changed, the rfcaddr.exe utility (also in C:\IBMCOM) must be run to update the running NBTCP.EXE instance. NBTCP.EXE also reads RFCNAMES.LST on startup.

Setting up this configuration comes with a big caveat. For whatever reason, IBM left a nice trap for the unwary. By default, NBTCP.EXE allocates enough space for zero RFCNAMES.LST entries. In other words, RFCNAMES.LST will be silently ignored.

To avoid this problem, it is necessary to change the “Maximum number of name-ip address pairs in names file” in the MPTS configuration interface to some reasonable non-zero value:

The setting is stored in PROTOCOL.INI. Instead of using the configuration GUI, users can also edit PROTOCOL.INI directly, changing the value of the NAMESFILE keyword in the [tcpbeui_nif] section. Changing the setting requires a reboot, because PROTOCOL.INI is only processed on startup.

NetBIOS Names as DNS Names

Most versions of the IBM protocol stack support an alternative method of NetBIOS name resolution which could be attractive in some setups but is extremely cumbersome. It is possible to use DNS to resolve RFC-encoded NetBIOS names.

A NetBIOS names is a sequence of 16 bytes (all bits are significant, hence the names need not be printable). RFC 1001 defines a method for converting a 16-byte NetBIOS name into a string of 32 ASCII characters. The ASCII encoding can be converted back to a 16-byte NetBIOS name. The ASCII name is defined such that it is also a valid domain name.

Using the example from RFC 1001, the NetBIOS name ‘FRED’ will be encoded as ‘EGFCEFEECACACACACACACACACACACACA’ — which, although extremely ugly, is a valid DNS name.

It is possible to set up DNS alias records which map these RFC-encoded names to IP addresses. When TCPBEUI tries to resolve NetBIOS names, it can RFC-encode them and then use standard TCP/IP name resolution. The downside of this scheme is that it requires control of DNS, and that for every machine several NetBIOS names with different last bytes must be registered. For example, for a SMB domain server, four RFC-encoded DNS entries would be needed.

Newer Windows versions support a radically simpler scheme, where the NetBIOS name is resolved as a DNS name without any encoding. This is not a generalized scheme, because NetBIOS names are not necessarily valid DNS names. However, if the OS only accepts machine names that are valid DNS names and at the same time valid NetBIOS names, this scheme works.

This scheme was introduced in Windows NT 4.0 (but not enabled by default; the details are complicated) and was used by default on Windows 2000 and later. IBM implemented it in Convenience Package 2 (MCP2/ACP2); it is detailed in README.ADD on the installation CD-ROM, in a section titled “TCPBEUI Unencoded Name Lookup on Domain Name Server”.

For whatever reason this capability is not enabled by default on OS/2. It is controlled by the ENABLEDNS keyword in the [tcpbeui_nif] section of the PROTOCOL.INI file. Also note that unlike Windows NT 4.0 and later, OS/2 does not support using IP addresses in place of machine names.

When ENABLEDNS = 1, TCPBEUI first tries to resolve RFC-encoded names, then DNS names. When ENABLEDNS = 2, TCPBEUI tries to resolve DNS names first, then RFC-encoded names. By default, ENABLEDNS = 0 and no unencoded name resolution takes place.

Worse yet, there is yet another catch. Setting ENABLEDNS to a non-zero value is not enough to enable DNS lookup. The capability is firmly tied to the lookup of RFC-encoded names (because that will be done either before or after looking up RFC-unencoded names). And lookup of RFC-encoded names is enabled by setting the “NetBIOS Domain Scope String”, which corresponds to the DOMAINSCOPE keyword in the [tcpbeui_nif] section of PROTOCOL.INI.

Note: The “NetBIOS Domain Scope String” is a DNS suffix. If a server has a domain name mysmb.domain.net, then the domain scope string should be ‘domain.net. When TCPBEUI looks up the NetBIOS name “MYSMB”, it will automatically add a ‘.domain.net’ suffix before resolving the name. The “Local NetBIOS Name Scope String” is completely unrelated—it has nothing to do with DNS and it enables the creation of multiple logically separate NetBIOS networks on top of a single TCP/IP network (as defined by the RFCs). The local NetBIOS scope is almost always empty; if you don’t know exactly what to set it to, you need to not set it.

The unencoded DNS name lookup fairly similar to what Windows does. The difference is that on Windows, no manual configuration is usually required; Windows (specifically Windows 2000 and later) performs the unencoded name lookup by default, and queries the DNS suffix through DHCP.

On OS/2, there is no built-in capability to set the TCPBEUI NetBIOS Domain Scope String based on DHCP options, although it is possible to write custom scripts to get the options from DHCP, rewrite PROTOCOL.INI, and reboot the machine. Such an approach in outlined in the Beyond DHCP Redbook.

Debugging

In TCPBEUI shipped with OS/2 Warp 4 and later, it is possible to get limited debug output from NBTCP.EXE by adding a DEBUGNBTCP = “YES” keyword in the [tcpbeui_nif] section of PROTOCOL.INI. This causes NBTCP.EXE to print debugging information to the screen. For this capability to be useful, NBTCP.EXE must be started after the OS boots up and not through a RUN= statement in CONFIG.SYS. This is no problem as long as NBTCP.EXE starts before the LAN Requester.

Note that NBTCP.EXE can also be started with the /d parameter to enable debugging, but the debug setting is soon overridden by the DEBUGNBTCP value from PROTOCOL.INI… which defaults to disabled. And which itself appears to be deleted when the MPTS configuration GUI is used.

Summary

There are two ways of configuring OS/2 TCPBEUI to find servers on a separate TCP/IP subnet in the absence of a dedicated NetBIOS name server. The first method works on OS/2 Warp Connect and later and requires the following:

• Add a NetBIOS name and hostname (or IP address) pair for each machine to the \IBMCOM\RFCNAMES.LST file, either using a text editor or the MPTS GUI interface
• Set the maximum number of entries in the names file to a sufficiently large non-zero value, either by using the MPTS GUI interface or modifying the NAMESFILE keyword in the [tcpbeui_nif] section of PROTOCOL.INI

The advantage of this method is that it works with old versions of TCPBEUI. The names list can also be dynamically updated with the help of rfcaddr.exe. The disadvantage is that every machine on a different subnet must be manually added to RFCNAMES.LST.

The alternative method only works on Convenience Package 2 and requires the following:

• Enable unencoded DNS name lookup, either in the MPTS GUI interface or by setting the ENABLEDNS keyword in the [tcpbeui_nif] section of PROTOCOL.INI to 2 (or, in an unlikely case, to 1)
• Enable DNS name lookup by setting the NetBIOS Domain Scope String, either in the MPTS GUI interface or by setting DOMAINSCOPE = "my.domain.net" (as appropriate) in the [tcpbeui_nif] section of PROTOCOL.INI

This newer method has the advantage that machine names do not need to be added manually, but the name resolution only works as long as all machines have the same domain name suffix. And, obviously, the DNS names must match NetBIOS names, although that is typically the case already.

Addendum: IBM TCP/IP 2.0 NetBIOS Kit

As mentioned previously, things looked a little different before LAN Server 4.0 and Warp Connect. The standard LAN Server 2.0/3.0 client only supported NETBEUI, and to get NetBIOS over TCP/IP support, one had to acquire the IBM TCP/IP base kit (1.2.x or 2.0) as well as an add-on NetBIOS kit. It is described in some detail in an IBM Redbook, TCP/IP 2.0 for OS/2 Installation and Interoperability. The Redbook refers to the older implementation as TCP/NetBIOS; this convention will be adopted here to distinguish between TCP/NetBIOS and TCPBEUI.

The old TCP/NetBIOS implementation was structured differently from TCPBEUI; there was only a small kernel driver (NBDRIVER.SYS) and almost everything else (notably all interfacing with TCP/IP) was done in the NBTCP.EXE process. This necessitated relatively high context switch overhead and IBM claimed that the newer TCPBEUI was about twice as fast as TCP/NetBIOS.

TCP/NetBIOS also took over NetBIOS functionality completely. Although the LAN Requester/Server could use dual NETBEUI and TCP/NetBIOS protocol stacks, NetBIOS applications had to use TCP/NetBIOS only.

Although TCP/NetBIOS had much the same capabilities as TCPBEUI, it was configured quite differently. It was not configurable through the MPTS interface at all; instead, command line parameters had to be passed to NBTCP.EXE. Additionally, NBTCP.EXE was not started from CONFIG.SYS but rather from STARTUP.CMD or possibly some other script executed after TCP/IP was brought up; therefore the TCP/NetBIOS configuration effectively resided in that script file.

TCP/NetBIOS supported much the same names and broadcast list files as TCPBEUI, but the locations and names of the files were not predefined. The full pathname was simply passed to NBTCP.EXE through the -n (names file) and -b (broadcast file) arguments.

The syntax was also slightly different—TCP/NetBIOS does not expect NetBIOS names to be enclosed in double quotes.

NBCP.EXE might be started as follows, after TCP/IP is already running:

START NBTCP -n C:\NAMES.LST 10.0.2.15 255.255.255.0

Note: The old NBTCP.EXE has a -d (debug) switch which does not match documentation. Or at least in the version of NBTCP.EXE shipped with CSD UN09313 it doesn’t. The -d switch expects an argument which appears to be a mask of debug flags controlling what gets written into an nbtcp.trc file. Passing -d ALL to NBTCP.EXE should log all possible messages.

And one more caveat: If the old NBTCP.EXE is not passed the local machine’s IP address correctly, the LAN Requester may fail to start with “duplicate network name” errors. However, when it is configured properly, TCP/NetBIOS plus the LAN Requester from LAN Server 3.0 can connect to SMB servers using NetBIOS over TCP/IP.