On ISA, probing, drivers and policy

[Mellvik]

This writeup was triggered by the discussion in #1500 and @ghaerr's summary in #1500 (comment) which is partly quoted below.

The discussion revealed a number of misunderstandings, misguided assumptions and lacking policy as far as where ELKS should be going. Also apparent was a lack of respect for the community effort ELKS is, an attitude that needs to be taken seriously because it destroys the idea of community development.

A word on ISA technology

It's an exaggeration right there. ISA is a name, not a technology. It's simply an extension of the PC main board bus with bus drivers (LS245) added for increased fanout stability. The LS245 goes tristate when not selected, making whatever is behind it invisible on the bus.

When an IO request is made on the ISA bus and the active address does not match an existing port, the state of the databus is technically undefined (floating). There are no pull-ups or pull-downs, the state (level) is determined by the components - which from the PC/AT and onwards means 'floating high'. It is thus safe to assume that reading a non-existent port on such systems will return 0xFF. Pre-AT systems apparently do not have this predictable behaviour.

ISA probing

Bus probing is the operation of determining whether a given IO address has a device assigned to it. Being 'just a bus' with no intelligence and no rules/procedures for identification and auto configuration, probing on the ISA bus is guesswork.

Probing has two levels. The first is to determine presence - whether anything is connected at a given port, the second is identification: what is connected. If the first is non-trivial, the second is non-non-trivial. The good news is that 100% reliable probing on both levels is possible, the bad news that it may take a lot of coding. No rules and thousands of different interfaces of all ages.

Case in point: When I started rewriting the NE2K driver some years back it had no probing, just an empty routine returning success. Looking at code from other systems I found a probing mechanism that worked (incidentally the same that @ghaerr is now suggesting). It worked fine but was a lot more complicated (more functionality) than desirable. And it had a serious drawback: It looks for a 8390 chip.

This may seem like a good thing except

1. All kinds of interfaces have 8390 chips, how do I know it's not a 3c503 or some WD8xxx we're talking to?
2. There are many variants of the 8390, will this work with some or all?
3. Most 'modern' NE2K interfaces use the RTL8019 chip, not the 8390. They're compatible, but how compatible? Will the mechanism work with those?

No insurmountable challenge, but the choice was easy: Use the simples probe possible - just like the probes used for the serial and parallel ports in ELKS. Which turned out to work fine for years - and still does, although it may need some minimal adjustments to accommodate for the undefined float of the early 8bit ISA bus.

Probe depth is a policy question, not a design consideration. Anything is possible given enough code. The probe routine in the 3c509 driver combines part of the initialization code and the probe and takes advantage of the fact the the EL3 supports ISA PlugAndPlay AND provides a convenient shortcut to its soft configuration management. This shortcut makes it possible to configure the interface completely at boot time, via /bootopts. It is also bloated - and thus candidate for encapsulation in appropriate #ifdefs.

The wd driver probe is almost as simple as the current ne2k probe and takes advantage of the fact that the MAC address PROM is directly available without initializing the device. This makes for a reasonably safe level 2 probing. Like the 3c509 driver, the actual probe routine includes basic initialization, which is a programming decision that has nothing to do with the actual ISA probing mechanism.

Policy

IOW - probing depth is a policy decision on a system where minimal footprint (and simplicity) is important. I have for some time run a version of the ne2k driver which identifies the RTL8019 if present and picks up the preset IRQ (and IO address) from the interface during probe. Getting the IRQ is useful, the IO address is not, since it will have to be correct in order to get there. So the addition is not sufficiently useful to publish.

I've also partly implemented PnP support for ELKS, in order to understand the nitti-gritty details of PnP since most of the interfaces (ethernet included) actually support PnP. It's interesting, it's ugly (PnP is known for that), and it will remain an experiment: Adding it will bloat ELKS, which is undesirable.

Why am I mentioning this? Because there are many ways to the goal of making device configuration simpler. And they all come at a cost.

The previous discussion

Finally, som comments on the misunderstandings in @ghaerr's summary of the previous discussion:

ne2k_probe:

        // Poke then peek at the base address of the interface.
        // If something is there, return 0.
        // No attempt is made to get details about the i/f.

        mov     net_port,%dx    // command register
        mov     $0x20,%al       // set page 0
        out     %al,%dx
        in      %dx,%al
        cmp     $0xff,%al       // cannot be FF
        jz      np_err
        cmp     $0,%al          // cannot be 0
        jz      np_err
        xor     %ax,%ax
        jmp     np_exit
np_err:
        mov     $1,%ax
np_exit:
        ret

The first thing to recognize here is that the NE2K driver ne2k_probe performs a proper ISA probe (while also rejecting 00 return values, which does not concern us here).

It does concern us to some degree: While the FF test is all we need (AT and up) to determine presence, the zero test following a valid command increases the likelihood that this is actually a ne2k.

However, the NE2K driver only uses the ISA probe to determine whether an NE2K NIC is present. This means that any card plugged in at the netport address that responds not equal to FF will be seen as an NE2K NIC. Therein lies the problem for early XT systems.

No, this is incorrect - see comment above. Reading FF means no device, not a device that returns FF. It may be an unwritten rule, but I don't think there are any devices out there that would respond to any input with a FF.
It would be reasonable to guess that if not floating hight, the older systems may be floating low. But if that was the case, the serial port interface probe should fail - if I'm reading the code correctly. They seem to work.

So let's figure out how the early ISA bus behaves when reading or writing nonexistent ports. Gathering such knowledge should take a couple of minutes at most using DEBUG on MSDOS.

Yes - that would be nice to know. But it doesn't change anything, since the the IBM 5160 is obviously returning some value between 1 and 254 inclusive, thus essentially failing the ISA-probe-only mechanism currently used to determine whether a NIC card is present. There should be an additional check that the card set at the netport address is indeed a NIC, rather than say, an IDE controller.

I disagree. Getting that knowledge would be very helpful because it would show what to look for: Is it a pattern, is it random, does a delayed read after write change anything, etc. Of course the simple 0x20 command in the ne2k case may be changed to something else to increase precision, like we're doing in the WD driver.

Both the wd and 3c NIC drivers perform more than a strict ISA probe to determine their chip is present - only the ne driver performs just an ISA probe, with no attempt at chip identification until past the probe routine returning -ENODEV.

See above, this is at best partly true. And the serial probes are even simpler than this. Then again, the likelihood of someone reusing those particular addresses would be minimal as opposed to a more 'public' address like 0x300.

I myself like to write software, not just for myself, but for others. Writing software itself is not enough, I want people to run it, without having to be a developer. I want more users to come to our project, download it and give it a try. If the software does not work, I feel many will quietly just leave without saying anything, and never come back - just like a bad restaurant.

It's easy to agree with that. But going to an italian restaurant and then complain about the food not being Thai, is, well, …

ELKS itself is advertised as an OS for any 8086 based system, PC, XT, AT and other "nearly compatibles". Many contributors have put in a lot of work to ensure the kernel boots and performs floppy identification and I/O with older BIOSes, including the very first PC BIOS. I feel that our standard distribution needs to work out-of-the-box on all PC, XT and AT hardware, which it does, except now we find for this issue it does not.

That's a great goal and not even (bloated) Linux has achieved that. To this day we have to adjust/edit configurations to make Linux work on many hardware platforms. I'm not convinced this is a reasonable (even attainable) goal. I think we need to be more pragmatic about this.

I'm not sure the extra code is useful enough to qualify the space it takes.

That could be. We might measure how much code we're talking about. I suspect less than 50 bytes.

Like I said above, this is not about 50 bytes, it's about policy. The problem at hand can most likely be fixed with 2 or 5 bytes once we know what it is.

We're coming from a place where it was expected that the user configures a system to match the hardware.

If desired, we can remove the network drivers from the default distribution, that will allow booting ELKS for all IBM PC users. However, it would be a shame to not allow others to see ELKS networking in operation, should they have hardware installed. We've put a lot of work into that.

It would indeed be a shame to break something running on hundreds, maybe thousands of systems in order to accommodate a rare problem on a rare (I just learned that 'esoteric' is an insult) platform, a problem that haven't been investigated yet, for lack of interest.

XT and before apparently not fully supported yet and will remain so until someone is sufficiently interested to participate.

XT is supposed to be fully supported.

It's an interesting statement given the obvious facts. Why not just admit that the XT is only partly supported until the problem has been identified and a solution built into the OS?

Endnote

There is a lot to be learned here. Not the least about the basics of community development. Attitude, respect, mindset, willingness to contribute, to share and much more. And how easy it is to just go after the symptoms instead of the problem. Needless to say, I'm not a supporter of that.

--M

[Mellvik]

This was my final contribution to ELKS. I'm checking out.

It was fun for a while. Good luck to all.

-M