view doc/Linux-DTR-RTS-flaw @ 173:df4bf4e06221

doc: several articles moved to other repositories
author Mychaela Falconia <falcon@freecalypso.org>
date Mon, 11 Sep 2023 06:51:05 +0000
parents 95c2a67e1219
children fb2f6497ba53
line wrap: on
line source

There is a fundamental flaw in the Linux kernel serial port handling subsystem
that affects anyone who builds special hardware in which DTR and/or RTS modem
control lines are repurposed for some non-traditional functions.  The flaw is
that whenever a serial port is opened under Linux, the kernel immediately and
unstoppably asserts DTR and RTS (their initial power-up state prior to software
action is negated on every sane serial port hardware implementation), *without*
giving userspace applications any ability to say "no, please don't do it".

As long as DTR and RTS modem control outputs from the DTE are used for their
original RS-232 functions, automatically raising both signals on serial port
open is harmless - and because these signals often need to be asserted in order
for serial communication to take place, having them raised automatically on
open (without requiring an explicit TIOCMBIS ioctl) is a convenience which many
applications rely on - if this standard kernel behaviour were to be changed
for the general case (outside of special quirk configurations), a lot of
applications will break.  Linux implements this standard behaviour as mandated
by POSIX and other similar standards, and those standards are in turn based on
the original 1970s UNIX where this architectural design originates.

However, this standard Linux behaviour (and going all the way back to 1970s in
the greater UNIX family) is a total killer for custom hardware designs in which
DTR and/or RTS outputs from the UART are repurposed for some totally different
functions.  Suppose that the hardware is wired in such a way that asserting the
control output causes an explosive charge to be set off, or causes a radio
transmitter to turn on (perhaps operating on some tightly regulated frequency
supporting mission-critical services, where spurious out-of-protocol
transmissions are not permissible), or applies a hard reset to some other
component that may be part of a live production system that must not be casually
reset - in all listed examples having such hardware wiring would be perfectly
safe in an OS-less environment where the custom application controls the custom
hardware as desired, without any OS inserting its own mind: the hardware design
of most serial ports (both traditional and USB-serial) guarantees that the
initial power-up state of both DTR and RTS outputs prior to software action will
always be negated, and the custom application thus gets to decide if and when
each of the two signals (independent of each other) should be asserted.

Everything works great if the application runs on the bare metal and directly
controls the hardware (or runs under something like DOS, which is the same as
running on bare metal for the present purpose of operating serial ports), but
add Linux into the equation, and things quickly begin to break.  The problem is
that the moment you open a serial port under Linux (and sadly, the same thing
happens under most other current OSes too), the kernel automatically asserts
both DTR and RTS immediately on the open operation itself, without giving
userspace applications any way to say "no, please don't do it".  Some people
have been proposing new termios flags that would suppress this auto-assertion
on subsequent opens, but you have to open the port first in order to do termios
or other ioctls on it, and if the auto-assertion of DTR and RTS on that initial
open causes irreparable damage, then you are screwed no matter what you do.

The only currently possible solution to this madness is to patch the kernel to
suppress this automatic assertion of DTR & RTS upon serial port open.  But one
cannot simply change the standard behaviour for all serial ports, as lots of
standard applications for classic serial communication (where DTR and RTS do
need to be asserted) will break in that case.  Instead the suppression of
automatic assertion of DTR & RTS on open needs to be conditionalized in some
way, so that the modified against-standards serial port open behaviour is
applied ONLY when special modem-control-repurposed hardware is being operated
on, and not for ordinary applications operating on ordinary serial ports.

Given the current state of Linux and what is possible in the current reality,
if a patch is to be applied to the kernel, creating the ability to exempt
certain serial port open operations from the standard POSIX requirement of
automatically asserting DTR & RTS, there are only 3 practically feasible ways
to communicate to the kernel that a given serial port (or a given individual
open operation on a serial port) should be exempt from automatic assertion of
DTR & RTS:

1) Create a new open flag like O_NODTR, or reuse/abuse some existing open flag
   like O_DIRECT which currently has no effect on serial ports.

2) Create a sysfs attribute that is attached to every serial port, controlling
   whether or not DTR & RTS should be automatically asserted on open, with the
   default being standards-mandated traditional UNIX behaviour of
   auto-assertion.

3) In special cases where the custom DTR/RTS-repurposed hardware is inseparably
   integrated (on the same custom PCB) with a USB-serial chip, such that the
   EEPROM controlling the USB VID:PID of the USB-serial device identifies not
   just the USB-serial converter part, but the entire product board as a whole,
   including the circuits that repurpose DTR and RTS for non-serial purposes,
   then the most sensible approach is to mark the USB-serial device as special
   and disable auto-assertion of DTR & RTS on this special device when the
   custom USB VID:PID is detected.

As it happens, our own FreeCalypso hardware gadget with repurposed DTR & RTS
that requires suppression of auto-assertion of these signals (the optional boot
control feature of our DUART28 adapter) falls into the last special category
above (custom USB-serial device unambiguously distinguished by a custom USB ID),
hence this special case is the one that I (Mother Mychaela) have been focusing
on the most - as humans, we all have a natural right to put our own self-
interest first.

I (Mother Mychaela) have no way of knowing whether or not there is even one
person alive on Earth today who has an active use case where a need exists to
suppress automatic assertion of DTR & RTS for some serial device, but that
device does not have the same quality of being inseparably integrated with a
custom USB ID as our DUART28C, i.e., an active use case where a need exists to
signal to the kernel "please don't auto-assert DTR & RTS on this serial port"
and moreover do this special signaling for "any" serial port, rather than one
identified by a custom USB VID:PID.  For all I know, I may very well be the
only person alive on Earth today who has an active need for auto-DTR/RTS
suppression - but I need it ONLY for a device that has a unique distinctive USB
VID:PID, not for "any" serial port.

I currently run Slackware Linux 14.2 as my personal OS, running Linux kernel
version 4.4.14 around which this version of Slackware was built - when I tried
running newer 4.4.x kernels, I was getting crashes which I could not debug.  I
currently run this elderly Linux kernel version with my own custom patch applied
to the ftdi_sio driver, a patch that adds support for FreeCalypso DUART28C
(custom USB ID) and applies the appropriate special quirk just for this USB ID,
not affecting any other devices - a quirk that suppresses automatic DTR & RTS
assertion on FT2232D Channel B, the UART channel on which these signals are
repurposed on DUART28 hardware.  Several different versions of this patch (made
to apply cleanly to several different kernel versions) can be found in the
linux-patch directory in the present source repository.

In 2020-09 I made a good-faith, due-diligence attempt to get the hardware
support patch for DUART28C (a patch to ftdi_sio driver that recognizes the new
USB ID and applies the necessary quirk, entirely contained inside this driver)
mainlined - I submitted the patch to ftdi_sio maintainer Johan Hovold.  I was
quickly met with hostility, with Johan telling me to redesign my hardware (he
was basically telling me to throw away 20 perfectly good boards) in some
different way that would be more in line with the 1970s UNIX worldview for DTR
and RTS, which is what Linux currently implements.

Some time later I was able to kinda-somewhat-partially convince Johan that the
current handling of DTR and RTS is a serious problem for some users, and he was
a little more agreeable to my patch - but instead of merging it as-is, he
proposed an expanded patch (getting into the tty subsystem, outside of just
USB-serial) that solves a more general problem.  Johan's proposed patch
introduced an internal flag telling the tty layer to suppress DTR & RTS
assertion on open, and a sysfs attribute (added to all classic serial and USB-
serial ports) that exposes this flag.  A patch to ftdi_sio that recognizes my
custom USB ID and sets this flag in the quirk function was still included in
that proposed patch series, so I was happy with the proposal.

However, Johan's sysfs proposal was quickly shot down by other kernel
maintainers who didn't like the sysfs approach, and Johan himself was not too
interested in defending his sysfs proposal either - instead he favors a termios
flag that would only affect second and subsequent repeated opens of a device,
after the initial open to set that flag.  Of course this termios flag idea does
not help at all, given that the very first open of the serial port would still
unstoppably assert DTR & RTS, causing irreparable damage - if these control
signals are wired to set off explosives, for example, the user's house would be
up in flames the moment he issues that magic stty command to set the new termios
flag, and Johan's assurances that second and subsequent opens of the same
serial port would not auto-assert DTR & RTS would be of little help to the poor
guy who just lost his house.

By the end of 2021-01 I realized that my battle against Johan and Greg K-H is
hopeless, so I give up.  The only workable solution at this point is for all
affected people to stop running unpatched mainline kernels and to apply our own
local patches instead, preferably with our own coordination amongst ourselves
so we have some degree of standardization among our kind.  The whole discussion
is archived here:

https://lore.kernel.org/linux-serial/X8iuCXYhOBVMGvXv@localhost/T/

I shall indefinitely, for as long as I am alive, maintain my ftdi_sio driver
patch that adds support for FreeCalypso DUART28C hardware.  And because I do
not know whether or not there exists even one person on Earth who would benefit
from an ability to suppress DTR & RTS assertion under Linux on "any" serial
port, outside of tightly integrated USB-based devices with custom USB IDs, I
also make the following conditional offer: *if* at least one person comes
forward to me and demonstrates that he or she has an active use case of the
kind I am talking about, *then* I will also dig up Johan's patch (the one
rejected by other maintainers) adding a sysfs attribute, providing a working
solution for "any" serial port, start actively supporting that sysfs patch, and
maybe even make another attempt at convincing kernel maintainers to mainline it.
But I will go down that path *only* if there is at least one person alive on
Earth (just one person would be enough) who would actively benefit from this
feature - otherwise there is no point.