--- a/doc/RVTMUX	Fri Nov 13 19:29:16 2015 +0000
+++ b/doc/RVTMUX	Sun Nov 15 01:42:50 2015 +0000
@@ -60,83 +60,177 @@
 send packets to RVT for transmission on this serial interface, and can also
 register to receive packets beginning with a particular type ID byte.
 
+Debug trace output
+==================
+
+All GSM device firmwares that are based on TI's Calypso chipset reference fw
+continuously emit quite voluminous debug trace output on their RVTMUX serial
+port, whether it is hidden or exposed on a given device.  Like all RVTMUX
+traffic, this debug trace output takes the form of binary packets as explained
+above, but the content of these packets is mostly ASCII with some binary header
+bytes prepended.  FreeCalypso host utility rvtdump captures all serial output
+from a GSM device's RVTMUX port, parses the packet structure and displays this
+output in line-oriented pure ASCII with all binary parts decoded.
+
+Test Mode commands
+==================
+
+The other major use of the RVTMUX interface is sending so-called Test Mode
+commands from an external host to a running GSM device.  Depending on the
+firmware version, a GSM device can be commanded to do any of the following
+things through this mechanism:
+
+* Exercise RF test modes, e.g., transmit continuously at a set frequency and
+  power level;
+* Read and write arbitrary memory locations in the Calypso ARM7 address space;
+* Read and write ABB chip registers;
+* Reboot or power off;
+* Access and manipulate the device's flash file system (FFS).
+
+In the segment of history of interest to us TI has produced two different
+target firmware components that can receive, interpret and act upon Test Mode
+command packets:
+
+* The original Test Mode component of Layer 1, called L1TM or TML1: this
+  component handles all RF test modes (needed for RF calibration on device
+  production lines), and originally it also implemented memory and ABB register
+  read and write commands, and provided access to TMFFS1 (see below).  In the
+  original implementation this component registered itself as the handler for
+  the "TM" RVTMUX channel (RVT packet type 0x14), so it would receive all TM
+  packets sent to the device.
+
+* Enhanced Test Mode (ETM) is a later invention.  It registers itself (instead
+  of the old TM in L1) with RVT as the handler for the "TM" RVTMUX channel, and
+  then provides a registration service of its own, such that various components
+  in the fw suite can register to receive external command packets passing
+  first through RVT, then through ETM, and can send responses passing through
+  ETM, then through RVT back to the external host.  If a given fw version
+  contains both ETM and L1TM, then L1TM registers itself with ETM; an external
+  host would send exactly the same binary command packets to exercise RF test
+  modes, but inside the firmware they now pass through ETM on their way to L1TM.
+
+The ETM_CORE module contained within ETM itself provides some low-level debug
+commands: by sending the right binary command packets to the GSM device via the
+RVTMUX serial channel, an external host can examine or modify any memory
+location and any hardware register, cause the device to reset, etc.  Prior to
+ETM some of these functions (but not all) could be exercised through older TM3
+commands, but in FreeCalypso we became familiar with the ETM versions of these
+commands long before the older ones because we got the ETM component in full
+source form, whereas our copy of TCS211 (TI's reference fw) has L1TM in a
+binary library.
+
+Our TCS211/leo2moko reference fw has both ETM and L1TM, thus it accepts both
+ETM and TM3 command packets.  ETM commands (including TMFFS2, see below) work
+on Pirelli's fw, but Mot/Compal's original fw for the C139 has only the
+original non-enhanced Test Mode, not ETM.
+
+FFS access via TM/ETM
+=====================
+
+One of the essential facilities provided in one form or another in all known
+incarnations of the Test Mode mechanism is the ability to access and manipulate
+the GSM device's flash file system (FFS).  See TIFFS-Overview for a description
+of this file system.  TI's TMFFS1 and TMFFS2 protocols provide a command and
+response packet interface to the FFS API functions inside the fw, and enable an
+external host connected to the GSM device via the RVTMUX channel to perform
+arbitrary read and write operations on the device file system.
+
+In the segment of history of interest to us TI has produced two different
+and entirely incompatible versions of the TMFFS protocol: TMFFS1 and TMFFS2.
+Or rather, what is now called TMFFS1 was originally just TMFFS, and then came
+TMFFS2.  TMFFS2 works only through ETM, whereas TMFFS1 predates ETM: in the
+original implementation the tm_ffs() function in the FFS code was called from
+L1TM code.
+
+Our copy of TCS211 reference fw includes the source for both TMFFS1 and TMFFS2;
+it is theoretically possible to build a firmware image that includes both TMFFS
+versions (they won't conflict because they respond to different command
+packets), but it is pretty clear that TI never intended to have both enabled
+at the same time.  Our copy of TCS211 came with TMFFS1 enabled and we didn't
+change it when we made the moko12 (leo2moko-r1) fw release for the Openmoko
+community (the previous proprietary mokoN firmwares also implement TMFFS1),
+but we have subsequently switched to TMFFS2 for our current TCS211-based work.
+
+Pirelli's fw implements TMFFS2: we don't have any source for this fw, but our
+FreeCalypso host utilities written to talk the TMFFS2 protocol based on our
+available TCS211 source work beautifully when run against Pirelli's fw.
+
 Use in FreeCalypso
 ==================
 
 The FreeCalypso project has adopted the same general firmware architecture as
 that exhibited by TI's standard firmwares from the Moko/Pirelli time frame.  We
 use TI's RiViera framework lifted directly out of the TCS211 reference fw, and
-that includes the RVT module and the RVTMUX interface it presents.  At the
-present time (early development stage, none of the actual GSM functionality has
-been integrated yet) this RVTMUX interface is put to the following uses in our
-own gsm-fw:
-
-* Debug trace output from various components sent via the rvf_send_trace()
-  function - it is the RiViera Trace output in the proper sense;
-
-* The ETM module and the associated FFS access protocol described below.
+that includes the RVT module and the RVTMUX interface it presents.  Our GSM fw
+emits the same 3 kinds of debug traces (RV, L1 and GPF) as the pre-existing
+firmwares with which we are seeking functional parity, and for Test Mode
+functionality we have the option of including ETM, TMFFS1 and/or TMFFS2 in our
+firmware builds.  (Both TMFFS versions require ETM in our implementation, and
+it is possible to build a firmware image with both included.)
 
-In the existing proprietary firmwares which serve as our reference, the RVTMUX
-serial channel is continuously spewing very voluminous debug output.  This debug
-output exhibits 3 different packet types: RV traces described above, and also
-L1 and G23 traces, each in its own format.  We expect that our own gsm-fw will
-become just like these reference versions in this regard, once we integrate
-those code layers.
+We have adopted ETM and TMFFS2 as the standard combination for FreeCalypso,
+i.e., ETM_CORE for memory and ABB register reads and writes and TMFFS2 for
+external FFS access.  We needed to develop our own host tools for operating on
+GSM device FFS via one of the two TMFFS protocols, and after studying the fw
+source implementing both, I (Space Falcon) came to the conclusion that TMFFS2
+is both more capable and more reliable; my guess is that TMFFS1 was likely kept
+around only because some of TI's crappy Weendoze host software depended on it.
+(See gsm-fw/services/ffs/tmffs.c if you would like to judge for yourself.)
 
-ETM and FFS access
-==================
+We have the following host tools for communicating with TI-based GSM firmwares
+(both our own and some of the existing proprietary ones):
+
+rvtdump		This tool produces a human-readable dump of all output emitted
+		by a TI-based GSM fw in the form of RVTMUX binary packets.  It
+		can also log this dump to a file.
 
-Another component which we have lifted out of the TCS211 reference fw is ETM,
-which stands for Enhanced Test Mode.  This module registers its own "top level"
-protocol over RVTMUX, and provides a registration service of its own, such that
-various components in the fw suite can register to receive external command
-packets passing first through RVT, then through ETM, and can send responses
-passing through ETM, then through RVT back to the external host.
+rvinterf	This tool is a superset of rvtdump: it not only dumps and/or
+		logs all output from the GSM fw, but also provides a mechanism
+		for sending command packets to it.
+
+Rvinterf is the engine behind the following host tools that send Test Mode
+commands to a target:
 
-The ETM_CORE module contained within ETM itself provides some low-level debug
-commands: by sending the right binary command packets to the GSM device via the
-RVTMUX serial channel, an external host can examine or modify any memory
-location and any hardware register, cause the device to reset, etc.
+fc-tmsh		This is our basic tool for sending Test Mode commands to a
+		running GSM fw.  It is strictly asynchronous in that commands
+		entered by the operator get sent to the target, and any response
+		packets received from the target are displayed as they come in.
+		The tool has no knowledge of any correspondence between commands
+		being sent and whatever responses they should elicit, i.e., it
+		is perfectly suited for experimental discovery of firmware
+		behaviour in response to Test Mode commands.
 
-The only other ETM-based functionality currently integrated in our gsm-fw
-besides ETM_CORE is TMFFS (Test Mode for FFS), which is the external access
-channel to the device file system - see TIFFS-Overview.  The TMFFS1 and TMFFS2
-protocols provide a command/response packet interface to the FFS API functions
-inside the fw, and enable an external host connected to the GSM device via the
-RVTMUX channel to perform arbitrary read and write operations on the device
-file system.
+		This tool was written before we realized that there was/is an
+		older, more basic Test Mode predating ETM, hence in many place
+		we say "ETM" when we really should have said "TM".  Oh well...
 
-TMFFS protocol versions
+fc-fsio		This tool speaks the TMFFS2 protocol and allows a user or
+		developer to perform a wide range of operations on the file
+		system of a GSM device.  It operates synchronously, i.e., it
+		sends ETM/TMFFS2 commands and expects responses in strict
+		lock-step; a single user command may translate into a large
+		number of ETM/TMFFS2 command packet exchanges.
+
+AT commands over RVTMUX
 =======================
 
-TI made two different and entirely incompatible versions of the TMFFS protocol
-for accessing a device's FFS via RVT/ETM: TMFFS1 and TMFFS2.  The fw sources
-available to us contain implementations of both versions, so we have the freedom
-to use whichever we like better for FreeCalypso.  After studying the fw source
-implementing the two TMFFS protocols, I (Space Falcon) came to the conclusion
-that TMFFS2 is both more capable and more reliable; my guess is that TMFFS1 was
-likely kept around only because some of TI's crappy Weendoze host software
-depended on it.  (See gsm-fw/services/ffs/tmffs.c if you would like to judge
-for yourself.)  Thus TMFFS2 is currently the "officially adopted" version for
-FreeCalypso.
+There is one more use to which we put the RVTMUX debug serial interface that is
+an original FreeCalypso invention: communicating with the AT command interpreter
+(ATI).  TI's original architecture assumes that if a product is to offer a
+standard AT command interface (the product is either a GSM/GPRS modem for which
+this AT command interface is the sole mode of usage or a feature phone that
+offers a data port as one of its features), then it will be presented on a
+dedicated UART separate from RVTMUX.
 
-Our fc-tmsh utility (see below and ../rvinterf/README) allows a developer-
-operator to send TMFFS "get version" queries to a running GSM fw in both
-ETM_FFS1 and ETM_FFS2 formats; this capability allows us to determine
-experimentally which protocol (if any) is implemented by a given proprietary
-firmware version.  Experiments reveal that Openmoko's moko11 firmware
-implements TMFFS1, whereas Pirelli's fw implements TMFFS2.
+However, many of our target devices have only one UART practically accessible,
+and even when we use Openmoko's modem as our development platform, the RVTMUX
+interface is more convenient because it connects externally, whereas the MODEM
+UART is connected to the application processor of the smartphone.  Therefore,
+we developed a way to pass AT commands over RVTMUX.  We created a new RVTMUX
+channel for this interface and assigned it RVT packet type 0x1A.  Packets sent
+from an external host to the GSM device carry AT commands and SMS string input,
+whereas packets flowing the other way carry ATI's responses to commands and
+asynchronous notifications such as incoming calls.
 
-The leo2moko-r1 firmware produced by the FreeCalypso project in 2013-10
-implements TMFFS1, simply because that was the selected configuration in the
-found Leonardo source that transitional fw is based on, and that release was
-made before I learned RVTMUX, FFS, ETM and TMFFS properly.  All future
-FreeCalypso firmwares will use TMFFS2, or at least that's the current plan.
-
-Host utility support
-====================
-
-As one would naturally expect, the FreeCalypso project has developed some host
-tools that allow a PC running GNU/Linux (or other Unix systems) to interface to
-running firmwares on GSM devices via RVTMUX.  See the rvinterf subtree of
-freecalypso-sw for the source and documentation.
+The host utility for talking AT commands to a FreeCalypso GSM device via RVTMUX
+is fc-shell; it works via rvinterf just like fc-fsio and fc-tmsh.