--- a/CHANGES	Wed May 29 05:40:51 2019 +0000
+++ b/CHANGES	Sat Jun 01 06:46:46 2019 +0000
@@ -6,6 +6,13 @@
 * New developer utility fc-bin2rftab can read or extract RF parameter tables
   from binary files and convert them into our FreeCalypso ASCII format.
 
+* The way in which fc-loadtool handles different flash configurations has been
+  completely redesigned; there should be no user impact on any previously
+  supported target, but the new architecture makes it much easier to add
+  support for new target devices and allows the possibility of different flash
+  chip types (autodetected at run time) on the same -h target.  See the new
+  doc/Loadtool-flash-support article for the details.
+
 * Iota poweroff command in the target-utils suite (primarily affecting the
   loadagent target program used by loadtools) now programs the VRPCAUX register
   like TI's TCS211 firmware does before sending the DEVOFF command.  This

--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/doc/Loadtool-flash-support	Sat Jun 01 06:46:46 2019 +0000
@@ -0,0 +1,219 @@
+fc-loadtool is our tool for reading and writing the non-volatile flash memory
+on all of our supported target devices, and the set of targets which it needs
+to support keeps growing.  Here are some of the challenges we have to deal with:
+
+* Some Calypso board designs use AMD-style flash, others use Intel-style flash.
+  Initially we only supported AMD-style flash chips that were used in our first
+  targets (Openmoko GTA02 and Pirelli DP-L10), then we got other targets that
+  have Intel-style flash.  So far we have not yet run into a case where both
+  kinds of flash can be encountered on the same target family, but our current
+  design supports this possibility.
+
+* All Calypso devices which we currently support have flash chips with non-
+  uniform sector geometries, i.e., the area that would otherwise be the first
+  or the last sector is subdivided into smaller sectors (erase units).  Both
+  "top boot" (small sectors at high addresses) and "bottom boot" (small sectors
+  at low addresses) geometries are found among our targets, as well as flashes
+  that have small sectors at both ends.  The exact sector geometry needs to be
+  known to the flash manipulation tool in order to perform correct flash erase
+  and program operations.
+
+* While most Calypso devices have a single flash chip providing a single bank
+  of flash (can be as small as 2 MiB or as big as 8 MiB), some of our targets
+  (our own FCDEV3B and the Pirelli DP-L10 phone from which the idea was copied)
+  provide two flash chip select banks of 8 MiB each.  To make the matters even
+  more complicated, all of that flash is actually a single 16 MiB chip that has
+  two chip selects instead of one, specifically designed for processors like
+  our Calypso that can only address a maximum of 8 MiB per chip select.
+
+* It is a fixed target property whether a given board is wired for only one
+  flash chip select or allows the possibility of dual-bank flash, and if a
+  second flash chip select is provided for, which Calypso chip select it is
+  wired to.
+
+Given the existence of the CFI (Common Flash Interface) standard and the fact
+that every flash chip we have encountered so far in a Calypso device does have
+a readable CFI structure, one may naively think that the most sensible way to
+support all of our possible flash configurations would be to read and parse the
+CFI structure in a device-agnostic way (i.e., without special cases for specific
+chip types) and thus support "everything".  But here are the problems with this
+simplistic approach:
+
+* On boards that have 16 MiB of flash in a Spansion S71PL129J or S71PL129N chip,
+  it makes the most sense for us to treat this big flash as two separate banks
+  of 8 MiB each - but the CFI structure describes a single 16 MiB flash chip.
+
+* AMD-style flashes with "top boot" geometries are among our repertoire of
+  devices to be supported, and they have their regions listed in the wrong order
+  in the CFI structure - one needs to look in the AMD-specific part outside of
+  the vendor-neutral geometry structure to see the true "top boot" geometry.
+
+* Intel-style flashes with independent read/write partitions such that each
+  partition has its own status register and its own "read array" vs. "read SR"
+  state require special handling in our architecture, but autoconfiguring this
+  quirk agnostically from CFI seems too difficult to me, and I wouldn't trust
+  it.
+
+Our previous architectural attempts
+===================================
+
+Initially we only supported two flash chip types, Samsung K5A32xx_T (Openmoko
+GTA02) and Spansion S71PL129N (Pirelli DP-L10) with strictly manual selection:
+-h gta02 selected one and -h pirelli selected the other via hardware parameter
+files.  There was an ID check to prevent bogosity from wrong manual selection,
+but no autodetection or autoconfiguration.  Then we added Compal target support;
+aside from Mot C155/156 which has partition quirks that were only discovered
+much later, these phones have simple Intel-style flashes without any of the CFI
+problems listed above, thus they were handled via CFI.  Thus we had a hybrid
+architecture: Openmoko, Pirelli and FCDEV3B targets were handled by way of
+manual selection and ID checks to catch errors, whereas Compal targets were
+handled by way of CFI-based autodetection and autoconfiguration.
+
+Then it was discovered that the 8 MiB Intel-style flash on the D-Sample board
+and on Mot C155/156 has partition quirks which our CFI-based autoconfiguration
+(looking at vendor-agnostic geometry bits only) could not take care of, and the
+solution was to move these targets from CFI-based autoconfiguration to the same
+kind of fixed device selection and configuration as was used for AMD flashes.
+At that point our flash handling architecture became a mess, and when I started
+questioning how to extend it further as the need arises to support more
+different flash chip types on a wide variety of Calypso targets, it became
+clear that a redesign was needed.
+
+Our current architecture
+========================
+
+In our current architecture the only flash configuration that is indicated
+statically in the hardware parameter files (selected with the -h option,
+practically meaning predefined target configurations) is board wiring
+information.  There are 3 possibilities that can be configured:
+
+flash single-4M base_addr		-- wired for 1 bank of up to 4 MiB
+flash single-8M base_addr		-- wired for 1 bank of up to 8 MiB
+flash dual-8M bank0_base bank1_base	-- wired for 2 banks of up to 8 MiB each
+
+Naturally the dual-8M configuration only makes sense for boards that are wired
+with a provision for a second flash bank, in which case the second bank base
+address will depend on the board wiring, i.e., which Calypso chip select it is.
+(Bank 0 base address will normally be 0x03000000, i.e., the alternate nCS0
+mapping that needs to be used when the boot ROM is mapped at 0.)  The choice
+between single-4M and single-8M needs to match whether or not the associated
+init script includes a "w16 fffef006 0008" command to enable ADD22.
+
+Beyond this board wiring configuration, the rest of flash support is based on a
+hard-coded table of all supported devices (a table that can grow indefinitely)
+plus autodetection amongst this supported set.  In other words, fc-loadtool will
+only operate on a given flash chip if it explicitly knows about that chip, but
+the set of supported chips can be indefinitely extended without hitting
+architectural barriers, and our autodetection logic will detect and handle any
+supported chip on any board target.
+
+Autodetection details
+=====================
+
+The flash chip autodetection operation proceeds as follows:
+
+* A sequence of writes is done to put the chip into the Read ID mode,
+  equivalent to the following hypothetical C code with base_addr being an
+  integer:
+
+  *(volatile uint16_t *)(base_addr + 0xAAA) = 0xAA;
+  *(volatile uint16_t *)(base_addr + 0x554) = 0x55;
+  *(volatile uint16_t *)(base_addr + 0xAAA) = 0x90;
+
+* 16-bit words at base_addr offsets of 0 and 2 (where the manufacturer and
+  device ID codes are expected to reside) are read, and this ID is looked up in
+  a table.  If the ID code is not known, we give up and don't allow any flash
+  operations.
+
+* For most ID codes, if we have found the code in our table, we know what device
+  we should expect.  But before we go ahead and assume that the command set and
+  the geometry are as we think based on the ID code, we also do a CFI check.
+  Specifically, we put the flash chip into CFI query mode, read a defined set
+  of word locations (can be different for each chip type), and require these
+  words to match our compiled-in table.  Thus we guard against the possibility
+  of some other flash chip having the same ID code (yes, there are known
+  instances of ID code reuse) but having a different geometry.
+
+* Some ID codes receive more complex handling.  Right now the only such case is
+  Spansion PL-J/PL-N flash.  PL129J and PL129N flashes have different geometries
+  and thus must be distinguished, but they have exactly the same ID codes and
+  can only be distinguished by CFI.  We have CFI match tables for PL129J and
+  for PL129N; we try to match the CFI bits provided by the chip against one
+  table first, and if it fails to match, we try the other.  (As an optimization,
+  we try the PL129N table first, as the N flash is the one found in real-world
+  Pirelli DP-L10 specimen and used on our FCDEV3B.)  If the CFI matches neither
+  table, we give up and don't allow any flash operations.
+
+The end effect of this logic is that we err on the side of caution: we only
+allow flash erase and program operations if we detect a flash chip which is
+fully known to us and fully matches our expectations, with both the ID codes
+and the CFI structure being as we expect.
+
+Adding support for new flash chip types
+=======================================
+
+All supported flash devices are listed in the fldevs.c source module; new
+devices that differ in geometry, command set or quirks need to be added there.
+The description of each flash device in fldevs.c also includes the CFI table
+that needs to matched to confirm the device in question.  A different module
+named flashid.c contains the autodetection function and the table of device ID
+codes; the latter table always needs to be extended, sometimes adding an
+entirely new device, othertimes adding a newly found ID code for some flash
+chip that is fully equivalent to an already supported one in terms of geometry,
+command set and relevant quirks.
+
+What do you do if you are an end user (not a FreeCalypso developer) and you got
+a Calypso device whose flash chip is not being recognized by fc-loadtool?
+Answer: you send the output of the "flash id" command (contains ID codes) and a
+dump of the CFI structure to Mother Mychaela for analysis.  To make a dump of
+the CFI structure, execute the following commands:
+
+loadtool> w16 030000aa 98
+loadtool> dump2bin 03000000 200 cfidump.bin
+
+Handling of dual-bank 16 MiB flash chips
+========================================
+
+The Calypso can only address a maximum of 8 MiB per chip select, thus 16 MiB or
+larger flash chips with a single chip select cannot be used in Calypso board
+designs.  However, there are some special 16 MiB flash chips that present
+themselves as two banks of 8 MiB each (even though the CFI structure describes
+a single 16 MiB chip), and such flash chips are used on the Pirelli DP-L10 and
+on our own FCDEV3B.
+
+The flash handling architecture of fc-loadtool allows two banks to be configured
+via a flash dual-8M setting in the hardware parameter file, and when that
+configuration is used (-h fcfam and -h pirelli), the two banks are treated as
+being entirely independent.  All regular flash commands operate only on the main
+bank, and a parallel set of flash2 commands operates on the secondary bank.
+The autodetection logic and the resulting configuration are done independently
+on each flash bank when it is first accessed, thus fc-loadtool would happily
+handle two separate flash chips of different types, even though such arrangement
+is not expected to occur in any Calypso device.  But when a PL129J or PL129N
+device is detected (the two dual-bank devices we currently support) on the
+autodetection probe of either bank, the operating geometry is configured
+appropriately based on which bank it is.
+
+Primary flash bank mapping at 0x03000000
+========================================
+
+When loadagent runs on the Calypso target controlled by fc-loadtool, the Calypso
+boot ROM will usually be mapped at 0, thus the alternate nCS0 mapping at
+0x03000000 needs to be used for flash access.  However, the Calypso chip (all
+versions we work with) has a little design bug in this part of the silicon:
+this alternate nCS0 mapping at 0x03000000 works only when the debug visibility
+bit in the API-RHEA control register (bit 6 in the FFFF:FB0E register) is set,
+and does not work otherwise.  This bit is initially set as the Calypso comes
+out of reset, and on most platforms we gain loadtool access via the boot ROM,
+hence the problem does not occur - but on Compal targets we gain loadtool
+access either through Compal's bootloader or via tfc139, and in both cases
+Compal's fw (either the full fw or the bootloader part) has already set the
+register in question to the runtime operational value of 0x2A (unchanged from
+TI's TCS211 reference fw), with the debug visibility bit cleared, hence the
+0x03000000 flash mapping no longer works.
+
+There are two possible solutions: we can write into the FFFF:FB10 register to
+disable the boot ROM and use the "regular" flash mapping at 0, which is what we
+used to do, or we can write into the FFFF:FB0E register and re-enable the debug
+visibility mode.  Right now we do the latter, allowing us to use the same
+0x03000000 flash mapping on all targets for consistency.