--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/doc/How-flash-really-works	Sat Dec 09 09:08:19 2023 +0000
@@ -0,0 +1,97 @@
+How NOR flash memory really works
+=================================
+
+The type of flash memory used in Calypso GSM devices is formally known as NOR
+flash.  Most embedded software programmers and tinkerers know the fundamental
+principle of how NOR flash works: any bit can be transitioned from a '1' to a
+'0' at any time in any combination (an operation called programming), but the
+opposite transition (from '0' to '1' bits, an operation called erasure) can only
+be done on fairly large sectors - you can erase a sector and make it all 1s,
+but changing bits from 0 to 1 individually or in any smaller granularity
+(smaller than a sector) is impossible.
+
+What many "software-minded" programmers and tinkerers don't realize, however,
+is that sector erasure is not an elementary or atomic operation that magically
+"makes all bits 1s" in one motion.  Instead it is a complex process with two or
+three substeps:
+
+1) Before starting the physical process of erasure, one has to go through all
+   bits in the to-be-erased sector and make them all 0s.  Any bits that are in
+   '1' state when the sector erase operation is commanded MUST be programmed to
+   '0' state before the actual erasure begins!  In the language of flash chip
+   industry, this step is called preprogramming.  In the case of flash chips
+   that are used in Calypso GSM devices (all known ones), this preprogramming
+   step is done internally by the chip, so that you as the user or software
+   developer are not aware of it - but it is there nonetheless.  The chip does
+   NOT magically "wave" all bits in the sector into '1' state, instead it first
+   makes them all '0' internally, and only then erases.
+
+2) Once every bit in the sector is in '0' state, the real physics of erasure
+   begins.  All bit cells in the sector are physically acted upon at once in
+   this step, and because it is a probabilistic process involving a Gaussian
+   distribution, all bit cells need to be in the fully programmed state before
+   they begin their shared journey toward the erased state.
+
+3) The step of preprogramming every bit to 0 prior to erasure prevents the
+   absolutely unacceptable condition of gross overerasure - but given the
+   Gaussian distribution, some bit cells may still get a little overerased.
+   Many (most? all? not sure) flash chips therefore implement a third internal
+   step before the software-visible "erase" operation is declared complete:
+   they go through all bit cells in the just-erased sector, check for
+   overerasure, and "soft-program" (move slightly to the right in the Vt
+   distribution) any overerased cells.  This step is called post-erase
+   conditioning or recovery.
+
+The above process was originally explained to me (Mother Mychaela) some years
+ago (around 2008, IIRC) by a Spansion support engineer on a conference call at
+my then day job - it was a project for a customer who was big and powerful
+enough to get top-tier support from chip vendors.  More recently, however, some
+other flash vendors have posted public documents that provide the same
+explanation - here is one from Renesas/Adesto:
+
+https://www.freecalypso.org/pub/embedded/flash/REN_an500_APN_20210702_1.pdf
+
+Even though the above document was written by Renesas (or more precisely, the
+part that was originally Adesto), the theory described therein applies just as
+well to Intel, Spansion and Samsung flash chips that are used in Calypso GSM
+devices.  For anyone who wishes to know how NOR flash memory really works, I
+strongly recommend reading that Renesas appnote - it is a good description.
+
+Additional note on terminology: describing the two states of a flash memory cell
+as '0' and '1', like I did above, is only a convenience for software-minded
+people.  A more proper view is to think in terms of a "programmed state" and an
+"erased state" for each bit cell.  History and tradition are such that flash
+chips return '0' on read in the programmed state and '1' in the erased state
+(this tradition probably originates from the fact that the actual NV storage
+element, a transistor, conducts read current in the erased state), at least for
+the main flash array - however, when flash memory elements are used for
+additional purposes such as write protection controls, it is best to think
+natively in terms of programmed and erased states.  For the latter kind of
+special applications, an opposite polarity may be applied in read-bit values.
+
+One straightforward take-away from this theory is that flash endurance is really
+about program-erase cycles, rather than number of program or number of erase
+operations.  Every time you give a sector erase command, every bit in that
+sector cycles through the fully programmed (0) state first before becoming
+erased (1), irrespective of whether or not you programmed into it on your own!
+Hence every bit-cell of the affected sector always goes through a full
+program-erase cycle, and all bits in a given sector are always cycled equally,
+irrespective of whether they get written with mostly-0s or mostly-1s in between
+erase cycles.
+
+Another situation where this raw physics gets exposed to the user is the case
+of special-purpose non-volatile bits in flash chips outside of the main flash
+memory array - for example, Persistent Protection Bits (PPBs) in some Spansion
+and Samsung flash chips.  While program and erase commands for the main flash
+array invoke chip-internal mechanisms that take care of everything and present
+a sane model of 0s and 1s to software, Spansion PL-J PPB program and erase
+commands expose raw guts: there is a command that applies a raw program pulse
+to a single PPB, and there is a command that applies a raw erase pulse to the
+NV memory element (like a little sector of its own) that holds all PPBs.
+Applying the erase pulse without preprogramming every PPB first would be very
+bad (see Renesas appnote about the badness of overerasure) - hence in a seeming
+paradox, one has to explicitly lock every sector before applying PPB erase
+pulses that will eventually unlock everything!
+
+Our flash ppb-erase-all command does implement the preprogramming step before
+actual erasure, and the present document (hopefully) explains why.