FreeCalypso > hg > freecalypso-tools
changeset 1012:11391cb6bdc0
patch from fixeria: doc change from SE K2x0 to K2xx
Since their discovery in late 2022, Sony Ericsson K200 and K220 phones
were collectively referred to as SE K2x0 in FreeCalypso documentation.
However, now that SE K205 has been discovered as yet another member
of the same family (same PCBA in different case), it makes more sense
to refer to the whole family as SE K2xx.
author | Mychaela Falconia <falcon@freecalypso.org> |
---|---|
date | Mon, 23 Sep 2024 12:23:20 +0000 |
parents | 6d9b10633f10 |
children | bc4dc383ff4a |
files | doc/Flash-write-protection doc/Loadtools-usage doc/SE-K2x0-FFS doc/SE-K2xx-FFS doc/TIFFS-IVA-usage loadtools/loadtool.help |
diffstat | 6 files changed, 190 insertions(+), 190 deletions(-) [+] |
line wrap: on
line diff
--- a/doc/Flash-write-protection Thu May 23 17:29:57 2024 +0000 +++ b/doc/Flash-write-protection Mon Sep 23 12:23:20 2024 +0000 @@ -22,7 +22,7 @@ ability to erase PPBs is irreversibly disabled, locking all write-protected sectors forever - but so far we (FreeCalypso community) have not yet encountered any devices in which any of these "hard" locks have been activated. There is, -however, at least one Calypso-based phone out there (Sony Ericsson K2x0 family) +however, at least one Calypso-based phone out there (Sony Ericsson K2xx family) in which the shipping state of the device includes some flash sector locks - but these locks are of the "soft" kind, removable by performing a PPB erase operation which is not further blocked.
--- a/doc/Loadtools-usage Thu May 23 17:29:57 2024 +0000 +++ b/doc/Loadtools-usage Mon Sep 23 12:23:20 2024 +0000 @@ -92,7 +92,7 @@ Pirelli DP-L10 -h pirelli Sony Ericsson J100 -h compal -c 1004 Sony Ericsson J120 -h chimei - Sony Ericsson K2x0 -h fcfam + Sony Ericsson K2xx -h fcfam TI D-Sample -h dsample TI Leonardo -h leonardo
--- a/doc/SE-K2x0-FFS Thu May 23 17:29:57 2024 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,184 +0,0 @@ -Flash file system in Sony Ericsson K200/220 phones -================================================== - -SE K200/220 phones are based on our familiar Calypso chipset, their firmware is -based on the standard chipset reference fw from TI, and they use TIFFS as their -flexible data storage mechanism. Their TIFFS instance is located in the first -3328 KiB of the 2nd flash bank; the geometry is 256x13 or 64x52 depending on -which flash chip was used in each given phone. (Some specimen have Spansion -S71PL129NB0 chips with 256 KiB flash sectors, others feature Samsung K5L2931CAM -chips with 64 KiB flash sectors.) - -Calibration data and self-regenerating ability -============================================== - -SE's FFS life cycle design on these phones is interesting in that vital per-unit -calibration data (RF, AFC and MADC calibrations) are stored in two places: -(1) in the FFS in TI's standard format and locations, and (2) in a dedicated -flash sector outside of FFS that appears to be written once at the factory and -never touched afterward. - -SE's firmware design allows the FFS to be fully erased without breaking the -phone: whenever the fw boots in a state where the FFS area of the flash is fully -erased (you can of course erase those FFS sectors with fc-loadtool, but it -appears that some official tools also provide the same operation) but the -separate factory calibration data sector is valid, the firmware formats and -initializes a new FFS, and furthermore copies all essential calibration records -from SE's proprietary sector structure into their standard TIFFS locations! - -But it gets even more interesting - when we examine FFS images read out of -various SE K2x0 specimen, we see two different occurrences: - -1) In some specimen the FFS content is exactly the same as what we would get by - erasing the FFS with fc-loadtool and letting the firmware regenerate a fresh - one on its next boot. We can thus reasonably assume that the FFS in these - specimen is indeed the product of such intervention at some point in the - phone's history. - -2) In other specimen we see some additional files beyond those that exist with - the life cycle of the previous paragraph. One readily noticeable addition - is the presence of /gsm/rf/tx/ramps.* files, which are absent in the self- - regenerated FFS where the phone's fw copied calibration records from SE's - dedicated calibration data sector. How are these Tx ramp tables different - from all other RF calibration records? Answer: in the present SE K2x0 - design, and also in all other Calypso phone and modem designs known to this - Mother, Tx ramps are calibrated per design and not per unit. Because the - correct Tx ramp tables are compiled into the firmware, having them in FFS is - redundant. - -Based on the above evidence, I (Mother Mychaela) hereby make the following -reconstruction of how these two kinds of FFS found in the wild likely came into -being: - -* In the case of Compal (Mot C1xx and SE J100) and Pirelli DP-L10 we know by - simple deductive reasoning that those firmwares must have had their l1_cust.c - code modified to read calibration records from their respective proprietary - flash data structures instead of TI's original FFS files. However, in the - case of SE K2x0 it is plausible that their l1_cust.c did NOT have to be - likewise modified - they could plausibly still have TI's original l1_cust.c - code that reads RF tables from FFS on initialization and writes them back - into FFS upon those special MISC_ENABLE commands that are issued at the end - of TI's original calibration procedures. - -* If my hypothesis above is correct, then it follows that when SE or their ODM - did their factory production line calibration, the results were initially - written into FFS and not into SE's separate factory calibration data sector. - If they are using unmodified l1_cust.c code from TI, then the presence of - redundant /gsm/rf/tx/ramps.* files is explained - TI's standard calibration - procedure results in these files being written into FFS because the canonical - l1_cust.c code classifies them as "Tx calibration" rather than "Tx config" - - see our RF_tables article. - -* In a separate step after all standard calibration procedures, they must have - copied all important calibration records into their non-standard flash sector - at 0x01FD0000, specifically to produce a more durable copy that can persist - through full erasure of FFS. Tx ramp tables were not included because they - are relatively large and redundant, not actually calibrated per unit. - -* Those SE K2x0 specimen whose FFS contains /gsm/rf/tx/ramps.* files and some - other files that aren't found in auto-regenerated FFS probably exhibit the - original FFS from the factory production line that was never subjected to - being blown away and regenerated! - -SE's non-standard extension to TIFFS -==================================== - -There is one aspect of SE K2x0 FFS that creates a pain point for our tools: -they made their own non-standard extension to TIFFS in terms of extended UCS-2 -filenames, and given our natural desire to be able to use our tiffs tool (TIFFS -IVA) to examine FFS images from these SE phones, we had to extend our tool so -that it parses SE's extension instead of throwing up errors like it did before. - -As explained in our TIFFS-Overview article, classic TIFFS requires every -elementary filename to consist of printable ASCII characters from the sensibly -narrow set of [A-Za-z0-9_.,+%$#-], further limited to a maximum of 20 -characters. However, SE K2x0 FFS images often contain some files that violate -this constraint; deeper examination reveals that SE (or their ODM) devised a -feature to allow filenames composed of UCS-2 characters, implemented as follows: - -* The first "character" of the filename written into TIFFS is \x02, i.e., NOT a - printable ASCII character; - -* The next "character" in the TIFFS object name is another binary byte encoding - the length of the UCS-2 string: for example, if the "high-level" name consists - of 6 UCS-2 characters, this second TIFFS object name "character" is \x06; - -* These two non-printable-ASCII chars are followed by a long string of [0-9A-F] - ASCII characters, encoding the "high-level" UCS-2 name in hex; - -* The whole arrangement ends with a terminating NUL, allowing the rest of TIFFS - to work unchanged. - -Seen from TIFFS perspective, these extended filenames created by SE K2x0 fw -violate traditional canon in two ways: - -1) The overall length of the mostly-hex-encoded filename string usually exceeds - the traditional 20 character limit; - -2) The set of allowed characters is grossly violated: not an innocuous extension - of allowing some additional "sane" characters, but non-printable-ASCII binary - chars that are barely compatible with C string functions by virtue of not - containing zero bytes. - -Please see the new section titled "Support for SE K2x0 extended filenames" in -our TIFFS-IVA-usage article for the explanation of how we handle these extended -filenames in our TIFFS IVA tool. - -fc-fsio considerations -====================== - -In addition to "in vitro" analysis with our tiffs tool, SE K2x0 FFS can also be -accessed "in vivo" with fc-fsio in the following two scenarios: - -1) SE K2x0 firmware does not have RVTMUX enabled by default, but they have - non-volatile flags in their FFS (which can be set via a hidden menu entered - via secret MMI code #*87223564#) that enable RVTMUX on the MODEM UART that - is brought out, and their RVTMUX includes ETM and TMFFS2 protocols. - -2) We can run TI's original FFS code against SE K2x0 FFS bodies by adding the - necessary configuration to our fc-xram based FFS editor tool described in - our TIFFS-Overview article. - -The expected behaviour in scenario 2 can be predicted statically by studying -the code, whereas scenario 1 calls for experimentation. Experiments with -RVTMUX-enabled SE K2x0 original fw reveal that it does behave (at least as far -as visible behaviour over TMFFS2) exactly like TI's original FFS code in terms -of readdir output when listing directories with extended filenames in them: as -long as they fit into TMFFS2 string buffer, these extended filenames are -returned just as one would naively expect. - -In light of these observations, fc-fsio has also been patched up to behave -gracefully when faced with such previously unexpected readdir results from the -target. Specifically: - -* The short form of 'ls' command will print whatever the target firmware - returns, including extended filenames; any non-printable-ASCII characters are - escaped in \x hex form. - -* 'ls -l' or 'll' will throw up an error if one attempts to list a directory - that contains one or more extended filenames, caught either at the level of - the readdir result not fitting into the buffer sized for standard TIFFS - filenames or (for SE K2x0 extended filenames of 4 or fewer UCS-2 characters) - at the level of this readdir result containing non-printable-ASCII chars. - -Finally, the following limitations need to be kept in mind: - -* There is no way to address an FFS object with an extended filename by entering - a pathname in fc-fsio commands; - -* File systems or subtrees containing extended filenames cannot be read out with - fc-fsio cpout command: it will fail just like ls -l; - -* rm-subtree and cleandir commands will likewise fail and stop whenever they - encounter a filename longer than 20 characters (invalid for the purpose of - pathname generation based on fixed-length buffers) anywhere in the subtree to - be deleted. - -The last limitation (inability to delete "bad" files) may seem like a serious -flaw in the design of fc-fsio at first, but please realize that the primary -mission of FreeCalypso is to work with our own hardware and firmware, not alien -- and the original FFS code from TI, which is what we use in FreeCalypso, will -never allow file system objects with such bad filenames to be created in the -first place. Therefore, the limitation of fc-fsio being unable to manipulate -certain files to the extent of not even being able to delete them is specific -to the peculiar scenario of operating on *alien* FFS from Sony Ericsson.
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/SE-K2xx-FFS Mon Sep 23 12:23:20 2024 +0000 @@ -0,0 +1,184 @@ +Flash file system in Sony Ericsson K2xx phones +================================================== + +SE K2xx phones are based on our familiar Calypso chipset, their firmware is +based on the standard chipset reference fw from TI, and they use TIFFS as their +flexible data storage mechanism. Their TIFFS instance is located in the first +3328 KiB of the 2nd flash bank; the geometry is 256x13 or 64x52 depending on +which flash chip was used in each given phone. (Some specimen have Spansion +S71PL129NB0 chips with 256 KiB flash sectors, others feature Samsung K5L2931CAM +chips with 64 KiB flash sectors.) + +Calibration data and self-regenerating ability +============================================== + +SE's FFS life cycle design on these phones is interesting in that vital per-unit +calibration data (RF, AFC and MADC calibrations) are stored in two places: +(1) in the FFS in TI's standard format and locations, and (2) in a dedicated +flash sector outside of FFS that appears to be written once at the factory and +never touched afterward. + +SE's firmware design allows the FFS to be fully erased without breaking the +phone: whenever the fw boots in a state where the FFS area of the flash is fully +erased (you can of course erase those FFS sectors with fc-loadtool, but it +appears that some official tools also provide the same operation) but the +separate factory calibration data sector is valid, the firmware formats and +initializes a new FFS, and furthermore copies all essential calibration records +from SE's proprietary sector structure into their standard TIFFS locations! + +But it gets even more interesting - when we examine FFS images read out of +various SE K2xx specimen, we see two different occurrences: + +1) In some specimen the FFS content is exactly the same as what we would get by + erasing the FFS with fc-loadtool and letting the firmware regenerate a fresh + one on its next boot. We can thus reasonably assume that the FFS in these + specimen is indeed the product of such intervention at some point in the + phone's history. + +2) In other specimen we see some additional files beyond those that exist with + the life cycle of the previous paragraph. One readily noticeable addition + is the presence of /gsm/rf/tx/ramps.* files, which are absent in the self- + regenerated FFS where the phone's fw copied calibration records from SE's + dedicated calibration data sector. How are these Tx ramp tables different + from all other RF calibration records? Answer: in the present SE K2xx + design, and also in all other Calypso phone and modem designs known to this + Mother, Tx ramps are calibrated per design and not per unit. Because the + correct Tx ramp tables are compiled into the firmware, having them in FFS is + redundant. + +Based on the above evidence, I (Mother Mychaela) hereby make the following +reconstruction of how these two kinds of FFS found in the wild likely came into +being: + +* In the case of Compal (Mot C1xx and SE J100) and Pirelli DP-L10 we know by + simple deductive reasoning that those firmwares must have had their l1_cust.c + code modified to read calibration records from their respective proprietary + flash data structures instead of TI's original FFS files. However, in the + case of SE K2xx it is plausible that their l1_cust.c did NOT have to be + likewise modified - they could plausibly still have TI's original l1_cust.c + code that reads RF tables from FFS on initialization and writes them back + into FFS upon those special MISC_ENABLE commands that are issued at the end + of TI's original calibration procedures. + +* If my hypothesis above is correct, then it follows that when SE or their ODM + did their factory production line calibration, the results were initially + written into FFS and not into SE's separate factory calibration data sector. + If they are using unmodified l1_cust.c code from TI, then the presence of + redundant /gsm/rf/tx/ramps.* files is explained - TI's standard calibration + procedure results in these files being written into FFS because the canonical + l1_cust.c code classifies them as "Tx calibration" rather than "Tx config" - + see our RF_tables article. + +* In a separate step after all standard calibration procedures, they must have + copied all important calibration records into their non-standard flash sector + at 0x01FD0000, specifically to produce a more durable copy that can persist + through full erasure of FFS. Tx ramp tables were not included because they + are relatively large and redundant, not actually calibrated per unit. + +* Those SE K2xx specimen whose FFS contains /gsm/rf/tx/ramps.* files and some + other files that aren't found in auto-regenerated FFS probably exhibit the + original FFS from the factory production line that was never subjected to + being blown away and regenerated! + +SE's non-standard extension to TIFFS +==================================== + +There is one aspect of SE K2xx FFS that creates a pain point for our tools: +they made their own non-standard extension to TIFFS in terms of extended UCS-2 +filenames, and given our natural desire to be able to use our tiffs tool (TIFFS +IVA) to examine FFS images from these SE phones, we had to extend our tool so +that it parses SE's extension instead of throwing up errors like it did before. + +As explained in our TIFFS-Overview article, classic TIFFS requires every +elementary filename to consist of printable ASCII characters from the sensibly +narrow set of [A-Za-z0-9_.,+%$#-], further limited to a maximum of 20 +characters. However, SE K2xx FFS images often contain some files that violate +this constraint; deeper examination reveals that SE (or their ODM) devised a +feature to allow filenames composed of UCS-2 characters, implemented as follows: + +* The first "character" of the filename written into TIFFS is \x02, i.e., NOT a + printable ASCII character; + +* The next "character" in the TIFFS object name is another binary byte encoding + the length of the UCS-2 string: for example, if the "high-level" name consists + of 6 UCS-2 characters, this second TIFFS object name "character" is \x06; + +* These two non-printable-ASCII chars are followed by a long string of [0-9A-F] + ASCII characters, encoding the "high-level" UCS-2 name in hex; + +* The whole arrangement ends with a terminating NUL, allowing the rest of TIFFS + to work unchanged. + +Seen from TIFFS perspective, these extended filenames created by SE K2xx fw +violate traditional canon in two ways: + +1) The overall length of the mostly-hex-encoded filename string usually exceeds + the traditional 20 character limit; + +2) The set of allowed characters is grossly violated: not an innocuous extension + of allowing some additional "sane" characters, but non-printable-ASCII binary + chars that are barely compatible with C string functions by virtue of not + containing zero bytes. + +Please see the new section titled "Support for SE K2xx extended filenames" in +our TIFFS-IVA-usage article for the explanation of how we handle these extended +filenames in our TIFFS IVA tool. + +fc-fsio considerations +====================== + +In addition to "in vitro" analysis with our tiffs tool, SE K2xx FFS can also be +accessed "in vivo" with fc-fsio in the following two scenarios: + +1) SE K2xx firmware does not have RVTMUX enabled by default, but they have + non-volatile flags in their FFS (which can be set via a hidden menu entered + via secret MMI code #*87223564#) that enable RVTMUX on the MODEM UART that + is brought out, and their RVTMUX includes ETM and TMFFS2 protocols. + +2) We can run TI's original FFS code against SE K2xx FFS bodies by adding the + necessary configuration to our fc-xram based FFS editor tool described in + our TIFFS-Overview article. + +The expected behaviour in scenario 2 can be predicted statically by studying +the code, whereas scenario 1 calls for experimentation. Experiments with +RVTMUX-enabled SE K2xx original fw reveal that it does behave (at least as far +as visible behaviour over TMFFS2) exactly like TI's original FFS code in terms +of readdir output when listing directories with extended filenames in them: as +long as they fit into TMFFS2 string buffer, these extended filenames are +returned just as one would naively expect. + +In light of these observations, fc-fsio has also been patched up to behave +gracefully when faced with such previously unexpected readdir results from the +target. Specifically: + +* The short form of 'ls' command will print whatever the target firmware + returns, including extended filenames; any non-printable-ASCII characters are + escaped in \x hex form. + +* 'ls -l' or 'll' will throw up an error if one attempts to list a directory + that contains one or more extended filenames, caught either at the level of + the readdir result not fitting into the buffer sized for standard TIFFS + filenames or (for SE K2xx extended filenames of 4 or fewer UCS-2 characters) + at the level of this readdir result containing non-printable-ASCII chars. + +Finally, the following limitations need to be kept in mind: + +* There is no way to address an FFS object with an extended filename by entering + a pathname in fc-fsio commands; + +* File systems or subtrees containing extended filenames cannot be read out with + fc-fsio cpout command: it will fail just like ls -l; + +* rm-subtree and cleandir commands will likewise fail and stop whenever they + encounter a filename longer than 20 characters (invalid for the purpose of + pathname generation based on fixed-length buffers) anywhere in the subtree to + be deleted. + +The last limitation (inability to delete "bad" files) may seem like a serious +flaw in the design of fc-fsio at first, but please realize that the primary +mission of FreeCalypso is to work with our own hardware and firmware, not alien +- and the original FFS code from TI, which is what we use in FreeCalypso, will +never allow file system objects with such bad filenames to be created in the +first place. Therefore, the limitation of fc-fsio being unable to manipulate +certain files to the extent of not even being able to delete them is specific +to the peculiar scenario of operating on *alien* FFS from Sony Ericsson.
--- a/doc/TIFFS-IVA-usage Thu May 23 17:29:57 2024 +0000 +++ b/doc/TIFFS-IVA-usage Mon Sep 23 12:23:20 2024 +0000 @@ -191,14 +191,14 @@ Just like regular cat, but takes an inode number instead of a pathname. Can be used to cat the old content of deleted or overwritten files. -Support for SE K2x0 extended filenames +Support for SE K2xx extended filenames ====================================== -As explained in the SE-K2x0-FFS article, Sony Ericsson K200/220 phones use +As explained in the SE-K2xx-FFS article, Sony Ericsson K2xx phones use TIFFS as their flexible data storage mechanism, but they also made their own non-standard extension to TIFFS in terms of extended UCS-2 filenames. -In order to allow listing full FFS trees from SE K2x0 phones with tiffs ls and +In order to allow listing full FFS trees from SE K2xx phones with tiffs ls and possibly also extracting them with tiffs xtr, we have made the following hackish extension to our TIFFS IVA: whenever either of these two commands encounters an "extended" filename (one which fails standard TIFFS filename validity checks but
--- a/loadtools/loadtool.help Thu May 23 17:29:57 2024 +0000 +++ b/loadtools/loadtool.help Mon Sep 23 12:23:20 2024 +0000 @@ -145,7 +145,7 @@ memories of the supported GSM devices. Most Calypso phones and modems have only one flash bank (as in chip select), and are manipulated with the flash command. FreeCalypso development boards and some complex phones (Pirelli -DP-L10, Sony Ericsson K2x0) have two flash banks (as in chip selects) of 8 MiB +DP-L10, Sony Ericsson K2xx) have two flash banks (as in chip selects) of 8 MiB each; they are manipulated with flash and flash2 commands. The following flash operations are available: