FreeCalypso > hg > freecalypso-tools

comparison doc/SE-K2x0-FFS @ 922:3152e23399a2

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
document SE K2x0 FFS quirks and our support for them
author Mychaela Falconia <falcon@freecalypso.org>
date Mon, 02 Jan 2023 00:50:19 +0000
parents
children
comparison
equal deleted inserted replaced
921:74d284add54d 922:3152e23399a2
1 Flash file system in Sony Ericsson K200/220 phones
2 ==================================================
3
4 SE K200/220 phones are based on our familiar Calypso chipset, their firmware is
5 based on the standard chipset reference fw from TI, and they use TIFFS as their
6 flexible data storage mechanism. Their TIFFS instance is located in the first
7 3328 KiB of the 2nd flash bank; the geometry is 256x13 or 64x52 depending on
8 which flash chip was used in each given phone. (Some specimen have Spansion
9 S71PL129NB0 chips with 256 KiB flash sectors, others feature Samsung K5L2931CAM
10 chips with 64 KiB flash sectors.)
11
12 Calibration data and self-regenerating ability
13 ==============================================
14
15 SE's FFS life cycle design on these phones is interesting in that vital per-unit
16 calibration data (RF, AFC and MADC calibrations) are stored in two places:
17 (1) in the FFS in TI's standard format and locations, and (2) in a dedicated
18 flash sector outside of FFS that appears to be written once at the factory and
19 never touched afterward.
20
21 SE's firmware design allows the FFS to be fully erased without breaking the
22 phone: whenever the fw boots in a state where the FFS area of the flash is fully
23 erased (you can of course erase those FFS sectors with fc-loadtool, but it
24 appears that some official tools also provide the same operation) but the
25 separate factory calibration data sector is valid, the firmware formats and
26 initializes a new FFS, and furthermore copies all essential calibration records
27 from SE's proprietary sector structure into their standard TIFFS locations!
28
29 But it gets even more interesting - when we examine FFS images read out of
30 various SE K2x0 specimen, we see two different occurrences:
31
32 1) In some specimen the FFS content is exactly the same as what we would get by
33 erasing the FFS with fc-loadtool and letting the firmware regenerate a fresh
34 one on its next boot. We can thus reasonably assume that the FFS in these
35 specimen is indeed the product of such intervention at some point in the
36 phone's history.
37
38 2) In other specimen we see some additional files beyond those that exist with
39 the life cycle of the previous paragraph. One readily noticeable addition
40 is the presence of /gsm/rf/tx/ramps.* files, which are absent in the self-
41 regenerated FFS where the phone's fw copied calibration records from SE's
42 dedicated calibration data sector. How are these Tx ramp tables different
43 from all other RF calibration records? Answer: in the present SE K2x0
44 design, and also in all other Calypso phone and modem designs known to this
45 Mother, Tx ramps are calibrated per design and not per unit. Because the
46 correct Tx ramp tables are compiled into the firmware, having them in FFS is
47 redundant.
48
49 Based on the above evidence, I (Mother Mychaela) hereby make the following
50 reconstruction of how these two kinds of FFS found in the wild likely came into
51 being:
52
53 * In the case of Compal (Mot C1xx and SE J100) and Pirelli DP-L10 we know by
54 simple deductive reasoning that those firmwares must have had their l1_cust.c
55 code modified to read calibration records from their respective proprietary
56 flash data structures instead of TI's original FFS files. However, in the
57 case of SE K2x0 it is plausible that their l1_cust.c did NOT have to be
58 likewise modified - they could plausibly still have TI's original l1_cust.c
59 code that reads RF tables from FFS on initialization and writes them back
60 into FFS upon those special MISC_ENABLE commands that are issued at the end
61 of TI's original calibration procedures.
62
63 * If my hypothesis above is correct, then it follows that when SE or their ODM
64 did their factory production line calibration, the results were initially
65 written into FFS and not into SE's separate factory calibration data sector.
66 If they are using unmodified l1_cust.c code from TI, then the presence of
67 redundant /gsm/rf/tx/ramps.* files is explained - TI's standard calibration
68 procedure results in these files being written into FFS because the canonical
69 l1_cust.c code classifies them as "Tx calibration" rather than "Tx config" -
70 see our RF_tables article.
71
72 * In a separate step after all standard calibration procedures, they must have
73 copied all important calibration records into their non-standard flash sector
74 at 0x01FD0000, specifically to produce a more durable copy that can persist
75 through full erasure of FFS. Tx ramp tables were not included because they
76 are relatively large and redundant, not actually calibrated per unit.
77
78 * Those SE K2x0 specimen whose FFS contains /gsm/rf/tx/ramps.* files and some
79 other files that aren't found in auto-regenerated FFS probably exhibit the
80 original FFS from the factory production line that was never subjected to
81 being blown away and regenerated!
82
83 SE's non-standard extension to TIFFS
84 ====================================
85
86 There is one aspect of SE K2x0 FFS that creates a pain point for our tools:
87 they made their own non-standard extension to TIFFS in terms of extended UCS-2
88 filenames, and given our natural desire to be able to use our tiffs tool (TIFFS
89 IVA) to examine FFS images from these SE phones, we had to extend our tool so
90 that it parses SE's extension instead of throwing up errors like it did before.
91
92 As explained in our TIFFS-Overview article, classic TIFFS requires every
93 elementary filename to consist of printable ASCII characters from the sensibly
94 narrow set of [A-Za-z0-9_.,+%$#-], further limited to a maximum of 20
95 characters. However, SE K2x0 FFS images often contain some files that violate
96 this constraint; deeper examination reveals that SE (or their ODM) devised a
97 feature to allow filenames composed of UCS-2 characters, implemented as follows:
98
99 * The first "character" of the filename written into TIFFS is \x02, i.e., NOT a
100 printable ASCII character;
101
102 * The next "character" in the TIFFS object name is another binary byte encoding
103 the length of the UCS-2 string: for example, if the "high-level" name consists
104 of 6 UCS-2 characters, this second TIFFS object name "character" is \x06;
105
106 * These two non-printable-ASCII chars are followed by a long string of [0-9A-F]
107 ASCII characters, encoding the "high-level" UCS-2 name in hex;
108
109 * The whole arrangement ends with a terminating NUL, allowing the rest of TIFFS
110 to work unchanged.
111
112 Seen from TIFFS perspective, these extended filenames created by SE K2x0 fw
113 violate traditional canon in two ways:
114
115 1) The overall length of the mostly-hex-encoded filename string usually exceeds
116 the traditional 20 character limit;
117
118 2) The set of allowed characters is grossly violated: not an innocuous extension
119 of allowing some additional "sane" characters, but non-printable-ASCII binary
120 chars that are barely compatible with C string functions by virtue of not
121 containing zero bytes.
122
123 Please see the new section titled "Support for SE K2x0 extended filenames" in
124 our TIFFS-IVA-usage article for the explanation of how we handle these extended
125 filenames in our TIFFS IVA tool.
126
127 fc-fsio considerations
128 ======================
129
130 In addition to "in vitro" analysis with our tiffs tool, SE K2x0 FFS can also be
131 accessed "in vivo" with fc-fsio in the following two scenarios:
132
133 1) SE K2x0 firmware does not have RVTMUX enabled by default, but they have
134 non-volatile flags in their FFS (which can be set via a hidden menu entered
135 via secret MMI code #*87223564#) that enable RVTMUX on the MODEM UART that
136 is brought out, and their RVTMUX includes ETM and TMFFS2 protocols.
137
138 2) We can run TI's original FFS code against SE K2x0 FFS bodies by adding the
139 necessary configuration to our fc-xram based FFS editor tool described in
140 our TIFFS-Overview article.
141
142 The expected behaviour in scenario 2 can be predicted statically by studying
143 the code, whereas scenario 1 calls for experimentation. Experiments with
144 RVTMUX-enabled SE K2x0 original fw reveal that it does behave (at least as far
145 as visible behaviour over TMFFS2) exactly like TI's original FFS code in terms
146 of readdir output when listing directories with extended filenames in them: as
147 long as they fit into TMFFS2 string buffer, these extended filenames are
148 returned just as one would naively expect.
149
150 In light of these observations, fc-fsio has also been patched up to behave
151 gracefully when faced with such previously unexpected readdir results from the
152 target. Specifically:
153
154 * The short form of 'ls' command will print whatever the target firmware
155 returns, including extended filenames; any non-printable-ASCII characters are
156 escaped in \x hex form.
157
158 * 'ls -l' or 'll' will throw up an error if one attempts to list a directory
159 that contains one or more extended filenames, caught either at the level of
160 the readdir result not fitting into the buffer sized for standard TIFFS
161 filenames or (for SE K2x0 extended filenames of 4 or fewer UCS-2 characters)
162 at the level of this readdir result containing non-printable-ASCII chars.
163
164 Finally, the following limitations need to be kept in mind:
165
166 * There is no way to address an FFS object with an extended filename by entering
167 a pathname in fc-fsio commands;
168
169 * File systems or subtrees containing extended filenames cannot be read out with
170 fc-fsio cpout command: it will fail just like ls -l;
171
172 * rm-subtree and cleandir commands will likewise fail and stop whenever they
173 encounter a filename longer than 20 characters (invalid for the purpose of
174 pathname generation based on fixed-length buffers) anywhere in the subtree to
175 be deleted.
176
177 The last limitation (inability to delete "bad" files) may seem like a serious
178 flaw in the design of fc-fsio at first, but please realize that the primary
179 mission of FreeCalypso is to work with our own hardware and firmware, not alien
180 - and the original FFS code from TI, which is what we use in FreeCalypso, will
181 never allow file system objects with such bad filenames to be created in the
182 first place. Therefore, the limitation of fc-fsio being unable to manipulate
183 certain files to the extent of not even being able to delete them is specific
184 to the peculiar scenario of operating on *alien* FFS from Sony Ericsson.