Reading factory RF calibration values out of Mot C1xx and SE J100 phones
========================================================================

Motorola C1xx and Sony Ericsson J100 phones are based on the Calypso+Iota+Rita
chipset from TI and their firmware is also loosely based on TI's reference, but
Compal (the ODM who produced these phones for Motorola and SE) made lots and
lots of changes moving away from TI's canonical way of doing things.  When it
comes to RF calibration, Mot/SE/Compal have performed it on each individual
unit on their factory production line just like all other GSM phone and modem
manufacturers, but instead of storing the results of this calibration in TI's
flash file system, Compal put these calibration values into a completely
different flash data structure of their own invention.

We don't know the proper name for Mot/SE/Compal's flash data structure that has
no counterpart in TI's canonical solution, but we know its location in the
flash:

* On C1xx phones with 2 MiB flash (most C11x/12x variants), it is the 8 KiB
  flash sector at 0x1FC000;

* On C139/140 phones and the rare C11x/12x variants with 4 MiB flash, and also
  on the SE J100, it is the 8 KiB flash sector at 0x3FC000;

* On C155/156 phones with 8 MiB flash, the data structure in question is
  contained in the first 8 KiB of the 64 KiB physical flash sector at 0x7E0000.

The flash sector in question contains record-structured data; we don't know the
meaning of most of these records, but we have been able to find the RF
calibration records among them and locate the actual calibration values of
interest inside those records.

In order to extract the RF calibration values from your C1xx or SE J100 phone
for use with FreeCalypso, you will need a dump of your phone's flash, or at
least a dump of the specific 8 KiB sector at the model-dependent offset given
above.  As a specific example, if your phone is a C139/140 or a C11x/12x
variant with 4 MiB flash (or SE J100) and you have a complete dump of that
flash, execute a command like the following:

c1xx-calextr -b rfbin flashdump.bin 0x3FC000

The c1xx-calextr utility locates the RF calibration records in the flash dump
binary, extracts those calibration values contained therein which we are able
to grok, and converts them to TI's canonical format for use with FreeCalypso
firmware.  The numeric argument after the flash image filename is the offset
within that image file where the magic sector should be sought, and the -b
option directs the tool to save the converted RF calibration tables in binary
format (the alternative is -a for ASCII format) in the directory named after
the option, named rfbin in this example.

If you use the binary output option as recommended here, the resulting output
directory will have two subdirectories in it, named rx and tx.  The rx
subdirectory will contain agcparams and calchan tables for each band, and the
tx subdirectory will contain a levels table for each band.  This directory
structure and these names for the binary files correspond directly to the
/gsm/rf directory subtree in the flash file system (FFS) of TI's canonical
solution, hence once the C1xx phone in question is converted to FreeCalypso
(i.e., runs FreeCalypso fw with an aftermarket FFS created for it), you can
upload the extracted and converted RF calibration values into it like this:

fc-fsio upload-subtree rfbin /gsm/rf

Rx channel correction values
============================

A GSM phone or modem needs to know how to derive the actual input signal level
in dBm from the power measurements reported by the DSP; the difference between
the two is called the "magic gain" (GMagic), and the firmware needs to know
what it is.  The primary GMagic value for each band is calibrated at the center
frequency of that band, and then there are channel-dependent corrections
applied.

In TI's canonical solution the complete ARFCN range of each band is divided
into up to 10 subbands, and each of these subbands gets its own channel
correction value.  The ARFCN boundaries between the subbands are defined by the
external calibration system and not by the firmware code, by virtue of being
given inside the Rx calchan table itself along with the correction values.

But Compal (all C1xx variants and SE J100) have made two changes:

* They increased the number of subbands from 10 (TI's canon) to 21 for the
  GSM850 band, 30 for the EGSM band, 63 for DCS and 50 for PCS, so that each
  subband is only 6 channels (1.2 MHz).

* The ARFCN boundaries for the subbands are not stored in the calibration
  records in the flash, but are fixed in the firmware instead.

Changing our FreeCalypso firmware to allow up to 63 Rx AGC subbands to match
Compal's fw architecture would be too disruptive, hence our current c1xx-calextr
implementation translates Compal's Rx channel correction values to TI/FC format
by combining groups of Compal's subbands into larger subbands, and making a
mean value out of the smaller subband correction values in Compal's factory
calibration record.

The AGC subbands defined by Compal's fw are listed below, with each numeric
line giving the ARFCN range of each subband; blank lines separate the groupings
made by c1xx-calextr.

850 MHz band:

128-134
135-140

141-146
147-152

153-158
159-164

165-170
171-176

177-182
183-188

189-194
195-200

201-206
207-212

213-218
219-224

225-230
231-236

237-242
243-248
249-251

900 MHz band:

0-6
7-12
13-18

19-24
25-30
31-36

37-42
43-48
49-54

55-60
61-66
67-72

73-78
79-84
85-90

91-96
97-102
103-108

109-114
115-120
121-124

975-975
976-981
982-987

988-993
994-999
1000-1005

1006-1011
1012-1017
1018-1023

1800 MHz band:

512-518
519-524
525-530
531-536
537-542
543-548

549-554
555-560
561-566
567-572
573-578
579-584

585-590
591-596
597-602
603-608
609-614
615-620

621-626
627-632
633-638
639-644
645-650
651-656

657-662
663-668
669-674
675-680
681-686
687-692

693-698
699-704
705-710
711-716
717-722
723-728

729-734
735-740
741-746
747-752
753-758
759-764

765-770
771-776
777-782
783-788
789-794
795-800
801-806

807-812
813-818
819-824
825-830
831-836
837-842
843-848

849-854
855-860
861-866
867-872
873-878
879-884
885-885

1900 MHz band:

512-518
519-524
525-530
531-536
537-542

543-548
549-554
555-560
561-566
567-572

573-578
579-584
585-590
591-596
597-602

603-608
609-614
615-620
621-626
627-632

633-638
639-644
645-650
651-656
657-662

663-668
669-674
675-680
681-686
687-692

693-698
699-704
705-710
711-716
717-722

723-728
729-734
735-740
741-746
749-752

753-758
759-764
765-770
771-776
777-782

783-788
789-794
795-800
801-806
807-810

Tx channel correction values
============================

A similar situation holds here: in TI's canon each band is divided into 8
subbands for the purpose of Tx channel-dependent corrections, but Mot/Compal
seem to be using smaller subbands: 13 for the GSM850 band, 18 for EGSM, 38 for
DCS and 30 for PCS.  We can see where these correction values are stored in the
calibration records in the flash (immediately after the Tx levels array), but
the ARFCN boundaries of Mot/Compal's Tx channel calibration subbands are not
known, and the semantics of the correction values themselves are not clear:
Mot/Compal's Tx channel correction values are centered around 0, whereas in
TI's canonical version they are centered around 128.

Because we are not able to grok Mot/Compal's Tx channel correction, we currently
ignore this part of their factory calibration, i.e., FreeCalypso fw will run
with all channel correction values set to 128, meaning no channel correction.
But since we do use the Tx levels table of APC DAC values from Mot/Compal's
factory records, and given that the tolerances for Tx power levels given in the
GSM 05.05 spec are quite generous, we expect to still be within these tolerances
despite the lack of channel correction.

In vivo approach: tried and failed
==================================

Before I figured out the format of Mot/Compal's factory calibration records in
their flash and wrote the c1xx-calextr "in vitro" extraction and conversion
tool, I tried an "in vivo" approach: reading the calibration values out from
the running firmware via TI's L1/RF Test Mode commands which are still present
in Mot/Compal's fw.  This approach successfully yielded the tables of Tx ramp
templates which are calibrated per design rather than per unit and thus compiled
into the fw and not present in the per-unit factory calibration records (these
extracted Tx ramps tables are now used by FC Magnetite fw when built for the
C139 target), but does not help with much of anything else:

* One can read the calibrated Tx levels table (rftr 16) for the low frequency
  band (850 or 900 MHz), but not for the high (1800 or 1900 MHz) band: in order
  to access the tables for the high band, one needs to issue an rfpw 7 command,
  but in Mot/Compal's version the latter command only loads the compiled-in
  tables and does not apply their non-TI calibration records.

* The Rx agcparams table returned in response to rftr 31 always has the GMagic
  field set to the fw's compiled-in value and not the calibrated one.

* The Rx calchan table (which Mot/Compal enlarged from 10 to 63 entries as
  explained earlier in this article) cannot be read out at all: the rftr 25
  command crashes the firmware, probably via a buffer overflow from the
  enlarged table.

* The Tx calchan table can be read out with rftr 17, but it does not make any
  sense: it still has 4 copies of a table of 8 subbands like in TI's canon,
  even though when we look at their factory calibration records, we can clearly
  see that the table of Tx channel correction values is also enlarged.  But the
  correction values themselves are centered around 0 in this strange table
  returned in response to rftr 17, and not around 128 like in TI's canon.

The fc-readcal utility was written before c1xx-calextr, and it was my original
idea of how to extract Mot/Compal's factory RF calibration values.  It features
a -c command line option for "Compal mode" which disables the reading of Rx
calchan and Tx calchan tables via rftr 25 and rftr 17, respectively (the former
crashes the fw, the latter has the wrong semantics), but because it issues
rfpw 7 commands for each band preceded by tms 1, it will only yield the
firmware's compiled-in values, and not any of the factory-calibrated ones.

Therefore, the fc-readcal method should not be used, and the c1xx-calextr method
described in the main body of this article should be used instead.