Reading factory RF calibration values out of Mot C1xx and SE J100 phones========================================================================Motorola C1xx and Sony Ericsson J100 phones are based on the Calypso+Iota+Ritachipset from TI and their firmware is also loosely based on TI's reference, butCompal (the ODM who produced these phones for Motorola and SE) made lots andlots of changes moving away from TI's canonical way of doing things. When itcomes to RF calibration, Mot/SE/Compal have performed it on each individualunit on their factory production line just like all other GSM phone and modemmanufacturers, but instead of storing the results of this calibration in TI'sflash file system, Compal put these calibration values into a completelydifferent flash data structure of their own invention.We don't know the proper name for Mot/SE/Compal's flash data structure that hasno counterpart in TI's canonical solution, but we know its location in theflash:* On C1xx phones with 2 MiB flash (most C11x/12x variants), it is the 8 KiB flash sector at 0x1FC000;* On C139/140 phones and the rare C11x/12x variants with 4 MiB flash, and also on the SE J100, it is the 8 KiB flash sector at 0x3FC000;* On C155/156 phones with 8 MiB flash, the data structure in question is contained in the first 8 KiB of the 64 KiB physical flash sector at 0x7E0000.The flash sector in question contains record-structured data; we don't know themeaning of most of these records, but we have been able to find the RFcalibration records among them and locate the actual calibration values ofinterest inside those records.In order to extract the RF calibration values from your C1xx or SE J100 phonefor use with FreeCalypso, you will need a dump of your phone's flash, or atleast a dump of the specific 8 KiB sector at the model-dependent offset givenabove. As a specific example, if your phone is a C139/140 or a C11x/12xvariant with 4 MiB flash (or SE J100) and you have a complete dump of thatflash, execute a command like the following:c1xx-calextr -b rfbin flashdump.bin 0x3FC000The c1xx-calextr utility locates the RF calibration records in the flash dumpbinary, extracts those calibration values contained therein which we are ableto grok, and converts them to TI's canonical format for use with FreeCalypsofirmware. The numeric argument after the flash image filename is the offsetwithin that image file where the magic sector should be sought, and the -boption directs the tool to save the converted RF calibration tables in binaryformat (the alternative is -a for ASCII format) in the directory named afterthe option, named rfbin in this example.If you use the binary output option as recommended here, the resulting outputdirectory will have two subdirectories in it, named rx and tx. The rxsubdirectory will contain agcparams and calchan tables for each band, and thetx subdirectory will contain a levels table for each band. This directorystructure and these names for the binary files correspond directly to the/gsm/rf directory subtree in the flash file system (FFS) of TI's canonicalsolution, hence once the C1xx phone in question is converted to FreeCalypso(i.e., runs FreeCalypso fw with an aftermarket FFS created for it), you canupload the extracted and converted RF calibration values into it like this:fc-fsio upload-subtree rfbin /gsm/rfRx channel correction values============================A GSM phone or modem needs to know how to derive the actual input signal levelin dBm from the power measurements reported by the DSP; the difference betweenthe two is called the "magic gain" (GMagic), and the firmware needs to knowwhat it is. The primary GMagic value for each band is calibrated at the centerfrequency of that band, and then there are channel-dependent correctionsapplied.In TI's canonical solution the complete ARFCN range of each band is dividedinto up to 10 subbands, and each of these subbands gets its own channelcorrection value. The ARFCN boundaries between the subbands are defined by theexternal calibration system and not by the firmware code, by virtue of beinggiven inside the Rx calchan table itself along with the correction values.But Compal (all C1xx variants and SE J100) have made two changes:* They increased the number of subbands from 10 (TI's canon) to 21 for the GSM850 band, 30 for the EGSM band, 63 for DCS and 50 for PCS, so that each subband is only 6 channels (1.2 MHz).* The ARFCN boundaries for the subbands are not stored in the calibration records in the flash, but are fixed in the firmware instead.Changing our FreeCalypso firmware to allow up to 63 Rx AGC subbands to matchCompal's fw architecture would be too disruptive, hence our current c1xx-calextrimplementation translates Compal's Rx channel correction values to TI/FC formatby combining groups of Compal's subbands into larger subbands, and making amean value out of the smaller subband correction values in Compal's factorycalibration record.The AGC subbands defined by Compal's fw are listed below, with each numericline giving the ARFCN range of each subband; blank lines separate the groupingsmade by c1xx-calextr.850 MHz band:128-134135-140141-146147-152153-158159-164165-170171-176177-182183-188189-194195-200201-206207-212213-218219-224225-230231-236237-242243-248249-251900 MHz band:0-67-1213-1819-2425-3031-3637-4243-4849-5455-6061-6667-7273-7879-8485-9091-9697-102103-108109-114115-120121-124975-975976-981982-987988-993994-9991000-10051006-10111012-10171018-10231800 MHz band:512-518519-524525-530531-536537-542543-548549-554555-560561-566567-572573-578579-584585-590591-596597-602603-608609-614615-620621-626627-632633-638639-644645-650651-656657-662663-668669-674675-680681-686687-692693-698699-704705-710711-716717-722723-728729-734735-740741-746747-752753-758759-764765-770771-776777-782783-788789-794795-800801-806807-812813-818819-824825-830831-836837-842843-848849-854855-860861-866867-872873-878879-884885-8851900 MHz band:512-518519-524525-530531-536537-542543-548549-554555-560561-566567-572573-578579-584585-590591-596597-602603-608609-614615-620621-626627-632633-638639-644645-650651-656657-662663-668669-674675-680681-686687-692693-698699-704705-710711-716717-722723-728729-734735-740741-746749-752753-758759-764765-770771-776777-782783-788789-794795-800801-806807-810Tx channel correction values============================A similar situation holds here: in TI's canon each band is divided into 8subbands for the purpose of Tx channel-dependent corrections, but Mot/Compalseem to be using smaller subbands: 13 for the GSM850 band, 18 for EGSM, 38 forDCS and 30 for PCS. We can see where these correction values are stored in thecalibration records in the flash (immediately after the Tx levels array), butthe ARFCN boundaries of Mot/Compal's Tx channel calibration subbands are notknown, and the semantics of the correction values themselves are not clear:Mot/Compal's Tx channel correction values are centered around 0, whereas inTI's canonical version they are centered around 128.Because we are not able to grok Mot/Compal's Tx channel correction, we currentlyignore this part of their factory calibration, i.e., FreeCalypso fw will runwith all channel correction values set to 128, meaning no channel correction.But since we do use the Tx levels table of APC DAC values from Mot/Compal'sfactory records, and given that the tolerances for Tx power levels given in theGSM 05.05 spec are quite generous, we expect to still be within these tolerancesdespite the lack of channel correction.In vivo approach: tried and failed==================================Before I figured out the format of Mot/Compal's factory calibration records intheir flash and wrote the c1xx-calextr "in vitro" extraction and conversiontool, I tried an "in vivo" approach: reading the calibration values out fromthe running firmware via TI's L1/RF Test Mode commands which are still presentin Mot/Compal's fw. This approach successfully yielded the tables of Tx ramptemplates which are calibrated per design rather than per unit and thus compiledinto the fw and not present in the per-unit factory calibration records (theseextracted Tx ramps tables are now used by FC Magnetite fw when built for theC139 target), but does not help with much of anything else:* One can read the calibrated Tx levels table (rftr 16) for the low frequency band (850 or 900 MHz), but not for the high (1800 or 1900 MHz) band: in order to access the tables for the high band, one needs to issue an rfpw 7 command, but in Mot/Compal's version the latter command only loads the compiled-in tables and does not apply their non-TI calibration records.* The Rx agcparams table returned in response to rftr 31 always has the GMagic field set to the fw's compiled-in value and not the calibrated one.* The Rx calchan table (which Mot/Compal enlarged from 10 to 63 entries as explained earlier in this article) cannot be read out at all: the rftr 25 command crashes the firmware, probably via a buffer overflow from the enlarged table.* The Tx calchan table can be read out with rftr 17, but it does not make any sense: it still has 4 copies of a table of 8 subbands like in TI's canon, even though when we look at their factory calibration records, we can clearly see that the table of Tx channel correction values is also enlarged. But the correction values themselves are centered around 0 in this strange table returned in response to rftr 17, and not around 128 like in TI's canon.The fc-readcal utility was written before c1xx-calextr, and it was my originalidea of how to extract Mot/Compal's factory RF calibration values. It featuresa -c command line option for "Compal mode" which disables the reading of Rxcalchan and Tx calchan tables via rftr 25 and rftr 17, respectively (the formercrashes the fw, the latter has the wrong semantics), but because it issuesrfpw 7 commands for each band preceded by tms 1, it will only yield thefirmware's compiled-in values, and not any of the factory-calibrated ones.Therefore, the fc-readcal method should not be used, and the c1xx-calextr methoddescribed in the main body of this article should be used instead.