Reconstruction of the history of Openmoko GSM modem firmware By Mychaela Falconia, Mother of FreeCalypso ******************************************* All mokoN modem firmwares are based on TCS211 reference fw from TI. TCS211 (short for TCS2.1.1) was TI's program name for the Calypso chipset and its associated official fw for GSM and GPRS, as distinct from other chipsets (and their associated fw) that came both before and after our beloved Calypso. All mokoN firmwares from moko3 onward are based on a TI TCS211 delivery dated 20070608 that must have been prepared specifically for OM (see below for my reasoning as to why it must be so), but the prior history is murky. It looks like there were two code deliveries which TI had prepared specifically for OM: the first was dated 20070419 and was used in moko1 and moko2, and the second (and last) was dated 20070608 and was used from moko3 onward. However, it appears that even earlier versions had once existed, as shown further below. All mokoN fw binaries that are archived in all_old_versions.tar.gz and moko10-11-bin.tar.bz2 are based on one of these two TI fw deliveries, either 20070419 or 20070608. We can tell so from the G23M component version strings that are embedded in the fw binary; these strings are emitted by the running fw when queried with AT%VER (one of the many TI-specific AT commands), but can also be seen more easily by running strings(1) on the fw binary and grepping for the right substrings or patterns. A good pattern to grep for can be as follows: [0-9][0-9]:[0-9][0-9]:[0-9][0-9] [0-9][0-9]/[0-9][0-9]/[0-9][0-9] The output of strings on moko11.bin (the official calypso-moko11.m0 image converted to straight binary with mokosrec2bin) passed through the above grep pattern is as follows: t30 a086 ** NONE ** 14:16:02 08/06/07 ss a086 ** NONE ** 14:15:52 08/06/07 sms a086 ** NONE ** 14:14:20 08/06/07 sim a086 ** NONE ** 14:13:54 08/06/07 rr a038 ** NONE ** 09:35:21 01/04/08 rlp a086 ** NONE ** 14:12:40 08/06/07 ra a086 ** NONE ** 14:12:32 08/06/07 mm a086 ** NONE ** 14:11:52 08/06/07 l2r a086 ** NONE ** 14:10:36 08/06/07 fad a086 ** NONE ** 14:08:06 08/06/07 dl a086 ** NONE ** 14:07:46 08/06/07 cc a086 ** NONE ** 14:07:10 08/06/07 alr a086 ** NONE ** 14:06:38 08/06/07 aci moko non_clearcase 08:30:23 24/02/09 These strings are produced by the make_ver.pl script in the firmware build system; you can find this script under g23m/system/busyb/tools in any of our TCS211 fw source trees. An explanation of the various fields is in order: * The first field is the name of the G23M protocol stack component to which the version string applies. Only G23M PS components get such version strings, and only the GSM ones (not GPRS) - not any other firmware components. * The second field is supposed to indicate who ran the build that resulted in the component in question being compiled from C into binary object form. The script takes the $USERNAME environment variable and truncates it to 4 characters max. All components that have been compiled from C to binary at TI rather than OM have either a086 or a038 in this field, but keep in mind that it is not the full username of whoever at TI did the build, but just the first 4 characters of it. Truncated username a038 appears only once in the case of the 20080401 update to rr.lib (see below), whereas a086 appears everywhere else, for both 20070419 and 20070608 deliveries as well as some mysterious earlier dates shown below - but because of the truncation we cannot conclude that it must have been the same TI person who built all of those different versions. The only component which was compiled from C to binary at OM rather than TI was aci; the corresponding username is sean (Sean Chiang, not Sean Moss-Pultz) for moko1 through moko9-beta1 and moko for the moko10/11 family. * The next part (always one of "non_clearcase" or "** NONE **" for all component version strings in all OM firmwares) is set as follows by TI's make_ver.pl script: - If the machine on which the build is run does not have ClearCase software installed, the string is set to "non_clearcase". - If the machine on which the build is run does have ClearCase sw installed, the string is set to the output of `cleartool pwv -s`. This output will be the familiar "** NONE **" string when the directory in which the build takes place is not a ClearCase view. Thus we can see that whoever at TI produced the 20070419 and 20070608 full deliveries and the 20080401 rr.lib update had ClearCase installed, but OM naturally did not have that exotic proprietary sw on their build system. * The generated timestamp is the time when the make_ver.pl script ran as part of the fw build; it is not derived from any source file modification timestamps. Back to the history of OM's firmwares, when we look at the component version strings in all fw versions from moko3 onward, we always see the same 20070608 dates for all components except aci and rr. aci was compiled from source at OM and its associated component version string gives us the date for OM's fw build, whereas rr can be either the original 20070608 version (matching the others) or the later 20080401 update as detailed further below. Doing the same strings and grep procedure on the moko1 and moko2 binaries from OM's all_old_versions.tar.gz archive, we can see that they were built with the earlier 20070419 binary libraries; here is moko1: t30 a086 ** NONE ** 14:54:20 19/04/07 ss a086 ** NONE ** 14:54:12 19/04/07 sms a086 ** NONE ** 14:52:56 19/04/07 sim a086 ** NONE ** 14:52:29 19/04/07 rr a086 ** NONE ** 14:51:31 19/04/07 rlp a086 ** NONE ** 14:51:09 19/04/07 ra a086 ** NONE ** 14:50:56 19/04/07 mm a086 ** NONE ** 14:50:24 19/04/07 l2r a086 ** NONE ** 14:49:16 19/04/07 fad a086 ** NONE ** 14:46:42 19/04/07 dl a086 ** NONE ** 14:46:17 19/04/07 cc a086 ** NONE ** 14:45:32 19/04/07 alr a086 ** NONE ** 14:45:00 19/04/07 aci sean non_clearcase 14:25:18 25/06/07 However, it also appears that there once existed even earlier versions than this 20070419. The evidence is contained in this OM Wiki page: http://wiki.openmoko.org/wiki/Neo_1973_and_Neo_FreeRunner_gsm_modem which gives the following %VER output "on an un-updated phase-0 GTA01Bv4 phone": %VER: aci sean non_clearcase 15:11:24 17/04/07 %VER: cc a086 ** NONE ** 10:16:39 28/03/07 %VER: dl a086 ** NONE ** 10:17:34 28/03/07 %VER: mm a086 ** NONE ** 10:23:09 28/03/07 %VER: rr a086 ** NONE ** 10:25:37 28/03/07 %VER: sim a086 ** NONE ** 10:26:45 28/03/07 %VER: sms a086 ** NONE ** 10:27:21 28/03/07 %VER: ss a086 ** NONE ** 10:28:52 28/03/07 %VER: alr a086 ** NONE ** 10:15:48 28/03/07 %VER: l2r a086 ** NONE ** 10:21:37 28/03/07 %VER: ra a086 ** NONE ** 10:25:01 28/03/07 %VER: rlp a086 ** NONE ** 10:25:12 28/03/07 %VER: fad a086 ** NONE ** 10:18:09 28/03/07 %VER: t30 a086 ** NONE ** 10:29:01 28/03/07 and the following "For a GTA01Bv3 phase 0 phone" with even earlier dates: %VER: aci sean non_clearcase 10:46:06 29/08/06 %VER: cc a086 non_clearcase 14:53:59 28/08/06 %VER: dl a086 non_clearcase 14:54:40 28/08/06 %VER: mm a086 non_clearcase 14:59:25 28/08/06 %VER: rr a086 non_clearcase 15:00:11 28/08/06 %VER: sim a086 non_clearcase 15:00:57 28/08/06 %VER: sms a086 non_clearcase 15:01:23 28/08/06 %VER: ss a086 non_clearcase 15:02:22 28/08/06 %VER: alr a086 non_clearcase 14:53:19 28/08/06 %VER: l2r a086 non_clearcase 14:58:02 28/08/06 %VER: ra a086 non_clearcase 14:59:50 28/08/06 %VER: rlp a086 non_clearcase 14:59:56 28/08/06 %VER: fad a086 non_clearcase 14:55:04 28/08/06 %VER: t30 a086 non_clearcase 15:02:27 28/08/06 (The third example given in that Wiki page has dates corresponding to the moko4 image in the all_old_versions.tar.gz archive.) It stands to reason that prior to OM getting an official fw delivery from TI custom-tailored specifically for them, they were most likely playing with whatever their parent company FIC already had. The a086 username in the G23M PS component version strings indicates that even these (presumed) pre-OM FIC fw versions had been compiled at TI rather than FIC, but it does not necessarily mean that the source for these components was withheld: it is possible that FIC did have the source for those components, but never modified them and never disturbed the build tree they got from TI, hence the components never got recompiled and the version strings never got regenerated. And it is equally possible that the early FIC was in the same sans-source situation with respect to these fw components as the later OM - we'll probably never know. It is my (Mychaela's) understanding (which can very easily be wrong) that prior to the OM project, FIC made traditional "dumb" phones in which the Calypso was the main processor, rather than smartphones like GTA0x in which it is reduced to a slave modem. (All OM stories I could find seem to indicate that the GTA01 was FIC's first experience with smartphone design, the first experience in which FIC made hw design choices which OM software folks were apparently disappointed by - there is never any mention of any FIC smartphones prior to OM, hence my reasoning that there probably weren't any.) It thus stands to reason that whatever Calypso TCS211 family fw FIC must have had prior to OM ought to have been built in the "dumbphone" configuration with Calypso-driven UI, rather than the AT-command-controlled modem config. TI's TCS211 firmware build system (called BuSyB) works very nicely *if* you are one of the lucky few people who were privileged enough to get a complete source delivery with nothing withheld, such that you can blow away the entire g23m/__out__ directory tree and do a full rebuild from source. In this case you can build the fw for any supported target in any supported config from the same source tree, all build products for that config will be neatly emitted under g23m/__out__/gsm_ where the part encodes the selected configuration, and the compiled objects built with different options for different configs will never erroneously mix. It's a very neat system *if* you are privileged to have the full source. But this once-neat system turns into a royal pita when you are working with a crippled fw delivery in which the source for some essential fw components has been withheld and only binary objects are given instead. It appears that these semi-src releases were produced via a 3-step process: 1. A full firmware build was done, starting from full source, with the configuration set to what is to be delivered to the customer in question. 2. After that initial full build was done and the g23m/__out__ directory was fully populated with build products, TI tweaked something in the XML voodoo under g23m/system/busyb from which the pdt_*.mak makefiles are generated. The pdt_*.mak appropriate for the specific customer's configuration (pdt_2091 in the case of Openmoko) was regenerated in the "delivered libs" configuration. Unlike the original makefile which must have built these libs from source in the first place, the new makefile treats some of the libs under g23m/__out__ as "sources": it expects them to be there like other starting "source" files, and contains no instructions for rebuilding them. 3. The semi-src package was shipped to the customer, with the complete g23m/__out__ build-output tree included, but with the withheld source pieces deleted. The problem should be obvious: not only is the ability to study and modify some fw components taken away, but the ability to change the build configuration is also lost: the binary libs which must be used for the lack of the corresponding source have been built for one specific config, and reside in a g23m/__out__/gsm_ directory whose name reflects that config. That being said, wherever there is a will, there is a way. As I was getting comfortable hacking my way around TI's TCS211 build system in 2015, I was able to take the pdt_2091 configuration that has been salvaged from the ruins of Openmoko (the AT-command-controlled modem config) and transform it into its phone-UI-enabled sibling called pdt_2092. What made this surgery possible is that as long as the GSM+GPRS services config remains unchanged (the binary-only libs impose the fixed config of GPRS enabled, FAX_AND_DATA enabled, tty disabled) and only the MMI config changes (ACI vs. MFW), the same binary libs still work, and the surgery involves BuSyB-generated makefiles and the g23m/__out__ tree. Given that I was able to perform this firmware functional role reassignment surgery in the ACI->MFW direction, perhaps whoever did the early Calypso fw work at FIC/OM (Sean Chiang?) was able to do the opposite MFW->ACI role reassignment surgery in a similar hacky manner? In any case, whatever fw versions and TI deliveries may have existed at FIC/OM in the past, only one version has survived into the present to the best of our knowledge: TI's 20070608 TCS211 fw delivery (the last complete fw delivery given to OM) with OM's changes that bring it up to moko11 revision level. This ware is preserved in the moko10-11-src.tar.xz archive, which contains the 20070419_N7.19_obj_rel_OM tree salvaged from the ruins of OM and a FreeCalypso- added README file that explains its quirks which would cause a lot of confusion otherwise. Note that despite having "20070419" in the directory name, the code contained therein is 20070608-based, with only two tidbits remaining from the previous 20070419 version: Pre-20070608 remnant 1: the comment in g23m/pdt_2091.mak (the makefile for building the fw) reads: ## generated 4/23/07 5:42 PM It is my (Mychaela's) guess that after getting the 20070419 delivery from TI, OM's Sean Chiang had edited g23m/system/busyb/sourcesets/aci.xml to add their ati_fic.c source module and regenerated pdt_2091.mak from BuSyB, producing the date we see in that header comment. When they later imported TI's 20070608 update, the entire g23m/system/busyb tree got overwritten with the new version, blowing away the addition of ati_fic.c to the aci.xml sourceset, but they never regenerated their pdt_2091.mak again - had they tried to do so, they would have had to apply the necessary fixes to the XML voodoo under g23m/system/busyb first, which they apparently never did. Pre-20070608 remnant 2: although the file timestamps have been reset to some 20081030 date (probably by OM's internal SVN), the timestamps in the COFF binary headers of all g23m/__out__/gsm_/obj/r2d_drv/*.obj R2D objects are from 20070419. We may never know what other differences may exist between TI's first 20070419 code version (which, to the best of our knowledge, has survived only as code bits linked into moko1 and moko2 fw images with no symbolic info whatsoever) and the better-preserved 20070608 version, but we know one difference: the R2D RiViera SWE (SoftWare Entity) was enabled in the 20070419 delivery, and disabled in 20070608. The R2D SWE is the driver for TI's 176x220 pixel color LCD which appears on their D-Sample phone development kit (connected to Calypso memory bus on chip select nCS3), and is clearly not appropriate for a modem like OM's where said hardware does not exist. At this point it needs to be noted that all of OM's firmwares including their last moko11 contain such code for non-existent hw (battery charging, keypad, piezoelectric buzzer), but R2D must have drawn OM's attention because of the bulk it adds to the fw image: about 256 KiB. (You can check for yourself and see that moko1 and moko2 are bigger than all later versions by about this much.) Because OM never regenerated their pdt_2091.mak makefile after having integrated TI's 20070608 update with R2D disabled, the following oddities result: * The code for R2D still gets compiled from C into object form as directed by the makefile, although the 20070419 COFF header timestamps on the R2D objects indicate that the corresponding C sources (which have survived) had never been touched after that date. * This R2D object code packaged into r2d_drv_*.lib gets included in the list of libraries presented to the linker. * The only reason why this R2D code does not end up in the final fw image is because the files under g23m/__out__/gsm_/config are the updated 20070608 versions, including the rv_swe.h header in which R2D is disabled. There is a source module called rvm_swe.c which includes rv_swe.h; enabling or disabling various RiViera SWEs in that header causes rvm_swe.c (which OM got in source form and which is set up to be recompiled from source in pdt_2091.mak) to pull in or not pull in those SWEs. * If someone were to regenerate the config headers from the un-updated pdt_2091.mak, the R2D code would get pulled into the link and included in the fw image once again. Besides the rvm_swe.c module which controls whether or not a given RiViera SWE gets pulled into the link, there is one more module whose code is affected by changes in the rv_swe.h config header: create_RVtasks.c (the code that starts these SWEs on boot) that ends up in main.lib. But main.lib is provided as a non-recompilable binary-only library in TI's 20070608 version, and the corresponding source appears to have been withheld from OM, or at least is missing in the source tree we have salvaged: there are no *.c files under chipsetsw/system/Main, only *.h and some assembly code. Thus TI gave OM a non-modifiable main.lib blob that has R2D disabled, and thus cannot be used to build firmware for "dumbphone" products in which it needs to be enabled. But then their previous 20070419 version probably had it enabled in the create_RVtasks.c code in main.lib, and if my hypothesis about FIC having built dumbphones prior to OM is correct, then they must have had R2D enabled too. At this point it should be noted that all of the wrongfully-withheld C source for the code that ends up in main.lib has now been fully and successfully reconstructed in the FreeCalypso project, by taking the corresponding *.c modules from the LoCosto source and massaging them until they compile into a perfect match to the binary objects from 20070608, hence this code is no longer binary-only and has become freely editable and reconfigurable. Analysis of OM's changes from TI's TCS211-20070608 baseline to mokoN ==================================================================== It stands to reason that when FIC or OM received their 20070419 and 20070608 fw deliveries from TI, those deliveries were most likely neatly packaged, probably as ZIPs. However, no surviving copy of any such intact original TI delivery has been found, instead all we have is what has survived from these two deliveries (mostly the 20070608 one) into the late-OM-era source tree that is presented in moko10-11-src.tar.xz. Because of OM's own changes to the code, we don't have a complete and pristine set of bits from 20070608: while most of the code is unchanged from 20070608 as will be shown shortly, wherever OM had changed some code, we no longer have what was there originally. How can we tell which bits were changed by OM and which are unchanged from TI's TCS211-20070608 baseline? The answer lies in the discrete object files under g23m/__out__/gsm_/obj and the timestamps embedded in their COFF headers. It is absolutely crucial that what we have is not merely a source tree, but a source + build products tree, with all of the intermediate steps between the starting source and the final fw image preserved. For every single module that comprises the firmware, whether it's a source-enabled module or a source- withheld binary-only one, whether it's been modified by OM or not, there is a discrete COFF object under that obj tree, and inside each of these objects there is a timestamp indicating when it was compiled. Looking at these COFF header timestamps on the individual objects, we can see that most of them have 20070608 dates, i.e., the corresponding C sources have not been touched at all at OM - otherwise the objects would have been recompiled with new timestamps. The only objects under the obj tree (as opposed to wholesale binary lib updates in the lib directory, see the separate section dealing with those) that have post-20070608 timestamps, indicating OM's local modifications, are the objects resulting from the following source locations: chipsetsw/drivers/drv_app/ffs/* chipsetsw/drivers/drv_app/uart/uartfax.c chipsetsw/services/Audio/audio_api.c g23m/condat/ms/src/aci/* The source modules listed above are the ONLY ones that have changed between TCS211-20070608 and moko10-beta1; all other objects have 20070608 timestamps. The following 5 C modules have additionally changed between moko10-beta1 and moko11, i.e., beyond the COFF timestamps seen in the obj tree: chipsetsw/drivers/drv_app/sim/sim.c chipsetsw/drivers/drv_app/uart/uartfax.c chipsetsw/drivers/drv_core/armio/armio.c g23m/condat/ms/src/aci/ati_bas.c g23m/condat/ms/src/aci/cmh_sims.c A more detailed look at these 20070608->mokoN changes: ACI modules: Most COFF objects here have 20070726 timestamps, corresponding to the build date of moko3 seen in the image in the all_old_versions.tar.gz archive, so that is the date when a full rebuild of the whole ACI was done - any detailed changes prior to that date are masked. The modules changed after this date are: ati_bas.c Contains hard-coded version ID strings returned in response to AT+CGxx queries, and has been edited for every fw revision as a result. The original AT+CGxx handling code from TCS211-20070608 has been lost; the only example we have of TI's original AT+CGxx handling code is the LoCosto version. ati_cmd.c OM's custom AT commands (AT@ family) were getting added here. ati_fic.c The implementation of those custom AT commands lives here. ati_src_uart.c The rfcap bogosity added in moko6 lives here. cmh_sims.c Moko10/11 change relative to previous fw versions, first appeared in moko10-beta2: the check suppressing the GSM APDU class for the AT+CSIM command has been removed, allowing GSM APDUs with this AT command. cmh_smsr.c SMS memory full indication added; the COFF object date is 20070929, thus agreeing with OM's all_version__CHANGELOG.txt which says that this feature was added in moko5. FFS code in chipsetsw/drivers/drv_app/ffs: There was a full rebuild of FFS code on 20071210, corresponding to moko6; the details of changes from TI's baseline (if any) are not known. There was a further recompile of dev.obj on 20081026, possibly corresponding to the dev.c vs. dev.c.org diff under chipsetsw/drivers/drv_app/ffs/board, but the 3 new device type additions in the actively-compiled version of dev.c are bogus, so the truth is very unclear here. It should also be mentioned that the BuSyB XML file that indicates which parts of the fw are to be rebuilt from source and which must be taken as binary libs (g23m/system/busyb/deliverydefs/dlvcfg0.xml) as it appears in the source tree salvaged from the ruins of OM has FFS_LIB_DLV and FFS_CUST_LIB_DLV set to 1, putting FFS in the class of binary-only libs. Did this dlvcfg0.xml come with the 20070608 delivery? Does it mean that this delivery had the FFS source omitted and OM were using the FFS source they got earlier with the 20070419 version? We may never know. uartfax.c: This module is the driver for the MODEM UART on which the AT command channel is presented; on OM GTA0x devices this UART is connected to the application processor, and OM had made some changes to improve the robustness of this interface, including an interrupt to the AP when the modem has something to send but the AP has CTS flow control stopped. The moko10-beta1 COFF object has a 20070806 timestamp, corresponding to somewhere between moko3 and moko4 - this must have been the original round of changes. Then there was a change in moko11 (i.e., a change from moko10 to moko11) to enable hardware flow control. These OM changes have been deemed by FreeCalypso to be good for all potential external hosts communicating with a Calypso modem, not just the GTA02 AP, and have therefore been retained in our own FreeCalypso fw on our own FreeCalypso hw products. armio.c: The GPIO line which OM assigned for signaling interrupts from the Calypso to the AP (Calypso GPIO 1) is the same Calypso GPIO which TI used to enable or disable the loudspeaker amplifier on their D-Sample and Leonardo boards. TI's original code in armio.c set this GPIO output to high on boot to enable the loudspeaker amplifier, and OM never noticed until their final days, changing it between moko10 and moko11. audio_api.c & sim.c: TI had made post-20070608 updates to the code in these two modules, but instead of giving OM a source patch, they gave them binary libs to plop in the place of built-from-source ones. OM's hired contractor Dieter Spaar (hired by OM to produce their moko10 and moko11 firmwares) had reconstructed the source changes, producing modified sources that compile into a perfect match to TI's binary object updates, apparently using a very similar process to my own much more extensive source-from- objects reconstruction later in FreeCalypso. A closer look at the TCS211-20070608 firmware itself ==================================================== Having examined the changes which OM made to the TCS211 fw they got from TI, let us take a closer look at the original 20070608 firmware itself, or at least those parts of it that have survived through OM, which is thankfully most of the code base as shown above by way of COFF timestamps. Let us begin by looking at which parts of the 20070608 fw delivery came as full source and which parts came as source-withheld binary-only objects. We are not going to look at the Nucleus RTOS, the compiler's RTS library (libc/libgcc equivalent for TI's proprietary TMS470 compiler) and GPF, which were distributed and used as precompiled binary libs even inside TI: these parts are outside of the scope of this article. It will suffice to say that these parts have already been successfully taken care of in our FreeCalypso Citrine firmware, and will be similarly taken care of in our future Selenite fw. Aside from the just-mentioned pieces which are outside of TCS211 proper, the following parts of the 20070608 fw delivery came as binary-only libs with the corresponding source wrongfully withheld (we have good reason to assume that the same situation held for the previous 20070419 delivery, as OM's pdt_2091.mak was built from the latter): * L1 (GSM Layer 1) * G23M protocol stack (layers 2&3) * CCD and ccddata.lib supporting the G23M PS * main.lib (initialization code) * bootloader.lib The code in bootloader.lib is especially peculiar, as it only duplicates the functionality provided by the Calypso boot ROM, i.e., in the Calypso silicon itself. When the Calypso ARM7 processor boots (in sensible hw designs like OM GTA0x that have the nIBOOT pin pulled down), it does not immediately start executing instructions from the fw image in flash (which could be bricked), but instead executes the on-die boot ROM code first. This boot ROM code eventually transfers control to the fw image in flash, but it does not do so immediately - instead it provides a time window during which the boot process can be interrupted and diverted by sending the right characters into either of the two UARTs. Thus there is a hardware unbricking mechanism built into the Calypso chip itself. However, all of TI's firmwares from the Calypso era contain a piece of code that duplicates this boot ROM functionality: it gets placed by the linker script in the first flash sector, separately from the rest of the fw, and it executes first, before the rest of the fw. This code similarly provides some time window for an external host running TI's FLUID (Flash Loader Utility Independent of Device) to interrupt and divert the boot process through some serial magic, and if no such serial magic is seen in the allotted time, the rest of the fw boots normally. We never got any source for this fw component, OM's TCS211-based fw has it in binary-only form (bootloader.lib), the serial protocol spoken by this bootloader is not documented anywhere and is unknown to us, and the far-end implementation of this protocol (fluid.exe) is also a binary sans source - albeit for Linux/ARM (for running on the GTA01/02 AP) rather than Windows! The solution taken in FreeCalypso is to simply remove this code altogether: because it does nothing but a poor job of duplicating the functionality provided much better (truly unbrickable) by the hardware itself, it can be removed without any loss. In the case of L1 and main.lib the problem of wrongfully withheld source has been solved through painstaking reconstruction, using disassembly of the binary objects from TCS211-20070608 and approximately-corresponding sources from the LoCosto version in which these components came as full source. This reconstruction has been made possible because even though the *.c files were deleted, all of the original L1 *.h files have been preserved, as well as the BSP *.h files used by the init code. The reconstruction proceeded by taking each individual *.c module from the LoCosto source, plopping it into the TCS211 environment with all TCS211 *.h files and compilation options unchanged, and massaging it until it passes compilation in this environment. Then the resulting compiled object was disassembled and diffed against the disassembly of the original TCS211 object, using a custom-developed disassembler that is very specifically tuned to the quirks and idiosyncrasies of TI's TMS470 compiler. The source under reconstruction was then further massaged until it compiles into an object that perfectly matches the original blob. The final match is bit for bit for most modules, and for others the instruction bits are different, but the logic has been carefully verified to match exactly nonetheless. In the case of the G23M protocol stack and CCD, our FreeCalypso solution is to combine the fully reconstructed chipsetsw portion of TCS211 fw (including our source-reconstructed L1) with the TCS3/LoCosto versions of the G23M PS, ACI and CCD, which are newer than OM's 20070608 (the LoCosto source dates from 20090327) and exist in the form of full C source. The result is a TCS2/TCS3 hybrid, and "hybrid" is the name of the corresponding configuration in our FC Magnetite fw. Back to our analysis of the TCS211-20070608 fw delivery that has survived into the present through the ruins of Openmoko, there are two additional peculiarities when it comes to the source vs. binary-only libs distinction. There are two components for which the corresponding source *is* present, but is not compiled, with the build system requiring the binary lib to be present as a "source" component; these two components are ALR and comlib. ALR is the bottom-most layer of the G23M PS that ties this PS to TI's L1; it thus sits between L1 and the rest of G23M. Yet while both the L1 source and the source for all other G23M components has been removed (the latter would have been under g23m/condat/ms/src), the source for ALR is right there under g23m/condat/ms/src/alr. In the course of the FC Magnetite project I performed an experiment in which I hacked the BuSyB XML voodoo to recompile ALR from this source under discussion, and confirmed that this source perfectly matches the familiar 20070608 binary alr.lib, compiling into the same code bit for bit except for the COFF header timestamp. The same match situation holds for comlib, which is a small library of a few odds and ends, the most interesting of which is the function that retrieves the phone's or modem's IMEISV from wherever it is stored in a given configuration. But what makes the presence of ALR in OM's source particularly interesting is that it is not merely unreferenced files: even though the build system does not use it and takes the blob version of alr.lib instead, OM's own code in ati_fic.c that implements their AT@BAND command (returns the GSM frequency band of the current serving cell) includes some *private* ALR header files and digs in ALR's private data structures to retrieve this current band info, a feat that absolutely requires having the source for ALR or at least its private internal header files! It almost looks as if the ALR source is present in OM's tree for no purpose other than to facilitate this dirty trick... So how come they got the source for ALR while having no source for any other radio protocol stack components, neither L1 below nor G23M above? Did they ask TI how to retrieve the current band and TI responded by sending them the ALR source with instructions on how to abuse its private internal headers and data structures? Did they get this source from their parent company FIC who got a more complete TI source delivery for their dumbphone projects - in which case how come it is a perfect match to the 20070608 object version? Or did TI accidentally overlook alr when they were sanitizing their source to delete the parts which they didn't want OM to have, and later OM folks stumbled across that source and figured out how to extract the current serving cell's frequency band from it? We'll probably never know. And finally this gem: the 3 directories named atb. bmi and mfw under g23m/condat/ms/src contain nothing less than TI's reference/demo/prototype UI code for dumbphones! Yup, that's right, dumbphone UI code in an Openmoko modem firmware source. At this point it will help to remember that TI's original TCS211 firmware prior to customization for OM (see below) targeted TI's D-Sample and Leonardo development boards, and the D-Sample kit consists of the main board plus a handset part with a big color LCD and a full keypad: https://www.freecalypso.org/members/falcon/pictures/D-Sample/ TI's reference/demo/prototype dumbphone UI code targets the 176x220 pixel color LCD on this D-Sample platform. We actually have one of these D-Sample kits (the one pictured) here at FreeCalypso HQ, but the sanitization of the TCS211 fw for OM has made it impossible to build a fw image targeting the D-Sample: the D-Sample board has TI's older Clara RF, and when TI moved on to Rita RF for their production chipset offering (used in OM GTA0x and many others), instead of making a new board with Rita RF to replace the D-Sample, they kept the support for Clara RF in the mainline firmware - but with L1 delivered only as binary objects in OM's version, this Clara RF support is lost. We have reconstructed recompilable C source for the Rita-targeting L1 we got, but it does not help with Clara RF for which we have no code at all, neither source nor linkable objects. Unable to build a fw image with the dumbphone UI enabled targeting the proper D-Sample hw, we were still able to get a glimpse of what this UI looks like by running the fw with the dumbphone UI code enabled on the GTA02 modem (yes, you read it right) and hacking the R2D code to send its LCD raster blits out through the RVTMUX serial channel, which is the headset jack on the GTA02. We also did a similar feat on the Pirelli DP-L10 hardware, using that phone's keypad to drive this virtual D-Sample UI. We got a glimpse of the UI this way, but the idea could not be taken further because the 176x220 pix, 16 bits per pixel blits were just too much for the poor serial interface to handle, even at the maximum 812500 baud rate. We were also able to resurrect an even older TI dumbphone UI configuration targeting an 84x48 pixel B&W LCD (C-Sample) and run it on Mot C139 hardware, but this code is too bitrotten - I want the D-Sample UI, not C-Sample. Further work in this direction will have to wait until we can build our own board that combines a D-Sample-like 176x220 pix color LCD with Rita RF for which we have the needed L1 driver code. TI's TCS211-20070608 delivery specific to OM's triband hardware =============================================================== The mechanism in TI's firmware build system that controls which pieces are to be compiled from source and which are to be used in the form of prebuilt binary libs operates on the granularity of groups of libs. In the case of L1, there are 3 divisions whose binary delivery status can be independently controlled: the L1_LIB_DLV symbol controls the main L1 code in l1_ext.lib & l1_int.lib, L1_CUST_LIB_DLV controls the delivery status of l1_custom_ext.lib and l1_custom_int.lib (the code in l1_rf12.[ch] where various compiled-in RF parameters can be tuned goes into l1_custom_int.lib), and L1_TPU_LIB_DLV controls the delivery status of tpudrv.lib, the library compiled from tpudrv12.[ch] - the code that needs to be edited for different TSPACT signal wiring in the RF section of the phone or modem board. But the insane aspect of TI's stripped semi-src delivery for OM is that they locked down not only the main part of L1 (L1_LIB_DLV), but the L1_CUST_LIB_DLV and L1_TPU_LIB_DLV parts as well! Thus instead of giving OM a firmware delivery that targets TI's own Leonardo board and letting OM change and recompile the TPU driver code themselves, TI must have built a special fw version for OM that is tweaked for OM's triband RFFE TSPACT signal wiring, and did it for both 20070419 and 20070608 fw deliveries. OM's hardware is unusually close (compared to other Calypso hw out there) to TI's Leonardo, such that a fw image built for the Leonardo target from TI's internal TCS211 mainline would *almost* work on the GTA0x modem. The only area where the fw absolutely must be tweaked when going from Leonardo to GTA0x is FIC's hobbling of the RF front end from quadband down to triband. There is a difference between TI's quadband RFFE and FIC's triband one in terms of how the TSPACT signals need to be driven: Signal Function on Leonardo Function on GTA0x modem hw -------------------------------------------------------------------------- TSPACT02 (FEM_7) Tx low band Tx high band TSPACT01 (FEM_8) Tx high band Rx PCS TSPACT04 (FEM_9) Rx GSM-850 Tx low band If you look at the TSPACT definitions in the chipsetsw/layer1/tpu_drivers/source0/tpudrv12.h header file in OM's source (remember that only *.c were deleted, but all of *.h survived), lines 291 through 311, you can clearly see that the code originally targeted TI's quadband RFFE on Leonardo and E-Sample boards, but was patched for OM's triband RFFE. Particularly notice the messed-up horizontal position of the closing parenthesis on lines 298 and 308, defining RU_850 and RU_1900, respectively. TI must have made these changes to tpudrv12.h prior to compiling the withheld tpudrv12.c source into the delivered tpudrv.lib, as the latter contains the bits for OM's triband RFFE with the same 20070608 COFF header timestamps as the rest of this fw delivery. It should also be noted that because OM likewise never got the source for l1_rf12.c, the compiled-in RF parameters set in this C module and also in the preserved l1_rf12.h header, which end up as bits in l1_cust.obj inside l1_custom_int.lib, have most likely never been tuned specifically for OM's hardware, and are probably TI's old Leonardo settings instead. (TI's Leonardo reference design is a few years older than OM's hardware.) Most of these compiled-in RF parameters get overridden by the per-unit RF tables that get individually calibrated on the device production line and written into FFS, thus it makes no practical difference that many of these compiled-in values are wildly off the mark for the actual hw this fw runs on - but it does mean that having this RF calibration done is a requirement. We already know from experience that the compiled-in VCXO parameters for the AFC algorithm are totally wrong for the VCXO on the GTA02 (and on our semi-clone thereof on the FCDEV3B), thus if no VCXO calibration is performed, the modem fails to acquire the frequency burst and thus fails to lock onto a cell. The only compiled-in RF parameters which survive intact through the production line calibration process are the Tx power ramp-up and ramp-down tables. These tables do get written into FFS when the production calibration station issues the me 104 command to the L1TM code in the fw at the end of each Tx band calibration, but their content is never modified from what is compiled into the fw image that ran on the board during the calibration procedure. Per TI's intent, these ramp tables are supposed to be calibrated in the lab during product development and compiled into the fw, rather than calibrated per unit on the device production line, hence the latter processes don't change them. We don't know if these ramp-up and ramp-down tables we've got as part of the 20070608 binary object version of l1_custom_int.lib are properly correct for the RF3166 PA used on GTA02 and FCDEV3B hardware, or if they've been calibrated for the earlier RF3133 PA that was apparently used on later Leonardo boards, or if they go back even earlier to the AWT6108 PA that was apparently used on the original Leonardo. We just hope that they've been calibrated for RF3133 rather than AWT6108, and that RF3133 and RF3166 are sufficiently similar. The fact that OM appear to have passed some kind of certification or type approval testing with this hw+fw combo gives us some additional peace of mind in this regard. OM's rfcap setting bogosity =========================== The G23M protocol stack needs to be told which GSM frequency bands are supported by the hardware it is running on, so it does not attempt to operate in the unsupported bands. TI's canonical way of communicating this information to the G23M PS is by way of a file in the flash file system called /gsm/com/rfcap. This file holds a data structure of 16 bytes and is read by the RR component of the G23M PS, and if the file is not present, a compiled-in default is used. In practice most of the fields in the rfcap file are never changed from the firmware's compiled-in defaults except one: the setting of which frequency bands are supported in the hardware and which aren't. OM's GTA0x hardware is either 900/1800/1900 MHz triband or 850/1800/1900 MHz triband; in FreeCalypso we call these hw band configurations tri900 and tri850, respectively. Given this hardware reality, the correct approach for Openmoko would have been to have their production line program the /gsm/com/rfcap file in the modem's FFS with either the tri900 setting or the tri850 one depending on the physical hw variant, and have the firmware read and obey this file, just like TI's original fw did prior to OM's hack. But this is not what OM did. Instead they inserted a dirty hack into their fw that always overwrites the /gsm/com/rfcap file on every boot (unconditionally) with a setting that says "I am quadband" - even though the actual hw is only triband. As the result, the fw wastes time scanning for cells in the unsupported 4th band that is blocked by the SAW filter in the low band Rx path (the 850 MHz band is blocked on 900 MHz hw units and vice-versa), and if the signal from a particularly strong cell does make it through the wrong-band filter (there are field reports of such events happening), then the modem will operate uncalibrated in that 4th band (the factory production line calibration was only done for the 3 properly supported bands), and may transmit at wrong power levels. OM put their rfcap hack in the ati_src_uart.c module, right before the io_SendMessage() call that prints the "AT-Command Interpreter ready" welcome string on the UART - the rfcap hack has absolutely no logical relation to the ATI source UART code, but I am guessing it was simply a code path which OM's modem fw hackers understood to execute on every boot and which they were able to modify easily. This rfcap hack was introduced in fw version moko6 and persisted through OM's final moko11 release; it has been removed with the first post-OM-company FreeCalypso fw release (leo2moko-r1). OM also made their hack in chipsetsw/drivers/drv_app/ffs/board/ffspcm.c, apparently without realizing that this source module is a dud - it is not included in the build at all - it appears to be a remnant from some TI internal program other than the one that led to the working TCS211 product. OM's AT@AUL brokenness ====================== TI's TCS211 fw includes an audio mode setting facility fully documented in the doc/Audio-mode-config article in the FreeCalypso host tools package. This facility works by way of audio mode configuration files being prepared in the development environment and then programmed into the FFS (flash file system) of individual device units on the production line; the running fw can then activate these prepared audio mode configs with the audio_mode_load() API call. OM had prepared an audio mode config file named /aud/para0.cfg which they programmed into the FFS of every GTA02 modem on the production line, and they added a custom AT command named AT@AUL that invokes the audio_mode_load() API to load this para0 config - the command name AT@AUL was certainly inspired by the aul command in TI's TMSH (Test Mode Shell, recreated in FreeCalypso as fc-tmsh) that performs the same operation. OM's /aud/para0.cfg audio configuration differs from TI's fw default (the settings that are in effect when no audio mode config table has been loaded) in two ways: * The Iota ABB sidetone gain is changed from -5 dB to disabled - OM's setup needs to have this Iota ABB sidetone disabled because they generate the needed sidetone in their Wolfson audio codec chip instead. * The DSP in the Calypso has FIR filters in the voice uplink and downlink paths, by default these filters do nothing (the default coefficients produce an identity transform), but OM's para0 config sets up a non-trivial set of coefficients for the voice downlink FIR filter. However, OM's efforts in this regard were all for naught, as their AT@AUL command does not work (always returns ERROR because the underlying audio_mode_load() API call fails) and can never work, not because there is anything wrong in the implementation of this AT@AUL command itself, but because of OM's incorrect FFS programming on the factory production line. TI's implementation of the audio mode facility in the RiViera Audio Service requires every /aud/*.cfg configuration to be accompanied by an /aud/*.vol volume setting file; OM's factory programmed their /aud/para0.cfg file into the FFS of every unit, but there is no corresponding /aud/para0.vol file. As a result this para0 configuration can never be activated, and AT@AUL is a non-functional command. Instead of figuring out and fixing their error with the lack of /aud/para0.vol, OM seem to have given up on the audio mode loading facility and used a different hacked-up way to disable the Iota ABB sidetone (see below), and apparently gave up on the FIR filter coefficient setting. Post-20070608 updates to TI's binary-only libs ============================================== We can clearly see from the timestamps in COFF object headers and *.lib directory metadata (classic UNIX ar format) that the last complete TI fw delivery used by OM was the one from 20070608. All TI code updates after this date were in the form of TI's support sending OM individual *.lib to plop into the build tree, replacing the original 20070608 versions. One of the first such updates (if not the very first) had to do with Iota ABB sidetone control mentioned above. Having failed with the audio mode setting approach, OM needed some other way to disable the sidetone in the Iota ABB. They could have used the audio_full_access_write() API call with the AUDIO_MICROPHONE_SPEAKER_LOOP_SIDETONE parameter to set the sidetone control directly, bypassing the audio mode table which they couldn't figure out how to make work, and audio_full_access_read() to read it back, but apparently they could not figure out how to use this API either. It appears that OM asked TI support how to control the sidetone, and instead of teaching OM how to use the existing standard APIs, TI support responded by sending them a binary blob version of audio.lib, replacing the previously-full- source Audio Service code, with 3 new API functions added (Side_Tone_Mute(), Side_Tone_Write() and Side_Tone_Read()) that are nothing more than trivial wrappers around the standard audio_full_access_write() and audio_full_access_read() API calls. OM then proceeded to implement their AT@ST sidetone setting and AT@ST? sidetone query commands around these Side_Tone_*() wrapper functions. Much later when OM's hired contractor Dieter Spaar had to put the fw into shape in order to produce moko10 and moko11 versions, he must have been bewildered by why a once-full-source component (Audio Service) was replaced with a binary-only update, and he solved the problem by reconstructing the source for the 3 added functions from disassembly of the blob version, similarly to the source reconstruction approach taken in FreeCalypso for the other larger blob components. The other case of a previously-full-source component replaced with a binary-only update was the SIM driver: source files sim.c and sim32.c under chipsetsw/drivers/drv_app/sim, compiled and packaged into sim_drv.lib. In this case the issue was a legitimate problem with some SIM cards not working, not OM's inability to figure out how to work some aspect of the system, but TI's support still responded by sending OM a binary-only sim_drv.lib update instead of a source patch to sim.c. This update fixed bug #666. However, the makefile was still set up to rebuild sim_drv.lib from the original source, hence the use of the binary-only version of this lib could be maintained only through file modification timestamps, and was very easily blown away. Apparently this update did get blown away at some point and bug #666 came back; the problem was finally solved by Dieter Spaar reverse-engineering TI's binary and patching the sim.c source to match the binary lib update. All other post-20070608 binary lib updates from TI involved components which had been binary-only to begin with, so it was a matter of replacing one binary lib version with another, with the makefile not containing any instructions for rebuilding that lib. The source + build products tree that has been salvaged from the ruins of OM and is presented in moko10-11-src.tar.xz has the updated binary lib versions under g23m/__out__/gsm_/lib (what goes into the link of the fw), while the original 20070608 versions have survived as unreferenced *.obj files under g23m/__out__/gsm_/obj. The first such binary-only lib to be updated was sndcp.lib, a GPRS-specific component of the G23M protocol stack. The update is dated 20070903 (the date in the COFF object headers and ar metadata) and must have occurred between moko4 and moko5, but the reason for it is not known, nor do we know what the code changes are as both versions are binary-only. OM's change log entry for moko5 lists "Pass the PTCRB certification" as a bullet point - perhaps the sndcp.lib change was needed in connection with this certification. We can only assume that the updated sndcp.lib was included consistently in all mokoN fw versions after moko5, but we cannot easily verify it because the GPRS-specific components of the PS don't have component version strings we can grep for in the fw binary. The next G23M PS component to be updated in this manner was rr.lib. This update is dated 20080401, and because RR is a GSM component (not GPRS), we can easily see whether a given mokoN version uses the updated 20080401 rr.lib or the original 20070608 version by way of the component version string. The update is described as fixing a SIM card write endurance problem and was introduced in moko8. In the case of both rr.lib and sndcp.lib we have no corresponding source for either the original 20070608 version or the updated one, but we have the full source for the TCS3/LoCosto version of the full G23M PS, which is newer overall than all of the versions used by OM. The proper way forward is to migrate to this newer full-source G23M PS version (TCS2/TCS3 hybrid configuration in FC Magnetite fw), and whatever problems may then occur, be they a repeat of old ones once encountered by OM or entirely new ones, will need to debugged via conventional sw debugging techniques and fixed at the source level. In the meantime, until we are able to migrate fully to the TCS2/TCS3 hybrid fw config, our current TCS2-only stable fw config uses the updated versions of both rr.lib and sndcp.lib just like moko10/11. The last binary lib update OM got from TI was a misguided attempt at fixing the infamous bug #1024. The problem was eventually found to be purely in the modem hardware, requiring some hw surgery to fix, and not involving the fw at all, but prior to this realization someone at TI-Taiwan support totally misunderstood the problem and sent OM patched versions of l1_ext.lib & l1_int.lib purporting to fix it. OM's Sean Chiang publicly posted the following explanation: http://lists.openmoko.org/pipermail/openmoko-devel/2008-April/002582.html Quoting the relevant parts: > moko9-beta1 is available now, which is for oscillating re-camping > problem [#bug 1024].The lib patch is provided from TI, which not yet > official approved by TI RD. [...] > > 1) The patch include lib and some not so important text file. Hency I > could not know what's the root cause for #1024 or workaround from them.I > would like to ask TI after the patch approved by TI *or* could really > solve our problem, maybe post the partial code on list. Thus we know that this 20080421 patch to a few L1 modules in l1_ext.lib & l1_int.lib had not gone through TI's internal quality control processes, and could very well have been a quick hack made by someone who had no clue what they were doing. It was also very quickly confirmed empirically (long before the actual hw problem was found) that the patch made absolutely no improvement with respect to the manifestation of bug #1024 behaviour on the affected units. Yet the use of these 20080421 L1 libs extended past the experimental moko9-beta1 fw, and this mysteriously-patched L1 version was used in both moko10 and moko11. I also used it in my leo2moko-r1 aka moko12 release, not knowing any better at the time. However, when I did my reconstruction of TCS211 L1, reconstructing the wrongfully withheld source for this L1 by taking source modules from the LoCosto version and massaging them until they compile into a perfect match to the original objects, for the 3 modules which are different between the original 20070608 version and the 20080421 patch the reconstruction of the 20070608 version came more naturally: whatever is in that 20080421 patch was not on TI's internal mainline, as once I reversed the Calypso->LoCosto changes, I got code that compiled into a perfect match to the original 20070608 version, rather than the 20080421 update. This source reconstruction discovery combined with the overwhelming evidence that the 20080421 patch to L1 was nothing but a misguided hack and the empirical fact that this L1 change never made any observable improvement led to my decision to consider the 20070608 version and my source reconstruction thereof to be the proper version of L1 to use, rather than the mysterious 20080421 version. The source-reconstructed FreeCalypso version of L1 matching 20070608 is used in the moko13 fw release for Openmoko devices and in all official FreeCalypso fw releases for the FCDEV3B. Another interesting aspect of all of the just listed binary lib updates (l1_ext.lib & l1_int.lib, rr.lib, sndcp.lib and sim_drv.lib) is that in each case only one or a few modules within the lib have actual code changes, while the other modules are absolutely unchanged (bit for bit) from the original 20070608 version. However, the COFF header and ar metadata timestamps on these unchanged modules are 20070827 instead of 20070608. Thus someone at TI's support must have done a complete rebuild of TCS211 fw from source on 20070827, from the same code version as OM's 20070608 and in a configuration matching OM's, at least in the case of the affected fw components, and then made the patches for OM from that rebuild. Considering how the 20070419 and 20070608 fw deliveries must have been built specifically for OM because of the TPU driver change, I wonder if the 20070827 build was also done specifically for OM's cases by TI's support group, or if it was a more generic TCS211 release from the same time frame - we can't tell because tpudrv.lib was never updated in this manner. We'll probably never know. The very last change to a binary-only lib in OM's tree did not come from TI at all, but was a binary patch to sim_b_lib.lib done by Dieter Spaar as part of his work that led to moko10 and moko11. TI's standard implementation of the AT+CSIM command disallows GSM APDUs, but OM found some practical need to lift this restriction. In order to make this change, Dieter had to remove the check in the ACI layer in cmh_sims.c, but there was also another check in the SIM entity of the G23M PS (sim_b_lib.lib) - defeating the latter required a binary patch to the code in question. mokoN version by version ======================== OM has published a purported change log in all_version__CHANGELOG.txt listing the evolutionary history of their mokoN firmwares, but it is full of easily provable factual errors. This final section of my historical reconstruction is an attempt to set the record straight. moko1 and moko2: The moko1 and moko2 images presented in the all_old_versions.tar.gz archive are very peculiar in that both exhibit build dates of 20070625 (3 days later after OM supposedly received the new code from TI from which they built moko3), and the time delta from the build timestamp in the moko1 image to the one in the moko2 image in less than 1 hour. Both images thus seem to be archival builds, recompiled well after their initial creation for archival purposes. moko3: OM's published change log says "Based on Moko2 and '''modified libs''' release from TI at 06/22/2007", but the build date of the new TI delivery in question is 20070608, not 20070622. But then 20070608 is the date when the fw delivery in question was compiled from source at TI; it is possible that 20070622 was when OM received this update after all bureaucratic red tape had been cleared. However, the build date of the moko3 image in the all_old_versions.tar.gz archive is 20070726, just over a month after OM supposedly got the new fw delivery from TI. The same 20070726 date appears in the COFF header timestamps on most ACI objects, meaning that the ACI component was fully recompiled on that date. Did it really take OM a whole month to produce a new fw version based on the new baseline delivery from TI? moko4: The stanza for moko4 in all_version__CHANGELOG.txt says: "Fix Bug 212, setting AT@ST is broken" The bug number seems to be incorrect, as bug #212 visible at http://docs.openmoko.org/trac/ticket/212 is something totally unrelated to the GSM modem at all. But the reference to AT@ST makes me wonder if this is the point in time at which the audio.lib update with the 3 new functions related to sidetone control got introduced. moko5: TI's post-20070608 updates to sim_drv.lib and sndcp.lib first appeared in this version. It also differs from all other mokoN versions both before and after in how it uses FFS programming to form its responses to AT+CGxx queries. In TI's canon, the identifying strings to be returned in response to AT+CGMI, AT+CGMM and AT+CGMR queries are stored in the flash file system (FFS) and read out of there by the fw (going through the legacy Condat PCM layer), with some compiled-in dummies being returned in the absence of these files in FFS. TI's canonical implementation of AT+CGxx (found pristine in the LoCosto source and in a modified but still recognizable form in the TSM30 src) simply returns the strings read out of PCM/FFS verbatim, without any alteration. OM's fw versions moko{1-4,6-11} always return hard-coded strings, ignoring the FFS programming altogether. moko5 reads the strings from PCM/FFS, but not does not return them directly: instead the strings that are returned are constructed via sprintf() from some hard-coded bits and from the bits read out of PCM/FFS. All GTA02 units that have passed through my hands or whose flash dumps I have seen have this factory programming in their FFS: /pcm/CGMI: "FIC/OpenMoko" /pcm/CGMM: "Neo1973 GTA02" /pcm/CGMR: "GTA02BV4/Moko5" The '/' in the /pcm/CGMR string separates the hw and fw versions, and the "Moko5" string in the latter part is there just so that if someone flashes moko5 into the modem, it will identify itself correctly - no other mokoN version displays this string from PCM/FFS. Newer FreeCalypso firmwares have TI's original AT+CGxx implementation restored, so they will return the manufacturer and model names from FFS instead of hard-coded strings. But I have also implemented the necessary special case code so they will identify themselves as "/FreeCalypso" instead of "/Moko5". moko6: OM decided that they didn't like what they did in moko5 in terms of ID strings, and went back to their previous way. But now they were also making several different builds of each mokoN version, differing only in the ID strings and identical otherwise, the different ID strings being for different hardware versions. For moko6 there are 3 different versions archived in all_old_versions.tar.gz: gta01bv4-moko6, gta02bv4-moko6 and gta02bv5-moko6. The same situation persisted for moko7 and moko8. The rfcap setting bogosity described earlier was also introduced starting with moko6. moko7: To the best of our knowledge, the only diff from moko6 to moko7 was the addition of the undocumented AT@SC command for changing the IMEI. However, this method of setting the IMEI is badly broken in several ways: * The IMEI input is taken in an obfuscated form. * Whoever wrote this code failed to understand the distinction between an IMEI and an IMEISV. The /pcm/IMEI file really stores the 16 digits of the IMEISV, despite the name of the file, but the AT@SC command takes 15 digits as input, plus obfuscation. What ends up happening is that the first 15 digits of the IMEISV get set to the ones given, and the last digit gets set to either 0 or 1 as an artifact of the obfuscation scheme. In contrast, OM's factory programming of /pcm/IMEI (which could not have been done through this AT@SC command and must have been done the proper way instead) always has the last digit set to 0, which is impossible with AT@SC if the obfuscation scheme makes it 1 for a given IMEI. * The write to /pcm/IMEI is done twice, first through the ffs_file_write() API and then through pcm_WriteFile(). The first write is bogus, but the second one is correct. This AT@SC bogon has remained through moko11, but has been removed in moko12 (leo2moko-r1). The factory IMEI programming must have been done the proper way through TMFFS1, not through this broken AT@SC mechanism. moko8: The stanza for moko8 in all_version__CHANGELOG.txt says: "TI updated the l2r.lib for SIM Card Write Endurance issue." The wrong lib is named. The one that got updated is rr.lib; l2r.lib never changed since the 20070608 full fw baseline delivery. It is also worth noting that L2R is one of the components of the GSM protocol stack devoted specifically to CSD calls; if one is not using CSD, the code in l2r.lib and other libs in this family never gets exercised. moko8 is also where the AT@BAND command got introduced - see the description of the ALR mystery earlier in this article. Another moko8 oddity is that of the 3 versions archived in all_old_versions.tar.gz, gta02bv5-moko8 and gta02bv6-moko8 exhibit 20080411 build dates, whereas the build date of gta01bv4-moko8 is 20080422, about 20 min later than moko9-beta1. But this oddity goes beyond the build dates: gta02bv[56]-moko8 include the 20080401 rr.lib update but gta01bv4-moko8 does not, and there is also a difference in str2ind versions: the str2ind version of gta01bv4-moko8 is &1206521396 (20080326 GMT/UTC), while that of gta02bv[56]-moko8 is &1207547076 (20080407 GMT/UTC). moko9-beta1: There was no moko9 except this -beta1. Contrary to what OM's all_version__CHANGELOG.txt says, moko9-beta1 was NOT built by Dieter Spaar (from scratch or otherwise), but was the last version produced by OM's original modem fw maintainer Sean Chiang. Its purpose was to test the l1_ext.lib & l1_int.lib versions sent by TI in an attempt to fix bug #1024 - see the detailed description earlier in this article. FWIW, moko9-beta1 does NOT include the 20080401 update to rr.lib and features the same str2ind version &1206521396 as gta01bv4-moko8, and the build dates of these two versions differ by only 20 min. OM's original modem fw maintainer Sean Chiang left the company after moko9-beta1, and when OM needed further work on their fw, they hired Dieter Spaar as an independent contractor to pick up the broken pieces left after Sean's departure and put them back together. It has also been said that Dieter had to rebuild the fw "from scratch", but the reality is a little different: it appears that the corresponding sources for moko7, moko8 and moko9-beta1 were lost, and Dieter's starting point was the corresponding source for moko6, perhaps slightly corrupted. The AT@SC and AT@BAND commands had been added after moko6, so Dieter had to reconstruct their implementation functions from disassembly of one of Sean's last mokoN binaries. The implementation functions for these two commands in ati_fic.c in the moko10/11 source salvaged from the ruins of OM contain comments clearly saying that they are reconstructed source seeking to match the disassembly of a previous lost-source version. moko10-beta1: This version is the first survived snapshot of Dieter's work toward moko10. The source + build products tree that has been salvaged from the ruins of OM corresponds to moko10-beta1, or more precisely, the build products part of this tree corresponds to moko10-beta1, while the source part corresponds to moko11. The previously-dropped updates to rr.lib and sim_drv.lib have been reinstated, and Dieter had already source-reconstructed the sidetone wrapper functions in audio_api.c by this point, thus we don't have a preserved copy of the audio.lib binary update which OM got from TI earlier: the audio.lib that appears in the moko10-beta1 tree has been rebuilt from the constituent objects by the makefile, with audio_api.obj coming from the compilation of Dieter's reconstructed audio_api.c source and all other objects unchanged from 20070608. moko10-beta2: The following two changes happened between moko10-beta1 and moko10-beta2: * Dieter reverse-engineered TI's sim_drv.lib update and patched the sim.c source to match, so the SIM driver can now be recompiled from source without blowing away the fix for bug #666. * The change that allows access to GSM APDUs with AT+CSIM (involving a source patch to ACI and a binary patch to sim_b_lib.lib) happened at this time. OM's all_version__CHANGELOG.txt erroneously refers to this moko10-beta2 as "Moko9-Beta2"; there was no moko9-beta2 in reality. moko10: This is the official firmware release that has been programmed into GTA02 devices at the factory in the final production batches. Note that many of the final-batch GTA02 devices were produced well after the moko11 release, but the factory production line still used moko10 - as far as I can tell, they never went to moko11. The only diff between moko10-beta2 and the production moko10 version is in the compiled-in AT+CGxx ID strings: the AT+CGMM string changed from "Neo1973 GTA02 Embedded GSM Modem" to "Neo1973 GTA01/GTA02 Embedded GSM Modem", and the AT+CGMR string changed from "HW: GTA02BV5, GSM: gsm_ac_gp_fd_pu_em_cph_ds_vc_cal35_ri_36_amd8_ts0-Moko10" to "GSM: gsm_ac_gp_fd_pu_em_cph_ds_vc_cal35_ri_36_amd8_ts0-Moko10". moko11-beta1: All moko10->moko11 functional changes are already present in this moko11-beta1 version, and these changes amount to just two: * armio.c change to set GPIO 1 to low rather than high on boot; * uartfax.c change to enable hardware flow control. moko11: This is the very last modem fw version produced by The Company prior to its demise. The only diff from moko11-beta1 to the final moko11 is the removal of "b1" from the compiled-in AT+CGMR string. After the demise of The Company, further development of the modem fw for Openmoko devices has been continued by FreeCalypso. moko12 (originally called leo2moko-r1): The diff from moko11 to moko12 consists of: * Removal of the rfcap and AT@SC bogons; * New ID strings (AT+CGMI and AT+CGMM strings read from FFS); * Clean recompilation from freely published source. moko13: This release has been made in order to bring Openmoko devices up to date with the recent FreeCalypso developments. Because FreeCalypso is a living and actively maintained FOSS project unlike OM's proprietary fw, the small changes are far too numerous to list (that's what the public Mercurial commit history is for), but here are the major ones: * L1 has been reverted from the mysterious 20080421 version (misguided attempt at fixing bug #1024) back to the original 20070608 version (used successfully in moko3 through moko8, and was certainly in use when the type approval or certification testing was done), and most of it has been recompiled from the reconstructed source. * Previous fw versions configured many of Calypso's GPIO and multifunction pins that are unconnected in the GTA01/02 modem as inputs, resulting in floating inputs. Moko13 configures all of these pins as outputs instead. * A number of changes visible only to those who use the RVTMUX interface on the headset jack to talk to the fw as it runs, and not just for flashing.