FreeCalypso > hg > freecalypso-citrine
changeset 28:cb00b90edaff
documentation write-ups imported from freecalypso-sw and updated for Citrine
author | Mychaela Falconia <falcon@freecalypso.org> |
---|---|
date | Sun, 12 Jun 2016 18:28:35 +0000 |
parents | 3ecd6054a7f7 |
children | c0f2d21307d1 |
files | doc/Compal-Howto doc/Current_Status doc/Freerunner-Howto doc/Pirelli-Howto doc/TCH-special-feature |
diffstat | 5 files changed, 703 insertions(+), 0 deletions(-) [+] |
line wrap: on
line diff
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/Compal-Howto Sun Jun 12 18:28:35 2016 +0000 @@ -0,0 +1,237 @@ +FreeCalypso Citrine firmware on Mot C11x/12x and C139/140 families +================================================================== + +NOTE: this write-up refers specifically to our work-in-progress Citrine fw. +The tcs211-c139 hack which we have produced in late 2015 is an entirely +different animal. + +Unlike tcs211-c139, Citrine can run equally "well" on both our preferred +C139/140 platform and the more primitive C11x/12x, but this fw is currently +much more limited: + +* tcs211-c139 includes TI's demo/prototype UI code and an LCD driver that works + with C139/140 LCD hardware; Citrine currently has no UI code at all, + expecting control via AT commands via the same serial cable you use for + flashing it. + +* TCS211 is TI's official production-quality firmware for the Calypso, whereas + our Citrine fw is only beginning to catch up to it - see the Current_Status + article for more information. + +The phones in this family have very little RAM: 256 KiB of Calypso on-chip RAM +(IRAM) on all variants, plus another 256 KiB of board-level RAM (XRAM) on +C11x/12x or 512 KiB of XRAM on C139/140. The tcs211-c139 port uses almost all +available IRAM and XRAM on the C139, hence porting it to C11x with even less +RAM was completely out of the question. Citrine currently has a lot less +functionality integrated, which naturally translates to lower memory +requirements - hence it is possible to build for the C11x. + +Because RAM is so precious on these feeble targets, running our own fw on them +absolutely requires flashing - fc-xram is not an option. Furthermore, we cannot +use an FFS-in-RAM configuration like we do on large-XRAM targets, and Motorola's +original FFS (flash file system) on these phones is not suitable for our needs - +unlike the situation on Openmoko modems. Therefore, we need to create and +maintain our own aftermarket FFS in a region of the device's flash memory which +we arbitrarily choose ourselves. + +If you are going to play with FreeCalypso firmwares on Mot C1xx targets, we +recommend that you devote a phone specifically for FreeCalypso and have another +phone to charge batteries. The process of flashing our firmware and creating +and maintaining the necessary aftermarket FFS on these targets is quite +involved, hence flashing a given phone back and forth between FreeCalypso and +Mot/Compal's official firmwares would be a total pita. However, none of our +firmwares (neither this one nor tcs211-c139) currently has working battery +charging code, hence you will need to use another phone running one of the +official fw versions to charge batteries. + +Compiling +========= + +The starting configuration file for building Citrine for targets in this family +is configs/c139-gsm-flash. If your phone is a C139 or C140, this default +config can be used as-is, although you are always welcome to edit it to taste. +If your phone is C11x or C12x, change the target setting from c139 to c11x. + +The two numbers on the 'feature aftermarket-ffs' line select the region of +flash where our aftermarket FFS will be placed. The default configuration +places our FFS in the region from 0x3C0000 through 0x3EFFFF. This configuration +is recommended because: + +* it does not conflict with the FFS maintained by Mot/Compal's fw (the two + locations are different), eliminating the possibility of one firmware trying + to use the FFS created by the other; + +* it is placed at the very end of the flash (or rather at the end of the main + flash zone with 64 KiB sectors), maximizing the room available for the + firmware code image. + +NOTE 1: our aftermarket FFS code cannot use 8 KiB flash sectors at the chip's +highest addresses. Therefore, the sectors with factory data (which we don't +know how to grok) are safely left untouched by our fw. + +NOTE 2: if your phone is a C11x/12x variant with 2 MiB of flash (some have +2 MiB, others have 4 MiB), directing the firmware to put its FFS at 0x3C0000 +will result in it being at 0x1C0000 in reality - the highest address bit does +nothing when the flash chip only has 2 MiB. + +NOTE 3: if your phone is C139/140, keeping the aftermarket FFS at 0x3C0000 is +doubly recommended as that is the location used by our tcs211-c139 build. + +Flashing +======== + +The flashing procedures can be divided into two parts: the steps which you need +to perform only once when you first convert a given phone from Mot/Compal's fw +to FreeCalypso vs. the steps which you need to perform each time you wish to +flash another image you just compiled. + +If you are starting with a "virgin" phone that never ran FreeCalypso before, +you will need to start by breaking in with fc-loadtool and possibly tfc139 - +see the Compal-unlock article in the FreeCalypso host tools package for more +details. Once you are in with loadtool and have made a backup of your original +flash content, your first step will be to reflash sector 0 (the dangerous one) +with a version of the bootloader code that has been patched to transfer control +to the main fw image in the way we need: + +loadtool> flash erase-program-boot compal-flash-boot-for-fc.bin + +The compal-flash-boot-for-fc.bin code image can be downloaded here: + +ftp://ftp.freecalypso.org/pub/GSM/FreeCalypso/compal-flash-boot-for-fc.bin + +It was made from one of Mot/Compal's original versions by applying a binary +patch to it; the source for this patch can be found in the retired +freecalypso-sw source repository on Bitbucket. + +This step of replacing the bootloader needs to be done only once - you don't +need to reflash this dangerous sector again when you reflash the main fw image. +The patched FreeCalypso bootloader is also the same for both the present Citrine +fw and tcs211-c139. + +The next step is to flash the main firmware image which you have just compiled: + +loadtool> flash erase 0x10000 0x160000 +loadtool> flash program-bin 0x10000 finlink/flashImage.bin + +Note that the main fw image is flashed at 0x10000 on these targets. It is +flashed at 0 on sane targets with the Calypso boot ROM enabled in the hardware, +but Compal phones have malicious wiring in their PCBs that makes them brickable +and imposes the requirement of having working boot code in sector 0 at all +times, with the main fw image pushed down to 0x10000. + +Finally, you should erase the flash region which you have allocated for the +aftermarket FFS: + +loadtool> flash erase 0x3C0000 0x30000 + +or if your phone only has 2 MiB of flash: + +loadtool> flash erase 0x1C0000 0x30000 + +Now you can close your loadtool session with an exit command, and the phone +will be cleanly powered off. + +The next time you need to reflash another FreeCalypso image, get in with +loadtool like this: + +fc-loadtool -h compal /dev/ttyXXX + +There is no more need for tfc139 or for the inefficient -c 1003 option to +fc-loadtool once you've replaced the bootloader with compal-flash-boot-for-fc. +Once you are in loadtool, just reflash the main fw image, and leave the +bootloader and FFS sectors alone. + +First boot of the firmware +========================== + +Connect the serial cable, but instead of running fc-loadtool, run rvinterf. +Press the red power button on the phone briefly just like you would for +fc-loadtool entry. Because there is no fc-loadtool running on the host end of +the serial cable, the boot path will *not* be diverted in the bootloader, and +the main fw image will run - and this time it will be the FreeCalypso firmware +you have compiled and flashed. The phone's LCD will remain dark as there is no +LCD driver code in this firmware, but you will see trace output in the rvinterf +window, telling you that the fw is running. + +Before you do anything else, you will need to run fc-fsio and initialize the +aftermarket FFS for our firmware. When running on Openmoko GTA0x and Pirelli +DP-L10 targets, our fw can use the original factory-programmed IMEISV and RF +calibration values (partial in the case of the Pirelli), but on Mot/Compal +phones these factory data are stored in a format which we haven't been able to +grok, hence we cannot make use of them. Therefore, you will have to set your +own IMEISV manually, and the radio will run uncalibrated. + +Initialize your aftermarket FFS as follows: + +fsio> format / +fsio> mk-std-dirs +fsio> set-imeisv fc XXXXXXXX-YYYYYY-ZZ (punctuation optional, place anywhere) +fsio> set-rfcap dual-eu (if you have 900+1800 MHz hardware) +or +fsio> set-rfcap dual-us (if you have 850+1900 MHz hardware) + +After you've initialized your FFS as above, you can exit fc-fsio, run fc-shell +and try some AT commands: + +AT+CMEE=2 -- enable verbose error responses +AT+CFUN=1 -- enable radio and SIM interfaces +AT+COPS=0 -- register to the default GSM network + +When you are done, you can power the phone off by sending a 'poweroff' command +through fc-shell. The only other way is to yank the battery, and doing the +latter is recommended anyway: when a phone with the present hack-firmware +flashed into it is powered off but still has the battery inserted, even a +momentary accidental press of the power button will cause it to power on and +boot, but there will be absolutely no visual indication, as the LCD stays dark. + +FreeCalypso GSM firmware on Mot C155/156 +======================================== + +One major difference between Mot C155/156 and the other two subfamilies is that +C155 and C156 have 2 MiB of XRAM, which is large enough to allow our small-ish +experimental firmware to run entirely from RAM, without flashing, just like on +the Pirelli DP-L10. + +If you are ready to play with our experimental GSM pseudo-modem fw on your +C155/156, the steps are as follows: + +1. Build the firmware in the c155-gsm-ramonly configuration - see the + Compiling document for more details. + +2. Connect your serial or USB-serial cable as usual; the phone needs to be + powered off at this point. + +3. Run a command like the following: + + fc-xram -h c155 /dev/ttyUSB0 finlink/ramImage.srec rvinterf + + If you are using an official FreeCalypso USB-serial cable from UberWaves, + you can speed up the code download by switching the serial line to 812500 + baud: + + fc-xram -h c155 -B 812500 /dev/ttyUSB0 finlink/ramImage.srec rvinterf + + Adjust the paths to your /dev/ttyUSBx or other serial device and your + ramImage.srec as appropriate, and add rvinterf logging or other options as + desired. Specifying rvinterf on the fc-xram command line directs fc-xram to + exec rvinterf and pass the serial channel to it immediately as soon as the + code image has been loaded into target RAM and jumped to; this direct + passing of the serial channel from fc-xram to rvinterf is appropriate + because the loaded fw will immediately start emitting binary trace packets + in TI's RVTMUX format. + +4. Momentarily press the red power button on the phone. + +Once the phone executes its boot code with fc-xram running, the boot path will +be diverted and our experimental firmware will be loaded into target device RAM +and jumped to. Our fw will now run, and the rvinterf process on the host will +maintain communication with it. + +Just like on the lower Mot/Compal subfamilies, we don't know how to extract the +factory-programmed IMEI and RF calibration data from Mot/Compal's proprietary +flash data structures, therefore, when our RAM-based firmware boots, it has no +IMEI and no RF calibration. Because this RAM-only configuration leaves the +flash completely alone and does not create a non-volatile FFS there, you will +need to set the IMEISV and RFCAP with fc-fsio on each boot. See the fc-fsio +commands given earlier, but skip the format command as the RAM-based FFS is +automatically formatted - but not otherwise initialized - upon firmware boot.
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/Current_Status Sun Jun 12 18:28:35 2016 +0000 @@ -0,0 +1,107 @@ +The goal of the Citrine firmware project is to replace the Windows-built +firmwares which have been produced in other subprojects under the FreeCalypso +umbrella - see leo2moko and tcs211-c139. Our leo2moko project has produced a +production quality modem fw image for the Openmoko GTA02, while a C139 reflashed +with tcs211-c139 is the first dumbphone in history that can still function as an +untethered phone after having had its fw replaced with an indie one that bears +no relation to the manufacturer's original - but those TCS211-based +Windows-built projects have severe limitations. Much of the firmware code base +in those versions is in the form of unmodifiable binary object libraries, and +the Windows-based configuration and build system is incompatible with the +long-term needs of FreeCalypso development. + +The present fw project (FreeCalypso Citrine) seeks to rectify the situation by +replacing the blob-laden, Windows-built firmware with a version that is built +from full source (no binary blobs) with gcc, with an entirely different +configuration mechanism that actually suits our needs. Because one of the key +goals of this project is to build the firmware from *full source*, the binary +object versions of L1 (GSM Layer 1) and G23M (layers 2&3 of the protocol stack) +featured in our reference TCS211 fw could not be reused. Instead this project +uses versions of L1 and G23M (and some other pieces) that have been lifted from +the firmware for TI's other chipset (LoCosto) and backported to Calypso. + +The current state of the project is that we have made remarkable progress, but +what we have right now is still not a satisfactory replacement for TCS211. +Specifically: + +* Only the bare minimal modem functionality for the voice+SMS subset has been + integrated so far. "Modem" means our fw can only be controlled via AT + commands; no UI code (as in LCD+keypad) has been integrated at all. But it + is not a true modem either as none of the data functions have been integrated + yet: no CSD, no fax, no GPRS. Thus it is an AT-command-controlled voice+SMS + pseudo-modem. + +* The firmware can be built for the following targets: + + Mot C11x/12x + Mot C139/140 + Mot C155/156 + Openmoko GTA01/02 + Pirelli DP-L10 + + All configurations are built from the same source tree. The firmware + functions identically on all supported targets. Because there is no UI code + integrated yet, the LCD stays dark and the buttons do nothing on those target + devices that have such hardware. + +* Most of our supported target devices have only one practically accessible + serial port (UART). Our firmware presents TI's RVTMUX interface on this + UART; the operator is expected to interface to it by running our rvinterf + tools on the host PC/laptop. One of the utilities in the rvinterf suite is + fc-shell; this tool is used to send AT commands to the running firmware, + which is the only way to control its operation. + +* With a valid SIM card inserted and a valid IMEISV configured, a GSM device + running our firmware can successfully connect to live commercial GSM networks, + make and receive voice calls, and send and receive SMS. + +* In the case of voice calls, the call downlink audio is routed to the phone's + earpiece speaker and the phone's microphone serves as the source for the + uplink audio, i.e., even though the LCD and keypad are dead with our fw, the + earpiece and mic continue to function as in a conventional phone. FR and EFR + codecs work correctly (EFR was broken until recently), but AMR does not work + with this fw for some not-yet-understood reason, hence by default our fw + currently advertises to the GSM network that the MS only supports FR, HR and + EFR codecs. + + There is also a highly experimental and minimally tested alternative mode + of operation in which the traffic channel carrying FR codec bits (260 bits + every 20 ms; it is not known whether or not this feature will also work with + EFR) is rerouted away from the internal vocoder to the external host, + such that you can receive the downlink voice bits digitally instead of + listening to them in the earpiece speaker, and you can substitute your own + uplink bits instead of the microphone-fed internal vocoder output. See the + TCH-special-feature write-up for more information. + +There are also two known bugs which manifest only intermittently, but the +misbehaviour does occur often enough that you will likely encounter it: + +* Sometimes something gets messed up in the voice uplink path such that the + downlink audio sounds just fine in the earpiece speaker of the phone running + FC Citrine, but the far end of the call hears only silence. Other times the + voice audio passes just fine in both directions. It is not currently + understood what factors determine whether it will work or not. + +* Sometimes the L1A task in the firmware (see the Firmware_Architecture + write-up) appears to stop running; the externally visible behaviour is that + the debug trace output from L1 suddenly stops while the rest of the firmware + keeps running. GSM firmware without working L1 is unusable, hence when the + fw gets into this state, the only remedy is a power cycle reboot. It is not + currently understood exactly what happens and under what conditions. + +Going forward, the current plan is to wait until our FCDEV3B hardware gets built +before any attempts will be made to go after the above-listed bugs. I, the +principal developer, am sick and tired of limping along with Compal, Pirelli +and Openmoko hardware when we have a design for our own FreeCalypso development +board which, when built, will be a much more convenient platform for firmware +development. + +Target-specific usage instructions +================================== + +If you would like to play with our work-in-progress firmware and check it out +for yourself, see the following target-specific instructions: + +Mot C1xx (Compal) Compal-Howto +Openmoko GTA01/02 Freerunner-Howto +Pirelli DP-L10 Pirelli-Howto
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/Freerunner-Howto Sun Jun 12 18:28:35 2016 +0000 @@ -0,0 +1,109 @@ +How to play with FreeCalypso GSM firmware on a Neo Freerunner +============================================================= + +We have two entirely different firmware offerings for the Freerunner: + +1. Leo2moko fw produced back in 2013-10: this is the only one suitable for + end users. We also have leo2moko-debug which is a slightly hacked-up + version of leo2moko with some additional debug features for developers; + this version is for developers only; end users should stick with the + original leo2moko-r1 aka moko12 from 2013-10. + +2. The work-in-progress full-source gcc-built FreeCalypso Citrine fw + you are looking at can be built for multiple targets, and the + gtamodem target is one of them - the original one, in fact. + +The flash+SRAM chip which FIC/Openmoko populated in their modems provides +plenty enough RAM for the firmware's data space requirements, but not enough +to run a complete firmware code image entirely from RAM, hence whichever fw +version you would like to exercise, you need to flash it. There are two ways +to flash modem firmware images in these smartphones: from inside the phone +(from the application processor) or externally through a special serial cable +inserted into the analog headset jack. The internal method is intended only +for end users flashing released production-quality images; developers and +tinkerers are expected to use the serial cable method. + +The serial cable wiring requirements for the GTA02 are the same as for Mot C1xx +phones, thus the same cable can be used for both. The FreeCalypso project has +endorsed UberWaves as our official vendor for serial cables; George at +UberWaves now makes serial cables that are specifically certified for use with +FreeCalypso. If you would like to order one, email uberwaves@gmail.com. + +Please see the Current_Status write-up for the current status of the present +Citrine firmware. As you can read there, this fw is currently nowhere near +being able to replace leo2moko. Therefore, if you are going to flash Citrine +into your FR's modem, we expect that you are using your FR as a poor man's +substitute for the not-yet-built FCDEV3B (a board we seek to build specifically +for developers and not for end users), and are NOT expecting this experimental +work-in-progress modem fw to work together with user-oriented application +processor software like QtMoko or SHR. + +If you would like to play with our experimental Citrine fw using a GTA02 modem +as the hw platform, here are the instructions: + +1. Build the firmware in the gtamodem-gsm configuration - see the Compiling + document for more details; + +2. You should get a flashImage.bin image built; now you need to flash it into + your FR's modem. The serial cable method is highly recommended: the only + thing you'll be able to do with our current not-fully-functional firmware is + play with it and observe the debug output, and the serial cable will be + needed for the latter part anyway. + +3. Run fc-loadtool the same way you would if you were flashing leo2moko; + +4. The actual flash programming commands are a little different because the + image is smaller and in a different format: + +flash erase 0 0x160000 +flash program-bin 0 finlink/flashImage.bin + +The second number in the flash erase command needs to be the size of +flashImage.bin rounded up to a multiple of 64 KiB (the flash sector size in the +GTA02 modem); 0x160000 is correct for the fw image size as of this writing, but +please double-check it yourself before flashing. The 0 argument in the +flash program-bin command is the flash offset at which the image should be +programmed: it will always be 0 for FreeCalypso flashable fw images for gtamodem +and other targets that have the Calypso boot ROM enabled in the hardware. + +Once you have flashed our experimental fw into your modem, you can power-cycle +the modem and see the new fw boot. You should have the serial cable connected, +the serial channel enabled from the Freerunner's AP side and either rvtdump or +rvinterf running on your PC or other development machine when you first power +your modem up with the experimental fw in it: this way you will see the debug +output as the firmware boots up. + +Once the firmware has booted, it needs to be controlled via AT commands. The +present fw presents its AT command interface on two channels on this target: on +the MODEM UART going to the Freerunner's application processor and via RVTMUX. +At the present stage of development, we highly recommend that you avoid running +any GSM-driving software on the AP and exercise our work-in-progress fw solely +through the external serial interface on the headset jack, using rvinterf and +fc-shell. The standard AT command interface on the dedicated MODEM UART is a +feature which we plan to address properly only when we build our planned FCDEV3B +hardware, which will bring both UARTs out to the external host. + +Assuming that you already have rvinterf running in a terminal window (you should +have started it before you gave the modem power-on command from the AP side), +to exercise our firmware further, you will need to open another terminal window +on your driving PC/laptop and run fc-shell. This program will connect to the +already running rvinterf process via a local socket, and it will enable you to +send various commands to the running fw on the target, the most important ones +being standard AT commands. Send the following sequence of AT commands to +bring up GSM functionality: + +AT+CMEE=2 -- enable verbose error responses +AT+CFUN=1 -- enable radio and SIM interfaces +AT+COPS=0 -- register to the default GSM network + +To reflash your modem back to stable and working leo2moko aka moko12, execute +the following fc-loadtool commands: + +flash erase 0 0x230000 +flash program-m0 leo2moko.m0 + +(Whichever firmware image you are flashing, the flash erase command needs to + cover the range of flash sectors this image will occupy. You can erase more + sectors up to 0x300000, the "natural" boundary of the flash area where fw + images live, but I prefer to erase only the needed number of sectors: it is + both faster and imposes less wear on the flash.)
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/Pirelli-Howto Sun Jun 12 18:28:35 2016 +0000 @@ -0,0 +1,66 @@ +How to play with FreeCalypso GSM firmware on a Pirelli DP-L10 +============================================================= + +One very useful special feature of the Pirelli DP-L10 is its very large RAM: +8 MiB. Having such large RAM allows us to run our experimental fw on this +target entirely from RAM, without touching the flash. When you compile a +FreeCalypso Citrine fw image for the Pirelli target, by default a ramImage will +be built instead of a flashImage. It is possible to build a flashable image of +the fw in the same configuration and program it into flash with fc-loadtool, +but doing so is not recommended: our current fw has no battery management code, +so the charging hardware circuit will never be enabled and the battery will +discharge even with a USB power source connected; keeping Pirelli's original +fw in flash will allow the phone to charge its battery and otherwise function +normally when you are not in the middle of a FreeCalypso firmware experiment. + +If you are ready to play with our experimental GSM pseudo-modem fw on your +Pirelli, the steps are as follows: + +1. Build the firmware in the pirelli-gsm-rvtat configuration - see the + Compiling document for more details. + +2. Connect a USB cable from your GNU/Linux PC/laptop to the phone. If the + phone was off but the battery is present, it will go through a charger-plug + power-on event; if the flash contains Pirelli's original fw, it will boot in + the charging mode. If the battery is not present, the Calypso won't power + on (it needs VBAT and can't run on VCHG power instead), but the /dev/ttyUSBx + device will still show up, as the CP2102 USB-serial chip inside the phone is + powered strictly from the USB side. + +3. Run a command like the following: + + fc-xram -h pirelli /dev/ttyUSB0 finlink/ramImage.srec rvinterf + + Adjust the paths to your /dev/ttyUSBx device and your ramImage.srec as + appropriate, and add rvinterf logging or other options as desired. + Specifying rvinterf on the fc-xram command line directs fc-xram to exec + rvinterf and pass the serial channel to it immediately as soon as the code + image has been loaded into target RAM and jumped to; this direct passing of + the serial channel from fc-xram to rvinterf is appropriate because the + loaded fw will immediately start emitting binary trace packets in TI's RVTMUX + format. + +4. Induce the phone to execute its Calypso boot path: if the battery was + removed, insert it now; if Pirelli's regular fw is running, execute its + power-off sequence. + +Once the Calypso chip in the Pirelli phone executes its boot path with fc-xram +running, the boot path will be diverted and our experimental firmware will be +loaded into target device RAM and jumped to. Our fw will now run, and the +rvinterf process on the host will maintain communication with it. + +To exercise our firmware further, you will need to open another terminal window +on your driving PC/laptop and run fc-shell. This program will connect to the +already running rvinterf process via a local socket, and it will enable you to +send various commands to the running fw on the target, the most important ones +being standard AT commands. Send the following sequence of AT commands to +bring up GSM functionality: + +AT+CMEE=2 -- enable verbose error responses +AT+CFUN=1 -- enable radio and SIM interfaces +AT+COPS=0 -- register to the default GSM network + +When you are done playing with our experimental fw, you can either yank the +battery and kill the host side rvinterf and fc-shell processes, or you can +issue a 'tgtreset' command at the fc-shell prompt. The latter will cause the +target to reset and boot back into its regular firmware.
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/TCH-special-feature Sun Jun 12 18:28:35 2016 +0000 @@ -0,0 +1,184 @@ +FreeCalypso Citrine firmware implements an optional special feature (needs to be +explicitly enabled at compile time) which we call TCH rerouting. When this +feature is enabled, it applies the following special handling to GSM voice +traffic channels (TCH): + +* All downlink TCH bits passing from the channel decoder to the vocoder block + (260 bits every 20 ms with the original FR codec) can be non-invasively + intercepted and forwarded to the external host connected to the RVTMUX serial + interface; + +* Using the same serial interface, the external host can supply substitute + uplink TCH bits which will be transmitted in the place of the built-in + vocoder output, i.e., the latter can be effectively suppressed. + +In order to use this feature, you need to compile our firmware in the voice+SMS +pseudo-modem configuration, i.e., the configuration in which the fw expects to +be controlled via AT commands wrapped in the RVTMUX binary packet serial +interface. You can use a target GSM device that has just one accessible serial +port (Mot C1xx and Pirelli DP-L10) or one that has two Calypso UARTs (Openmoko +GTA02 or our future FCDEV3B), but in the latter case you will be using only one +UART - whichever one you have configured to be RVTMUX. + +Whatever system you are building that will act as the source and sink for TCH +bits will need to interface to the FreeCalypso GSM device via a serial port in +the RVTMUX binary packet format. Your system will need to send RVTMUX packets +with AT commands inside them in order to command the FC GSM device to register +with the network and to dial and/or answer calls, and you will need to send +RVTMUX packets of a different kind in order to supply the uplink TCH bits +during calls. In the other direction, your system will receive responses to +the AT commands you send, asynchronous notifications of incoming calls and SMS, +downlink TCH bits and various debug trace output from our FreeCalypso firmware. +The last part (debug trace output) can be simply ignored and discarded if you +wish, but we strongly recommend that you provide a way to view and/or log it +for debugging purposes. + +Please see the RVTMUX document in the FreeCalypso host tools package for general +background information regarding the RVTMUX binary packet interface; this +document should be considered required reading for anyone interested in working +with the TCH rerouting special feature. + +All packets transferred over the RVTMUX interface begin and end with 0x02. If +a payload byte within a packet equals 0x02 or 0x10, it needs to be prepended +with 0x10 as a transparency escape; all other payload bytes are sent literally. +The first byte within each RVTMUX packet after the opening 0x02 is the packet +type; the two packet types you will need to handle (both generate and receive) +are 0x1A for AT commands and 0x1C for TCH configuration commands. + +To send an AT command to the FreeCalypso GSM device, prepend the 0x1A packet +type in front of the "AT" characters, wrap the packet with 0x02 bytes on both +ends, and send it to the modem. Responses to AT commands and asynchronous +notification messages such as "RING" for incoming calls will be sent to the +host as RVTMUX packets also beginning with the 0x1A packet type; they will be +interspersed among other packet types, mostly debug trace output. Your system +will need to receive the RVTMUX serial byte stream continuously, parsing the +packet structure and looking at the type of each packet (the first byte after +the opening 0x02) in order to detect if the modem has sent something you may be +interested in. + +If you wish to receive a copy of all downlink TCH bits on the serial channel, +you will need to send the following 5-byte command packet to the modem: + +0x02: opening flag +0x1C: RVTMUX packet type for TCH +0x11: TCH_CONFIG_REQ command opcode +0x01: payload byte indicating that the "forward downlink" state should be + set to enabled +0x02: closing flag + +The modem will respond with a TCH_CONFIG_CONF confirmation message (opcode +0x12), and then during all voice calls your external host will receive the +following packet every 20 ms: + +0x02: opening flag +0x1C: RVTMUX packet type for TCH +0x15: TCH_DLBITS_IND opcode +- 40 (decimal) bytes of payload - +0x02: closing flag + +The 40 bytes of payload sent in every TCH_DLBITS_IND packet directly correspond +to the 20 16-bit words provided by the Calypso DSP in its a_dd_0 buffer. The +first 3 words (6 bytes) contains the DSP's own status information (not fully +understood by us yet, but we let you see what the DSP tells us without redacting +anything out), and the remaining 17 words (34 bytes) are supposed to contain +the TCH bits received from the GSM network in the FR codec format. Each DSP +API word is sent in the big-endian byte order, i.e., the most significant byte +followed by the least significant byte. + +If you wish to send your own TCH uplink bits, replacing the output of the +built-in vocoder with your own alternate uplink data, you will need to send +your uplink TCH bits to the modem in packets of the following format: + +0x02: opening flag +0x1C: RVTMUX packet type for TCH +0x13: TCH_ULBITS_REQ opcode +- 33 or 34 (decimal) bytes of payload - +0x02: closing flag + +Sending 260 bits requires only 33 bytes, but the DSP operates in terms of +16-bit words, hence 17 of those words are used. The least significant byte of +the last word (i.e., the very last byte with our big-endian transmission order) +is not expected to be used, but if you send 34 bytes rather than 33, you will +have control over every bit going into the DSP API RAM in this case. + +There is a queue inside the firmware in which these TCH uplink data blocks are +stored; this queue is filled by the serial packet receiving handler and drained +by the L1S (synchronous) code that executes at the right times in the GSM TDMA +multiplex when uplink TCH transmission is expected. Up to 4 blocks can be +queued up; as each queued-up block is transmitted on the air (more precisely, +as it is passed to the DSP for channel encoding and transmission), a +TCH_ULBITS_CONF short packet (consisting of just the opcode and nothing more) +is sent to the host. These confirmation packets can be used to pace the sending +of further TCH_ULBITS_REQs. + +Testing +======= + +The just-described mechanism has been tested as follows: + +1. I placed a call to WWV (+1-303-499-7111), and after verifying with my ear + that the downlink audio was good, I recorded the downlink TCH bits on that + call into a file with the tch record command in fc-shell. + +2. I placed a call to another phone (running over a live commercial GSM network) + and played the saved recording from WWV into the call uplink with the + tch play command in fc-shell. + +3. The audio heard on the other end of the call in the previous step: the + recording from WWV was definitely recognizable, but it didn't sound perfect, + i.e., it was rather garbled. + +[NOTE: the experiment described above was performed with an older version of + the firmware which is now codenamed Citrine, namely, the version with L1-2014. + I have not played with the TCH rerouting feature again since the transition to + L1-2016.] + +Further debugging of this mechanism will require two things which I currently +lack: (1) proper understanding of the workings of the GSM 06.10 FR codec and +(2) a test GSM network (as in OpenBTS/OpenBSC/etc) that could be used instead +of live commercial ones, so we could see exactly what the test MS is +transmitting on the air and what the BTS transmits in the downlink. + +Host side reference implementation +================================== + +If you are going to implement your own system for talking to FreeCalypso GSM +pseudo-modems via the RVTMUX binary packet interface, we strongly recommend +that you use our rvinterf and fc-shell Unix/Linux host utilities as your +starting point. You can find their source code in the freecalypso-tools Hg +repository on Bitbucket. + +The following test commands have been added to fc-shell for exercising the +experimental TCH rerouting mechanism: + +tch record <filename> + + Sends a TCH_CONFIG_REQ packet to the target, commanding the firmware to + start forwarding TCH downlink bits to the external host, and starts + recording the bits it receives in the named file. The file is written + with the same ordering of GSM 06.10 bits as used by the popular libgsm + implementation of this codec, i.e., the bits received from the GSM + device (ultimately coming from TI's DSP) are reordered before being + written into the file. It is only a reordering of bits with no change + in the information content. + + I was hoping that the resulting files could be played with the SoX play + command under Slackware Linux, but all I got was garbled audio, and my + audio-fu is not good enough to figure out what is wrong. + +tch record stop + + Stops TCH downlink recording and closes the file into which the bytes + were being written; until the file is thus closed, it may not be + actually written out to the file system. + +tch play <filename> + + Plays GSM 06.10 FR speech frames from the named file in libgsm format + (same as written by the tch record command) into the call uplink. + +tch play stop + + Terminates the TCH UL play-from-file operation. This command is + normally not needed, as the play session will end automatically when + the end of file is reached.