FreeCalypso > hg > fc-magnetite
changeset 94:596d86109e44
initial round of documentation
| author | Mychaela Falconia <falcon@freecalypso.org> |
|---|---|
| date | Mon, 03 Oct 2016 04:26:16 +0000 |
| parents | 6475bde1b170 |
| children | 48792a467305 |
| files | README doc/C139-Howto doc/Compiling doc/Freerunner-Howto doc/Pirelli-Howto |
| diffstat | 5 files changed, 423 insertions(+), 0 deletions(-) [+] |
line wrap: on
line diff
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/README Mon Oct 03 04:26:16 2016 +0000 @@ -0,0 +1,153 @@ +FreeCalypso Magnetite firmware project +====================================== + +This source tree contains yet another firmware offering created under the +FreeCalypso umbrella. The key qualities of this firmware offering are: + +Negatives: + + * Builds with TI's proprietary TMS470 compiler and thus requires Wine; + * Some of the fw components are still in the form of binary blobs - + but see below on the deblobbing progress. + +Positives: + + * Can be built to run on Motorola C139, Openmoko GTA02 and Pirelli DP-L10 + targets, as well as the to-be-built FCDEV3B, all from the same source tree; + + * Works as solidly as the TCS211 "golden" reference from TI, on all of the + supported targets - deep sleep works, voice calls work in all codec modes + including AMR, the DSP dynamic download mechanism does its magic, the call + audio passes reliably in both directions. + +The present FC Magnetite firmware is built on the principle of starting with +the known working TCS211 code base, without any major restructuring, and making +small incremental evolutionary changes, testing at every step to ensure that +nothing breaks. It is the direct opposite of the "rebuild from the ground up" +approach taken with our previous Citrine firmware (aka "gsm-fw"), the approach +that produced very disappointing results. + +Functionality +============= + +TI's GSM mobile station firmware architecture supports two ways in which the +GSM device may be controlled: via AT commands from an external host and/or via +a local UI on devices with LCD & keypad hardware. (I said "and/or" because the +two mechanisms can coexist.) At the present time, however, only the AT command +mode of operation is supported in FreeCalypso Magnetite. The effect of this +limitation is that if you run this fw on a Motorola C139 or a Pirelli DP-L10, +the phone's LCD will stay dark, the buttons won't do anything, and the +phone-turned-modem can only be operated via AT commands sent via FreeCalypso +host utility fc-shell. + +The demo/prototype phone UI code in TI's reference fw delivery is written for +a 176x220 pixel 16-bit color LCD (such an LCD was used on TI's official +development platform called D-Sample), whereas our available Mot C139 and +Pirelli targets have significantly smaller LCDs: 96x64 and 128x128 pixels, +respectively. Prior to the D-Sample, TI had used 84x48 pixel black&white +(1 bit per pixel) LCDs, and this old C-Sample LCD code is still there, albeit +bitrotten. In late 2015 this author made a very dirty hack to resurrect TI's +old C-Sample UI code and get it to display on the C139 phone LCD (84x48 is a +proper subset of 96x64) - you can find this hack in the tcs211-c139 Hg tree. + +The upshot of this LCD situation is that porting TI's phone UI code to run on +Mot C139 or Pirelli DP-L10 will require major rework of the affected parts of +the firmware. While I would like to do it eventually, I am not willing to do +it right now - I would like to get this code running in its pristine state on +its native 176x220 pix LCD *before* I hack the holy **** out of it for the +C139/Pirelli port. I do have a real TI-made D-Sample kit with the right LCD, +but unfortunately the main board has an older version of the core chipset for +which we lack the necessary fw support, hence it doesn't help. Therefore, we +will need to build our own Calypso board with a 176x220 pix 16-bit color LCD, +get the UI-enabled GSM firmware running on that board, and only then proceed +with C139 and/or Pirelli ports if such are still desired. + +For now, modem or pseudo-modem operation with control via AT commands is all +we have. + +Build system +============ + +Even though FC Magnetite is essentially unchanged TCS211 code base and builds +using TI's proprietary TMS470 compiler under Wine, the build system is entirely +new. TI's TCS211 build system, called BuSyB, works by way of a Java tool +generating a customized makefile for each desired build configuration, based on +lots of magic contained in a big repository of XML files. There also a bunch +of Perl scripts involved. The Java tool that does the heavy lifting exists +only as compiled Java bytecode sans source, and the surrounding Perl scripts +aren't very understandable either. And the whole thing thoroughly assumes a +Windows environment (drive letters, backslashes, case-insensitive file system) +throughout. As a result, when working with TCS211 fw with its original build +system, we had to treat these BuSyB-generated makefiles almost as being blobs in +themselves: regenerating a makefile from XML magic required major effort, there +were some bugs in the makefile generation which we couldn't fix and thus we had +to edit the makefiles manually after each regeneration - it was an utter mess, +and absolutely not an acceptable way to build a forward-looking, community- +serving project. + +In FC Magnetite I have recreated the relevant parts of the TCS211 build system, +using Bourne shell magic instead of Java and XML. Just like TI's BuSyB, ours +is a makefile generation system: in order to compile the firmware in a +particular desired configuration, you run a shell script to select the config +you would like. This shell script will create a dedicated build directory tree +to fully contain this build, and populate it with generated Makefiles and some +other bits - then you go into the just-created build directory and run make +there. The source and build trees are thus cleanly separated. See +doc/Compiling for detailed instructions. + +Another key difference from our previous TCS211-based firmware offerings is that +even though we still have to run TI's compiler binaries under Wine, the Wine +invokation has been moved from the top (root) of the build process to the +bottom leaves. With our previous TCS211-based works you would run Wine at the +top, and then the entire build process would proceed in the Windows environment, +using Windows versions of make and other nonsense. Not so in FC Magnetite: +in this firmware project all shell scripts, Makefiles, Perl scripts and other +build system accessories run at the native Unix level, and Wine is only invoked +at the lowest level by individual tool wrappers: for example, TI's compiler +binary cl470.exe is encapsulated in a Unix shell script called cl470 that +invokes Wine to run the Windows binary, presenting the illusion of a native +Unix tool to all upper levels. + +As yet another defenestration measure, all source files are checked into this +tree with Unix line endings. + +Blob status +=========== + +A long-term FreeCalypso goal is to have our phone/modem firmware rebuild fully +from source without any blobs, but this goal has not been achieved yet. While +we do have what *seems* to be a suitable replacement source (or feasible ability +to reconstruct such) for every piece of TCS211 fw that came in binary-only form, +actually making this replacement without breaking functionality is a very +non-trivial endeavor. Our previous attempt to rebuild the firmware from the +ground up, using source pieces lifted from different available leaks and +building with gcc so that no TMS470 COFF blobs could be used, produced very +disappointing results. + +Instead the new FreeCalypso firmware approach implemented in FC Magnetite is to +approach the blob-free goal incrementally. The new Magnetite build system is +specifically designed to enable the transition from the use of blobs to +recompilation from source to be made with very fine granularity, down to the +level of individual object modules within libs if necessary. We tackle one +binary-only component at a time, either reconstructing the missing source from +disassembly or adapting the source from a different version as works best in +each individual case, and we make a test build of the firmware using the +reconstructed or fitted component instead of the original blob. If the firmware +still works (doesn't break), we make this deblobbing transition permanent and +move on to the next component. + +As of this writing, most of Layer 1 and a few housekeeping parts of the fw have +already been deblobbed, i.e., are now recompiled from source. The G23M protocol +stack is our next deblobbing target - we have a newer version of it in full +source form, and we are hoping to be able to retrofit this newer G23M version +into the TCS211 environment in order to replace the binary blob version we are +using currently. + +Further reading +=============== + +For various instructions and notes specific to this FreeCalypso Magnetite +firmware, look in the doc directory. For more information about the overall +FreeCalypso project and our hardware building aspirations, go to our website: + +https://www.freecalypso.org/
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/C139-Howto Mon Oct 03 04:26:16 2016 +0000 @@ -0,0 +1,146 @@ +Running FreeCalypso Magnetite firmware on the Motorola C139 +=========================================================== + +Mot C139 phones are brickable - because the Calypso boot ROM is disabled by PCB +wiring, the ability to reflash a phone with new firmware critically depends on +there being a particular kind of boot code in flash sector 0 at all times - a +particular kind of boot code that allows the boot process to be interrupted and +diverted to external code loaded via the headset jack serial port. + +The FreeCalypso project has adopted one specific version of the flash sector 0 +boot code (produced by applying a binary patch to one of Compal/Motorola's +original versions) for use with all of our firmwares for this target. No matter +which FreeCalypso firmware you are running - Citrine, Magnetite or tcs211-c139 - +you flash your FC fw image at offset 0x10000 while keeping this boilerplate boot +code at the beginning of the flash: + +ftp://ftp.freecalypso.org/pub/GSM/FreeCalypso/compal-flash-boot-for-fc.bin + +Because neither FC Citrine nor FC Magnetite implements any phone UI or puts +anything at all on the LCD, when a C139 phone is flashed with one of our +firmwares, it will behave very oddly: + +* Whenever the phone is off but the battery is inserted, even a momentary + accidental press of the power button will launch a full power-on and firmware + boot - without any visible indication whatsoever as the LCD stays dark! + +* Once the firmware has booted from a press of the power button - even a + momentary accidental press - there is no way to make it shut down and power + off except by sending a power-off command via the headset jack serial port. + So it will just keep running until the battery runs down, once again with the + LCD dark and no visible indication of any kind that it's on. + +Additional considerations are: + +* Flashing a given phone back and forth between FreeCalypso and Mot/Compal's + official firmware is a royal pita, so if you are going to play with + FreeCalypso on a C139, it would be the easiest to dedicate a phone + specifically for FC experiments; + +* We haven't got firmware-controlled battery charging working yet, so you will + need another phone running one of the official fw versions to charge + batteries. + +Converting a phone to FreeCalypso +================================= + +Start by installing FreeCalypso host tools on your PC/laptop or whatever host +you will use to talk to C139 phones, if you haven't already. If you are +starting with an unhacked C139 phone running one of the official firmware +versions, the procedure for flashing and bringing up FreeCalypso for the first +time is as follows: + +* Note down your phone's factory IMEI. After you get FreeCalypso firmware + flashed and running, you will need to set your own IMEISV, as our fw doesn't + know how to grok Mot/Compal's flash data structures where they store theirs. + You can set whatever IMEISV you like, but if you would like to keep the + factory one, it would be the easiest to have it noted down on a piece of + paper. If you have a labelmaker, you can print a sticky label with the IMEI + and stick it on the side of the phone where you can easily see it later while + playing with FreeCalypso. + +* Get in with fc-loadtool, preceded with tfc139 if necessary - see FC host tools + documentation. + +* Once you are in with fc-loadtool, i.e., at the loadtool> prompt, reflash the + boot sector with the FreeCalypso version: + +loadtool> flash erase-program-boot compal-flash-boot-for-fc.bin + +* Flash the FC Magnetite firmware image you have compiled: + +loadtool> flash erase 0x10000 0x230000 +loadtool> flash program-bin 0x10000 fwimage.bin + +* Erase the flash sectors to be used for the FFS (flash file system) by + FreeCalypso firmwares: + +loadtool> flash erase 0x3C0000 0x30000 + +* Exiting fc-loadtool cleanly will cause it to power off the phone: + +loadtool> exit + +Reflashing between different FreeCalypso firmwares +================================================== + +By the conventions established in the FreeCalypso project, all of our firmwares +for the C139 target have the following in common: + +* They all stay out of the boot sector and expect to receive control from the + boot code in the same manner (boot entry point at 0x10058, exception vectors + at 0x10000), thus there is no need to reflash the dangerous boot sector when + going from one FC firmware to another. + +* They all use the same aftermarket FFS configuration of 3 sectors of 64 KiB + each (64x3) at 0x3C0000. This FFS location is deliberately different from + the one used by Mot/Compal's firmwares, eliminating the possibility of one fw + trying to use the FFS created by the other, and by putting our FFS toward the + end of the flash we maximize the amount of flash space available for our + firmware code images. But even though we don't share our FFS with + Mot/Compal's official firmwares, we do share the same FFS between all of + FreeCalypso firmware projects - thus once you have initialized your FFS (see + below) with one FC firmware version, it will work with the others as well. + +If you need to reflash your C139 from one FC firmware version to another, +simply get in with fc-loadtool -h compal (no more need for the inefficient +-c 1003 or -c 1004 options or for tfc139) and reflash just the fw image part: + +loadtool> flash erase 0x10000 0x230000 +loadtool> flash program-bin 0x10000 fwimage.bin + +First boot of the firmware +========================== + +Connect the serial cable, but instead of running fc-loadtool, run rvinterf. +Press the red power button on the phone briefly just like you would for +fc-loadtool entry. Because there is no fc-loadtool running on the host end of +the serial cable, the boot path will *not* be diverted in the bootloader, and +the main fw image will run - and this time it will be the FreeCalypso firmware +you have compiled and flashed. The phone's LCD will remain dark as there is no +LCD driver code in this firmware, but you will see trace output in the rvinterf +window, telling you that the fw is running. + +Before you do anything else, you will need to run fc-fsio and initialize the +aftermarket FFS for our firmware: + +fsio> format / +fsio> mk-std-dirs +fsio> set-imeisv fc XXXXXXXX-YYYYYY-ZZ (punctuation optional, place anywhere) +fsio> set-rfcap dual-eu (if you have 900+1800 MHz hardware) +or +fsio> set-rfcap dual-us (if you have 850+1900 MHz hardware) + +After you've initialized your FFS as above, you can exit fc-fsio, run fc-shell +and try some AT commands: + +AT+CMEE=2 -- enable verbose error responses +AT+CFUN=1 -- enable radio and SIM interfaces +AT+COPS=0 -- register to the default GSM network + +When you are done, you can power the phone off by sending a 'poweroff' command +through fc-shell. The only other way is to yank the battery, and doing the +latter is recommended anyway: when a phone with the present hack-firmware +flashed into it is powered off but still has the battery inserted, even a +momentary accidental press of the power button will cause it to power on and +boot, but there will be absolutely no visual indication, as the LCD stays dark.
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/Compiling Mon Oct 03 04:26:16 2016 +0000 @@ -0,0 +1,108 @@ +Preparing the development and build environment +=============================================== + +In order to compile our FreeCalypso Magnetite firmware, you will need a +Unix/Linux system. Even though we are using a compiler which we got in the +form of Windows .exe binaries and thus have to use Wine (see below), everything +that we have built on top of it is Unix-based. This author uses Slackware Linux +version 13.37, 32-bit. + +You will need to install the following four pieces of software on whatever +machine you will use to run the FC Magnetite build process: + +1. Wine: self-explanatory. + +2. FreeCalypso Wine environment: + + ftp://ftp.freecalypso.org/pub/GSM/TI_src/wine/installed-env.tar.xz + + Extract the content of the above tarball into your ~/.wine/drive_c + directory - that's all there is to it! + +3. nowhine wrapper around Wine: + + ftp://ftp.freecalypso.org/pub/GSM/TI_src/wine/nowhine.c + + Note that Wine may produce different whines on your system than it + does on mine, in which case if you wish to be relieved of those + whines, you'll need to edit my nowhine.c hack for your situation. + +4. mokosrec2bin flash image file format conversion utility: + + ftp://ftp.freecalypso.org/pub/GSM/GTA02/gsm-fw/mokosrec2bin.c + +Note that the four host software pieces above are exactly the same as what has +been needed to build our previous TCS211-based fw works such as leo2moko-debug +and tcs211-c139 - thus if you have built those previously, you should already +have all of the necessary host tools. + +Compiling the local helper utilities +==================================== + +(cd helpers; make) + +Do the above. Most of the build helper scripts used in the FC Magnetite build +system are written in Bourne shell, but a few were easier to implement in C. +You need to compile these C helper utilities before you can run an actual FC +Magnetite firmware build, but these utilities are totally ad hoc and specific +to the needs of our fw build system, hence they are not meant to be installed +globally on your system - instead they stay within the fc-magnetite tree. You +just need to run make in the helpers directory once before any actual firmware +builds. + +Actually building the firmware +============================== + +In order to build our FreeCalypso Magnetite firmware for a particular target in +a particular configuration, run a command like this from the top level of the +fc-magnetite tree: + +./configure.sh gtamodem l1reconst + +The first argument to the configure.sh script selects the target, and the second +argument selects the configuration. As of this writing, the following targets +are supported: + +c139 Motorola C139 +fcdev3b FreeCalypso FCDEV3B (hardware designed but not built yet) +gtamodem The Calypso GSM/GPRS modem in Openmoko GTA01/02 smartphones +pirelli Pirelli DP-L10 + +For the available configurations (the second required argument to the configure +script), look in the configs directory. As of this writing, the most +interesting configuration is l1reconst - it was named so because it rebuilds L1 +from the reconstructed source. + +Each configuration is built in its own directory; by default the build directory +is named build-$TARGET-$CONFIG, i.e., for the example configure.sh line above, +the resulting build directory will be build-gtamodem-l1reconst. You can change +the name of this directory by appending a BUILD_DIR=dir argument to the +./configure.sh line after the two required arguments. + +To actually compile the firmware, cd into the created build directory and run +make there. Unfortunately the use of TI's proprietary compiler via Wine makes +the build very slow - it takes about 42 minutes on my machine. When it's all +done, the flashable firmware image will be in fwimage.bin. This image is to be +flashed with fc-loadtool at address 0x10000 on the C139 and at address 0 on all +other targets. + +When building firmware for the Pirelli or for future FreeCalypso hardware that +will use the same high capacity flash+pSRAM chip, one can build either a +flashable image or a RAM-loadable one - or both. Because this part of the +build system is common with other targets for which only flash images can be +produced, the Makefile always builds the flashable image by default - +fwimage.bin is always meant for flash and never for RAM. To build a RAM- +loadable image when the target allows it, run 'make ram' - the image will be in +ramimage.srec, which you can then load and run on the target with FreeCalypso +host tool fc-xram. + +Running on the hardware +======================= + +In order to run the firmware you have built on your Calypso phone or modem +(flash or run in RAM as appropriate), you will need to use FreeCalypso host +tools. As of this writing, the latest packaged release is this one: + +ftp://ftp.freecalypso.org/pub/GSM/FreeCalypso/fc-host-tools-r4.tar.bz2 + +Please see target-specific notes for more details.
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/Freerunner-Howto Mon Oct 03 04:26:16 2016 +0000 @@ -0,0 +1,12 @@ +Running FreeCalypso Magnetite firmware on the Neo Freerunner +============================================================ + +Because FC Magnetite is essentially the same code as TCS211, it should function +identically to the standard moko12 (leo2moko-r1) firmware. If you would like +to try it, compile the fw image as described in the Compiling write-up and then +flash it with fc-loadtool like this: + +loadtool> flash erase 0 0x230000 +loadtool> flash program-bin 0 fwimage.bin + +That's all there is to it, folks - not much more to write here.