+
−FreeCalypso Magnetite firmware project
+
−======================================
+
−
+
−This source tree contains yet another firmware offering created under the
+
−FreeCalypso umbrella.  The key qualities of this firmware offering are:
+
−
+
−Negatives:
+
−
+
−  * Builds with TI's proprietary TMS470 compiler and thus requires Wine;
+
−  * Some of the fw components are still in the form of binary blobs -
+
−    but see below on the deblobbing progress.
+
−
+
−Positives:
+
−
+
−  * Officially supports our own FCDEV3B and Openmoko GTA01/02 modem targets,
+
−    as well as "hack-only" support for Motorola C139 and Pirelli DP-L10,
+
−    all from the same source tree;
+
−
+
−  * Works as solidly as the TCS211 "golden" reference from TI, on all of the
+
−    supported targets - deep sleep works (on non-broken hardware), voice calls
+
−    work in all codec modes including AMR, the DSP dynamic download mechanism
+
−    does its magic, the call audio passes reliably in both directions.
+
−
+
−The present FC Magnetite firmware is built on the principle of starting with
+
−the known working TCS211 code base, without any major restructuring, and making
+
−small incremental evolutionary changes, testing at every step to ensure that
+
−nothing breaks.  It is the direct opposite of the "rebuild from the ground up"
+
−approach taken with our previous Citrine firmware aka "gsm-fw".
+
−
+
−Functionality
+
−=============
+
−
+
−TI's GSM mobile station firmware architecture supports two ways in which the
+
−GSM device may be controlled: via AT commands from an external host and/or via
+
−a local UI on devices with LCD & keypad hardware.  (I said "and/or" because the
+
−two mechanisms can coexist.)  The code we got from TI (TCS211) is very solid
+
−and mature in the modem configuration (control via AT commands only, no UI, no
+
−battery management, no traditional handset on/off control), but the additional
+
−code layers that are needed for handset products but not for modems are in a
+
−very rough "proof of concept" condition, nowhere close to a usable product.
+
−
+
−The same situation currently exists in FreeCalypso Magnetite.  As explained
+
−below in the Build system section, a key feature of FC Magnetite is that many
+
−different firmware configurations can be built from the same source tree.  One
+
−of the configuration choices is that you can build the fw either with or without
+
−the phone UI layers.  The work we have done earlier in the tcs211-c139 side
+
−project (late 2015) has been integrated into Magnetite, properly conditionalized
+
−so that TI's original configuration is fully preserved when the target != c139.
+
−However, if you build the UI-enabled configuration for any target other than
+
−Mot C139, it will compile, but it won't do anything useful: it will try to
+
−display TI's 176x220 pixel color UI on the D-Sample LCD on Calypso chip select
+
−nCS3, but this hardware doesn't exist on any of our supported targets.  (The
+
−Pirelli in particular has flash on that chip select instead, so it may interfere
+
−with FFS operation.)
+
−
+
−Further work on the handset firmware configuration (UI, battery management and
+
−other currently missing functionality required for a usable phone) will have to
+
−wait until we build our own Handset Motherboard Prototype (HSMBP) with a 176x220
+
−pixel 16-bit color LCD, replicating TI's D-Sample - I personally am not too
+
−interested in doing this handset fw work on the very crippled Mot C139 or on
+
−the Pirelli with its own host of issues, although others in the community are
+
−more than welcome to take a shot at it if someone is interested.
+
−
+
−Until then, the primary current focus of the FC Magnetite project is the
+
−AT-command-controlled modem configuration.  Both TI's original modem fw (TCS211)
+
−and our recreation thereof in this Magnetite project support not only voice
+
−calls and SMS, but also CSD, fax and GPRS.  This advanced functionality is
+
−fully supported on our own GSM MS development board (FCDEV3B), where both
+
−Calypso UARTs are presented directly to the developer, as well as on the
+
−embedded Calypso modem in Openmoko GTA01/02 smartphones, where the AT command
+
−channel with CSD, fax and GPRS capabilities is connected to the phone's
+
−application processor.
+
−
+
−Build system
+
−============
+
−
+
−Even though FC Magnetite is essentially unchanged TCS211 code base and builds
+
−using TI's proprietary TMS470 compiler under Wine, the build system is entirely
+
−new.  TI's TCS211 build system, called BuSyB, works by way of a Java tool
+
−generating a customized makefile for each desired build configuration, based on
+
−lots of magic contained in a big repository of XML files.  There are a bunch of
+
−Perl scripts involved as well.  The Java tool that does the heavy lifting exists
+
−only as compiled Java bytecode sans source, and the surrounding Perl scripts
+
−aren't very understandable either.  And the whole thing thoroughly assumes a
+
−Windows environment (drive letters, backslashes, case-insensitive file system)
+
−throughout.  As a result, when working with TCS211 fw with its original build
+
−system, we had to treat these BuSyB-generated makefiles almost as being blobs in
+
−themselves: regenerating a makefile from XML magic required major effort, there
+
−were some bugs in the makefile generation which we couldn't fix and thus we had
+
−to edit the makefiles manually after each regeneration - it was an utter mess,
+
−and absolutely not an acceptable way to build a forward-looking, community-
+
−serving project.
+
−
+
−In FC Magnetite I have recreated the relevant parts of the TCS211 build system,
+
−using Bourne shell magic instead of Java and XML.  Just like TI's BuSyB, ours
+
−is a makefile generation system: in order to compile the firmware in a
+
−particular desired configuration, you run a shell script to select the config
+
−you would like.  This shell script will create a dedicated build directory tree
+
−to fully contain this build, and populate it with generated Makefiles and some
+
−other bits - then you go into the just-created build directory and run make
+
−there.  The source and build trees are thus cleanly separated.  See
+
−doc/Compiling for detailed instructions.
+
−
+
−Another key difference from our previous TCS211-based firmware offerings is that
+
−even though we still have to run TI's compiler binaries under Wine, the Wine
+
−invokation has been moved from the top (root) of the build process to the
+
−bottom leaves.  With our previous TCS211-based works you would run Wine at the
+
−top, and then the entire build process would proceed in the Windows environment,
+
−using Windows versions of make and other nonsense.  Not so in FC Magnetite:
+
−in this firmware project all shell scripts, Makefiles, Perl scripts and other
+
−build system accessories run at the native Unix level, and Wine is only invoked
+
−at the lowest level by individual tool wrappers: for example, TI's compiler
+
−binary cl470.exe is encapsulated in a Unix shell script called cl470 that
+
−invokes Wine to run the Windows binary, presenting the illusion of a native
+
−Unix tool to all upper levels.
+
−
+
−As yet another defenestration measure, all source files are checked into this
+
−tree with Unix line endings.
+
−
+
−Blob status
+
−===========
+
−
+
−A long-term FreeCalypso goal is to have our phone/modem firmware rebuild fully
+
−from source without any blobs, but this goal has not been achieved yet.  While
+
−we do have what *seems* to be a suitable replacement source (or feasible ability
+
−to reconstruct such) for every piece of TCS211 fw that came in binary-only form,
+
−actually making this replacement without breaking functionality is a very
+
−non-trivial endeavor.  Our previous attempt to rebuild the firmware from the
+
−ground up, using source pieces lifted from different available leaks and
+
−building with gcc so that no TMS470 COFF blobs could be used - see FreeCalypso
+
−Citrine - has produced only a very limited subset of the original functionality,
+
−and until very recently even the most basic function of voice calls did not work
+
−reliably.
+
−
+
−Instead the new FreeCalypso firmware approach implemented in FC Magnetite is to
+
−approach the blob-free goal incrementally.  The new Magnetite build system is
+
−specifically designed to enable the transition from the use of blobs to
+
−recompilation from source to be made with very fine granularity, down to the
+
−level of individual object modules within libs if necessary.  We tackle one
+
−binary-only component at a time, either reconstructing the missing source from
+
−disassembly or adapting the source from a different version as works best in
+
−each individual case, and we make a test build of the firmware using the
+
−reconstructed or fitted component instead of the original blob.  If the firmware
+
−still works (doesn't break), we make this deblobbing transition permanent and
+
−move on to the next component.
+
−
+
−See doc/Modem-configs for the current status of the deblobbing effort and for
+
−the description of the currently available configurations.
+
−
+
−Further reading
+
−===============
+
−
+
−For various instructions and notes specific to this FreeCalypso Magnetite
+
−firmware, look in the doc directory.  For more information about the overall
+
−FreeCalypso project and our hardware building aspirations, go to our website:
+
−
+
−https://www.freecalypso.org/