+
−FreeCalypso Magnetite firmware project
+
−======================================
+
−
+
−As of A.D. 2018, FC Magnetite is the primary official Calypso firmware source
+
−tree for the FreeCalypso family of projects.  This maintained and evolving
+
−source can be built in many different configurations for several different
+
−hardware targets, and is currently used in the following ways:
+
−
+
−* The official production firmwares for our fully supported GSM+GPRS modem
+
−  products (FCDEV3B and the modem part of Openmoko devices) are built from
+
−  this source tree.  See doc/Modem-configs for this mode of usage.
+
−
+
−* The work in progress toward a future complete FreeCalypso phone handset
+
−  (also known as a Libre Dumbphone) is being done in this source tree.
+
−  See doc/Handset-goal for more info about this project direction.
+
−
+
−* One can play with FreeCalypso and get a taste for it using certain hardware
+
−  that used to be readily available and which most people in the phone hacking
+
−  scene already have: Motorola C1xx and Pirelli DP-L10.  Firmware images of the
+
−  "hacking toy" sort can be built for these devices from the present source
+
−  tree, and one can exercise most of the functions and capabilities of
+
−  FreeCalypso (with the notable exception of CSD and GPRS) on this historical
+
−  but available-to-most-people hardware.  See doc/C1xx-Howto, doc/Pirelli-Howto
+
−  and doc/Voice-pseudo-modem for more information.
+
−
+
−* If anyone besides the Mother has an original TI-made D-Sample hardware kit
+
−  for Calypso software development, we are doing the best we can to support
+
−  running our FreeCalypso firmware on this original TI hardware - see
+
−  doc/D-Sample for more information.
+
−
+
−Functionality
+
−=============
+
−
+
−TI's GSM mobile station firmware architecture supports two ways in which the
+
−GSM device may be controlled: via AT commands from an external host and/or via
+
−a local UI on devices with LCD & keypad hardware.  (I said "and/or" because the
+
−two mechanisms can coexist.)  The code we got from TI (TCS211 and TCS3.2) is
+
−very solid and mature in the modem configuration (control via AT commands only,
+
−no UI, no battery management, no traditional handset on/off control), but the
+
−additional code layers that are needed for handset products but not for modems
+
−came in a very rough "proof of concept" condition, nowhere close to a usable
+
−product.
+
−
+
−In FC Magnetite we further maintain and support the solid code base we got for
+
−the modem functionality, and we are also working to improve the support for
+
−handset products and bring it into a practically usable state.  At the present
+
−time the handset firmware work is on hold, paused until we design and build the
+
−first prototype version of our own FreeCalypso handset hardware.  The reason
+
−for this hold is that we are not able to use TI's original development board
+
−(see doc/D-Sample for the issues), hence we need to build our own hardware that
+
−will serve in its place.  Until then we focus on maintaining and supporting the
+
−good working modem functionality.
+
−
+
−Both TI's original modem fw (TCS211) and our recreation thereof in this
+
−Magnetite project support not only voice calls and SMS, but also CSD, fax and
+
−GPRS.  This advanced functionality is fully supported on our own GSM MS
+
−development board (FCDEV3B), where both Calypso UARTs are presented directly to
+
−the developer, as well as on the embedded Calypso modem in Openmoko GTA01/02
+
−smartphones, where the AT command channel with CSD, fax and GPRS capabilities
+
−is connected to the phone's application processor.
+
−
+
−Build system
+
−============
+
−
+
−Even though FC Magnetite is essentially unchanged TCS211 code base and builds
+
−using TI's proprietary TMS470 compiler under Wine, the build system is entirely
+
−new.  TI's TCS211 build system, called BuSyB, works by way of a Java tool
+
−generating a customized makefile for each desired build configuration, based on
+
−lots of magic contained in a big repository of XML files.  There are a bunch of
+
−Perl scripts involved as well.  The Java tool that does the heavy lifting exists
+
−only as compiled Java bytecode sans source, and the surrounding Perl scripts
+
−aren't very understandable either.  And the whole thing thoroughly assumes a
+
−Windows environment (drive letters, backslashes, case-insensitive file system)
+
−throughout.  As a result, when working with TCS211 fw with its original build
+
−system, we had to treat these BuSyB-generated makefiles almost as being blobs in
+
−themselves: regenerating a makefile from XML magic required major effort, there
+
−were some bugs in the makefile generation which we couldn't fix and thus we had
+
−to edit the makefiles manually after each regeneration - it was an utter mess,
+
−and absolutely not an acceptable way to build a forward-looking, community-
+
−serving project.
+
−
+
−In FC Magnetite I have recreated the relevant parts of the TCS211 build system,
+
−using Bourne shell magic instead of Java and XML.  Just like TI's BuSyB, ours
+
−is a makefile generation system: in order to compile the firmware in a
+
−particular desired configuration, you run a shell script to select the config
+
−you would like.  This shell script will create a dedicated build directory tree
+
−to fully contain this build, and populate it with generated Makefiles and some
+
−other bits - then you go into the just-created build directory and run make
+
−there.  The source and build trees are thus cleanly separated.  See
+
−doc/Compiling for detailed instructions.
+
−
+
−Another key difference from our previous TCS211-based firmware offerings is that
+
−even though we still have to run TI's compiler binaries under Wine, the Wine
+
−invokation has been moved from the top (root) of the build process to the
+
−bottom leaves.  With our previous TCS211-based works you would run Wine at the
+
−top, and then the entire build process would proceed in the Windows environment,
+
−using Windows versions of make and other nonsense.  Not so in FC Magnetite:
+
−in this firmware project all shell scripts, Makefiles, Perl scripts and other
+
−build system accessories run at the native Unix level, and Wine is only invoked
+
−at the lowest level by individual tool wrappers: for example, TI's compiler
+
−binary cl470.exe is encapsulated in a Unix shell script called cl470 that
+
−invokes Wine to run the Windows binary, presenting the illusion of a native
+
−Unix tool to all upper levels.
+
−
+
−As yet another defenestration measure, all source files are checked into this
+
−tree with Unix line endings.
+
−
+
−Two versions of the G23M protocol stack
+
−=======================================
+
−
+
−A major component of all functional firmware configurations is the mass of code
+
−that implements layers 2 and 3 of the GSM+GPRS protocol stack, called G23M.  In
+
−FC Magnetite we have the option of using one of two different versions of this
+
−key firmware component:
+
−
+
−* The original G23M version from Openmoko: this version is believed to be very
+
−  stable because it has been used (successfully to the best of our knowledge)
+
−  by Openmoko, and it came from TI's TCS211 program that officially targeted
+
−  the Calypso chipset - but it exists only as binary object libraries with no
+
−  corresponding source.  The lack of source means no ability to change the
+
−  feature configuration (what is enabled and what is disabled), very difficult
+
−  to debug if something does go wrong - all of the usual problems of software
+
−  without source.
+
−
+
−* A newer version from TI's TCS3/LoCosto program.  This version came to us in
+
−  the form of full C source, and because this fw component is chipset-
+
−  independent (unlike L1), we have successfully produced a TCS2/TCS3 hybrid in
+
−  which the new G23M code from TCS3.2 is grafted onto the chipsetsw foundation
+
−  from TCS211, which we have reconstructed into full C source form as well.
+
−
+
−Each of the two G23M PS versions also has its own version of the Application
+
−Control Interface (ACI) layer to go with it, which we call aci2 and aci3 -
+
−except that in the case of ACI, we have the full source for both versions.
+
−Furthermore, in the case of firmware builds that have the phone UI layers
+
−enabled (BMI and MFW, sitting on top of ACI), we likewise have different TCS2
+
−and TCS3 versions of these BMI and MFW layers, also with full source for both
+
−versions.  Thus the chipsetsw foundation is always the same TCS211, but then we
+
−bifurcate: we can build either a pure TCS211 config using TCS211 versions of
+
−G23M PS (blobs), ACI (source) and MFW+BMI (source), or a TCS2/TCS3 hybrid config
+
−using the new TCS3 versions of G23M PS, ACI and MFW+BMI, all in full C source
+
−form.
+
−
+
−The new TCS2/TCS3 hybrid config is the way forward, as our goal is to have no
+
−blobs in our firmware - having the full source is a prerequisite for maintaining
+
−a software product.  FC Magnetite supports building both configurations in order
+
−to facilitate the transition.  As of this writing, the hybrid config is believed
+
−to have reached feature parity with Openmoko-mimicking classic and l1reconst
+
−configs, i.e., unless we missed or broke something, every feature that worked
+
−correctly on Openmoko's original modem should work just as well with our hybrid
+
−firmware.  All functionality (both standard and OM-custom) that can be tested
+
−in our lab appears to work flawlessly; the new hybrid modem fw now needs to be
+
−tested by a larger number of users in real-life usage.
+
−
+
−Other blobs
+
−===========
+
−
+
−Our TCS2/TCS3 hybrid firmware is built almost entirely from source; the only
+
−components which are linked in the form of prebuilt objects or libraries are
+
−certain glue components in GPF (specifically OSL and OSX), the Nucleus PLUS RTOS
+
−and the TMS470 compiler's equivalent of libc/libgcc.  In the case of Nucleus we
+
−have another version of it in full source form, but switching to a different
+
−version of Nucleus is outside the scope of FC Magnetite - see the next section.
+
−In the case of OSL and OSX, we have a reconstructed source and one can build a
+
−Magnetite config (hybrid-osl) that uses these reconstructed versions of OSL and
+
−OSX instead of the blobs, but we are not 100% confident in the quality of this
+
−reconstruction, hence this hybrid-osl config is currently considered
+
−experimental, not production.
+
−
+
−Blob-free, gcc-built version
+
−============================
+
−
+
−For those who require or desire a firmware version that is completely free of
+
−any blobs and builds with a FLOSS compiler, we now have a new experimental
+
−firmware project called FreeCalypso Selenite:
+
−
+
−https://www.freecalypso.org/hg/fc-selenite/
+
−
+
−FC Selenite is based on FC Magnetite (just the modem functionality, no work-in-
+
−progress phone UI, and only the hybrid code version), but it uses the new
+
−full-source version of Nucleus and our reconstructed versions of OSL and OSX,
+
−and it offers the option of building either with the original TMS470 compiler
+
−or with gcc.  The gcc-built version also uses entirely new implementations of
+
−all assembly components and linker script magic, as the two toolchain
+
−environments are entirely different in this regard.
+
−
+
−This new Selenite firmware is expected to fully supplant our earlier and now
+
−completely unmaintained Citrine fw (our first attempt at gcc-built fw), but it
+
−is still highly experimental: as of this writing, GPRS does not work at all in
+
−Selenite-gcc (Citrine does not have any GPRS support either), and even the
+
−GSM-only config with GPRS disabled exhibits some intermittent glitches.  Thus
+
−the stable production fw is still TMS470-built FC Magnetite for the time being.
+
−
+
−Further reading
+
−===============
+
−
+
−For various instructions and notes specific to this FreeCalypso Magnetite
+
−firmware, look in the doc directory.  For more information about the overall
+
−FreeCalypso project and our hardware building aspirations, go to our website:
+
−
+
−https://www.freecalypso.org/