FreeCalypso Magnetite firmware project
======================================

As of A.D. 2018, FC Magnetite is the primary official Calypso firmware source
tree for the FreeCalypso family of projects.  This maintained and evolving
source can be built in many different configurations for several different
hardware targets, and is currently used in the following ways:

* The official production firmwares for our fully supported GSM+GPRS modem
  products (FCDEV3B and the modem part of Openmoko devices) are built from
  this source tree.  See doc/Modem-configs for this mode of usage.

* The work in progress toward a future complete FreeCalypso phone handset
  (also known as a Libre Dumbphone) is being done in this source tree.
  See doc/Handset-goal for more info about this project direction.

* One can play with FreeCalypso and get a taste for it using certain hardware
  that used to be readily available and which most people in the phone hacking
  scene already have: Motorola C1xx and Pirelli DP-L10.  Firmware images of the
  "hacking toy" sort can be built for these devices from the present source
  tree, and one can exercise most of the functions and capabilities of
  FreeCalypso (with the notable exception of CSD and GPRS) on this historical
  but available-to-most-people hardware.  See doc/C1xx-Howto, doc/Pirelli-Howto
  and doc/Voice-pseudo-modem for more information.

* If anyone besides the Mother has an original TI-made D-Sample hardware kit
  for Calypso software development, we are doing the best we can to support
  running our FreeCalypso firmware on this original TI hardware - see
  doc/D-Sample for more information.

Functionality
=============

TI's GSM mobile station firmware architecture supports two ways in which the
GSM device may be controlled: via AT commands from an external host and/or via
a local UI on devices with LCD & keypad hardware.  (I said "and/or" because the
two mechanisms can coexist.)  The code we got from TI (TCS211 and TCS3.2) is
very solid and mature in the modem configuration (control via AT commands only,
no UI, no battery management, no traditional handset on/off control), but the
additional code layers that are needed for handset products but not for modems
came in a very rough "proof of concept" condition, nowhere close to a usable
product.

In FC Magnetite we further maintain and support the solid code base we got for
the modem functionality, and we are also working to improve the support for
handset products and bring it into a practically usable state.  At the present
time the handset firmware work is on hold, paused until we design and build the
first prototype version of our own FreeCalypso handset hardware.  The reason
for this hold is that we are not able to use TI's original development board
(see doc/D-Sample for the issues), hence we need to build our own hardware that
will serve in its place.  Until then we focus on maintaining and supporting the
good working modem functionality.

Both TI's original modem fw (TCS211) and our recreation thereof in this
Magnetite project support not only voice calls and SMS, but also CSD, fax and
GPRS.  This advanced functionality is fully supported on our own GSM MS
development board (FCDEV3B), where both Calypso UARTs are presented directly to
the developer, as well as on the embedded Calypso modem in Openmoko GTA01/02
smartphones, where the AT command channel with CSD, fax and GPRS capabilities
is connected to the phone's application processor.

Build system
============

Even though FC Magnetite is essentially unchanged TCS211 code base and builds
using TI's proprietary TMS470 compiler under Wine, the build system is entirely
new.  TI's TCS211 build system, called BuSyB, works by way of a Java tool
generating a customized makefile for each desired build configuration, based on
lots of magic contained in a big repository of XML files.  There are a bunch of
Perl scripts involved as well.  The Java tool that does the heavy lifting exists
only as compiled Java bytecode sans source, and the surrounding Perl scripts
aren't very understandable either.  And the whole thing thoroughly assumes a
Windows environment (drive letters, backslashes, case-insensitive file system)
throughout.  As a result, when working with TCS211 fw with its original build
system, we had to treat these BuSyB-generated makefiles almost as being blobs in
themselves: regenerating a makefile from XML magic required major effort, there
were some bugs in the makefile generation which we couldn't fix and thus we had
to edit the makefiles manually after each regeneration - it was an utter mess,
and absolutely not an acceptable way to build a forward-looking, community-
serving project.

In FC Magnetite I have recreated the relevant parts of the TCS211 build system,
using Bourne shell magic instead of Java and XML.  Just like TI's BuSyB, ours
is a makefile generation system: in order to compile the firmware in a
particular desired configuration, you run a shell script to select the config
you would like.  This shell script will create a dedicated build directory tree
to fully contain this build, and populate it with generated Makefiles and some
other bits - then you go into the just-created build directory and run make
there.  The source and build trees are thus cleanly separated.  See
doc/Compiling for detailed instructions.

Another key difference from our previous TCS211-based firmware offerings is that
even though we still have to run TI's compiler binaries under Wine, the Wine
invokation has been moved from the top (root) of the build process to the
bottom leaves.  With our previous TCS211-based works you would run Wine at the
top, and then the entire build process would proceed in the Windows environment,
using Windows versions of make and other nonsense.  Not so in FC Magnetite:
in this firmware project all shell scripts, Makefiles, Perl scripts and other
build system accessories run at the native Unix level, and Wine is only invoked
at the lowest level by individual tool wrappers: for example, TI's compiler
binary cl470.exe is encapsulated in a Unix shell script called cl470 that
invokes Wine to run the Windows binary, presenting the illusion of a native
Unix tool to all upper levels.

As yet another defenestration measure, all source files are checked into this
tree with Unix line endings.

Two versions of the G23M protocol stack
=======================================

A major component of all functional firmware configurations is the mass of code
that implements layers 2 and 3 of the GSM+GPRS protocol stack, called G23M.  In
FC Magnetite we have the option of using one of two different versions of this
key firmware component:

* The original G23M version from Openmoko: this version is believed to be very
  stable because it has been used (successfully to the best of our knowledge)
  by Openmoko, and it came from TI's TCS211 program that officially targeted
  the Calypso chipset - but it exists only as binary object libraries with no
  corresponding source.  The lack of source means no ability to change the
  feature configuration (what is enabled and what is disabled), very difficult
  to debug if something does go wrong - all of the usual problems of software
  without source.

* A newer version from TI's TCS3/LoCosto program.  This version came to us in
  the form of full C source, and because this fw component is chipset-
  independent (unlike L1), we have successfully produced a TCS2/TCS3 hybrid in
  which the new G23M code from TCS3.2 is grafted onto the chipsetsw foundation
  from TCS211, which we have reconstructed into full C source form as well.

Each of the two G23M PS versions also has its own version of the Application
Control Interface (ACI) layer to go with it, which we call aci2 and aci3 -
except that in the case of ACI, we have the full source for both versions.
Furthermore, in the case of firmware builds that have the phone UI layers
enabled (BMI and MFW, sitting on top of ACI), we likewise have different TCS2
and TCS3 versions of these BMI and MFW layers, also with full source for both
versions.  Thus the chipsetsw foundation is always the same TCS211, but then we
bifurcate: we can build either a pure TCS211 config using TCS211 versions of
G23M PS (blobs), ACI (source) and MFW+BMI (source), or a TCS2/TCS3 hybrid config
using the new TCS3 versions of G23M PS, ACI and MFW+BMI, all in full C source
form.

The new TCS2/TCS3 hybrid config is the way forward, as our goal is to have no
blobs in our firmware - having the full source is a prerequisite for maintaining
a software product.  FC Magnetite supports building both configurations in order
to facilitate the transition.  As of this writing, the hybrid config is believed
to have reached feature parity with Openmoko-mimicking classic and l1reconst
configs, i.e., unless we missed or broke something, every feature that worked
correctly on Openmoko's original modem should work just as well with our hybrid
firmware.  All functionality (both standard and OM-custom) that can be tested
in our lab appears to work flawlessly; the new hybrid modem fw now needs to be
tested by a larger number of users in real-life usage.

Other blobs
===========

Our TCS2/TCS3 hybrid firmware is built almost entirely from source; the only
components which are linked in the form of prebuilt objects or libraries are
certain glue components in GPF (specifically OSL and OSX), the Nucleus PLUS RTOS
and the TMS470 compiler's equivalent of libc/libgcc.  In the case of Nucleus we
have another version of it in full source form, but switching to a different
version of Nucleus is outside the scope of FC Magnetite - see the next section.
In the case of OSL and OSX, we have a reconstructed source and one can build a
Magnetite config (hybrid-osl) that uses these reconstructed versions of OSL and
OSX instead of the blobs, but we are not 100% confident in the quality of this
reconstruction, hence this hybrid-osl config is currently considered
experimental, not production.

Blob-free, gcc-built version
============================

For those who require or desire a firmware version that is completely free of
any blobs and builds with a FLOSS compiler, we now have a new experimental
firmware project called FreeCalypso Selenite:

https://www.freecalypso.org/hg/fc-selenite/

FC Selenite is based on FC Magnetite (just the modem functionality, no work-in-
progress phone UI, and only the hybrid code version), but it uses the new
full-source version of Nucleus and our reconstructed versions of OSL and OSX,
and it offers the option of building either with the original TMS470 compiler
or with gcc.  The gcc-built version also uses entirely new implementations of
all assembly components and linker script magic, as the two toolchain
environments are entirely different in this regard.

This new Selenite firmware is expected to fully supplant our earlier and now
completely unmaintained Citrine fw (our first attempt at gcc-built fw), but it
is still highly experimental: as of this writing, GPRS does not work at all in
Selenite-gcc (Citrine does not have any GPRS support either), and even the
GSM-only config with GPRS disabled exhibits some intermittent glitches.  Thus
the stable production fw is still TMS470-built FC Magnetite for the time being.

Further reading
===============

For various instructions and notes specific to this FreeCalypso Magnetite
firmware, look in the doc directory.  For more information about the overall
FreeCalypso project and our hardware building aspirations, go to our website:

https://www.freecalypso.org/