FreeCalypso firmware direction

[Mychaela Falconia]

Hello FreeCalypso community,

This post is a periodic update on our firmware situation, which I feel
is in need of clarification for many of our community members.  On the
one hand, we do have a firmware version that builds with gcc from full
source without any blobs, called FC Citrine - yet I am not doing any
active work on it, and instead am working on our FC Magnetite firmware
which builds with TI's proprietary compiler under Wine and still has
some parts in the form of blobs.  Why so?

Let me begin my answer by reassuring everyone that I do desire to have
a fully functional, production quality firmware built from full source
(no blobs) with a free compiler (no Wine) that can properly replace
(in terms of functionality and stability) our current Wine-built,
blob-laden "official" firmware for Standard Modem functionality.
However, I do not believe that the specific approach taken in what
eventually became Citrine (originally just "FreeCalypso GSM firmware")
is the right way to reach that goal.  Please remember that what
eventually became Citrine was my first attempt at putting together a
libre fw for the Calypso, with key design decisions made back in
2013-2014 when I was just learning my way around TI's firmware
architecture.  Perfection does not always happen on the first try, and
sometimes the right thing to do is to put the first cut aside as a
lesson learned and do the job in a different way.

My current vision is that our hopefully-not-too-distant future fw that
will provide 100% of the Standard Modem functionality with 100%
stability (i.e., zero visible regression relative to the Wine-built,
blob-laden fw that currently serves as our Official modem fw) in a
blob-free and proprietary-tool-free manner will be neither Citrine nor
Magnetite, but will be named after some other gem; my current tentative
planned name for it is Selenite.  FC Selenite will evolve from FC
Magnetite, leaning on the lessons learned from my first fw attempt
that became Citrine.  However, before FC Selenite can become a reality,
certain preliminary steps have to happen first:

* Our current FC Magnetite is not really a single fw offering, but a
  family of fw offerings which are all built in the same source tree.
  The two major modem fw configs we have currently are l1reconst and
  hybrid: l1reconst is our current production config which still has
  the original all-blobs version of the G23M PS from 20070608, and
  hybrid is what is going to be our way forward.

* My planned future FC Selenite fw which will replace Citrine in terms
  of being 100% blob-free and built with gcc is going to evolve from
  the current hybrid config in the Magnetite environment, therefore,
  the latter intermediate step needs to be brought to production-level
  quality and stability before further steps toward the Selenite goal
  can be made.

* At each point in time there is *some* fw version that serves as the
  official production fw for Standard Modem functionality, i.e.,
  officially recommended for the few remaining users of Openmoko
  devices and programmed into newly made FCDEV3B boards (and other FC
  hardware in the future) on the production line.  Right now this
  official production fw is Magnetite l1reconst; it is my hope that
  some day it will become Magnetite hybrid (i.e., the hope is that the
  hybrid config will reach the necessary level of functionality and
  stability), and eventually Selenite.

* Right now my main focus is on hardware work, with the goal of
  building the next batch of FCDEV3B boards, and I am also doing some
  other work in the Magnetite tree toward the goal of producing a
  moko13 fw release for Openmoko devices in the next few days.  After
  this work is done, it will be a good time to work on polishing and
  stabilizing the hybrid config of Magnetite, i.e., tracking down and
  fixing the known bugs, extensive testing to catch those that haven't
  been noticed yet, and adding the missing bits of functionality to
  the TCS3 version of ACI this hybrid config requires.

In the current Magnetite hybrid config the following fw components are
still in the form of blobs rather than rebuilt from source:

* init.obj: the corresponding source init.c has been lost and needs to
  be reconstructed.  This is the part I'm working on right now, and
  the deblobbed init.c will go into both l1reconst and hybrid configs
  of Magnetite, i.e., stable l1reconst users will see it too.  Various
  pieces of this init module have been reconstructed previously back
  in 2013 in my first FC GSM fw attempt (now Citrine), but now I need
  to do this reconstruction more properly.

* GPRS-specific parts of L1: these have never been deblobbed before,
  as there is no GPRS in Citrine.  But I don't expect any major
  difficulty there, as they are L1 modules like the others, and I have
  had very good success with the deblobbing of the rest of L1.  Once
  deblobbed, these parts will also appear in both l1reconst and hybrid
  configs.

* All of GPF except CCD is currently used in the form of TI's prebuilt
  binary libraries; this part of the fw is so stable and config-
  independent that it was distributed and used as prebuilt libs even
  inside TI.  We have what appears to be the exact corresponding source
  for most objects in these libs, but it still remains to be verified
  that this source is indeed 100% corresponding.

* There are some small parts of GPF for which TI withheld the source;
  these parts are OSL and OSX.  Both OSL and OSX have already been
  reconstructed from disassembly back in 2014 and these reconstructed
  C sources are currently used in Citrine (which cannot use any blobs),
  but just like with init.c, I need to revisit this source
  reconstruction and likely fix a few things.

* All Magnetite configs including hybrid use a version of the Nucleus
  PLUS RTOS that exists only as binary libs.  We have a different
  (slightly newer) version of Nucleus PLUS in C source form, and this
  latter version is used in Citrine.  A transition from the blob
  version of Nucleus to the source version will need to happen as part
  of the transition from Magnetite to Selenite *after* all other parts
  above have been deblobbed (and proven solidly working after the
  deblobbing) in the Magnetite environment.

So where does Citrine fit into the picture?  Answer: according to my
current vision, it does not fit anywhere at all *if* one's goal is
100% standard modem functionality, i.e., direct replacement for the
mainstream proprietary modems, including 100% feature parity with
Openmoko's modem as it existed prior to FreeCalypso.  The above is
what I call the Standard Modem functionality, and what we have in
Citrine simply doesn't cut it.

However, not everyone is interested in Standard Modem functionality;
some people are interested quite specifically in doing some very non-
standard applications, having the Calypso modem do something different
from the standard vanilla functionality.  For those non-standard
applications the modular, bottom-up architecture of Citrine may be a
very good fit, and it is my understanding that some of our community
members are already working on some non-standard GSM applications
using Citrine firmware as their starting point.

Which brings me to the following: since my own personal interest is
strictly in the Standard Modem functionality (and eventually a
Pirelli-like Libre Dumbphone for my purse for which the Standard Modem
is a prerequisite), I do not see myself doing any more work on Citrine,
as I've already decided instead on the Magnetite->Selenite direction
as the way toward my goals.  Therefore, if someone else in our
community would like to fully take ownership of the Citrine subproject,
I would be happy to do a formal handover (oh the pun).

Hasta la Victoria, Siempre,
Mychaela aka The Mother