src/g23m-aci/aci/cmh_smss.c: sAT_PercentCMGS() disabled for !FF_MMI_RIV
like in TCS211 and in Citrine
line source
+ − There exists a set of C header files, needed for TI's GSM firmware to build,
+ − called cdginc. They are not needed for building everything up to and including
+ − GPF and L1, but they are needed in order to build the core G23 protocol stack
+ − components.
+ −
+ − However, these cdginc headers are generated files, not human-written sources,
+ − and the process by which they get created is very messy:
+ −
+ − 1. Contained as part of the ultimate source for the firmware, there is a set of
+ − XML files which give definitions for Air Interface Messages (AIMs) and
+ − Service Access Points (SAPs).
+ −
+ − 2. Each *.aim file is "compiled" into an MDF (message definition file) and each
+ − *.sap file gets similarly "compiled" into a PDF (primitive definition file).
+ − (Despite sharing the same acronym and filename suffix, these "primitive
+ − definition files" have nothing to do with PDF as in Portable Document
+ − Format.) This "compilation" is done by way of XSLT: an XSLT processor is
+ − invoked, its inputs being the source *.{aim,sap} files and a set of *.xsl
+ − files defining the transformation. (The latter can be found under
+ − gpf/util/sape/xslt in the "peek" LoCosto source.)
+ −
+ − 3. A special TI/Condat-developed program called ccdgen (which we only have in
+ − the form of an M$ Windows binary sans source, ccdgen.exe) reads the *.mdf
+ − and *.pdf files produced in the previous step, performs some unknown
+ − processing (unknown because we have no source for this tool), and writes out
+ − most of the C header files which appear in the cdginc directory. (The
+ − exception is that a few of these header files seem to be produced directly
+ − by the XSLT step.)
+ −
+ − ==================
+ − XSLT and Java woes
+ − ==================
+ −
+ − When I (Space Falcon) tried to reproduce the above steps for FreeCalypso,
+ − problems began at the XSLT step. The XSLT processor used in TI's build flow is
+ − an old version of Xalan-J from Apache. J stands for Java - yikes. Thus TI's
+ − build flow actually runs java with a set of *.jar files which comprise Xalan-J.
+ −
+ − I looked to see if the use of Xalan-J (and thus of Java) was required, or if
+ − one could use any XSLT processor, including non-Java implementations. Not so
+ − fast: TI's *.xsl files for the needed transformation call some functions
+ − (please forgive my probably incorrect terminology: both XSLT and Java are as
+ − foreign and unfamiliar to me as Japanese or Arabic or ... - you get the idea)
+ − which seem to have been implemented by TI as custom Java classes, falling under
+ − com.ti.xslt.extension - the latter live in gpf/tools/lib/java/xalan-ext.jar in
+ − TI's semi-source trees (both Leonardo and LoCosto), if you would like to see
+ − for yourself.
+ −
+ − That xalan-ext.jar file with TI's "XSLT extension" classes contains Java
+ − bytecode, not source. Thus one of the required pieces for the *.{aim,sap} ->
+ − *.{mdf,pdf} build step effectively exists only in the form of compiled code
+ − sans source. It is of course an impairment to freedom, and as I quickly
+ − discovered, not only in philosophical terms, but also in practice: as I will
+ − show in a moment, there appears to be a bug in there which we lack the ability
+ − to fix.
+ −
+ − Of course TI ran their java invokation for XSLT under Winblows. (As a side
+ − note, I successfully ran TI's entire Winblows environment, including this step,
+ − under Wine when I did leo2moko - but I wasn't trying to extract any individual
+ − step and get it to run by itself, instead I ran TI's *entire Winblows env*
+ − under a single top-level wine invokation.) But Java bytecode is supposed to be
+ − platform-independent, right? So I tried running the java command from pdt_*.mak
+ − makefiles in the Leonardo version, using their set of XSLT/xalan jars as-is,
+ − under Slackware Linux without Wine, using the Linux-native version of Java that
+ − came with Slackware.
+ −
+ − I started with the AIM->MDF part. The operation succeeded on a few of the
+ − files, but then failed on others. The error had something to do with filename
+ − and pathname manipulation. Some of the com.ti.xslt.extension functions called
+ − by TI's xslt transforms seem to be responsible for turning short filenames into
+ − absolute pathnames and then into file:// URLs, and it appears that TI's
+ − implementation of these functions assumes that absolute pathnames will have
+ − Weendoze drive letters, and breaks on Unix absolute pathnames which lack that
+ − nonsense. And the part responsible for the bug is a piece of Java bytecode in
+ − a jar sans source, remember? I didn't get as far as trying the SAP->PDF part.
+ −
+ − I reason that someone who knows the world of Java could probably reverse-eng
+ − that bytecode and fix the bug with a binary patch, or rewrite an alternate
+ − implementation. Reversing Java bytecode might not even be necessary: someone
+ − who understands XSLT could probably figure out what functionality is expected
+ − from these extension functions, and then reimplement that (most likely trivial)
+ − functionality anew. But XSLT is just as foreign to me as Java; they both might
+ − as well be Japanese or Arabic or some other super-hard foreign language.
+ −
+ − Given that my goal is to produce free GSM firmware, I decided that taking a
+ − very long detour to learn XSLT and Java just so we can regenerate TI's *.[mp]df
+ − files from *.{aim,sap} is not worth it, and imported prebuilt *.[mp]df files
+ − from the LoCosto source along with *.{aim,sap}.
+ −
+ − ============================
+ − Different versions of cdginc
+ − ============================
+ −
+ − Most of the hard work in the FreeCalypso project involves reconciliation between
+ − our two reference versions: TCS211 (aka Leonardo) and LoCosto. Our TCS211
+ − reference version (in the form of leo2moko) already runs on one of our target
+ − platforms and works beautifully, but it has the entire GSM protocol stack in
+ − binary-only libs. The LoCosto version is full source (aside from Nucleus and
+ − some parts of GPF which have already been taken care of), but targets the wrong
+ − chipset, and has that nasty SBuild crap instead of pdt_*.mak.
+ −
+ − The versions of cdginc used in the TCS211 and LoCosto semi-src trees differ in
+ − the following ways:
+ −
+ − 1. The starting *.aim and *.sap files are different: the LoCosto versions are
+ − newer.
+ −
+ − 2. Slightly different versions of ccdgen.exe are used: the version featured in
+ − the TCS211 version from Sotovik identifies itself as 2.5.5, whereas the one
+ − featured in the LoCosto "peek" find identifies itself as 2.5.5A. Aside from
+ − some cosmetic differences, one substantive difference was found in the
+ − generated output: the so-called mtx tables (don't ask me what they are, as I
+ − don't understand it myself) are emitted in a different format. (Ccdgen
+ − version 2.5.5A generates the new format by default, and has a command line
+ − option to revert to the old format.)
+ −
+ − These "mtx" generated header files are included by some ccddata modules (see
+ − ../ccd/README for more info), and the only C source we have for these modules
+ − (for all of CCD, in fact) comes from the LoCosto version. This version of
+ − the ccddata C source expects "mtx" cdginc headers in the new format, hence
+ − that is the format we need to use.
+ −
+ − 3. One additional input to ccdgen besides the *.[mp]df files is a "settings"
+ − file called fflags.h. It has the form of a C header file with #define and
+ − #undef lines (the rest is just comments), but as far as I can tell, it never
+ − gets fed to a C preprocessor, only to ccdgen. The starting *.{aim,sap} files
+ − contain some "options", and for each of these options, fflags.h must give a
+ − yes/no answer in the form of $define or #undef. This "settings" file is
+ − mandatory: if it is not given on the ccdgen command line, or if it has
+ − neither a #define nor a #undef for some "option" defined in the *.{aim,sap}
+ − files, ccdgen aborts with an error.
+ −
+ − It appears that the version of *.{aim,sap} featured in the TCS211 fw has only
+ − one option named TI_DUAL_MODE, which needs to be disabled, as the version of
+ − fflags.h in that tree has only one non-comment line:
+ −
+ − #undef TI_DUAL_MODE
+ −
+ − But the LoCosto version of fflags.h (which appears here as fflags-locosto.h) is
+ − much more extensive, and all of the options listed therein appear in the
+ − *.{aim,sap} files and are in need of explicit enabling or disabling - as I
+ − found out when I tried running ccdgen on LoCosto *.[mp]df files with the TCS211
+ − version of fflags.h - it immediately failed with a bunch of errors about certain
+ − options not being set one way or the other.
+ −
+ − ============
+ − New features
+ − ============
+ −
+ − It appears that all of the options enabled in the LoCosto version of fflags.h
+ − correspond to new features of the G23 protocol stack which do not exist at all
+ − in our TCS211 reference version (leo2moko). How can we tell what features are
+ − present or absent in our TCS211 version if all we have is binary libs sans
+ − source? For a long time I thought the problem was unsolvable, but then I found
+ − the answer in an obscure place: the "relic" pdt_*.mak files *other than* the
+ − actively used pdt_2091.mak present in the source from Sotovik. The source we
+ − have is "sanitized" in that the C source files for all of L1 and G23 have been
+ − removed, and the makefile in pdt_2091.mak has no compilation stanzas for these
+ − modules either: this generated makefile is set up to take the corresponding
+ − binary-only libs as "sources". However, the other (unused) pdt_*.mak files
+ − have been built (back in TI's development environment, presumably) in the "full
+ − source present" configuration, and list not only the names and paths for all of
+ − the deleted source files, but also their complete compilation lines with all -I
+ − directories and -D options!
+ −
+ − Looking at the latter compilation lines, one can see that none of the options
+ − related to GSM REL99 or TI's new "multiband" stuff (seen both in fflags.h and
+ − throughout the L1 and G23 sources we are back-porting from the LoCosto "peek"
+ − find) are present at all in our TCS211/leo2moko reference version.
+ −
+ − ===============================
+ − Approach chosen for FreeCalypso
+ − ===============================
+ −
+ − I have not yet figured out whether the new apparently-high-level GSM PS features
+ − found in the LoCosto version we are working with actually depend on some LoCosto
+ − hardware or DSP ROM code feature not present on the Calypso, or if they can be
+ − back-ported to the Calypso just fine. I also currently have very little
+ − understanding as to their merit, i.e., practical value or usefulness for a GSM
+ − cellphone end user.
+ −
+ − The approach I'm taking for the initial version is to recreate the TCS211
+ − configuration that already works on the Neo Freerunner in the form of leo2moko
+ − as closely as possible, and that means setting all of the newer options to the
+ − disabled state. Or at least that is the approach I am following until and
+ − unless I run into some problem with it; if and when the latter happens, I'll
+ − re-evaluate my course.
+ −
+ − However, simply using the set of cdginc files from TCS211 (or even regenerating
+ − them from the TCS211 versions of *.[mp]df with the new ccdgen.exe to get mtx
+ − tables in the format expected by our version of CCD source) with the G23
+ − protocol stack C sources from the LoCosto version will likely lead to problems,
+ − or least more hassle than it's worth, hence I decided to bite the bullet and
+ − use the *.{aim,sap} and *.[mp]df files from LoCosto.
+ −
+ − Using the LoCosto versions of *.[mp]df, I ran ccdgen.exe (version 2.5.5A, set
+ − to generate mtx tables in the new format) with two different fflags.h
+ − configurations: "locosto" (unchanged from the "peek" find) and "conservative".
+ − The latter is a configuration in which every existing option is set to disabled:
+ − I took fflags-locosto.h and changed every #define to #undef.
+ −
+ − The symlink is currently set to compile the GSM fw using the "conservative"
+ − version of cdginc headers. As I said above, I'll proceed with this
+ − configuration until and unless I hit a problem, and then re-evaluate this
+ − course if need be.
+ −
+ − =====================
+ − ccdgen binary problem
+ − =====================
+ −
+ − We only have a ccdgen.exe binary, but no source. The Winblows binary in
+ − question happens to run fine under Wine, at least on my Slackware machine, but
+ − needless to say, asking every FreeCalypso user who wishes to compile her own
+ − GSM fw from source to run a sans-source Weendoze binary under Wine and pray
+ − that it works is not an attractive proposition. Therefore, as a workaround I
+ − have checked the generated cdginc files into Hg as if they were source files.
+ −
+ − Of course this workaround is not a proper solution either, but it is the best
+ − we can do for the time being, until and unless someone either finds the missing
+ − source for ccdgen or figures out its logic and writes a from-scratch functional
+ − replacement.