FreeCalypso > hg > freecalypso-sw
view gsm-fw/cdg/README @ 836:6c5d3fecbc67
gdi: audio.c compiles
author | Space Falcon <falcon@ivan.Harhan.ORG> |
---|---|
date | Sat, 25 Apr 2015 19:21:20 +0000 |
parents | c2c4c6b5a5c5 |
children |
line wrap: on
line source
There exists a set of C header files, needed for TI's GSM firmware to build, called cdginc. They are not needed for building everything up to and including GPF and L1, but they are needed in order to build the core G23 protocol stack components. However, these cdginc headers are generated files, not human-written sources, and the process by which they get created is very messy: 1. Contained as part of the ultimate source for the firmware, there is a set of XML files which give definitions for Air Interface Messages (AIMs) and Service Access Points (SAPs). 2. Each *.aim file is "compiled" into an MDF (message definition file) and each *.sap file gets similarly "compiled" into a PDF (primitive definition file). (Despite sharing the same acronym and filename suffix, these "primitive definition files" have nothing to do with PDF as in Portable Document Format.) This "compilation" is done by way of XSLT: an XSLT processor is invoked, its inputs being the source *.{aim,sap} files and a set of *.xsl files defining the transformation. (The latter can be found under gpf/util/sape/xslt in the "peek" LoCosto source.) 3. A special TI/Condat-developed program called ccdgen (which we only have in the form of an M$ Windows binary sans source, ccdgen.exe) reads the *.mdf and *.pdf files produced in the previous step, performs some unknown processing (unknown because we have no source for this tool), and writes out most of the C header files which appear in the cdginc directory. (The exception is that a few of these header files seem to be produced directly by the XSLT step.) ================== XSLT and Java woes ================== When I (Space Falcon) tried to reproduce the above steps for FreeCalypso, problems began at the XSLT step. The XSLT processor used in TI's build flow is an old version of Xalan-J from Apache. J stands for Java - yikes. Thus TI's build flow actually runs java with a set of *.jar files which comprise Xalan-J. I looked to see if the use of Xalan-J (and thus of Java) was required, or if one could use any XSLT processor, including non-Java implementations. Not so fast: TI's *.xsl files for the needed transformation call some functions (please forgive my probably incorrect terminology: both XSLT and Java are as foreign and unfamiliar to me as Japanese or Arabic or ... - you get the idea) which seem to have been implemented by TI as custom Java classes, falling under com.ti.xslt.extension - the latter live in gpf/tools/lib/java/xalan-ext.jar in TI's semi-source trees (both Leonardo and LoCosto), if you would like to see for yourself. That xalan-ext.jar file with TI's "XSLT extension" classes contains Java bytecode, not source. Thus one of the required pieces for the *.{aim,sap} -> *.{mdf,pdf} build step effectively exists only in the form of compiled code sans source. It is of course an impairment to freedom, and as I quickly discovered, not only in philosophical terms, but also in practice: as I will show in a moment, there appears to be a bug in there which we lack the ability to fix. Of course TI ran their java invokation for XSLT under Winblows. (As a side note, I successfully ran TI's entire Winblows environment, including this step, under Wine when I did leo2moko - but I wasn't trying to extract any individual step and get it to run by itself, instead I ran TI's *entire Winblows env* under a single top-level wine invokation.) But Java bytecode is supposed to be platform-independent, right? So I tried running the java command from pdt_*.mak makefiles in the Leonardo version, using their set of XSLT/xalan jars as-is, under Slackware Linux without Wine, using the Linux-native version of Java that came with Slackware. I started with the AIM->MDF part. The operation succeeded on a few of the files, but then failed on others. The error had something to do with filename and pathname manipulation. Some of the com.ti.xslt.extension functions called by TI's xslt transforms seem to be responsible for turning short filenames into absolute pathnames and then into file:// URLs, and it appears that TI's implementation of these functions assumes that absolute pathnames will have Weendoze drive letters, and breaks on Unix absolute pathnames which lack that nonsense. And the part responsible for the bug is a piece of Java bytecode in a jar sans source, remember? I didn't get as far as trying the SAP->PDF part. I reason that someone who knows the world of Java could probably reverse-eng that bytecode and fix the bug with a binary patch, or rewrite an alternate implementation. Reversing Java bytecode might not even be necessary: someone who understands XSLT could probably figure out what functionality is expected from these extension functions, and then reimplement that (most likely trivial) functionality anew. But XSLT is just as foreign to me as Java; they both might as well be Japanese or Arabic or some other super-hard foreign language. Given that my goal is to produce free GSM firmware, I decided that taking a very long detour to learn XSLT and Java just so we can regenerate TI's *.[mp]df files from *.{aim,sap} is not worth it, and imported prebuilt *.[mp]df files from the LoCosto source along with *.{aim,sap}. ============================ Different versions of cdginc ============================ Most of the hard work in the FreeCalypso project involves reconciliation between our two reference versions: TCS211 (aka Leonardo) and LoCosto. Our TCS211 reference version (in the form of leo2moko) already runs on one of our target platforms and works beautifully, but it has the entire GSM protocol stack in binary-only libs. The LoCosto version is full source (aside from Nucleus and some parts of GPF which have already been taken care of), but targets the wrong chipset, and has that nasty SBuild crap instead of pdt_*.mak. The versions of cdginc used in the TCS211 and LoCosto semi-src trees differ in the following ways: 1. The starting *.aim and *.sap files are different: the LoCosto versions are newer. 2. Slightly different versions of ccdgen.exe are used: the version featured in the TCS211 version from Sotovik identifies itself as 2.5.5, whereas the one featured in the LoCosto "peek" find identifies itself as 2.5.5A. Aside from some cosmetic differences, one substantive difference was found in the generated output: the so-called mtx tables (don't ask me what they are, as I don't understand it myself) are emitted in a different format. (Ccdgen version 2.5.5A generates the new format by default, and has a command line option to revert to the old format.) These "mtx" generated header files are included by some ccddata modules (see ../ccd/README for more info), and the only C source we have for these modules (for all of CCD, in fact) comes from the LoCosto version. This version of the ccddata C source expects "mtx" cdginc headers in the new format, hence that is the format we need to use. 3. One additional input to ccdgen besides the *.[mp]df files is a "settings" file called fflags.h. It has the form of a C header file with #define and #undef lines (the rest is just comments), but as far as I can tell, it never gets fed to a C preprocessor, only to ccdgen. The starting *.{aim,sap} files contain some "options", and for each of these options, fflags.h must give a yes/no answer in the form of $define or #undef. This "settings" file is mandatory: if it is not given on the ccdgen command line, or if it has neither a #define nor a #undef for some "option" defined in the *.{aim,sap} files, ccdgen aborts with an error. It appears that the version of *.{aim,sap} featured in the TCS211 fw has only one option named TI_DUAL_MODE, which needs to be disabled, as the version of fflags.h in that tree has only one non-comment line: #undef TI_DUAL_MODE But the LoCosto version of fflags.h (which appears here as fflags-locosto.h) is much more extensive, and all of the options listed therein appear in the *.{aim,sap} files and are in need of explicit enabling or disabling - as I found out when I tried running ccdgen on LoCosto *.[mp]df files with the TCS211 version of fflags.h - it immediately failed with a bunch of errors about certain options not being set one way or the other. ============ New features ============ It appears that all of the options enabled in the LoCosto version of fflags.h correspond to new features of the G23 protocol stack which do not exist at all in our TCS211 reference version (leo2moko). How can we tell what features are present or absent in our TCS211 version if all we have is binary libs sans source? For a long time I thought the problem was unsolvable, but then I found the answer in an obscure place: the "relic" pdt_*.mak files *other than* the actively used pdt_2091.mak present in the source from Sotovik. The source we have is "sanitized" in that the C source files for all of L1 and G23 have been removed, and the makefile in pdt_2091.mak has no compilation stanzas for these modules either: this generated makefile is set up to take the corresponding binary-only libs as "sources". However, the other (unused) pdt_*.mak files have been built (back in TI's development environment, presumably) in the "full source present" configuration, and list not only the names and paths for all of the deleted source files, but also their complete compilation lines with all -I directories and -D options! Looking at the latter compilation lines, one can see that none of the options related to GSM REL99 or TI's new "multiband" stuff (seen both in fflags.h and throughout the L1 and G23 sources we are back-porting from the LoCosto "peek" find) are present at all in our TCS211/leo2moko reference version. =============================== Approach chosen for FreeCalypso =============================== I have not yet figured out whether the new apparently-high-level GSM PS features found in the LoCosto version we are working with actually depend on some LoCosto hardware or DSP ROM code feature not present on the Calypso, or if they can be back-ported to the Calypso just fine. I also currently have very little understanding as to their merit, i.e., practical value or usefulness for a GSM cellphone end user. The approach I'm taking for the initial version is to recreate the TCS211 configuration that already works on the Neo Freerunner in the form of leo2moko as closely as possible, and that means setting all of the newer options to the disabled state. Or at least that is the approach I am following until and unless I run into some problem with it; if and when the latter happens, I'll re-evaluate my course. However, simply using the set of cdginc files from TCS211 (or even regenerating them from the TCS211 versions of *.[mp]df with the new ccdgen.exe to get mtx tables in the format expected by our version of CCD source) with the G23 protocol stack C sources from the LoCosto version will likely lead to problems, or least more hassle than it's worth, hence I decided to bite the bullet and use the *.{aim,sap} and *.[mp]df files from LoCosto. Using the LoCosto versions of *.[mp]df, I ran ccdgen.exe (version 2.5.5A, set to generate mtx tables in the new format) with two different fflags.h configurations: "locosto" (unchanged from the "peek" find) and "conservative". The latter is a configuration in which every existing option is set to disabled: I took fflags-locosto.h and changed every #define to #undef. The symlink is currently set to compile the GSM fw using the "conservative" version of cdginc headers. As I said above, I'll proceed with this configuration until and unless I hit a problem, and then re-evaluate this course if need be. ===================== ccdgen binary problem ===================== We only have a ccdgen.exe binary, but no source. The Winblows binary in question happens to run fine under Wine, at least on my Slackware machine, but needless to say, asking every FreeCalypso user who wishes to compile her own GSM fw from source to run a sans-source Weendoze binary under Wine and pray that it works is not an attractive proposition. Therefore, as a workaround I have checked the generated cdginc files into Hg as if they were source files. Of course this workaround is not a proper solution either, but it is the best we can do for the time being, until and unless someone either finds the missing source for ccdgen or figures out its logic and writes a from-scratch functional replacement.