FreeCalypso > hg > freecalypso-docs

comparison Firmware-deblobbing @ 19:f68ca40fa5c1

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Firmware-deblobbing document written
author Mychaela Falconia <falcon@freecalypso.org>
date Fri, 13 Sep 2019 07:28:00 +0000
parents
children 14b8e532c966
comparison
equal deleted inserted replaced
18:7ba5c951803c 19:f68ca40fa5c1
1 The state of blobs vs. blob-free firmware in FreeCalypso
2 ========================================================
3
4 As of 2019, we have 3 different firmware versions for Standard Modem
5 functionality:
6
7 * Magnetite hybrid is the current production firmware version. The only blobs
8 are Nucleus, OSL and OSX glue components of GPF, and TI's proprietary TMS470
9 compiler. Everything other than Nucleus and OSL/OSX is compiled from source,
10 but the compiler is TI's proprietary TMS470. The same Magnetite source tree
11 also supports other configurations (maintained only for regression testing)
12 which have more blobs, as well as handset configurations which are currently
13 frozen for lack of suitable development hardware.
14
15 * Selenite-470 is FC Selenite built with TI's TMS470 compiler: all code is
16 compiled from source, no blobs other than the compiler and its RTS library
17 (libc/libgcc equivalent). The blob version of Nucleus is replaced with a
18 different (slightly newer) version in full source form, while the blob
19 versions of OSL and OSX glue components have been replaced with reconstructed
20 sources, i.e., a reconstruction based on disassembly. This fw version is
21 currently considered experimental, not cleared for production, and the reason
22 is OSL/OSX: the reconstruction is of uncertain correctness and known to be
23 incomplete.
24
25 * Selenite-gcc is FC Selenite built with gcc. This version has no blobs
26 whatsoever (there is no support in the gcc+binutils toolchain for TI's TMS470
27 ABI, thus it is physically impossible to include any of TI's COFF blobs in
28 the link), and it is built with a FLOSS compiler. However, it is even more
29 experimental and not-for-production than Selenite-470: not only is the OSL/OSX
30 issue still there, but there is also widespread breakage from the use of a
31 different compiler which was never anticipated by the original developers.
32 The old FC Citrine firmware (unmaintained since 2016) also suffers from all
33 of the same problems, plus additional ones, and therefore should not be
34 considered at all.
35
36 When it comes to the firmware, right now those OSL and OSX glue components of
37 GPF form the biggest bone in our collective throat. Just 10 C source files are
38 missing:
39
40 os_com.c
41 os_drv.c
42 os_evt.c
43 os_isr.c
44 os_mem.c
45 os_mis.c
46 os_pro.c
47 os_sem.c
48 os_tim.c
49 osx.c
50
51 If we can ever find these 10 missing files (does not even need to be exactly
52 the same version as in TCS211 GPF), then Selenite-470 would immediately become
53 the new production firmware replacing Magnetite (the Nucleus change isn't the
54 problem, it's OSL and OSX), and the way would be cleared to begin work on
55 bringing Selenite-gcc up to par. But in the absence of these 10 files, the
56 following two interlocking problems get in the way of FC Selenite:
57
58 1) I (Mother Mychaela) am not willing to skip Selenite-470 and jump directly to
59 Selenite-gcc, as doing so would violate the fundamental principle of
60 incrementality: we need to be making one small change at a time, requiring
61 full stability after each incremental change before going to the next.
62
63 2) I am not able to produce a reconstructed C source for certain parts of OSL
64 which would result in correct object code when compiled with TMS470: the
65 issue is potential race conditions in the OSL timer code. The existing COFF
66 object code avoids the race, I can produce C code for compilation with gcc
67 which also avoids the race, but I don't know the requisite magic for C code
68 to be compiled with TMS470.
69
70 At this point you are going to ask - OK, so what do we do if we can't find
71 those 10 missing files? The Mother's current plan is as follows: if these 10
72 files are still missing when we get our handset UI development board built, I
73 will create a third firmware source tree (not Magnetite, not Selenite, but to
74 be named after some other mineral) with the following key properties:
75
76 * Just like Selenite, it will be hybrid only, no legacy blob-laden
77 configurations;
78
79 * Both modem and handset configurations will be included;
80
81 * The compiler will be TMS470 - sorry, no gcc;
82
83 * The version of Nucleus will be the new source-enabled one, same as Selenite;
84
85 * I will do some careful surgery on the OSL/OSX blobs to make them work with
86 the new version of Nucleus.
87
88 The result of these listed key properties is that the new firmware tree will
89 have blobs for OSL and OSX and will use the TMS470 compiler as required by
90 these blobs, but absolutely everything else will be source-enabled. This
91 situation will then persist until someone can wave a few million dollars in
92 front of TI to convince their execs to direct their archivists to dig up the 10
93 missing files, or until the world civilization collapses into a Mad Max world,
94 allowing us to seize those archives with a Spetsnaz unit without police
95 interference.
96
97 Special modem applications
98 ==========================
99
100 The above plan states that the third firmware source tree will be created as
101 described if the original OSL and OSX source files are still missing when we
102 get our handset UI development board built. The reason for this coupling is
103 that when we get this UIDB built, the floodgates will open for intensive
104 handset UI development. The latter work will need to be done without the
105 clutter of Magnetite, yet Selenite is blocked by the lack of the 10 missing
106 files - hence the case for the third firmware source tree as described above.
107
108 Alternatively, a third fw source tree similar to the one described (but perhaps
109 without the handset configuration) can be created if someone commissions
110 significant work on modem firmware, work that is significant enough to call for
111 a source tree that is as stable as Magnetite, but free of the clutter.
112
113 cdginc header files
114 ===================
115
116 Another area of deblobbing that hasn't been done yet, but can be done when and
117 if a serious need arises, is the cdginc header file set. The cdginc files which
118 are currently used for our hybrid config aren't blobs in the strict sense: they
119 are C header files included by the sources being recompiled, but they have been
120 auto-generated (from true human-editable sources which we do have) by a tool
121 (ccdgen) which currently exists only as a Windows binary sans source.
122
123 If anyone needs to make changes to cdginc, the proper course of action should
124 be to hire a Windows reverser to reverse ccdgen.exe and to produce a perfect
125 form, fit and function replacement.