FreeCalypso > hg > freecalypso-docs

annotate Firmware-deblobbing @ 34:dd94e04b9539

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
new RFFE-notes article replaces old Quadband-ideas
author Mychaela Falconia <falcon@freecalypso.org>
date Tue, 29 Sep 2020 07:26:45 +0000
parents f68ca40fa5c1
children 14b8e532c966
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
19
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
1 The state of blobs vs. blob-free firmware in FreeCalypso
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
2 ========================================================
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
3
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
4 As of 2019, we have 3 different firmware versions for Standard Modem
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
5 functionality:
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
6
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
7 * Magnetite hybrid is the current production firmware version. The only blobs
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
8 are Nucleus, OSL and OSX glue components of GPF, and TI's proprietary TMS470
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
9 compiler. Everything other than Nucleus and OSL/OSX is compiled from source,
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
10 but the compiler is TI's proprietary TMS470. The same Magnetite source tree
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
11 also supports other configurations (maintained only for regression testing)
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
12 which have more blobs, as well as handset configurations which are currently
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
13 frozen for lack of suitable development hardware.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
14
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
15 * Selenite-470 is FC Selenite built with TI's TMS470 compiler: all code is
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
16 compiled from source, no blobs other than the compiler and its RTS library
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
17 (libc/libgcc equivalent). The blob version of Nucleus is replaced with a
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
18 different (slightly newer) version in full source form, while the blob
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
19 versions of OSL and OSX glue components have been replaced with reconstructed
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
20 sources, i.e., a reconstruction based on disassembly. This fw version is
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
21 currently considered experimental, not cleared for production, and the reason
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
22 is OSL/OSX: the reconstruction is of uncertain correctness and known to be
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
23 incomplete.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
24
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
25 * Selenite-gcc is FC Selenite built with gcc. This version has no blobs
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
26 whatsoever (there is no support in the gcc+binutils toolchain for TI's TMS470
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
27 ABI, thus it is physically impossible to include any of TI's COFF blobs in
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
28 the link), and it is built with a FLOSS compiler. However, it is even more
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
29 experimental and not-for-production than Selenite-470: not only is the OSL/OSX
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
30 issue still there, but there is also widespread breakage from the use of a
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
31 different compiler which was never anticipated by the original developers.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
32 The old FC Citrine firmware (unmaintained since 2016) also suffers from all
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
33 of the same problems, plus additional ones, and therefore should not be
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
34 considered at all.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
35
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
36 When it comes to the firmware, right now those OSL and OSX glue components of
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
37 GPF form the biggest bone in our collective throat. Just 10 C source files are
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
38 missing:
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
39
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
40 os_com.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
41 os_drv.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
42 os_evt.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
43 os_isr.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
44 os_mem.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
45 os_mis.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
46 os_pro.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
47 os_sem.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
48 os_tim.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
49 osx.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
50
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
51 If we can ever find these 10 missing files (does not even need to be exactly
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
52 the same version as in TCS211 GPF), then Selenite-470 would immediately become
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
53 the new production firmware replacing Magnetite (the Nucleus change isn't the
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
54 problem, it's OSL and OSX), and the way would be cleared to begin work on
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
55 bringing Selenite-gcc up to par. But in the absence of these 10 files, the
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
56 following two interlocking problems get in the way of FC Selenite:
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
57
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
58 1) I (Mother Mychaela) am not willing to skip Selenite-470 and jump directly to
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
59 Selenite-gcc, as doing so would violate the fundamental principle of
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
60 incrementality: we need to be making one small change at a time, requiring
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
61 full stability after each incremental change before going to the next.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
62
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
63 2) I am not able to produce a reconstructed C source for certain parts of OSL
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
64 which would result in correct object code when compiled with TMS470: the
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
65 issue is potential race conditions in the OSL timer code. The existing COFF
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
66 object code avoids the race, I can produce C code for compilation with gcc
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
67 which also avoids the race, but I don't know the requisite magic for C code
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
68 to be compiled with TMS470.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
69
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
70 At this point you are going to ask - OK, so what do we do if we can't find
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
71 those 10 missing files? The Mother's current plan is as follows: if these 10
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
72 files are still missing when we get our handset UI development board built, I
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
73 will create a third firmware source tree (not Magnetite, not Selenite, but to
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
74 be named after some other mineral) with the following key properties:
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
75
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
76 * Just like Selenite, it will be hybrid only, no legacy blob-laden
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
77 configurations;
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
78
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
79 * Both modem and handset configurations will be included;
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
80
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
81 * The compiler will be TMS470 - sorry, no gcc;
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
82
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
83 * The version of Nucleus will be the new source-enabled one, same as Selenite;
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
84
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
85 * I will do some careful surgery on the OSL/OSX blobs to make them work with
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
86 the new version of Nucleus.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
87
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
88 The result of these listed key properties is that the new firmware tree will
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
89 have blobs for OSL and OSX and will use the TMS470 compiler as required by
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
90 these blobs, but absolutely everything else will be source-enabled. This
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
91 situation will then persist until someone can wave a few million dollars in
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
92 front of TI to convince their execs to direct their archivists to dig up the 10
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
93 missing files, or until the world civilization collapses into a Mad Max world,
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
94 allowing us to seize those archives with a Spetsnaz unit without police
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
95 interference.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
96
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
97 Special modem applications
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
98 ==========================
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
99
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
100 The above plan states that the third firmware source tree will be created as
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
101 described if the original OSL and OSX source files are still missing when we
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
102 get our handset UI development board built. The reason for this coupling is
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
103 that when we get this UIDB built, the floodgates will open for intensive
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
104 handset UI development. The latter work will need to be done without the
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
105 clutter of Magnetite, yet Selenite is blocked by the lack of the 10 missing
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
106 files - hence the case for the third firmware source tree as described above.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
107
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
108 Alternatively, a third fw source tree similar to the one described (but perhaps
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
109 without the handset configuration) can be created if someone commissions
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
110 significant work on modem firmware, work that is significant enough to call for
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
111 a source tree that is as stable as Magnetite, but free of the clutter.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
112
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
113 cdginc header files
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
114 ===================
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
115
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
116 Another area of deblobbing that hasn't been done yet, but can be done when and
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
117 if a serious need arises, is the cdginc header file set. The cdginc files which
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
118 are currently used for our hybrid config aren't blobs in the strict sense: they
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
119 are C header files included by the sources being recompiled, but they have been
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
120 auto-generated (from true human-editable sources which we do have) by a tool
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
121 (ccdgen) which currently exists only as a Windows binary sans source.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
122
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
123 If anyone needs to make changes to cdginc, the proper course of action should
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
124 be to hire a Windows reverser to reverse ccdgen.exe and to produce a perfect
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
125 form, fit and function replacement.