FreeCalypso > hg > freecalypso-docs

annotate Firmware-deblobbing @ 23:14391ad53281

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
FCDEV3B-repackaging article removed for legal reasons The idea expressed in that article, namely the idea that some party other than Mother Mychaela could be permitted to create a derived work based on FCDEV3B board design and have it be accepted into the FreeCalypso family, is no longer allowed by our current stance on the matters of intellectual property, particularly Falconia IP. For technical content, the new FC-modem-family and Quadband-ideas articles should fully supplant this old FCDEV3B-repackaging article.
author Mychaela Falconia <falcon@freecalypso.org>
date Wed, 23 Oct 2019 00:43:21 +0000
parents f68ca40fa5c1
children 14b8e532c966
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
19
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
1 The state of blobs vs. blob-free firmware in FreeCalypso
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
2 ========================================================
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
3
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
4 As of 2019, we have 3 different firmware versions for Standard Modem
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
5 functionality:
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
6
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
7 * Magnetite hybrid is the current production firmware version. The only blobs
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
8 are Nucleus, OSL and OSX glue components of GPF, and TI's proprietary TMS470
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
9 compiler. Everything other than Nucleus and OSL/OSX is compiled from source,
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
10 but the compiler is TI's proprietary TMS470. The same Magnetite source tree
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
11 also supports other configurations (maintained only for regression testing)
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
12 which have more blobs, as well as handset configurations which are currently
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
13 frozen for lack of suitable development hardware.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
14
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
15 * Selenite-470 is FC Selenite built with TI's TMS470 compiler: all code is
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
16 compiled from source, no blobs other than the compiler and its RTS library
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
17 (libc/libgcc equivalent). The blob version of Nucleus is replaced with a
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
18 different (slightly newer) version in full source form, while the blob
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
19 versions of OSL and OSX glue components have been replaced with reconstructed
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
20 sources, i.e., a reconstruction based on disassembly. This fw version is
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
21 currently considered experimental, not cleared for production, and the reason
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
22 is OSL/OSX: the reconstruction is of uncertain correctness and known to be
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
23 incomplete.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
24
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
25 * Selenite-gcc is FC Selenite built with gcc. This version has no blobs
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
26 whatsoever (there is no support in the gcc+binutils toolchain for TI's TMS470
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
27 ABI, thus it is physically impossible to include any of TI's COFF blobs in
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
28 the link), and it is built with a FLOSS compiler. However, it is even more
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
29 experimental and not-for-production than Selenite-470: not only is the OSL/OSX
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
30 issue still there, but there is also widespread breakage from the use of a
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
31 different compiler which was never anticipated by the original developers.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
32 The old FC Citrine firmware (unmaintained since 2016) also suffers from all
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
33 of the same problems, plus additional ones, and therefore should not be
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
34 considered at all.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
35
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
36 When it comes to the firmware, right now those OSL and OSX glue components of
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
37 GPF form the biggest bone in our collective throat. Just 10 C source files are
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
38 missing:
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
39
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
40 os_com.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
41 os_drv.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
42 os_evt.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
43 os_isr.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
44 os_mem.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
45 os_mis.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
46 os_pro.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
47 os_sem.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
48 os_tim.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
49 osx.c
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
50
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
51 If we can ever find these 10 missing files (does not even need to be exactly
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
52 the same version as in TCS211 GPF), then Selenite-470 would immediately become
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
53 the new production firmware replacing Magnetite (the Nucleus change isn't the
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
54 problem, it's OSL and OSX), and the way would be cleared to begin work on
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
55 bringing Selenite-gcc up to par. But in the absence of these 10 files, the
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
56 following two interlocking problems get in the way of FC Selenite:
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
57
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
58 1) I (Mother Mychaela) am not willing to skip Selenite-470 and jump directly to
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
59 Selenite-gcc, as doing so would violate the fundamental principle of
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
60 incrementality: we need to be making one small change at a time, requiring
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
61 full stability after each incremental change before going to the next.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
62
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
63 2) I am not able to produce a reconstructed C source for certain parts of OSL
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
64 which would result in correct object code when compiled with TMS470: the
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
65 issue is potential race conditions in the OSL timer code. The existing COFF
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
66 object code avoids the race, I can produce C code for compilation with gcc
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
67 which also avoids the race, but I don't know the requisite magic for C code
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
68 to be compiled with TMS470.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
69
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
70 At this point you are going to ask - OK, so what do we do if we can't find
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
71 those 10 missing files? The Mother's current plan is as follows: if these 10
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
72 files are still missing when we get our handset UI development board built, I
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
73 will create a third firmware source tree (not Magnetite, not Selenite, but to
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
74 be named after some other mineral) with the following key properties:
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
75
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
76 * Just like Selenite, it will be hybrid only, no legacy blob-laden
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
77 configurations;
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
78
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
79 * Both modem and handset configurations will be included;
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
80
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
81 * The compiler will be TMS470 - sorry, no gcc;
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
82
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
83 * The version of Nucleus will be the new source-enabled one, same as Selenite;
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
84
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
85 * I will do some careful surgery on the OSL/OSX blobs to make them work with
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
86 the new version of Nucleus.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
87
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
88 The result of these listed key properties is that the new firmware tree will
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
89 have blobs for OSL and OSX and will use the TMS470 compiler as required by
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
90 these blobs, but absolutely everything else will be source-enabled. This
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
91 situation will then persist until someone can wave a few million dollars in
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
92 front of TI to convince their execs to direct their archivists to dig up the 10
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
93 missing files, or until the world civilization collapses into a Mad Max world,
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
94 allowing us to seize those archives with a Spetsnaz unit without police
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
95 interference.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
96
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
97 Special modem applications
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
98 ==========================
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
99
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
100 The above plan states that the third firmware source tree will be created as
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
101 described if the original OSL and OSX source files are still missing when we
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
102 get our handset UI development board built. The reason for this coupling is
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
103 that when we get this UIDB built, the floodgates will open for intensive
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
104 handset UI development. The latter work will need to be done without the
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
105 clutter of Magnetite, yet Selenite is blocked by the lack of the 10 missing
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
106 files - hence the case for the third firmware source tree as described above.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
107
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
108 Alternatively, a third fw source tree similar to the one described (but perhaps
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
109 without the handset configuration) can be created if someone commissions
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
110 significant work on modem firmware, work that is significant enough to call for
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
111 a source tree that is as stable as Magnetite, but free of the clutter.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
112
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
113 cdginc header files
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
114 ===================
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
115
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
116 Another area of deblobbing that hasn't been done yet, but can be done when and
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
117 if a serious need arises, is the cdginc header file set. The cdginc files which
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
118 are currently used for our hybrid config aren't blobs in the strict sense: they
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
119 are C header files included by the sources being recompiled, but they have been
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
120 auto-generated (from true human-editable sources which we do have) by a tool
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
121 (ccdgen) which currently exists only as a Windows binary sans source.
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
122
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
123 If anyone needs to make changes to cdginc, the proper course of action should
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
124 be to hire a Windows reverser to reverse ccdgen.exe and to produce a perfect
f68ca40fa5c1 Firmware-deblobbing document written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
125 form, fit and function replacement.