FreeCalypso > hg > freecalypso-reveng

annotate objgrep/README @ 392:35009c936a4a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
compal/melody-extr: first attempt at actual melody extraction
author Mychaela Falconia <falcon@freecalypso.org>
date Fri, 01 Apr 2022 06:03:47 +0000
parents 10a9a0ca9d07
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
176
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
1 We have TI's reference firmware for the Calypso/Iota/Rita chipset (Leonardo) in
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
2 the form of linkable COFF objects and some source pieces, but when it comes to
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
3 practically usable "dumbphones" based on this chipset, we only have the binary
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
4 fw images read out of flash, without any kind of symbolic info.
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
5
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
6 The tools in this directory perform a kind of grep operation, searching an
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
7 unknown binary fw image for the bits of code or data contained in a linkable
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
8 COFF object. The objective was to determine whether or not our "reference"
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
9 Leonardo objects could be found verbatim in the set of proprietary firmwares
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
10 from Compal and Foxconn (Pirelli DP-L10) that run on our "dumbphone" targets.
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
11
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
12 The tools are as follows:
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
13
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
14 objgrep
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
15
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
16 This tool extracts one section (e.g., .text or .const, to be specified
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
17 on the command line) from a "needle" COFF object and searches for it in
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
18 the "haystack" unknown binary. The byte positions in the sought-for
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
19 object section where relocs are to be applied at linking time are masked
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
20 as appopriate for each reloc type, and the section is expected to start
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
21 on a 4-byte-aligned boundary in the unknown binary. If a match is
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
22 found, objgrep can print out the list of symbol addresses in the
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
23 sought-for and found section, and it can also deduce some symbols
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
24 external to the module or belonging to the module's other sections by
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
25 looking where the relocs that were masked for the match point to in the
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
26 unknown binary.
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
27
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
28 In order for this form of grep to be effective, the section being
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
29 searched for should be "meaty", i.e., mostly code or constant data with
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
30 some interspersed relocs. If the sought-for section is very small, fits
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
31 the same pattern after reloc masking as other unrelated bits of code,
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
32 or consists mostly of relocs, the most likely result will be a useless
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
33 false hit.
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
34
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
35 objgrep-fe
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
36
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
37 This program is a front-end to objgrep. It reads a line-based text file
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
38 listing the objects and sections to be grepped for, and invokes objgrep
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
39 for each listed section. The output of objgrep is captured through a
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
40 pipe; objgrep-fe collates together the symbol addresses found with each
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
41 individual objgrep hit and produces a sorted symbol listing.
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
42
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
43 Results
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
44 =======
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
45
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
46 The idea proved quite successful in the case of Pirelli DP-L10 firmware,
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
47 specifically version D910.0.3.98: this fw appears to have been built with
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
48 exactly the same RTS, Nucleus and GPF libraries that are featured in our
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
49 Leonardo semi-src as "very stable blobs", i.e., *.lib files in the source tree
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
50 itself, rather than blobs under g23m/__out__ for which TI's closed source
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
51 police excluded the corresponding source. Every object that comes from these
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
52 libraries in our leo2moko build was also found in Pirelli's fw.
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
53
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
54 It is worth noting that the GPF libraries in particular contain a few objects
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
55 with embedded second-granularity timestamps, courtesy of the C compiler's
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
56 __DATE__ and __TIME__ preprocessor definitions, i.e., the timestamp strings
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
57 with times to the second are emitted into the code image built with these
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
58 libraries. These timestamped objects were found in Pirelli's fw with our
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
59 objgrep tools along with the rest of GPF, proving beyond any doubt that this fw
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
60 has been built with exactly the same GPF libs as our leo2moko.
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
61
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
62 This confirmation in the case of Pirelli's fw is very reassuring because this
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
63 fw has received a lot of real-life testing: I've been using a Pirelli running
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
64 its original proprietary fw (as no free fw exists yet, for this or any other
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
65 dumbphone) as my personal everyday cellphone for over a year now. That is a
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
66 lot more real life experience than I can get with anything Openmoko-based, and
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
67 it is reassuring to know that the GPF libraries we have painstakingly
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
68 reconstructed are used not only in the largely-untested moko firmware, but also
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
69 in the much more real-life-tested Pirelli DP-L10 fw.
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
70
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
71 Attemping the same grep against Compal's fw yielded far fewer hits, however.
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
72 A lot of RTS modules were found, but very little from Nucleus or GPF libs.
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
73 Nucleus' tct and tmt assembly modules were found, but not much else. Manual
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
74 examination of Compal's INC_Initialize() function (which is easy to locate even
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
75 in a totally unknown fw binary, as it's only one ARM->Thumb call veneer away
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
76 from the boilerplate code at the boot entry point) has revealed that it's the
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
77 same code, but compiled slightly differently, probably a slightly newer C
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
78 compiler version. (The version in our reference libs saves one more call-
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
79 preserved register than necessary; the version that appears in Compal's fw is
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
80 fully optimal in this regard.) I reason that the same compiler difference must
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
81 be responsible for the great scarcity of hits in general, as these kinds of
10a9a0ca9d07 objgrep/README written
Michael Spacefalcon <msokolov@ivan.Harhan.ORG>
parents:
diff changeset
82 compiler changes would produce differences in just about every module.