FreeCalypso > hg > freecalypso-tools

annotate doc/Nonfree-phones-user-data @ 995:74024eb17e04

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
fc-loadtool help: improve language regarding 16 MiB flash chips In FC project history, 16 MiB flash originally meant Pirelli DP-L10. Then we got FCDEV3B with the same flash (our own design), but now we are discovering more Calypso devices that used such large flash, both late Calypso era (Sony Ericsson K2x0) as well as much earlier ones (FIC FLUID devices.txt file with 2004 dates, Leonardo+ rev 5). Hence we need to migrate to more generic or neutral language in associated documentation, without giving elevated status to specific examples that drove our early project history.
author Mychaela Falconia <falcon@freecalypso.org>
date Sun, 03 Dec 2023 21:11:12 +0000
parents ac527de89a0d
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
808
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
1 Using FreeCalypso tools to access user data on non-free phones
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
2 ==============================================================
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
3
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
4 If you are using a Motorola C1xx or Pirelli DP-L10 phone as your daily driver,
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
5 running its original proprietary fw for the lack of anything better, you can
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
6 still use FreeCalypso tools to get to some of your user data, namely, your
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
7 phonebook and your SMS store. You can even do the same with a non-Calypso
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
8 phone if you can move the SIM in and out of it. This article tells you how to
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
9 do such tricks.
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
10
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
11 Phonebook howto
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
12 ===============
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
13
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
14 Whenever you have to use a phone whose firmware is not fully liberated, you
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
15 should always store your phonebook contacts on the SIM, rather than "on the
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
16 phone". SIM phonebook storage format is strictly defined by standard specs,
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
17 leaving no room for proprietary firmwares to apply any wacky proprietary ideas
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
18 of their own - OTOH, "on the phone" storage really means storing your data in
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
19 proprietary fw data structures in only-Cthulhu-knows-what format.
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
20
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
21 If you have your contacts stored in your SIM phonebook, you can manipulate the
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
22 latter with FreeCalypso tools: you can dump your SIM phonebook and save it on
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
23 your larger host computer, you can restore these backups back to the SIM, and
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
24 you can freely edit your SIM phonebook. If your phone is a supported Calypso
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
25 model, you can perform all of these manipulations without removing your SIM -
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
26 you only need to shut down the phone's regular fw for the duration of the
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
27 operation, just like you would do for flashing with fc-loadtool. The other way
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
28 is to remove your SIM from the phone and stick it into a standalone smart card
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
29 "reader" device - this path is totally independent of all phones.
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
30
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
31 You will need to have both the present FC host tools package and the separate
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
32 FC SIM tools package installed on your host machine. See the instructions in
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
33 the SIM-manipulation article for how to run fc-simint for the purpose of
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
34 operating on a SIM card inside your Calypso phone. Once you have landed at the
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
35 simtool> prompt, see the doc/User-oriented-commands article in the fc-sim-tools
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
36 repository for instructions on how to enter your PIN (if your SIM requires one)
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
37 and how to manipulate your phonebooks.
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
38
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
39 SMS storage howto
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
40 =================
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
41
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
42 Motorola's firmware always stores received SMS on the SIM, whereas sent SMS are
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
43 stored in their FFS (flash file system, see Compal-FFS article) in their own
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
44 proprietary format. You can transfer your received SMS to your larger host
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
45 computer for longer-term archival as follows:
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
46
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
47 1) Shut down the phone's regular fw and run fc-simint to operate on the SIM;
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
48
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
49 2) Once you are at the simtool> prompt, enter your PIN if your SIM requires one,
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
50 and then save the SIM SMS store to a Unix host file with the save-sms-bin
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
51 command, as explained in the doc/User-oriented-commands article in the
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
52 fc-sim-tools repository. You can then issue an sms-erase-all command in the
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
53 same seesion to clean out the SIM SMS store.
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
54
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
55 3) The files saved by the fc-simint/fc-simtool save-sms-bin command are binary,
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
56 but the message content can then be fully decoded to human-readable ASCII
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
57 (or ISO 8859-1 or UTF-8 if desired) with our pcm-sms-decode utility,
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
58 described later in this article.
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
59
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
60 To retrieve your sent SMS from Compal's FFS, you will need to similarly shut
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
61 down the phone's regular fw and read out the FFS sectors with fc-loadtool. You
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
62 can then use our tiffs utility to parse the FFS structure, but the format of the
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
63 actual files used by Compal's proprietary fw implementation has not been studied
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
64 in detail by the Mother, thus you will need to use your own brain cells for
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
65 further decoding.
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
66
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
67 If you use a Pirelli DP-L10 phone rather than Motorola C1xx, Pirelli's fw gives
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
68 you the option of storing received SMS either on the SIM or "on the phone", the
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
69 latter meaning Pirelli's FFS. Sent SMS are always stored in Pirelli's FFS.
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
70
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
71 If you use SIM storage for received SMS on your Pirelli phone, then the
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
72 procedure for retrieving those SMS is the same as for Mot C1xx, as detailed
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
73 above. However, with this phone model, configuring the fw to store received SMS
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
74 in its FFS ("on the phone") is generally preferable: Pirelli's fw maintains a
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
75 file in its FFS (/pcm/SMS) whose binary format is exactly the same as the
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
76 standards-defined format for storing SMS on the SIM, thus reading out /pcm/SMS
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
77 with fc-fsio is just as good as reading out EF_SMS with fc-simint or fc-simtool,
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
78 but is much more convenient: fc-fsio operates on the phone while its regular fw
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
79 is running, whereas direct SIM manipulation with fc-simint requires shutting
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
80 down the phone's regular fw. Pirelli's fw also accesses its SMS store faster
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
81 when FFS storage is used, rather than SIM.
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
82
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
83 Decoding received SMS binary files with pcm-sms-decode
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
84 ======================================================
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
85
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
86 Whether you have retrieved your SMS store from a SIM with fc-simint/fc-simtool
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
87 save-sms-bin command or read out Pirelli's /pcm/SMS file with fc-fsio, the
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
88 binary format is exactly the same: a bundle of 176-byte records directly abutted
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
89 together, with each individual record having the standard format defined in GSM
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
90 TS 11.11 and its successor 3GPP TS 51.011. Looking at raw hex dumps won't be
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
91 very helpful, as the reversed nibble format used for phone numbers and SC
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
92 timestamps is not directly human-readable, and the message bodies will usually
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
93 be in the form of packed septets - hence a special decoding program is needed.
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
94
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
95 Our pcm-sms-decode utility does the needed job of reading these binary files and
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
96 fully decoding them into human-readable ASCII, or ISO 8859-1 or UTF-8 if your
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
97 host system supports such extended character sets and you have received SMS
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
98 content that uses beyond-ASCII characters. pcm-sms-decode was added to FC host
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
99 tools suite in early 2020; it is based on and shares most of its code with our
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
100 earlier sms-pdu-decode utility from 2018. (The latter is meant for decoding SMS
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
101 PDUs retrieved from FreeCalypso AT command modems with fcup-smdump.)
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
102
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
103 pcm-sms-decode uses the same backslash escapes in its output and accepts the
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
104 same -e, -u and -h options as sms-pdu-decode - please refer to the description
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
105 of sms-pdu-decode in the User-phone-tools article for the details.
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
106
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
107 By default, pcm-sms-decode numbers the records being decoded as 0 to N-1, where
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
108 N is the total number of records in the binary file given to it (file size equal
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
109 to N*176 bytes). As of fc-host-tools-r15, newly added -s option causes it to
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
110 number these same records as 1 to N instead. The original 0-based record
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
111 numbering was implemented when the tool was used solely to decode /pcm/SMS from
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
112 Pirelli's fw (before the introduction of fc-simtool), but now that we have
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
113 fc-simtool and fc-simint, and given that the SIM interface protocol and SIM file
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
114 system definition use 1-based record numbers, we need to support the same
ac527de89a0d pcm-sms-decode documentation
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
115 convention in pcm-sms-decode for consistency.