Using FreeCalypso tools to access user data on non-free phones
==============================================================

If you are using a Motorola C1xx or Pirelli DP-L10 phone as your daily driver,
running its original proprietary fw for the lack of anything better, you can
still use FreeCalypso tools to get to some of your user data, namely, your
phonebook and your SMS store.  You can even do the same with a non-Calypso
phone if you can move the SIM in and out of it.  This article tells you how to
do such tricks.

Phonebook howto
===============

Whenever you have to use a phone whose firmware is not fully liberated, you
should always store your phonebook contacts on the SIM, rather than "on the
phone".  SIM phonebook storage format is strictly defined by standard specs,
leaving no room for proprietary firmwares to apply any wacky proprietary ideas
of their own - OTOH, "on the phone" storage really means storing your data in
proprietary fw data structures in only-Cthulhu-knows-what format.

If you have your contacts stored in your SIM phonebook, you can manipulate the
latter with FreeCalypso tools: you can dump your SIM phonebook and save it on
your larger host computer, you can restore these backups back to the SIM, and
you can freely edit your SIM phonebook.  If your phone is a supported Calypso
model, you can perform all of these manipulations without removing your SIM -
you only need to shut down the phone's regular fw for the duration of the
operation, just like you would do for flashing with fc-loadtool.  The other way
is to remove your SIM from the phone and stick it into a standalone smart card
"reader" device - this path is totally independent of all phones.

You will need to have both the present FC host tools package and the separate
FC SIM tools package installed on your host machine.  See the instructions in
the SIM-manipulation article for how to run fc-simint for the purpose of
operating on a SIM card inside your Calypso phone.  Once you have landed at the
simtool> prompt, see the doc/User-oriented-commands article in the fc-sim-tools
repository for instructions on how to enter your PIN (if your SIM requires one)
and how to manipulate your phonebooks.

SMS storage howto
=================

Motorola's firmware always stores received SMS on the SIM, whereas sent SMS are
stored in their FFS (flash file system, see Compal-FFS article) in their own
proprietary format.  You can transfer your received SMS to your larger host
computer for longer-term archival as follows:

1) Shut down the phone's regular fw and run fc-simint to operate on the SIM;

2) Once you are at the simtool> prompt, enter your PIN if your SIM requires one,
   and then save the SIM SMS store to a Unix host file with the save-sms-bin
   command, as explained in the doc/User-oriented-commands article in the
   fc-sim-tools repository.  You can then issue an sms-erase-all command in the
   same seesion to clean out the SIM SMS store.

3) The files saved by the fc-simint/fc-simtool save-sms-bin command are binary,
   but the message content can then be fully decoded to human-readable ASCII
   (or ISO 8859-1 or UTF-8 if desired) with our pcm-sms-decode utility,
   described later in this article.

To retrieve your sent SMS from Compal's FFS, you will need to similarly shut
down the phone's regular fw and read out the FFS sectors with fc-loadtool.  You
can then use our tiffs utility to parse the FFS structure, but the format of the
actual files used by Compal's proprietary fw implementation has not been studied
in detail by the Mother, thus you will need to use your own brain cells for
further decoding.

If you use a Pirelli DP-L10 phone rather than Motorola C1xx, Pirelli's fw gives
you the option of storing received SMS either on the SIM or "on the phone", the
latter meaning Pirelli's FFS.  Sent SMS are always stored in Pirelli's FFS.

If you use SIM storage for received SMS on your Pirelli phone, then the
procedure for retrieving those SMS is the same as for Mot C1xx, as detailed
above.  However, with this phone model, configuring the fw to store received SMS
in its FFS ("on the phone") is generally preferable: Pirelli's fw maintains a
file in its FFS (/pcm/SMS) whose binary format is exactly the same as the
standards-defined format for storing SMS on the SIM, thus reading out /pcm/SMS
with fc-fsio is just as good as reading out EF_SMS with fc-simint or fc-simtool,
but is much more convenient: fc-fsio operates on the phone while its regular fw
is running, whereas direct SIM manipulation with fc-simint requires shutting
down the phone's regular fw.  Pirelli's fw also accesses its SMS store faster
when FFS storage is used, rather than SIM.

Decoding received SMS binary files with pcm-sms-decode
======================================================

Whether you have retrieved your SMS store from a SIM with fc-simint/fc-simtool
save-sms-bin command or read out Pirelli's /pcm/SMS file with fc-fsio, the
binary format is exactly the same: a bundle of 176-byte records directly abutted
together, with each individual record having the standard format defined in GSM
TS 11.11 and its successor 3GPP TS 51.011.  Looking at raw hex dumps won't be
very helpful, as the reversed nibble format used for phone numbers and SC
timestamps is not directly human-readable, and the message bodies will usually
be in the form of packed septets - hence a special decoding program is needed.

Our pcm-sms-decode utility does the needed job of reading these binary files and
fully decoding them into human-readable ASCII, or ISO 8859-1 or UTF-8 if your
host system supports such extended character sets and you have received SMS
content that uses beyond-ASCII characters.  pcm-sms-decode was added to FC host
tools suite in early 2020; it is based on and shares most of its code with our
earlier sms-pdu-decode utility from 2018.  (The latter is meant for decoding SMS
PDUs retrieved from FreeCalypso AT command modems with fcup-smdump.)

pcm-sms-decode uses the same backslash escapes in its output and accepts the
same -e, -u and -h options as sms-pdu-decode - please refer to the description
of sms-pdu-decode in the User-phone-tools article for the details.

By default, pcm-sms-decode numbers the records being decoded as 0 to N-1, where
N is the total number of records in the binary file given to it (file size equal
to N*176 bytes).  As of fc-host-tools-r15, newly added -s option causes it to
number these same records as 1 to N instead.  The original 0-based record
numbering was implemented when the tool was used solely to decode /pcm/SMS from
Pirelli's fw (before the introduction of fc-simtool), but now that we have
fc-simtool and fc-simint, and given that the SIM interface protocol and SIM file
system definition use 1-based record numbers, we need to support the same
convention in pcm-sms-decode for consistency.