--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/doc/User-oriented-commands	Sun Mar 14 07:57:09 2021 +0000
@@ -0,0 +1,554 @@
+This document describes those commands and functions of fc-simtool which can be
+exercised by end users on any regular operator-issued SIM, without requiring a
+special programmable SIM with admin privileges.  The Mother's plans for future
+development include a companion fc-simint utility that will operate on SIM cards
+inside Calypso phones; the intent is that all of the end-user-oriented commands
+of fc-simtool described in this document will also be replicated in fc-simint.
+
+Understanding SIM PIN1
+======================
+
+Every standard SIM card has a secret code called PIN1; this secret code can be
+anywhere between 4 and 8 digits in length, with 4-digit PINs being most common.
+In terms of persistent non-volatile state, SIM PIN1 can be enabled or disabled.
+When SIM PIN1 is disabled, all regular functions of the card are enabled, as in
+being able to power up the phone with the SIM in it and connect to the GSM
+network with your subscriber identity, and being able to read and write SIM user
+data content like phonebooks and stored messages - all of these functions are
+enabled from the moment you turn on the phone with the SIM in it (or power the
+SIM up by itself in a smart card "reader" driven by fc-simtool), without the
+user ever being asked for a PIN, such that you can forget that the PIN even
+exists - this situation in very common nowadays.  But when SIM PIN1 is enabled,
+the smart chip in the SIM will not allow you access to any of the data stored
+on the card and will not allow any GSM authentication operations until and
+unless you send the correct PIN to the SIM in the VERIFY CHV command.
+
+If you forgot your PIN1, the only way to reset it is to enter another secret
+code (always 8 digits in length) called PUK1.  If the SIM is made according to
+standards, then its PUK1 is set to a random number during either physical
+manufacturing or administrative programming of the card and then remains
+unchangeable afterward.  Therefore, in an ideal world if someone forgot their
+PIN1 and don't have their PUK1 either, they should be able to obtain PUK1 from
+the cellular operator who issued the SIM - but whether or not today's operators
+will actually help such hapless users (without forcing them to get a new SIM)
+is another question altogether.  PUK1 is often printed on the big (credit-card-
+sized) plastic piece on which SIM cards are initially delivered - but it doesn't
+help if you originally got your SIM many ages ago and no longer have that
+souvenir plastic piece.
+
+The standard protocol for communicating with SIM cards provides 5 special
+commands that are dedicated to working with PIN1, and so does fc-simtool:
+
+verify-pin1 XXXX
+
+This command tells the SIM that you are attempting to prove knowledge
+of PIN1, presenting a string of digits.  If the PIN digits you specify match
+the PIN1 secret code stored inside the SIM, the card unlocks access to its
+primary functions.  If the digits you send are wrong, the SIM decrements its
+non-volatile attempt counter, giving you a total of 3 attempts (irrespective of
+card power-downs between attempts) to enter the correct PIN.  If PIN1 is entered
+incorrectly 3 times in a row, this PIN is blocked, and the only way to unblock
+it is via PUK1.
+
+enable-pin1 XXXX
+
+This command changes the non-volatile state of the PIN1 enable/disable flag,
+such that from now on the SIM will require PIN1 to be provided on every card
+power-up before it will allow GSM authentication and access to user data.  The
+enable-pin1 operation itself requires correct PIN1 digits to be provided.
+
+disable-pin1 XXXX
+
+This command changes the non-volatile state of the PIN1 enable/disable flag,
+such that from now on the SIM will NOT require PIN1 to be provided on every
+card power-up, and will instead be live immediately without needing proof of
+card owner's identity.  The disable-pin1 operation itself requires correct PIN1
+digits to be provided.
+
+change-pin1 old-PIN new-PIN
+
+This command tells the SIM that you wish to change PIN1 secret code to some new
+digits.  Knowledge of the old PIN1 is required for this operation to succeed.
+
+unblock-pin1 PUK1-secret-code new-PIN1
+
+This command tells the SIM that you are attempting to prove knowledge
+of PUK1 and to set new PIN1.  If PUK1 is given correctly, the new PIN1 will be
+set.  If you enter wrong PUK1, the SIM decrements its non-volatile attempt
+counter, giving you a total of 10 attempts (irrespective of card power-downs
+between attempts) to enter the correct code.  If PUK1 is entered incorrectly 10
+times in a row, it is blocked and the card should be considered bricked beyond
+recovery.
+
+Understanding SIM PIN2
+======================
+
+GSM standards provide support for a very rarely used feature that works in the
+spirit of "parental controls": if you authenticate to the SIM with PIN2 secret
+code (which has to be different from PIN1 for meaningful security), you can
+edit a SIM-resident list of so-called Fixed Dialing Numbers (FDN), and then all
+standard phones that implement this feature per the spec will refuse to allow
+ordinary users (authenticated with PIN1 or with no PIN at all) to call any
+numbers other than those programmed in FDN.
+
+This whole "parental control" feature is totally silly and is not expected to be
+of any practical use, but the whole purpose of fc-simtool is to allow every
+feature of SIM cards to be exercised, hence we provide the necessary support.
+The following commands work just like their PIN1 counterparts:
+
+verify-pin2 XXXX
+change-pin2 old-PIN new-PIN
+unblock-pin2 PUK2-secret-code new-PIN2
+
+Unlike PIN1, PIN2 cannot be disabled per traditional SIM card standards.
+
+Getting basic info from the SIM
+===============================
+
+The following commands are available for retrieving basic info from the SIM:
+
+iccid
+
+This command retrieves the ICCID (Integrated Circuit Card ID) record from the
+SIM - it is a number of up to 20 digits (although 19-digit ICCIDs are most
+common) that identifies the SIM card as a physical artifact.  If your SIM is of
+the traditional operator-issued kind, as opposed to a developer-oriented
+programmable SIM from vendors like Sysmocom who have different ideas, this ICCID
+will usually be the SIM card ID number printed on the physical plastic, along
+with a barcode representation of the same number.
+
+imsi
+
+This command retrieves the IMSI (International Mobile Subscriber Identity) from
+the SIM - it is the most fundamental ID token by which GSM phones present
+themselves to networks, and they even use the first 5 or 6 digits of the IMSI
+to decide which network they should try connecting to first.
+
+It should also be noted that if your SIM has FDN (Fixed Dialing Numbers) enabled
+and the card implements GSM SIM specs to the letter, including the idiotic
+parts, then you will need to issue a rehab-imsi command before you can read the
+IMSI record - see the FDN section further in this document.
+
+sst
+
+Every SIM card is required to have an essential data record (an EF in technical
+terms) called the SIM Service Table, or SST.  This SST indicates which services
+are allocated and activated on the given SIM.  Our sst command lists all
+allocated service numbers, listing just a plain number if the service is both
+allocated and activated (the usual case), or a number with a '^' suffix if the
+service is allocated but not activated.  You will need to look in the 3GPP TS
+51.011 spec to make sense of these service numbers.
+
+user-sum
+
+This command displays a user-friendly summary of user-oriented services present
+on the SIM.  It reads SST to get the list of available and activated services,
+but it considers only user-oriented ones (as opposed to SIM services dealing
+with GSM network functions or serving operators' interests rather than users'),
+and it displays them in a user-friendly manner.  For each present SIM phonebook
+(ADN, FDN, SDN) and for the SMS store, user-sum displays the storage capacity
+provided by the SIM (number of phonebook entries or messages), and for each of
+the various phonebooks, the allocated number of alpha tag bytes is also
+displayed.
+
+The number of bytes allocated for the alpha tag in SIM phonebooks determines
+the maximum length of the name field in each phonebook entry.  These name fields
+can be written either in GSM7 encoding (GSM 03.38 aka 3GPP 23.038) or in UCS-2;
+when GSM7 encoding is used, no SMS-style septet packing is applied - instead the
+high bit of each byte is simply cleared.  Therefore, the maximum number of
+characters in a phonebook entry name field usually equals the number of bytes
+allocated for the alpha tag on the SIM, except for names containing ASCII
+characters [\]^ and {|}~ which get expanded to 2-character escape sequences in
+GSM7 encoding.
+
+uicc-dir
+
+If your SIM card functions not only as a classic GSM 11.11 SIM, but also as a
+UICC with USIM/ISIM or other UICC-based applications, it will have a file named
+EF_DIR in its file system, listing those applications.  fc-simtool uicc-dir
+command dumps the content of this file in a human-readable form - but please
+note that fc-simtool only speaks the classic GSM 11.11 protocol to the SIM, and
+not the UICC protocol.  EF_DIR does not officially exist in the classic GSM SIM
+spec, hence the dir command in fc-uicc-tool (speaking the UICC protocol) is the
+official way to read and dump the content of EF_DIR.
+
+Manipulating SIM phonebooks
+===========================
+
+GSM SIM specs allow for several different phonebooks to be present on the card:
+
+* ADN (Abbreviated Dialing Numbers) is the main SIM phonebook.  Each SIM card
+  issuer decides how much storage space they allocate to ADN (how many records);
+  the SIM spec maximum is 254 records, and many issuers' SIMs do provide this
+  many records or close to this limit.
+
+* FDN (Fixed Dialing Numbers) is the "parental control" phonebook.  The FDN
+  phonebook can only be written to after authenticating with PIN2, and when it
+  is enabled (enabling FDN is done by "invalidating" ADN, an operation which
+  also requires PIN2), spec-compliant phones allow only numbers in FDN to be
+  called.
+
+* SDN (Service Dialing Numbers) is a service-provider-controlled phonebook: it
+  can only be written if you have special admin privileges (ADM authentication
+  method is card-vendor-dependent), and it is read-only to ordinary users.
+
+* MBDN (Mailbox Dialing Numbers) is a late addition to GSM SIM specs - it is a
+  special phonebook that stores the number for Voice Mail and other related
+  esoteric services.
+
+* MSISDN is a phonebook-like file that stores the subscriber's own phone
+  number(s).  Most classic GSM phones have a menu command for showing your own
+  number, usually called "My number" or something like that; this menu command
+  displays the first record stored in the MSISDN phonebook.  Most network
+  operators update this MSISDN record over the air (using special SMS-encoded
+  commands) when you activate service or get a new phone number without changing
+  your SIM, but this MSISDN store in the SIM also has some interesting
+  properties:
+
+  + Per the spec the MSISDN phonebook is writable by ordinary users, not just
+    admins, and the Mother's experience with real T-Mobile SIMs is that they do
+    indeed allow the user to write anything into MSISDN.
+
+  + Most SIM card issuers allocate multiple records for MSISDN, not just one.
+    It is not clear if ordinary end user phones would do anything useful with
+    the extra records if one were to write something there.
+
+fc-simtool provides a unified set of commands and data formats for working with
+all SIM phonebooks: all pb-* commands take the name of the phonebook to be
+operated on as their first argument.  The following commands are available:
+
+pb-dump PBNAME
+
+This command dumps the full content of the selected phonebook on the terminal.
+The data format for representing SIM phonebook content in UNIX-based text files
+and dumps is described in the SIM-data-formats document in the freecalypso-docs
+repository.
+
+pb-dump PBNAME > outfile
+
+This form of the pb-dump command dumps the full content of the selected
+phonebook, but saves it in the named file instead of sending it to the terminal.
+This form is ideal for making backups of large SIM phonebooks.
+
+pb-dump-rec PBNAME rec
+
+This command dumps a single record from a potentially large phonebook.
+
+pb-dump-rec PBNAME start-rec end-rec
+
+This command dumps the specified range of records from a potentially large
+phonebook.
+
+pb-restore PBNAME filename
+
+This command reads a phonebook data file in the format described in the
+SIM-data-formats document and uploads it into the named SIM phonebook.  Every
+record in the SIM phonebook is overwritten with an UPDATE RECORD command; those
+record indices which do not appear in the data file being restored get blank
+records (0xFF in every byte) written into them.
+
+pb-update PBNAME filename
+
+This command reads a phonebook data file in the format described in the
+SIM-data-formats document and uploads it into the named SIM phonebook, writing
+only those record indices which appear in the data file - each record from the
+data file gets written into the SIM with an UPDATE RECORD command, while all
+other record locations remain untouched.
+
+pb-update-imm PBNAME rec phone-number [alpha-tag]
+
+This command writes a single phonebook entry directly from the command line,
+without going through a data file.  The specific record index to write into must
+always be specified (there is no built-in "find first empty record" function),
+and the entry format for both the phone number and the alpha tag is more relaxed
+compared to the very strict format required in data files:
+
+* The phone number can begin with a '+' character for international format;
+
+* The comma-separated TON/NPI byte is optional and will usually be omitted in
+  ordinary usage - this byte will default to 0x91 if the number begins with '+'
+  or to 0x81 otherwise;
+
+* Double-quotes around the alpha tag argument are required only if it contains
+  spaces or other problematic characters, and can be omitted otherwise;
+
+* If the alpha tag is empty, the last argument can be omitted altogether.
+
+pb-update-imm-hex PBNAME rec phone-number alpha-tag-hex
+
+This command is like pb-update-imm, but the alpha tag argument (required for
+this command) is given in hex - intended for creating phonebook entries with
+UCS-2 alpha tags.
+
+pb-erase PBNAME
+
+This command fully erases the named phonebook.
+
+pb-erase-one PBNAME rec
+
+This command erases the specified individual record in the named phonebook.
+
+pb-erase-range PBNAME start-rec end-rec
+
+This command erases the specified range of records in the named phonebook.  The
+starting record must be identified by number (SIM record numbers are 1-based);
+the ending record argument may be either a number or the "end" keyword.
+
+Enabling and disabling FDN
+==========================
+
+The Fixed Dialing Numbers (FDN) mechanism is normally disabled.  The protocol
+prescribed by GSM SIM specs is that FDN is enabled when the regular ADN
+phonebook is invalidated, and is disabled (unrestricted dialing allowed)
+otherwise.  fc-simtool provides commands for invalidating and rehabilitating
+ADN, thereby enabling and disabling FDN:
+
+inval-adn
+
+This command invalidates ADN and thereby enables FDN.
+
+rehab-adn
+
+This command rehabilitates ADN and thereby disables FDN.
+
+The SIM will only allow inval-adn and rehab-adn operations after you have
+successfully authenticated with PIN2 - see verify-pin2 command description.
+
+GSM SIM specs also stipulate a certain hack to prevent FDN-ignorant phones from
+making "forbidden" unrestricted calls: the specs stipulate that when a SIM
+powers up in an FDN-enabled state (ADN is invalidated), the "smart" logic in
+the SIM invalidates two essential files EF_IMSI and EF_LOCI (needed for GSM
+operation), requiring the phone (ME) to rehabilitate these two files at the
+beginning of every SIM session when FDN is in use.  The thinking must have been
+that if a given ME knows how to do these extra rehab-imsi, rehab-loci steps,
+then it also knows about FDN and will honor it.  Our answer: OK, whatever - but
+we do provide rehab-imsi and rehab-loci commands in fc-simtool.  These
+operations require only CHV1 access, thus PIN1 or no PIN at all depending on
+whether or not PIN1 is enabled - no need for PIN2.
+
+Last Number Dialed (LND)
+========================
+
+Traditional SIMs include a cyclic file that is intended to be updated whenever
+an outgoing call is dialed - but it is up to individual phone designs whether
+they actually update this LND cyclic store or not.  This SIM LND store has the
+same record format as phonebooks, carrying only phone numbers and optional alpha
+tags - there are no fields for date & time, call duration or status as in call
+answered or not.  Because of the limitations of this SIM LND store, most phone
+designs do not use it, and instead go with their own implementation of call
+history lists.
+
+Because this LND store is a cyclic file, not linear fixed like phonebooks, it
+does not allow random access writes: it allows random access reads like all
+regular record-based files, but the only write operation allowed by the SIM
+interface protocol and the SIM file system architecture is writing a new record
+that becomes the new #1, shifting all previous records down and losing the
+oldest one.  Because of this write access limitation, we do not provide the same
+set of operations on LND as for regular phonebooks - but we still provide good
+tinkering ability.  The following commands are available:
+
+lnd-dump
+
+This command dumps the content of the LND store on the terminal, in the same
+format as pb-dump for regular phonebooks.
+
+If you have had your SIM for a very long time, having used it in different
+phones with different firmwares, it may be interesting to look at the output of
+lnd-dump - you may have LND records that were generated ages ago by other
+phones if your current one does not write into SIM LND.
+
+lnd-dump > outfile
+
+This form of the lnd-dump command produces the same dump format, but saves it
+in the named file instead of sending it to the terminal.
+
+lnd-restore filename
+
+This command reads the named phonebook data file (presumably written previously
+with lnd-dump) and writes it into EF_LND on the SIM.  This command works by
+first constructing a full binary image of the desired EF_LND content, then
+writing every record in the reverse order from the last index to the first.
+
+lnd-write phone-number [alpha-tag]
+
+This command writes a new record into the LND cyclic store just like a standard
+phone would do when making a record of a new outgoing call.  The two arguments
+(one required and one optional) are the same as for pb-update-imm.
+
+lnd-erase
+
+This command erases the EF_LND cyclic store, making it appear as if no outgoing
+calls have ever been recorded.  It works by writing a blank record (0xFF in
+every byte) N times, where N is the size of the cyclic store in records.
+
+Manipulating stored SMS
+=======================
+
+The fundamental operating model of all message stores for SMS (whether SIM or
+phone-based) is that received messages accumulate (and possibly sent ones too,
+if they are stored in this manner), the limited available memory fills up, and
+then the user needs to clean out the accumulated messages, preferably also
+archiving them by transferring to a larger computer for longer-term storage.
+Given this fundamental operating model, we only need to provide commands for
+dumping the content of the message store and for cleaning it out - there is no
+real need to implement commands for writing messages into the store.
+
+The extent of special support for the SIM SMS store in fc-simtool is rather
+minimal because it just so happened that we already have external tools that do
+a major part of the work.  Some phone firmwares, particularly that of the
+Pirelli DP-L10 phone currently used by the Mother, implement their on-the-phone
+SMS storage by way of a file in their local flash file system whose binary
+format just happens to be exactly the same as the binary format of SIM-based
+EF_SMS if all 176-byte records are simply abutted together in the host-based
+binary representation.  A few release cycles ago we added a new utility named
+pcm-sms-decode to our FreeCalypso host tools suite; this utility reads a binary
+file in this "EF_SMS records concat" format and performs the quite involved job
+of fully decoding all messages into human-readable form.  Given that we have
+this external pcm-sms-decode utility, all we need to do in fc-simtool is save
+all records of EF_SMS into a single concatenated binary file, and let
+pcm-sms-decode do the rest.
+
+Our dedicated commands for working with the SIM SMS store are as follows:
+
+save-sms-bin host-filename
+
+This command saves the full content of EF_SMS in the named file in the host file
+system in binary format, suitable for further decoding with pcm-sms-decode.
+
+sms-erase-all
+
+This command erases every record entry in EF_SMS.
+
+sms-erase-one rec
+
+This command erases the specified individual record in EF_SMS.
+
+sms-erase-range start-rec end-rec
+
+This command erases the specified range of records in EF_SMS.  The starting
+record must be identified by number (SIM record numbers are 1-based); the
+ending record argument may be either a number or the "end" keyword.
+
+Manipulating SMS parameters
+===========================
+
+SIM cards have an SMS parameter store in the form of record-based file EF_SMSP.
+Its most essential function is to specify the Service Centre Address for
+outgoing SMS, but it can also be put to a few other uses:
+
+* The primary SMSP record that gives the SC address also typically includes PID
+  and DCS parameters.  The only sensible settings that can function as a
+  general-purpose default are PID=0x00 and DCS=0x00, but some SIMs have been
+  seen in the field that set bogus PID and DCS via their SMSP.  It appears that
+  most end user phones ignore these settings, and they have no effect when
+  outgoing SMS are submitted to an AT command modem in PDU mode, but these
+  settings do affect our TI-based AT command modem in text mode - if they are
+  bogus on the SIM, they need to be fixed, either with fc-simtool or in the
+  actual AT modem session with AT+CSMP.
+
+* The same primary SMSP record can also specify a default validity period in
+  one-byte relative VP format.
+
+* Just like the situation with MSISDN, even though only the first record of
+  EF_SMSP is used in practice, most SIM issuers allocate room for a few records.
+  These extra SMSP records are almost always blank,
+
+fc-simtool provides the following commands for working with EF_SMSP:
+
+smsp-dump
+
+This command dumps the full content of EF_SMSP (all records) on the terminal,
+using a lossless text-based format similar to the one we use for phonebooks.
+To illustrate our smsp format by way of examples, here is the output of
+smsp-dump from old T-Mobile USA SIMs that have classic GSM 11.11 SIM
+functionality:
+
+#1: SC=12063130004,0x91 PID=0x00 DCS=0x00 "T-Mobile"
+#2: ""
+#3: ""
+#4: ""
+
+Here is the output from an Austrian S-Budget Mobile SIM from circa-2017:
+
+#1: SC=4365009000000,0x91 PID=0xFF DCS=0xFF VP=173 ""
+#2: ""
+
+As one can see from these examples, T-Mobile allocated 4 records for their
+EF_SMSP, whereas S-Budget Mobile allocated only 2 records for theirs.
+(Sysmocom webshop SIMs sysmoUSIM-SJS1 and sysmoISIM-SJA2 also have 2 records in
+their EF_SMSP.)  Yet only the first record is actually used, and the remaining
+ones are blank.  Note that unlike pb-dump, smsp-dump does not skip blank
+records: it displays every record (the design rationale is that the total number
+of EF_SMSP records is expected to be small), and a blank record is simply one
+that has no parameters present and has an empty alpha tag.
+
+The following parameters may be present in each SMSP record, appearing in the
+smsp-dump output in the same order in which they appear in the SIM binary
+record:
+
+DA=	TP-Destination_Address
+SC=	TS-Service_Centre_Address
+PID=	TP-Protocol_Identifier
+DCS=	TP-Data_Coding_Scheme
+VP=	TP-Validity_Period
+
+The phone numbers in DA= and SC= parameters are emitted in the same format as
+in pb-dump, PID= and DCS= are emitted in hexadecimal with a 0x prefix, and VP=
+is emitted in decimal.  The alpha tag is always emitted at the end of the ASCII
+line, just like in pb-dump.
+
+smsp-dump > outfile
+
+This form of the smsp-dump command produces the same dump of EF_SMSP, but saves
+it in the named file instead of sending it to the terminal.
+
+smsp-restore filename
+
+This command reads a file written by smsp-dump and writes it back to the SIM.
+Both decimal and 0x-prefixed hexadecimal forms are accepted for all 3 of PID=,
+DCS= and VP= parameters.
+
+smsp-set rec params
+
+This command writes a single record into SMSP directly from the command line,
+without going through a data file.  The record index to write to must be given,
+followed by one or more parameters as in DA=, SC=, PID=, DCS= or VP=.  DA= and
+SC= phone numbers can be entered in the same relaxed form as in the
+pb-update-imm command, and the remaining 3 parameters can be either decimal or
+0x-prefixed hexadecimal.  This command leaves the alpha tag field blank.
+
+smsp-set-tag rec alpha-tag params
+
+This command is just like smsp-set, but adds an alpha tag argument.
+
+smsp-erase-all
+
+This command erases every record entry in EF_SMSP.
+
+smsp-erase-one rec
+
+This command erases the specified individual record in EF_SMSP.
+
+smsp-erase-range start-rec end-rec
+
+This command erases the specified range of records in EF_SMSP.  The starting
+record must be identified by number (SIM record numbers are 1-based); the
+ending record argument may be either a number or the "end" keyword.
+
+Identifying MVNO SIMs
+=====================
+
+Many SIMs, particularly those from MVNOs, are programmed by their issuers to
+cause phones to display the name of the MVNO or some other party rather than
+the standard PLMN name decoded from the connected network's MCC-MNC.  This
+"personalization" programming can appear in EF_SPN (old style) or in EF_PNN and
+EF_OPL (newer style).  fc-simtool provides commands to display the content of
+these SIM files in human-readable form:
+
+spn
+pnn-dump
+opl-dump
+
+These commands take no arguments, and their human-readable output is not
+explained in detail here.  If you need to understand the meaning of various
+fields in detail, please refer to 3GPP TS 51.011.