FreeCalypso > hg > sms-coding-utils

annotate doc/SMS-PDU-decoding @ 31:19476164c54d

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
doc/SMS-PDU-decoding: document imported utilities
author Mychaela Falconia <falcon@freecalypso.org>
date Fri, 14 Jun 2024 18:48:58 +0000
parents
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
31
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
1 The decoding part of the present sms-coding-utils suite consists of two
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
2 programs: sms-pdu-decode and pcm-sms-decode. Their functions are as follows:
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
3
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
4 * The input to sms-pdu-decode is an ASCII text stream (stdin or read from a
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
5 file) in which every SMS PDU to be decoded appears as a long hex string.
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
6 This input can originate from the GSM 07.05 interface on a FreeCalypso GSM MS
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
7 (fcup-smdump utility in FC host tools), in which case every GSM 03.40 TPDU
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
8 will be preceded by an SC address field - the original use case for
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
9 sms-pdu-decode, run it without special options. In the other alternative,
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
10 the input to sms-pdu-decode can originate from test scenarios on the network
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
11 side of GSM (SMSC development and testing), in which case input SMS PDUs will
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
12 be pure GSM 03.40 TPDUs, without SC address prefix - use sms-pdu-decode -n
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
13 option in this case.
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
14
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
15 * The input to pcm-sms-decode is a binary file with 176 bytes per record,
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
16 corresponding to the format of EF_SMS elementary file on SIM cards. This
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
17 program can be used to decode readouts of this EF_SMS file made with
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
18 fc-simtool, or readouts of /pcm/SMS file in the flash file system of Pirelli
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
19 DP-L10 phone, which uses the same format - the latter use case arose first in
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
20 chronological order of FreeCalypso development, hence the name of the utility.
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
21
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
22 Common options: character set and dump format control
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
23 =====================================================
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
24
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
25 By default, sms-pdu-decode and pcm-sms-decode only emit 7-bit ASCII characters
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
26 in their output; any GSM7 or UCS-2 characters which fall outside of this plain
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
27 ASCII repertoire are converted into backslash escapes. This conservative
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
28 default behaviour can be modified as follows:
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
29
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
30 -e option extends the potential output character repertoire from 7-bit ASCII to
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
31 8-bit ISO 8859-1. Any 8859-1 high characters are emitted as single bytes,
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
32 i.e., are NOT encoded in UTF-8 - this option is intended for non-UTF-8
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
33 environments.
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
34
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
35 -u option extends the potential output character repertoire to all of Unicode,
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
36 and changes the output encoding to UTF-8.
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
37
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
38 Regardless of whether the source message character set is GSM7 or UCS-2 and
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
39 irrespective of -e or -u options, any backslash characters are always escaped
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
40 as \\, and any CR characters are represented as \r. Additional backslash
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
41 escape encodings depend on the source message character set:
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
42
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
43 * If the source message character set is GSM7, the following additional
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
44 backslash escapes can be emitted:
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
45
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
46 - In the absence of -u option, the Euro currency symbol is converted to \E;
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
47
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
48 - Any GSM7 escape characters (0x1B) that aren't part of a valid escape
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
49 sequence for [\]^ or {|}~ or \E are represented as \e;
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
50
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
51 - Any GSM7 characters that either can't be represented in the output character
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
52 set (ASCII or ISO 8859-1) or are outright invalid per GSM 03.38 are
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
53 represented as \xX, where xX is the original GSM7 code point in 2-digit
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
54 hexadecimal form between 00 and 7F;
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
55
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
56 - Invalid GSM7 escape sequences are emitted as \e\xX.
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
57
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
58 * If the source message character set is UCS-2, the following additional
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
59 backslash escapes can be emitted:
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
60
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
61 - Invalid UCS-2 characters falling onto control character code points are
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
62 emitted as \u00XX;
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
63
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
64 - UCS-2 characters that can't be represented in ASCII or ISO 8859-1 (when
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
65 running without -u option) are emitted as \uXXXX;
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
66
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
67 - If UTF-16 surrogate pairs are detected in the input, the encoded high-plane
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
68 Unicode character is reconstructed and emitted as \UXXXXXX in the absence
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
69 of -u option, or as the appropriate UTF-8 byte sequence with -u.
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
70
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
71 -h option causes the user data portion of every message to be displayed as a
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
72 raw hex dump; in the case of GSM7-encoded messages, this hex dump shows the
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
73 unpacked septets.
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
74
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
75 sms-pdu-decode specifics
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
76 ========================
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
77
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
78 The input to the program may contain additional text besides SMS PDUs in the
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
79 form of long hex strings; all lines that are not hex strings are passed through
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
80 to the output. Every input line that is purely a string of directly abutted hex
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
81 bytes is taken to be an SMS PDU in need of decoding, and the full decoding
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
82 operation is attempted. The following additional options are available besides
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
83 the common -e, -u and -h options documented above:
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
84
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
85 -n By default, sms-pdu-decode expects every hex-encoded SMS PDU to begin
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
86 with an SC address field, followed by a GSM 03.40 TPDU - the format used
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
87 on GSM 07.05 interface in PDU mode and in SIM SMS storage. With -n
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
88 option, sms-pdu-decode expects pure GSM 03.40 TPDUs instead, without
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
89 SC address prefix.
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
90
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
91 -p Keep all hex-encoded PDU lines in the output: for each encountered hex
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
92 PDU, first the original hex line is output, then the decoding result.
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
93
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
94 pcm-sms-decode specifics
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
95 ========================
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
96
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
97 This program reads a binary file; the file to be read must be named on the
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
98 command line. The output is ASCII (or an extended character set with -e or -u
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
99 options as described in the common section above), naming each dumped record as
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
100 "Record #%u" and showing its content. For a binary file of N records, the
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
101 default record numbering is from 0 to N-1: this numbering order is natural to
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
102 this Mother's native world of CompSci, and I implemented it when I originally
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
103 wrote pcm-sms-decode for the purpose of decoding /pcm/SMS readouts from Pirelli
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
104 DP-L10 FFS. However, when I later wrote fc-simtool and pcm-sms-decode acquired
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
105 a second use case of decoding SIM EF_SMS readouts, a mismatch became apparent:
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
106 the record numbering used in READ RECORD and UPDATE RECORD commands on the
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
107 SIM-ME interface is 1..N instead of 0..N-1. pcm-sms-decode -s option switches
19476164c54d doc/SMS-PDU-decoding: document imported utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff changeset
108 the record numbering scheme to 1..N to match the SIM application.