FreeCalypso > hg > freecalypso-tools

comparison doc/TCH-bit-access @ 8:2748f257312b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
doc/TCH-bit-access: new write-up
author Mychaela Falconia <falcon@freecalypso.org>
date Tue, 14 Jun 2016 07:13:46 +0000
parents
children 3de3b34189be
comparison
equal deleted inserted replaced
7:08804864172a 8:2748f257312b
1 It has been discovered that the DSP ROM in the Calypso GSM baseband processor
2 implements one nifty feature which is not used at all in standard phone or modem
3 operation, but which can be used for all kinds of interesting hacks: the traffic
4 channel (TCH) bits coming out of the GSM 05.03 channel decoder in the downlink
5 direction (to be fed to one of several speech decoders) can be read out of the
6 DSP's API RAM in real time, and in the uplink direction the user can feed her
7 own bits to the input of the GSM 05.03 channel encoder, effectively suppressing
8 the output of the internal vocoder.
9
10 The DSP mechanism in question is known to work in TCH/FS and TCH/EFS channel
11 modes, corresponding to the FR and EFR codecs; it is not currently known if it
12 can also be used in TCH/HS, data traffic or AMR modes. I am aware though of an
13 anecdotal report that someone tried to make this feature work with AMR, but was
14 unsuccessful - hence we should be prepared for the possibility that the hack
15 is not possible with AMR.
16
17 In order to make use of this TCH bit access feature, one needs 3 things:
18
19 1) Firmware on the Calypso (the ARM part) that reads downlink bits from the DSP
20 and writes uplink bits into it while doing everything else that a GSM fw must
21 do in order to operate the MS;
22
23 2) Some protocol for passing these TCH bits into and out of the Calypso device;
24
25 3) A source for TCH UL bits and a sink for TCH DL bits on the external host.
26
27 In the case of FreeCalypso, we have defined our own protocol for passing TCH
28 bits into and out of Calypso GSM devices running one of our firmwares in the
29 form of an extension to TI's RVTMUX interface, i.e., we have defined a new
30 RVTMUX channel for this TCH interface and defined the packet types and formats
31 to be sent over the wire. On the Calypso side this interface is implemented in
32 FreeCalypso GSM firmwares (currently only in Citrine, but we can also make a
33 special version of TCS211 with this feature added if the need arises), and on
34 the host tools side there is support in rvinterf and fc-shell.
35
36 The TCH bit access mechanism in FreeCalypso has been designed with an objective
37 of presenting to the user exactly what TI's DSP presents to us, i.e., standing
38 out of the way as much as possible. TI's DSP presents TCH downlink bits and
39 accepts TCH uplink bits in the form of an array of 16-bit words; the bit order
40 within these words corresponds to the GSM 05.03 channel encoder bit order (with
41 a couple of TI-specific quirks documented below) and NOT that of the GSM 06.10
42 or EFR codecs. On the RVTMUX serial interface between the Calypso device and
43 the external host we transfer each TCH frame as a block of 33 bytes; our Calypso
44 firmwares translate between these bytes and the DSP's 16-bit words, but do not
45 reorder or change the bits in any way.
46
47 On the host tools side our fc-shell utility provides user commands to save TCH
48 DL bits into a file and to play TCH UL bits from a file; in the present version
49 these files are written and read in an ASCII-based hex format. In these ASCII
50 files each TCH frame is represented as a string of 66 hexadecimal digits, and
51 these hex digits correspond directly to the 33 bytes being read out of or
52 written into DSP API words. Therefore, in order to generate and/or interpret
53 these hexadecimal strings correctly, you (the user) need to understand the bit
54 order and mapping used by TI's implementation of the GSM 05.03 channel encoder.
55
56 Recall from the GSM specs that the 260 bits which comprise one speech frame are
57 not all treated equally, instead they are divided into 182 class 1 bits which
58 are protected by a convolutional encoder and 78 class 2 bits which are
59 transmitted without any forward error correction. Furthermore, the first 50 of
60 the class 1 bits are also protected by a CRC. The order in which the bits
61 appear on TI's DSP interface corresponds to this division, known as the order of
62 subjective importance.
63
64 Now let's look at the actual bit order and mapping which you need to understand
65 in order to make sense of the hex strings in tch record and tch play files.
66 The bit numbering is from the most significant bit to the least significant bit,
67 i.e., in our string of 66 hex digits the most significant bit of the leftmost
68 digit is bit 0 and the least significant bit of the rightmost digit is bit 263.
69 TI's DSP assigns these bits as follows:
70
71 * Bits 0 through 181 correspond to the 182 protected (class 1) bits in the
72 standard GSM 05.03 order;
73
74 * Bits 182 through 185 are unused - put zeros there if you are generating them
75 yourself;
76
77 * Bits 186 through 263 correspond to the 78 unprotected (class 2) bits in the
78 standard GSM 05.03 order.
79
80 Uplink testing
81 ==============
82
83 The uplink sending mechanism can be exercised as follows:
84
85 1. Record a speech sample in the GSM 06.10 codec format using any Unix/Linux
86 audio tool that can write the de facto standard libgsm format. For example,
87 using SoX:
88
89 rec -c1 recording.gsm
90
91 SoX will write the recording in the GSM 06.10 libgsm format based on the
92 .gsm suffix at the end of the recording file name; the -c1 option is needed
93 to disable stereo, otherwise the recording will be slowed down 2x.
94
95 2. Convert it from libgsm standard format into our ad hoc hex strings for
96 playing into the TCH uplink:
97
98 fc-fr2tch recording.gsm recording.tch-ul
99
100 3. In fc-shell, when you are in an established voice call, issue this command:
101
102 tch play recording.tch-ul
103
104 You should now hear the speech sample you recorded in step 1 above on the other
105 end of the GSM call. Please note, though, that for this example to work, the
106 call must be connected in the FR codec mode, not EFR. If you try it on an EFR
107 call, the vocoder on the other end of the call will try to interpret your FR
108 codec bits per EFR, resulting in garbage. In order to make this procedure work
109 properly with EFR, you would need to generate a speech sample in EFR format and
110 then put it in the correct hex string form with the correct bit order for
111 feeding to TI's GSM 05.03 channel encoder implementation, and at the present
112 moment the tools to do this feat are lacking.
113
114 The fc-fr2tch step above is new with the current version of FreeCalypso host
115 tools. My original implementation of the tch play command in fc-shell read
116 libgsm GSM 06.10 speech files directly, but I changed it to read hex strings
117 which correspond literally to what goes into TI's implementation of the 05.03
118 channel encoder to make it more general:
119
120 1) to support playing EFR speech bits into TCH/EFS channels;
121 2) to support people who may want to pass totally non-speech data over traffic
122 channels in TFO mode - see below.
123
124 Downlink testing
125 ================
126
127 When you are in an established voice call in fc-shell, you can record the
128 downlink TCH bits as follows:
129
130 tch record <name of file to put the recording into>
131
132 If you would like to record an entire call from the beginning, issue the
133 tch record command as above before the ATD or ATA command that dials or answers
134 the call. Either way, whether you are recording a call from the beginning or
135 from the middle, you need to eventually stop your recording with this command:
136
137 tch record stop
138
139 You can issue this stop command before or after the call is terminated, but
140 until you issue this tch record stop command, the output file is not closed and
141 thus may not be written to the file system.
142
143 The recording is written in an ASCII format that is similar but not identical
144 to what the tch play command takes as its input. Here is an example of what
145 you might get from tch record:
146
147 C214 281D 0063 81C008000008200046000000000000000000000000000007056608060B0A010B09
148 C204 0540 003A 9391480D3051F4BD81DF5EB35069BEBC4AEDF756351C4C19689BB1CA4EA5D4F5F5
149 C204 05A2 0047 65B80F9E690F7C8CE4ED80DEF69DA6436518AB99ABCE5815E6B562C5CE4EAC5DC5
150 C204 04CA 0044 A4483744B04371ED0334ECB350AF28C639B7F095519EF0242D299B6405124F77A5
151 C214 2544 0066 83800400000C1000E80000000000000000000000000000300674A07070F080D0B2
152 C200 4E8A 0000 07071DF0F83B9521EE61CFF095AA8C0E560300F6A5573C31F3E00601ED4AAE7E2F
153 C200 4C94 0000 077385ADE20450B2E410961D6C5B0A173ACF9E2D38D77C28CED8495D88AA4DE72E
154 C200 4ABC 0000 077F0CF86004132DA5C0A6D5A4B0BD4B28159A07D8F4282DC6AAAB27503BC02701
155
156 The example above is quoted from an actual recording made from a call to WWV at
157 +1-303-499-7111, a time of day service. This example shows 5 garbled frames
158 followed by 3 good ones.
159
160 In each line the first 3 space-separated 16-bit words are status words from the
161 DSP, and the rest is a hex string of 66 digits giving the 260-bit frame with 4
162 extra dummy bits at the boundary between the protected and unprotected portions.
163
164 Now here comes the unpleasant part: radio systems are inherently error-prone,
165 hence if you are naively expecting every received downlink frame to be a valid
166 speech frame that matches exactly what the other end transmitted, you will be
167 disappointed. The downlink frame stream will always consist of both valid and
168 invalid frames, and in the standard processing chain inside TI's DSP the speech
169 decoder block that receives these frames from the channel decoder will look at
170 the status words written by the latter in order to decide what to do with each
171 frame. But unfortunately I don't know the details, as the DSP is a mostly
172 undocumented black box.
173
174 Our FreeCalypso firmwares forward the downlink frame status words from the DSP
175 to the external host and my current implementation of the tch record command
176 writes them into the file because it is valuable information which should not
177 be discarded, but unfortunately I don't know their meaning. The first status
178 word consists of bit flags, the last one seems to be some kind of error count
179 (what kind of errors? I don't know), and the middle one is totally unknown.
180
181 There is also an fc-tch2fr utility that does sort of an inverse of fc-fr2tch,
182 except that fc-tch2fr expects files written by tch record which contain the
183 status words at the beginning - the files written by fc-fr2tch and read by
184 tch play don't have these status words; they just have 66 hex digits per line
185 giving the bits to be fed to the GSM 05.03 block in the DSP. The fc-tch2fr
186 utility works by disregarding the status words; the result will be a .gsm file
187 which you can play with SoX etc, but it will produce horrible sounds wherever
188 some garbled frames were received on the call downlink.
189
190 Passing non-speech data over TCH
191 ================================
192
193 If you are incredibly lucky and you happen to live in a part of the world where
194 the local GSM network implements TFO (tandem-free operation, see GSM 02.53),
195 i.e., if your GSM network transparently passes codec frames from one end of the
196 call to the other without transcoding back and forth in the middle, you might
197 be able to send arbitrary non-speech data bits over this TCH connection. If
198 you are going to attempt such a feat, you are mostly on your own as the GSM
199 network in my part of the world does not support TFO, but here are some general
200 ideas:
201
202 * Heed the difference between the 182 protected bits and the 78 unprotected
203 bits of each 260-bit frame. The bits in the first class are less likely to
204 be corrupted by radio errors. Also the first 50 out of these class 1 bits
205 are protected by a CRC, so if they get corrupted, there will be a flag bit
206 somewhere in the DSP status words - but I don't know which bit it is...
207
208 * Given the lack of detailed information about the exact meaning of the DSP
209 status words, implement some error checking of your own, e.g., an in-band
210 CRC within your frame payload so you can check on the receiving end if you
211 got an uncorrupted frame from your data source or not.
212
213 * Expect to lose some frames - even if your radio environment is pristine and
214 error-free, some frames will still be lost because their timeslots had to be
215 stolen by FACCH on either end of the call. You will need to implement
216 acknowledgements, retransmissions and retries - the usual stuff.
217
218 EFR differences
219 ===============
220
221 TCH/FS and TCH/EFS channel modes corresponding to the FR and EFR codecs,
222 respectively, share the same channel encoder that takes 260-bit frames as input.
223 An EFR speech frame is only 244 bits, but as the GSM 05.03 spec explains, each
224 EFR frame gets expanded to 260 bits by adding some CRC and repetition bits.
225 One would thus expect that the behaviour of the channel encoder block should be
226 strictly identical between TCH/FS and TCH/EFS channel modes, but it appears that
227 TI's implementation exhibits one difference in behaviour between the two modes.
228
229 It appears that TI's implementation of the GSM 05.03 channel encoder computes
230 the EFR-specific CRC bits internally, such that bits [65,72] of each uplink
231 frame fed to the DSP in the TCH/EFS mode need to be 0 and will be replaced by
232 the DSP with CRC bits computed per the standard. It also appears that their
233 implementation of the channel decoder verifies this CRC, sets a status bit
234 indicating whether the check succeeded or failed, and then zeroes these bits
235 out, i.e., the original bits in these CRC bit positions are lost. No problem
236 if what you are passing over TCH/EFS is indeed EFR speech frames, but it will
237 be a problem if you are passing non-speech data over your TCH and put something
238 else in those bits which the spec allocates to CRC.