FreeCalypso > hg > freecalypso-tools
view doc/TCH-bit-access @ 472:368ffb8a08e5
CHANGES: CC= and CFLAGS= override-ability documented
author | Mychaela Falconia <falcon@freecalypso.org> |
---|---|
date | Tue, 12 Feb 2019 00:30:37 +0000 |
parents | 2748f257312b |
children | 3de3b34189be |
line wrap: on
line source
It has been discovered that the DSP ROM in the Calypso GSM baseband processor implements one nifty feature which is not used at all in standard phone or modem operation, but which can be used for all kinds of interesting hacks: the traffic channel (TCH) bits coming out of the GSM 05.03 channel decoder in the downlink direction (to be fed to one of several speech decoders) can be read out of the DSP's API RAM in real time, and in the uplink direction the user can feed her own bits to the input of the GSM 05.03 channel encoder, effectively suppressing the output of the internal vocoder. The DSP mechanism in question is known to work in TCH/FS and TCH/EFS channel modes, corresponding to the FR and EFR codecs; it is not currently known if it can also be used in TCH/HS, data traffic or AMR modes. I am aware though of an anecdotal report that someone tried to make this feature work with AMR, but was unsuccessful - hence we should be prepared for the possibility that the hack is not possible with AMR. In order to make use of this TCH bit access feature, one needs 3 things: 1) Firmware on the Calypso (the ARM part) that reads downlink bits from the DSP and writes uplink bits into it while doing everything else that a GSM fw must do in order to operate the MS; 2) Some protocol for passing these TCH bits into and out of the Calypso device; 3) A source for TCH UL bits and a sink for TCH DL bits on the external host. In the case of FreeCalypso, we have defined our own protocol for passing TCH bits into and out of Calypso GSM devices running one of our firmwares in the form of an extension to TI's RVTMUX interface, i.e., we have defined a new RVTMUX channel for this TCH interface and defined the packet types and formats to be sent over the wire. On the Calypso side this interface is implemented in FreeCalypso GSM firmwares (currently only in Citrine, but we can also make a special version of TCS211 with this feature added if the need arises), and on the host tools side there is support in rvinterf and fc-shell. The TCH bit access mechanism in FreeCalypso has been designed with an objective of presenting to the user exactly what TI's DSP presents to us, i.e., standing out of the way as much as possible. TI's DSP presents TCH downlink bits and accepts TCH uplink bits in the form of an array of 16-bit words; the bit order within these words corresponds to the GSM 05.03 channel encoder bit order (with a couple of TI-specific quirks documented below) and NOT that of the GSM 06.10 or EFR codecs. On the RVTMUX serial interface between the Calypso device and the external host we transfer each TCH frame as a block of 33 bytes; our Calypso firmwares translate between these bytes and the DSP's 16-bit words, but do not reorder or change the bits in any way. On the host tools side our fc-shell utility provides user commands to save TCH DL bits into a file and to play TCH UL bits from a file; in the present version these files are written and read in an ASCII-based hex format. In these ASCII files each TCH frame is represented as a string of 66 hexadecimal digits, and these hex digits correspond directly to the 33 bytes being read out of or written into DSP API words. Therefore, in order to generate and/or interpret these hexadecimal strings correctly, you (the user) need to understand the bit order and mapping used by TI's implementation of the GSM 05.03 channel encoder. Recall from the GSM specs that the 260 bits which comprise one speech frame are not all treated equally, instead they are divided into 182 class 1 bits which are protected by a convolutional encoder and 78 class 2 bits which are transmitted without any forward error correction. Furthermore, the first 50 of the class 1 bits are also protected by a CRC. The order in which the bits appear on TI's DSP interface corresponds to this division, known as the order of subjective importance. Now let's look at the actual bit order and mapping which you need to understand in order to make sense of the hex strings in tch record and tch play files. The bit numbering is from the most significant bit to the least significant bit, i.e., in our string of 66 hex digits the most significant bit of the leftmost digit is bit 0 and the least significant bit of the rightmost digit is bit 263. TI's DSP assigns these bits as follows: * Bits 0 through 181 correspond to the 182 protected (class 1) bits in the standard GSM 05.03 order; * Bits 182 through 185 are unused - put zeros there if you are generating them yourself; * Bits 186 through 263 correspond to the 78 unprotected (class 2) bits in the standard GSM 05.03 order. Uplink testing ============== The uplink sending mechanism can be exercised as follows: 1. Record a speech sample in the GSM 06.10 codec format using any Unix/Linux audio tool that can write the de facto standard libgsm format. For example, using SoX: rec -c1 recording.gsm SoX will write the recording in the GSM 06.10 libgsm format based on the .gsm suffix at the end of the recording file name; the -c1 option is needed to disable stereo, otherwise the recording will be slowed down 2x. 2. Convert it from libgsm standard format into our ad hoc hex strings for playing into the TCH uplink: fc-fr2tch recording.gsm recording.tch-ul 3. In fc-shell, when you are in an established voice call, issue this command: tch play recording.tch-ul You should now hear the speech sample you recorded in step 1 above on the other end of the GSM call. Please note, though, that for this example to work, the call must be connected in the FR codec mode, not EFR. If you try it on an EFR call, the vocoder on the other end of the call will try to interpret your FR codec bits per EFR, resulting in garbage. In order to make this procedure work properly with EFR, you would need to generate a speech sample in EFR format and then put it in the correct hex string form with the correct bit order for feeding to TI's GSM 05.03 channel encoder implementation, and at the present moment the tools to do this feat are lacking. The fc-fr2tch step above is new with the current version of FreeCalypso host tools. My original implementation of the tch play command in fc-shell read libgsm GSM 06.10 speech files directly, but I changed it to read hex strings which correspond literally to what goes into TI's implementation of the 05.03 channel encoder to make it more general: 1) to support playing EFR speech bits into TCH/EFS channels; 2) to support people who may want to pass totally non-speech data over traffic channels in TFO mode - see below. Downlink testing ================ When you are in an established voice call in fc-shell, you can record the downlink TCH bits as follows: tch record <name of file to put the recording into> If you would like to record an entire call from the beginning, issue the tch record command as above before the ATD or ATA command that dials or answers the call. Either way, whether you are recording a call from the beginning or from the middle, you need to eventually stop your recording with this command: tch record stop You can issue this stop command before or after the call is terminated, but until you issue this tch record stop command, the output file is not closed and thus may not be written to the file system. The recording is written in an ASCII format that is similar but not identical to what the tch play command takes as its input. Here is an example of what you might get from tch record: C214 281D 0063 81C008000008200046000000000000000000000000000007056608060B0A010B09 C204 0540 003A 9391480D3051F4BD81DF5EB35069BEBC4AEDF756351C4C19689BB1CA4EA5D4F5F5 C204 05A2 0047 65B80F9E690F7C8CE4ED80DEF69DA6436518AB99ABCE5815E6B562C5CE4EAC5DC5 C204 04CA 0044 A4483744B04371ED0334ECB350AF28C639B7F095519EF0242D299B6405124F77A5 C214 2544 0066 83800400000C1000E80000000000000000000000000000300674A07070F080D0B2 C200 4E8A 0000 07071DF0F83B9521EE61CFF095AA8C0E560300F6A5573C31F3E00601ED4AAE7E2F C200 4C94 0000 077385ADE20450B2E410961D6C5B0A173ACF9E2D38D77C28CED8495D88AA4DE72E C200 4ABC 0000 077F0CF86004132DA5C0A6D5A4B0BD4B28159A07D8F4282DC6AAAB27503BC02701 The example above is quoted from an actual recording made from a call to WWV at +1-303-499-7111, a time of day service. This example shows 5 garbled frames followed by 3 good ones. In each line the first 3 space-separated 16-bit words are status words from the DSP, and the rest is a hex string of 66 digits giving the 260-bit frame with 4 extra dummy bits at the boundary between the protected and unprotected portions. Now here comes the unpleasant part: radio systems are inherently error-prone, hence if you are naively expecting every received downlink frame to be a valid speech frame that matches exactly what the other end transmitted, you will be disappointed. The downlink frame stream will always consist of both valid and invalid frames, and in the standard processing chain inside TI's DSP the speech decoder block that receives these frames from the channel decoder will look at the status words written by the latter in order to decide what to do with each frame. But unfortunately I don't know the details, as the DSP is a mostly undocumented black box. Our FreeCalypso firmwares forward the downlink frame status words from the DSP to the external host and my current implementation of the tch record command writes them into the file because it is valuable information which should not be discarded, but unfortunately I don't know their meaning. The first status word consists of bit flags, the last one seems to be some kind of error count (what kind of errors? I don't know), and the middle one is totally unknown. There is also an fc-tch2fr utility that does sort of an inverse of fc-fr2tch, except that fc-tch2fr expects files written by tch record which contain the status words at the beginning - the files written by fc-fr2tch and read by tch play don't have these status words; they just have 66 hex digits per line giving the bits to be fed to the GSM 05.03 block in the DSP. The fc-tch2fr utility works by disregarding the status words; the result will be a .gsm file which you can play with SoX etc, but it will produce horrible sounds wherever some garbled frames were received on the call downlink. Passing non-speech data over TCH ================================ If you are incredibly lucky and you happen to live in a part of the world where the local GSM network implements TFO (tandem-free operation, see GSM 02.53), i.e., if your GSM network transparently passes codec frames from one end of the call to the other without transcoding back and forth in the middle, you might be able to send arbitrary non-speech data bits over this TCH connection. If you are going to attempt such a feat, you are mostly on your own as the GSM network in my part of the world does not support TFO, but here are some general ideas: * Heed the difference between the 182 protected bits and the 78 unprotected bits of each 260-bit frame. The bits in the first class are less likely to be corrupted by radio errors. Also the first 50 out of these class 1 bits are protected by a CRC, so if they get corrupted, there will be a flag bit somewhere in the DSP status words - but I don't know which bit it is... * Given the lack of detailed information about the exact meaning of the DSP status words, implement some error checking of your own, e.g., an in-band CRC within your frame payload so you can check on the receiving end if you got an uncorrupted frame from your data source or not. * Expect to lose some frames - even if your radio environment is pristine and error-free, some frames will still be lost because their timeslots had to be stolen by FACCH on either end of the call. You will need to implement acknowledgements, retransmissions and retries - the usual stuff. EFR differences =============== TCH/FS and TCH/EFS channel modes corresponding to the FR and EFR codecs, respectively, share the same channel encoder that takes 260-bit frames as input. An EFR speech frame is only 244 bits, but as the GSM 05.03 spec explains, each EFR frame gets expanded to 260 bits by adding some CRC and repetition bits. One would thus expect that the behaviour of the channel encoder block should be strictly identical between TCH/FS and TCH/EFS channel modes, but it appears that TI's implementation exhibits one difference in behaviour between the two modes. It appears that TI's implementation of the GSM 05.03 channel encoder computes the EFR-specific CRC bits internally, such that bits [65,72] of each uplink frame fed to the DSP in the TCH/EFS mode need to be 0 and will be replaced by the DSP with CRC bits computed per the standard. It also appears that their implementation of the channel decoder verifies this CRC, sets a status bit indicating whether the check succeeded or failed, and then zeroes these bits out, i.e., the original bits in these CRC bit positions are lost. No problem if what you are passing over TCH/EFS is indeed EFR speech frames, but it will be a problem if you are passing non-speech data over your TCH and put something else in those bits which the spec allocates to CRC.