FreeCalypso > hg > freecalypso-tools
view doc/TCH-bit-access @ 908:3aca343be11c
CHANGES: add TCH changes since fc-host-tools-r17
author | Mychaela Falconia <falcon@freecalypso.org> |
---|---|
date | Thu, 29 Dec 2022 09:54:09 +0000 |
parents | 3de3b34189be |
children | 8f7c50e1fa3b |
line wrap: on
line source
It has been discovered that the DSP ROM in the Calypso GSM baseband processor implements one nifty feature which is not used at all in standard phone or modem operation, but which can be used for all kinds of interesting hacks: the traffic channel (TCH) bits coming out of the GSM 05.03 channel decoder in the downlink direction (to be fed to the channel mode-appropriate speech decoder) can be read out of the DSP's API RAM in real time, and in the uplink direction the user can feed her own bits to the input of the GSM 05.03 channel encoder, effectively suppressing the output of the internal vocoder. The DSP mechanism in question is known to work in TCH/FS and TCH/EFS channel modes, corresponding to FR1 and EFR codecs; it also appears to work for TCH/HS (HR1 codec), but we (FreeCalypso) haven't tested it because almost no one uses that infamous HR1 codec - the commercial GSM network in our part of the world gives you a full-rate channel if your phone does not support AMR. It would be possible to implement HR1 in our own test GSM network, but the effort that would be required is difficult to justify. Exploring TCH tap modes with AMR or CSD traffic channels is likewise a subject for further study. In order to make use of this TCH bit access feature, one needs 3 things: 1) Firmware on the Calypso (the ARM part) that reads downlink bits from the DSP and writes uplink bits into it while doing everything else that a GSM fw must do in order to operate the MS; 2) Some protocol for passing these TCH bits into and out of the Calypso device; 3) A source for TCH UL bits and a sink for TCH DL bits on the external host. In the case of FreeCalypso, we have defined our own protocol for passing TCH bits into and out of Calypso GSM devices running one of our firmwares in the form of an extension to TI's RVTMUX interface, i.e., we have defined a new RVTMUX channel for this TCH interface and defined the packet types and formats to be sent over the wire. On the Calypso side the special functionality in question was originally implemented in FC Citrine firmware in 2016 and then set aside for some years; when the right time came to resurrect this feature in late 2022, it turned out that the original implementation from 2016 was slightly incorrect, and the new implementation in FC Tourmaline fw is slightly different. On the host tools side the RVTMUX-based TCH interface is supported in rvinterf and fc-shell; the new version as of fc-host-tools-r18 supports both 2016 and 2022 versions of this over-the-wire interface. The TCH bit access mechanism in FreeCalypso has been designed with an objective of presenting to the user exactly what TI's DSP presents to us, i.e., standing out of the way as much as possible. TI's DSP presents TCH downlink bits and accepts TCH uplink bits in the form of an array of 16-bit words; the bit order within these words corresponds to the GSM 05.03 channel encoder bit order (with a couple of TI-specific quirks documented below) and NOT that of the GSM 06.10 or EFR codecs. On the RVTMUX serial interface between the Calypso device and the external host we transfer each TCH frame as a block of 33 bytes; our Calypso firmwares translate between these bytes and the DSP's 16-bit words, but do not reorder or change the bits in any way. On the host tools side our fc-shell utility provides user commands to save TCH DL bits into a file and to play TCH UL bits from a file; in the present version these files are written and read in an ASCII-based hex format. In these ASCII files each TCH frame is represented as a string of 66 hexadecimal digits, and these hex digits correspond directly to the 33 bytes being read out of or written into DSP API words. Therefore, in order to generate and/or interpret these hexadecimal strings correctly, you (the user) need to understand the bit order and mapping used by TI's implementation of the GSM 05.03 channel encoder. As of late 2022, there is a new TCH-tap-modes article in our freecalypso-docs repository that covers in detail the format of TI's DSP buffers for TCH DL and UL bits, as well as all known information about TCH DL status words and bit flags. But here is our original description from 2016: Recall from the GSM specs that the 260 bits which comprise one speech frame are not all treated equally, instead they are divided into 182 class 1 bits which are protected by a convolutional encoder and 78 class 2 bits which are transmitted without any forward error correction. Furthermore, the first 50 of the class 1 bits are also protected by a CRC. The order in which the bits appear on TI's DSP interface corresponds to this division, known as the order of subjective importance. Now let's look at the actual bit order and mapping which you need to understand in order to make sense of the hex strings in tch record and tch play files. The bit numbering is from the most significant bit to the least significant bit, i.e., in our string of 66 hex digits the most significant bit of the leftmost digit is bit 0 and the least significant bit of the rightmost digit is bit 263. TI's DSP assigns these bits as follows: * Bits 0 through 181 correspond to the 182 protected (class 1) bits in the standard GSM 05.03 order; * Bits 182 through 185 are unused - put zeros there if you are generating them yourself; * Bits 186 through 263 correspond to the 78 unprotected (class 2) bits in the standard GSM 05.03 order. TCH DL recording ================ When you are in an established voice call in fc-shell, you can record the downlink TCH bits as follows: tch record <name of file to put the recording into> If you would like to record an entire call from the beginning, issue the tch record command as above before the ATD or ATA command that dials or answers the call. Either way, whether you are recording a call from the beginning or from the middle, you need to eventually stop your recording with this command: tch record stop You can issue this stop command before or after the call is terminated, but until you issue this tch record stop command, the output file is not closed and thus may not be written to the file system. The recording is written in an ASCII line-based format with one line for every received TCH DL frame, but the exact format of each written line will depend on which firmware version is in use. If you are running ancient Citrine firmware that emits its TCH DL output in the old format from 2016 (now known to be incomplete and thus unusable for proper decoding), fc-shell will likewise write its ASCII output in the old format, which won't be covered further as it is deprecated and not practically useful. However, if you are running current FreeCalypso firmware with the resurrected (late 2022) version of the TCH tap feature, each TCH DL frame will be sent by the fw and received by fc-shell in the new over-the-wire format, and fc-shell will write the recording file in the new ASCII format documented in the TCH-tap-modes article in freecalypso-docs. Once you have captured a TCH DL recording, what can you do with it? If the recording came from an FR1 call, you will need to pass it through an Rx DTX handler for FR1 (see GSM 06.11, 06.12 and 06.31 specs) before you can pass it to a naive GSM 06.10 decoder such as classic Unix libgsm, and if the recording came from an EFR call, you will need to pass it to a proper EFR (not AMR!) decoder that includes the necessary EFR Rx DTX handler. Neither of the two just-mentioned library pieces (neither the Rx DTX handler for FR1 nor a proper, not-same-as-AMR implementation of GSM EFR) could be found among the existing body of FOSS as of 2022, thus we (FreeCalypso and Themyscira Wireless) implemented our own. Please look for our GSM codec libraries & utilities package, which is expected to reach its first official release some time in early 2023. Inside our gsm-codec-lib package you will find gsmfr-dlcap-* and gsmefr-dlcap-* utilities that read TCH downlink capture files written by fc-shell tch record and perform various decoding operations - please refer to further documentation within that package. Please don't use the old fc-tch2fr utility - the function it performs is now known to be a bogo-transform, and it can't grok the new TCH DL recording format which you will get with current FreeCalypso fw. TCH UL play =========== The uplink sending mechanism can be exercised as follows: 1. If you are going to be in an FR1 call, prepare a speech sample in the GSM 06.10 codec format using any Unix/Linux audio tool that can write the de facto standard libgsm format. For example, using SoX: rec -c1 recording.gsm SoX will write the recording in the GSM 06.10 libgsm format based on the .gsm suffix at the end of the recording file name; the -c1 option is needed to disable stereo, otherwise the recording will be slowed down 2x. Alternatively, you can use our new gsmfr-encode utility (gsm-codec-lib package) to encode from WAV into GSM 06.10, or gsmfr-encode-r for raw BE input instead of WAV. OTOH, if you are going to be in an EFR call rather than FR1, you will need to prepare a speech sample in the EFR codec format instead. You will need to use Themyscira gsmefr-encode or gsmefr-encode-r utilities, or convert from AMR (MR122 mode only, no DTX) with our gsm-amr2efr utility. 2. Convert your speech sample from libgsm standard format (FR1) or Themyscira gsmx format (EFR) into our ad hoc hex strings for playing into a TCH uplink: fc-fr2tch recording.gsm recording.tch-ul or fc-efr2tch recording.gsmx recording.tch-ul 3. In fc-shell, when you are in an established voice call, issue this command: tch play recording.tch-ul You should now hear the speech sample you recorded in step 1 above on the other end of the GSM call. Needless to say, the TCH mode of the call (TCH/FS or TCH/EFS) needs to match the codec in which your to-be-played recording was prepared, otherwise the other end of the call will receive garbage! Controlling the selection of speech codec for calls =================================================== One very obvious shortcoming of the present facilities for voice TCH redirection is that we only support FR1 and EFR codecs, but not AMR. However, most GSM networks prefer to use AMR when the MS supports it - and in regular operation with a speaker & mic user connection (as opposed to TCH tap modes), our current FreeCalypso firmwares do support AMR when running on Calypso C035 silicon with DSP ROM version 3606. (DSP ROM version 3416 together with the respective patch version also appears to have working AMR, at least in light testing, although of course we do NOT recommend it for production use.) Therefore, if you wish to play with EFR, you need to tell the network (via the Bearer Capabilities information element in CC messages) that your MS does not support AMR, and if you wish to play with FR1, you need to tell the network that your MS only supports FR1 and no others. The outstanding issue here is that we need to implement some mechanism in our FreeCalypso firmwares, probably a custom AT command, that will modify the Bearer Capabilities IE (artificially restrict the set of supported codecs) on a per-call basis. Until we implement the necessary support, the only current way to create such codec-restricted operation is by writing a /pcm/MSCAP file into FFS with the desired bit mask of supported codecs - but this method is hugely inconvenient because this file is read only on fw boot, thus every modification requires a full reboot cycle.