view doc/Calypso-TCH-downlink @ 442:6fa27df6903b

libtwamr: add file header public const datum
author Mychaela Falconia <falcon@freecalypso.org>
date Thu, 09 May 2024 07:19:42 +0000
parents 03b0702f4463
children e512f0d25409
line wrap: on
line source

It has been discovered that the implementation of standard signal processing
chains for speech TCH downlink and uplink in the DSP ROM in the Calypso GSM
baseband processor allows these signal processing chains to be tapped at certain
points, as detailed in the TCH-tap-modes article in our freecalypso-docs Hg
repository.  There is a mechanism to capture the stream of received traffic
frames on TCH DL, and there is another mechanism by which an externally supplied
stream can be "played" into TCH UL.

I (Mother Mychaela) previously played with this functionality back in 2016, and
it's been mostly shelved since then.  This functionality became interesting
once again in late 2022: now that we have a proper set of codec libraries (the
present package) and a proper understanding of Rx DTX handling requirements, we
can take another shot at decoding TCH downlink captures taken from Calypso GSM
MS.

The overall functionality is described in the TCH-tap-modes article in
freecalypso-docs; the mechanism for capturing TCH DL bits from Calypso DSP is
split between FreeCalypso GSM MS firmware (added to FC Tourmaline as of
2022-12-13) and the fc-shell utility in the FC host tools package, updated as
of fc-host-tools-r18 to support the new FreeCalypso fw.  There is also a set of
utilities included in the present GSM codec libraries & utilities package for
parsing and decoding these Calypso TCH DL captures; the present document
describes these utilities.

As explained in the TCH-tap-modes article in freecalypso-docs, the mechanism
for capturing TCH DL is currently implemented for TCH/FS, TCH/HS and TCH/EFS,
corresponding to FR1, HR1 and EFR codecs.  However, further parsing and decoding
support has only been implemented for FR1 and EFR codecs in the present package,
in the form of the following utilities:

gsmfr-dlcap-parse	This program reads a TCH/FS DL capture file and parses
			it for human analysis.  All input fields are passed
			through to the output, but the program also computes
			the ternary SID flag of GSM 06.31 section 6.1.1 from
			the payload bits (for comparison against what the DSP
			wrote in its status word 0) and prints all broken-down
			parameter fields of each GSM 06.10 FR1 codec frame.

gsmfr-dlcap-gsmx	This program reads a TCH/FS DL capture file and converts
			it into an extended-libgsm (gsmx) file containing a mix
			of FR1 codec frames and Themyscira BFI markers.  The
			latter BFI markers will be emitted in those frame
			positions where FACCH was received instead of speech,
			or where the DSP otherwise indicated BFI=1.  The gsmx
			output from this utility needs to be fed to gsmfr-decode
			from the present package, so that our FR1 Rx DTX
			preprocessor will take care of SIDs and BFIs, completing
			the required GSM MS processing chain for TCH/FS DL.

gsmefr-dlcap-parse	This program reads a TCH/EFS DL capture file and parses
			it for human analysis.  All input fields are passed
			through to the output, but the program also computes
			the ternary SID flag of GSM 06.81 section 6.1.1 from
			the payload bits (for comparison against what the DSP
			wrote in its status word 0) and prints all broken-down
			parameter fields of each EFR codec frame.  Finally, each
			triplicated bit group of GSM 05.03 section 3.1.1.2 is
			printed as an octal digit, to aid human analysis of how
			the DSP writes these bits in its a_dd_0 buffer.

gsmefr-dlcap-gsmx	This program reads a TCH/EFS DL capture file and
			converts it into a gsmx binary file, containing a mix
			of EFR codec frames and Themyscira BFI markers.  The
			latter BFI markers will be emitted in those frame
			positions where FACCH was received instead of speech,
			or where the DSP otherwise indicated BFI=1.  The gsmx
			output from this utility needs to be fed to
			gsmefr-decode (or gsmefr-decode-r) from the present
			package.

gsmefr-dlcap-dec	This program reads a TCH/EFS DL capture file and feeds
			it directly to the EFR reference decoder implemented in
			libgsmefr, without going through a gsmx intermediary.

Additional notes:

* The new gsmfr-dlcap-gsmx utility described above replaces the old fc-tch2fr
  utility from FC host tools - the latter should now be considered a bogon.
  The required GSM MS processing chain for TCH/FS DL includes the step of Rx
  DTX handler between the output of GSM 05.03 channel decoder and the input of
  GSM 06.10 speech decoder; the old chain of fc-tch2fr followed by libgsm
  decoding omitted this critical step and thus produced very unkind-on-ears
  sounds.

* gsmefr-dlcap-dec has been written as a bold attempt to replicate the complete
  Rx DTX handler and speech decoder (the part of TCH DL processing chain that
  sits past the a_dd_0 buffer) as they are implemented inside TI's DSP.  Such a
  feat won't be possible for FR1 codec (other than by a Herculean effort of full
  static reversing of the DSP ROM) because there is no bit-exact definition of
  FR1 Rx DTX functions in GSM specs, but for EFR there is a bit-exact reference
  implementation from ETSI.  *If* TI's DSP matches this bit-exact reference
  (there are some aspects of Rx DTX handling where this bit-exact reference is
  considered to be an example rather than normative, see GSM 06.61), then there
  is a chance we could replicate TI's DSP chain externally - but only if we can
  figure out exactly how the bits of a_dd_0[0] drive the logic of their Rx DTX
  handler.  The Mother's plan is to capture the DSP's decoded speech output from
  MCSI on an FCDEV3B using a small FPGA board with a PCM-to-UART logic function,
  while simultaneously capturing TCH DL bits in the a_dd_0 buffer, then run
  gsmefr-dlcap-dec on the captured TCH DL booty and see if we can replicate the
  DSP's end output - but until then, this gsmefr-dlcap-dec program should be
  treated as an unfinished experiment in progress.

* In the case of FR1 codec, there is no prescribed bit-exact definition for the
  Rx DTX handler (GSM 06.11, 06.12 and 06.31 specs define general requirements,
  but aren't bit-exact in most aspects), and the way in which we (Themyscira
  Wireless) have implemented our FR1 Rx DTX handler (libgsmfr2 in the present
  package) perfectly matches our gsmx binary file format for good vs bad frames.
  Therefore, in the case of FR1 codec there is nothing to be gained by skipping
  gsmx and calling library functions directly, and thus there is no FR1
  counterpart to gsmefr-dlcap-dec - just use gsmfr-dlcap-gsmx followed by
  gsmfr-decode or gsmfr-decode-r.

* In addition to TCH DL capture files, gsmfr-dlcap-parse also accepts the hex
  output from fc-vm2hex, originating from TCS211 voice memo recordings,
  including fc-vm2hex output in the case of VM recordings made in DTX mode.
  However, if the objective is to play that VM recording and not just look at
  parsed bits, the correct approach is to convert the VM file to gsmx with
  fc-vm2gsmx, and then decode with gsmfr-decode.  Using fc-vm2hex followed by
  gsmfr-dlcap-gsmx instead of fc-vm2gsmx won't work!