The original set of Themyscira Wireless utilities for FR and EFR codecs uses an
ad hoc binary file format to represent streams of FR or EFR codec frames - see
Binary-file-format article.  However, a newer hexadecimal format has now been
standardized as Themyscira Wireless Technical Specification TW-TS-005:

https://www.freecalypso.org/specs/tw-ts-005-v010003.txt

The standard has two annexes intended for practical use:

* TW-TS-005 Annex A defines a representation format for FR and EFR codecs;
* TW-TS-005 Annex B defines a representation format for HR codec.

The present version of ThemWi GSM codec libraries & utilities suite includes
some utilities that operate on TW-TS-005 Annex A hex files; support for Annex B
will appear in a future version when our work on GSM-HR codec integration
progresses further.

TW-TS-005 Annex A vs gsmx binary format
=======================================

For working with FR and EFR codecs, our original binary file format has one
major defect: it cannot represent bad traffic frames (in GSM 06.31 & 06.81
definition, i.e., BFI=1) that have payload data bits included, as happens in
well-designed GSM networks that use GSM 08.60 TRAU-UL frames or TW-TS-001
enhanced RTP transport.  This file format deficiency leads to the following
downstream defects:

* The combination of "bad traffic frame" and "accepted SID frame" (again,
  GSM 06.31 & 06.81 terminology) gets incorrectly treated as "unusable frame"
  rather than "invalid SID frame" as the specs decree.

* In the case of EFR, the reference decoder C code that forms the basis for
  Themyscira libgsmefr makes use of "fixed codebook excitation pulses" portion
  of bad frames during speech (as opposed to comfort noise) state - but these
  bits were lost to file format shortcoming.

The new hexadecimal format of TW-TS-005 Annex A solves this shortcoming: each
frame is stored as a hex line that directly corresponds to a single RTP payload,
hence the full capabilities of TW-TS-001 extended RTP format are made available
in a file at rest.

Because we have so many existing utilities that read and write gsmx binary
files, and this binary format is so entrenched in Themyscira development
environment, we are not doing a "forklift" migration of all of our tools to the
new format.  Instead we are taking a more tempered approach:

* For the decoding operation (taking a frame stream from an Rx Radio Subsystem
  and producing linear PCM output) that is most affected by the shortcomings of
  gsmx format, we have new utilities that read TW-TS-005 Annex A input, while
  the old gsmx-reading utilities are still preserved and maintained;

* For most other workflows (for example, encoding of new speech) conversion
  utilities between the two formats (described below) are deemed sufficient;

* New developments such as TFO transform use TW-TS-005 Annex A format natively.

Human-readable dump decoding of TW-TS-005 hex files
===================================================

A line-based hexadecimal file format with one line per stored codec frame is
inherently more human-readable than a binary file, but we also desire a more
complete decoding such as that produced by gsmrec-dump, showing all codec
parameters and frame metadata flags.  tw5a-dump produces such decoding for
TW-TS-005 Annex A hex files; there will also be a corresponding tw5b-dump
utility for TW-TS-005 Annex B when we finish integrating GSM-HR codec support.

Conversion utilities (FR and EFR codecs)
========================================

gsmx-to-tw5a and tw5a-to-gsmx utilities do what their names suggest: convert
FR/EFR speech recordings or test sequences between gsmx (binary) and TW-TS-005
Annex A (hex) formats.  Important semantic notes:

* gsmx-to-tw5a emits basic RTP format (no TEH) for all good frames, while each
  BFI marker record is converted to a TEH-only No_Data frame.

* tw5a-to-gsmx is the lossy conversion: distinction between basic and extended
  RTP formats is lost, ditto for TAF without BFI, all BFIs become BFI-no-data.

A conversion from gsmx to tw5a back to gsmx is lossless, but not the other way
around.