FreeCalypso > hg > gsm-codec-lib
annotate doc/TW-TS-005 @ 581:e2d5cad04cbf
libgsmhr1 RxFE: store CN R0+LPC separately from speech
In the original GSM 06.06 code the ECU for speech mode is entirely
separate from the CN generator, maintaining separate state. (The
main intertie between them is the speech vs CN state variable,
distinguishing between speech and CN BFIs, in addition to the
CN-specific function of distinguishing between initial and update
SIDs.)
In the present RxFE implementation I initially thought that we could
use the same saved_frame buffer for both ECU and CN, overwriting
just the first 4 params (R0 and LPC) when a valid SID comes in.
However, I now realize it was a bad idea: the original code has a
corner case (long sequence of speech-mode BFIs to put the ECU in
state 6, then SID and CN-mode BFIs, then a good speech frame) that
would be broken by that buffer reuse approach. We could eliminate
this corner case by resetting the ECU state when passing through
a CN insertion period, but doing so would needlessly increase
the behavioral diffs between GSM 06.06 and our version.
Solution: use a separate CN-specific buffer for CN R0+LPC parameters,
and match the behavior of GSM 06.06 code in this regard.
author | Mychaela Falconia <falcon@freecalypso.org> |
---|---|
date | Thu, 13 Feb 2025 10:02:45 +0000 |
parents | d9f6b3125259 |
children |
rev | line source |
---|---|
549
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
1 The original set of Themyscira Wireless utilities for FR and EFR codecs uses an |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
2 ad hoc binary file format to represent streams of FR or EFR codec frames - see |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
3 Binary-file-format article. However, a newer hexadecimal format has now been |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
4 standardized as Themyscira Wireless Technical Specification TW-TS-005: |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
5 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
6 https://www.freecalypso.org/specs/tw-ts-005-v010003.txt |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
7 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
8 The standard has two annexes intended for practical use: |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
9 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
10 * TW-TS-005 Annex A defines a representation format for FR and EFR codecs; |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
11 * TW-TS-005 Annex B defines a representation format for HR codec. |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
12 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
13 The present version of ThemWi GSM codec libraries & utilities suite includes |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
14 some utilities that operate on TW-TS-005 Annex A hex files; support for Annex B |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
15 will appear in a future version when our work on GSM-HR codec integration |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
16 progresses further. |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
17 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
18 TW-TS-005 Annex A vs gsmx binary format |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
19 ======================================= |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
20 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
21 For working with FR and EFR codecs, our original binary file format has one |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
22 major defect: it cannot represent bad traffic frames (in GSM 06.31 & 06.81 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
23 definition, i.e., BFI=1) that have payload data bits included, as happens in |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
24 well-designed GSM networks that use GSM 08.60 TRAU-UL frames or TW-TS-001 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
25 enhanced RTP transport. This file format deficiency leads to the following |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
26 downstream defects: |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
27 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
28 * The combination of "bad traffic frame" and "accepted SID frame" (again, |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
29 GSM 06.31 & 06.81 terminology) gets incorrectly treated as "unusable frame" |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
30 rather than "invalid SID frame" as the specs decree. |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
31 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
32 * In the case of EFR, the reference decoder C code that forms the basis for |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
33 Themyscira libgsmefr makes use of "fixed codebook excitation pulses" portion |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
34 of bad frames during speech (as opposed to comfort noise) state - but these |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
35 bits were lost to file format shortcoming. |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
36 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
37 The new hexadecimal format of TW-TS-005 Annex A solves this shortcoming: each |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
38 frame is stored as a hex line that directly corresponds to a single RTP payload, |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
39 hence the full capabilities of TW-TS-001 extended RTP format are made available |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
40 in a file at rest. |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
41 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
42 Because we have so many existing utilities that read and write gsmx binary |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
43 files, and this binary format is so entrenched in Themyscira development |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
44 environment, we are not doing a "forklift" migration of all of our tools to the |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
45 new format. Instead we are taking a more tempered approach: |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
46 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
47 * For the decoding operation (taking a frame stream from an Rx Radio Subsystem |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
48 and producing linear PCM output) that is most affected by the shortcomings of |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
49 gsmx format, we have new utilities that read TW-TS-005 Annex A input, while |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
50 the old gsmx-reading utilities are still preserved and maintained; |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
51 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
52 * For most other workflows (for example, encoding of new speech) conversion |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
53 utilities between the two formats (described below) are deemed sufficient; |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
54 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
55 * New developments such as TFO transform use TW-TS-005 Annex A format natively. |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
56 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
57 Human-readable dump decoding of TW-TS-005 hex files |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
58 =================================================== |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
59 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
60 A line-based hexadecimal file format with one line per stored codec frame is |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
61 inherently more human-readable than a binary file, but we also desire a more |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
62 complete decoding such as that produced by gsmrec-dump, showing all codec |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
63 parameters and frame metadata flags. tw5a-dump produces such decoding for |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
64 TW-TS-005 Annex A hex files; there will also be a corresponding tw5b-dump |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
65 utility for TW-TS-005 Annex B when we finish integrating GSM-HR codec support. |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
66 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
67 Conversion utilities (FR and EFR codecs) |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
68 ======================================== |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
69 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
70 gsmx-to-tw5a and tw5a-to-gsmx utilities do what their names suggest: convert |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
71 FR/EFR speech recordings or test sequences between gsmx (binary) and TW-TS-005 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
72 Annex A (hex) formats. Important semantic notes: |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
73 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
74 * gsmx-to-tw5a emits basic RTP format (no TEH) for all good frames, while each |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
75 BFI marker record is converted to a TEH-only No_Data frame. |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
76 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
77 * tw5a-to-gsmx is the lossy conversion: distinction between basic and extended |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
78 RTP formats is lost, ditto for TAF without BFI, all BFIs become BFI-no-data. |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
79 |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
80 A conversion from gsmx to tw5a back to gsmx is lossless, but not the other way |
d9f6b3125259
document TW-TS-005 utilities
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
81 around. |