FreeCalypso > hg > gsm-codec-lib
annotate doc/EFR-rationale @ 581:e2d5cad04cbf
libgsmhr1 RxFE: store CN R0+LPC separately from speech
In the original GSM 06.06 code the ECU for speech mode is entirely
separate from the CN generator, maintaining separate state. (The
main intertie between them is the speech vs CN state variable,
distinguishing between speech and CN BFIs, in addition to the
CN-specific function of distinguishing between initial and update
SIDs.)
In the present RxFE implementation I initially thought that we could
use the same saved_frame buffer for both ECU and CN, overwriting
just the first 4 params (R0 and LPC) when a valid SID comes in.
However, I now realize it was a bad idea: the original code has a
corner case (long sequence of speech-mode BFIs to put the ECU in
state 6, then SID and CN-mode BFIs, then a good speech frame) that
would be broken by that buffer reuse approach. We could eliminate
this corner case by resetting the ECU state when passing through
a CN insertion period, but doing so would needlessly increase
the behavioral diffs between GSM 06.06 and our version.
Solution: use a separate CN-specific buffer for CN R0+LPC parameters,
and match the behavior of GSM 06.06 code in this regard.
author | Mychaela Falconia <falcon@freecalypso.org> |
---|---|
date | Thu, 13 Feb 2025 10:02:45 +0000 |
parents | 69b9a1eeb5a2 |
children |
rev | line source |
---|---|
122
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
1 Problem in need of solving |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
2 ========================== |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
3 |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
4 At the time of the undertaking of Themyscira libgsmefr project (late 2022), |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
5 there did not exist any readily available library solution for GSM EFR codec. |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
6 The community of FOSS offers classic libgsm from 1990s for FR1 codec (it's an |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
7 implementation of GSM 06.10, on top of which we had to implement our own Rx DTX |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
8 handler) and opencore-amrnb for AMR (based on Android OpenCORE framework) - but |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
9 nothing for EFR. This situation creates a problem for anyone seeking to deploy |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
10 their own GSM network with a voice interface to PSTN or other networks: such |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
11 voice interface generally requires implementing a transcoder, and doing the |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
12 latter in turn requires a library that implements the codec to be supported. |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
13 In the present situation, anyone who wishes to implement a speech transcoder |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
14 for GSM networks can easily support FR1 and AMR codecs, but not EFR. |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
15 |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
16 EFR is more than just 12k2 mode of AMR! |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
17 ======================================= |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
18 |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
19 There is a common misconception in the GSM hacker community that EFR is nothing |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
20 but the highest 12k2 mode of AMR, and that any library that implements AMR, |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
21 such as opencore-amrnb, is thus sufficient to support EFR as well. However, |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
22 the reality is more complex: |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
23 |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
24 * If an AMR encoder operates with DTX disabled, such that the output contains |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
25 only speech frames and no SID, and the mode is forced to 12k2, then indeed a |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
26 simple reshuffling of bits will produce speech frames that can be fed to an |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
27 EFR decoder on the other end. Note that the two encoders (EFR and AMR 12k2) |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
28 will produce *different* encoded speech parameters from the same input, and |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
29 the decoded speech output on the other end will also be different, but the |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
30 two versions are expected to be equally good for human ears. |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
31 |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
32 * In the other direction, if an EFR input stream contains only good speech |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
33 frames (no SID and no lost, FACCH-stolen or DTX-suppressed frames), one can |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
34 likewise do a simple bit reordering and feed these frames to an AMR decoder. |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
35 The output of this AMR decoder will once again be different from a proper |
181
3a0ee08a3b9d
doc/EFR-rationale: typo fix
Mychaela Falconia <falcon@freecalypso.org>
parents:
127
diff
changeset
|
36 (bit-exact) EFR decoder for the same speech parameter inputs, but as long as |
122
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
37 the EFR input stream is all good speech frames, the output will be good enough |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
38 for human ears. |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
39 |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
40 * The real problem occurs when the EFR input stream contains SID frames and BFI |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
41 frame gaps, as will always happen in reality if this stream is an uplink from |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
42 a GSM call. AMR SID mechanism is different from that of EFR, and an AMR |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
43 decoder will NOT recognize EFR SID frames. A quick experiment confirms that |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
44 when a real GSM EFR uplink RTP capture is converted to AMR by non-SID-aware |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
45 bit reshuffling and then fed to amrnb-dec from opencore-amrnb, unpleasant |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
46 sounds appear in the output whenever GSM uplink goes into SID. |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
47 |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
48 EFR reference code from ETSI |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
49 ============================ |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
50 |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
51 A published-source bit-exact implementation of GSM EFR encoder and decoder, |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
52 complete with all beyond-speech functions of DTX, VAD, comfort noise generation, |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
53 error concealment etc does exist in the form of reference code from ETSI. |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
54 However, this code has never been turned into a usable codec library by anyone |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
55 prior to us (at least not by anyone who freely published their work), and doing |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
56 such librification (producing an EFR analogue to what Android OpenCORE people |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
57 did with AMR) is no easy feat! The original EFR code from ETSI exhibits two |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
58 problems which need to be remedied in the librification project: |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
59 |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
60 1) The original code maintains all codec state in global variables (lots of |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
61 them) that are scattered throughout. 3GPP reference code for AMR (naturally |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
62 later than EFR in chronological order) is better in this regard (in the AMR |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
63 version they gathered their global vars into structs and pass pointers to |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
64 these structs, although still many separately-malloc'ed structs instead of |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
65 single unified encoder state and decoder state), but we need the EFR version |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
66 for correct handling of all beyond-speech aspects, and this version is all |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
67 global vars. |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
68 |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
69 2) These reference codes from ETSI/3GPP (both EFR and AMR versions, it seems) |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
70 were intended to serve as simulations, not as production code, and the code |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
71 is very inefficient. |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
72 |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
73 Themyscira libgsmefr |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
74 ==================== |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
75 |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
76 Libgsmefr presented in this code repository is our current solution for EFR. |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
77 It is a library styled after classic libgsm for FR1, but its guts consist of a |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
78 librified derivative of ETSI EFR code. The problem of global vars has been |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
79 solved in this library version - they've been gathered into one unified struct |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
80 for encoder state and another unified struct for decoder state - but the problem |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
81 of poor performance (significantly worse than opencore-amrnb) still remains for |
b33f2168fdec
doc/EFR-rationale article written
Mychaela Falconia <falcon@freecalypso.org>
parents:
diff
changeset
|
82 now. |
127
4af99bf8671a
doc/EFR-rationale: add future roadmap section
Mychaela Falconia <falcon@freecalypso.org>
parents:
122
diff
changeset
|
83 |
4af99bf8671a
doc/EFR-rationale: add future roadmap section
Mychaela Falconia <falcon@freecalypso.org>
parents:
122
diff
changeset
|
84 Future roadmap |
4af99bf8671a
doc/EFR-rationale: add future roadmap section
Mychaela Falconia <falcon@freecalypso.org>
parents:
122
diff
changeset
|
85 ============== |
4af99bf8671a
doc/EFR-rationale: add future roadmap section
Mychaela Falconia <falcon@freecalypso.org>
parents:
122
diff
changeset
|
86 |
313
69b9a1eeb5a2
doc/EFR-rationale: update future roadmap
Mychaela Falconia <falcon@freecalypso.org>
parents:
181
diff
changeset
|
87 When this article was originally written in late 2022, my thoughts were that we |
69b9a1eeb5a2
doc/EFR-rationale: update future roadmap
Mychaela Falconia <falcon@freecalypso.org>
parents:
181
diff
changeset
|
88 would go in the direction of a single library supporting both AMR and EFR, |
69b9a1eeb5a2
doc/EFR-rationale: update future roadmap
Mychaela Falconia <falcon@freecalypso.org>
parents:
181
diff
changeset
|
89 sharing most code in common and handling the differences between EFR and MR122 |
69b9a1eeb5a2
doc/EFR-rationale: update future roadmap
Mychaela Falconia <falcon@freecalypso.org>
parents:
181
diff
changeset
|
90 similarly to how most proprietary implementations have done it, following the |
69b9a1eeb5a2
doc/EFR-rationale: update future roadmap
Mychaela Falconia <falcon@freecalypso.org>
parents:
181
diff
changeset
|
91 "alternative" of GSM 06.54 chapter 10. However, that plan has been revised; |
69b9a1eeb5a2
doc/EFR-rationale: update future roadmap
Mychaela Falconia <falcon@freecalypso.org>
parents:
181
diff
changeset
|
92 our current approach (as of 2024-04) is that we are developing a separate |
69b9a1eeb5a2
doc/EFR-rationale: update future roadmap
Mychaela Falconia <falcon@freecalypso.org>
parents:
181
diff
changeset
|
93 library for AMR (libtwamr, librifying 3GPP AMR code in the same way how we |
69b9a1eeb5a2
doc/EFR-rationale: update future roadmap
Mychaela Falconia <falcon@freecalypso.org>
parents:
181
diff
changeset
|
94 librified ETSI EFR in libgsmefr), while libgsmefr remains as it is, a pristine |
69b9a1eeb5a2
doc/EFR-rationale: update future roadmap
Mychaela Falconia <falcon@freecalypso.org>
parents:
181
diff
changeset
|
95 implementation of GSM-EFR in its *original* bit-exact form. |
69b9a1eeb5a2
doc/EFR-rationale: update future roadmap
Mychaela Falconia <falcon@freecalypso.org>
parents:
181
diff
changeset
|
96 |
69b9a1eeb5a2
doc/EFR-rationale: update future roadmap
Mychaela Falconia <falcon@freecalypso.org>
parents:
181
diff
changeset
|
97 Please refer to AMR-EFR-philosophy article for more information. |