# HG changeset patch # User Mychaela Falconia # Date 1713165151 0 # Node ID 6b479cfb06a4510901420676345d89c349fb3c4c # Parent e0d42e87da96cb3a12992011457afbc03352a638 beginning of libgsmfr2 documentation diff -r e0d42e87da96 -r 6b479cfb06a4 doc/FR1-library-API --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/FR1-library-API Mon Apr 15 07:12:31 2024 +0000 @@ -0,0 +1,211 @@ +Libgsmfr2 general usage +======================= + +The external public interface to Themyscira libgsmfr2 consists of a single +header file ; it should be installed in some system include +directory. + +The dialect of C used by all Themyscira GSM codec libraries is ANSI C (function +prototypes), const qualifier is used where appropriate, and the interface is +defined in terms of types; includes . The +use of old libgsm defined types (gsm_byte, gsm_frame and gsm_signal) has been +abolished in the migration from libgsm+libgsmfrp to libgsmfr2. + +GSM 06.10 encoder and decoder +============================= + +Both the encoder and the decoder are stateful; each running instance of either +element needs its own state structure. However, this GSM 06.10 component of +libgsmfr2 shares a peculiar property with old libgsm from which it was derived: +the same state structure (struct gsmfr_0610_state) is used by both entities. +Needless to say, each given instance of struct gsmfr_0610_state must be used +for only one purpose, either for the encoder or for the decoder; mixing calls +to encoder and decoder functions with the same state structure is an invalid +operation with undefined results. + +State structures for the basic encoder or decoder are allocated with this +function: + +struct gsmfr_0610_state *gsmfr_0610_create(void); + +This function allocates dynamic memory for the state structure with malloc() +(the size of the struct is internal to the library and not exposed) and returns +a pointer to the allocated and initialized struct if successful, or NULL if +malloc() fails. The state structure is malloc'ed as a single chunk, hence when +you are done with it, simply free() it. + +The initialization or reset portion of gsmfr_0610_create() operation can always +be repeated with this function: + +void gsmfr_0610_reset(struct gsmfr_0610_state *state); + +Immediately after gsmfr_0610_create() or gsmfr_0610_reset(), the "virgin" state +structure can be used either for the encoder or for the decoder; however, once +that state struct has been passed to functions of either group, it can only be +used for that functional group. + +Encoder specifics +----------------- + +The most elementary single-frame processing function of libgsmfr2 GSM 06.10 +encoder is: + +void gsmfr_0610_encode_params(struct gsmfr_0610_state *st, const int16_t *pcm, + struct gsmfr_param_frame *param); + +The input is an array of 160 linear PCM samples (left-justified in int16_t), +and the output is this structure: + +struct gsmfr_param_frame { + int16_t LARc[8]; + int16_t Nc[4]; + int16_t bc[4]; + int16_t Mc[4]; + int16_t xmaxc[4]; + int16_t xMc[4][13]; +}; + +Most of the time the following wrapper function is more useful: + +void gsmfr_0610_encode_frame(struct gsmfr_0610_state *st, const int16_t *pcm, + uint8_t *frame); + +The output is a 33-byte buffer, filled with the encoded GSM-FR speech frame in +the RTP format specified in ETSI TS 101 318 and IETF RFC 3551. + +If the optional encoder homing feature is desired, call this function right +after the call to gsmfr_0610_encode_frame() or gsmfr_0610_encode_params(): + +void gsmfr_0610_encoder_homing(struct gsmfr_0610_state *st, const int16_t *pcm); + +This function checks to see if the PCM frame (160 linear PCM samples) is an EHF; +if the input frame is indeed EHF, the function calls gsmfr_0610_reset(). + +Decoder specifics +----------------- + +The internal native form of the 06.10 decoder once again uses +struct gsmfr_param_frame: + +void gsmfr_0610_decode_params(struct gsmfr_0610_state *st, + const struct gsmfr_param_frame *param, + int16_t *pcm); + +The more commonly used RTP-format version is: + +void gsmfr_0610_decode_frame(struct gsmfr_0610_state *st, const uint8_t *frame, + int16_t *pcm); + +Please note: + +1) The basic GSM 06.10 decoder is just that: there is no SID recognition or DTX + handling, every possible input bit pattern will be interpreted and decoded + as a GSM 06.10 speech frame. + +2) There is no decoder homing function at this layer, and no check for DHF. + +3) The RTP signature nibble 0xD is ignored (not checked) by + gsmfr_0610_decode_frame(). + +Rx DTX preprocessor block +========================= + +The Rx DTX preprocessor is its own stateful element, independent from the 06.10 +decoder to which it is usually coupled. Libgsmfr2 provides a "fulldec" wrapper +that incorporates both elements, but the ability to use the Rx DTX preprocessor +by itself still remains, unchanged from our previous libgsmfrp offering. One +potential application for this preprocessor by itself, without immediately +following it with the GSM 06.10 decode step, is the possibility of implementing +the TFO/TrFO transform of 3GPP TS 28.062 section C.3.2.1.1 for GSM-FR: our Rx +DTX preprocessor does exactly what that section calls for, specifically in +"case 1" where the input UL frame stream may contain SIDs and BFI frame gaps, +but the output must be 100% valid frames and SID-free. + +The state structure for this block is struct gsmfr_preproc_state, and it is +allocated with this function: + +struct gsmfr_preproc_state *gsmfr_preproc_create(void); + +Like other state structures in Themyscira GSM codec libraries, this opaque +state is malloc'ed as a single chunk and can be simply freed afterward. A +reset function is also provided: + +void gsmfr_preproc_reset(struct gsmfr_preproc_state *state); + +Preprocessing good frames +------------------------- + +For every good traffic frame (BFI=0) you receive from the radio subsystem, you +need to call this preprocessor function: + +void gsmfr_preproc_good_frame(struct gsmfr_preproc_state *state, + uint8_t *frame); + +The second argument is both input and output, i.e., the frame is modified in +place. If the received frame is not SID (specifically, if the SID field +deviates from the SID codeword by 16 or more bits, per GSM 06.31 section 6.1.1), +then the frame (considered a good speech frame) will be left unmodified (i.e., +it is to be passed unchanged to the GSM 06.10 decoder), but preprocessor state +will be updated. OTOH, if the received frame is classified as either valid or +invalid SID per GSM 06.31, then the output frame will contain comfort noise +generated by the preprocessor using a PRNG, or a silence frame in one particular +corner case. + +GSM-FR RTP (originally libgsm) 0xD magic: the upper nibble of the first byte +can be anything on input to gsmfr_preproc_good_frame(), but the output frame +will always have the correct magic in it. + +Handling BFI conditions +----------------------- + +If you received a lost/missing frame indication instead of a good traffic frame, +call this preprocessor function: + +void gsmfr_preproc_bfi(struct gsmfr_preproc_state *state, int taf, + uint8_t *frame_out); + +TAF is a flag defined in GSM 06.31 section 6.1.1; if you don't have this flag, +pass 0 - you will lose the function of comfort noise muting in the event of +prolonged SID loss, but all other Rx DTX functions will still work the same. + +With this function the 33-byte frame buffer is only an output, i.e., prior +buffer content is a don't-care and there is no provision for making any use of +erroneous frames like in EFR. The frame generated by the preprocessor may be +substitution/muting, comfort noise or silence depending on the state. + +GSM-FR full decoder +=================== + +The full decoder is a high-level feature of libgsmfr2, incorporating both the +Rx DTX preprocessor block and the GSM 06.10 decoder block. The state structure +for the full decoder (struct gsmfr_fulldec_state) internally incorporates both +struct gsmfr_0610_state and gsmfr_preproc_state, but because it is implemented +inside libgsmfr2, it is still malloc'ed as a single chunk and can thus be +released with a single free() call. The functions for allocating and +initializing this state follow the established pattern: + +struct gsmfr_fulldec_state *gsmfr_fulldec_create(void); + +void gsmfr_fulldec_reset(struct gsmfr_fulldec_state *state); + +The reset function internally calls gsmfr_0610_reset() and +gsmfr_preproc_reset(), initializing both processing blocks. + +Frame processing functions are also straightforward: + +void gsmfr_fulldec_good_frame(struct gsmfr_fulldec_state *state, + const uint8_t *frame, int16_t *pcm); + +void gsmfr_fulldec_bfi(struct gsmfr_fulldec_state *state, int taf, + int16_t *pcm); + +These functions follow the same pattern as gsmfr_preproc_good_frame() and +gsmfr_preproc_bfi(), but the output is a 160-sample linear PCM buffer. Also +note that the frame input to gsmfr_fulldec_good_frame() is const, unlike the +situation with gsmfr_preproc_good_frame() - the copying into a scratchpad +buffer (on the stack) happens inside this "fulldec" wrapper. + +The "fulldec" layer also adds the decoder homing feature: +gsmfr_fulldec_good_frame() detects decoder homing frames and invokes +gsmfr_fulldec_reset() when required, and also implements EHF output per the +spec. diff -r e0d42e87da96 -r 6b479cfb06a4 doc/FR1-library-desc --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/FR1-library-desc Mon Apr 15 07:12:31 2024 +0000 @@ -0,0 +1,81 @@ +Themyscira libgsmfr2 is our new (2024) library offering for GSM FRv1 codec, +replacing the previous combination of libgsm (classic 1990s free sw offering) +and libgsmfrp (our add-on). That combination appeared satisfactory at first +because of how the decoder processing chain is defined for FRv1 (the Rx DTX +handler forms a modular piece, passing a frame of 260 bits to unmodified "pure" +GSM 06.10 decoder), but use of legacy libgsm presents some difficulties: + +* Inconvenience and inconsistency: for all other supported GSM speech codecs, + Themyscira libraries provide the complete solution, not depending on anyone + else's software, but for FRv1 we depended on a library that was created in a + different era for non-GSM applications but just happens, almost by accident, + to be a bit-exact implementation of GSM 06.10 spec. + +* Poor design in frame packing and unpacking functions: the operations of + bit-shuffling a GSM-FR codec frame between the array of parameters form + (76 words) and the packed RTP format used in IP-based GSM RAN (33 bytes) are + stateless pure functions, but their implementations in libgsm (gsm_explode() + and gsm_implode()) require a state structure. (Libgsm supports WAV49 format + in addition to the RTP-adopted one, and WAV49 packing is stateful - but this + WAV49 feature is of no use and no relevance in real GSM applications.) + +* No ability to implement homing: the internal state structure used by libgsm + is set to the home state when it is allocated with gsm_create(), but there is + no reset function, and such function cannot be implemented externally when + the state structure is private to the library and not exposed. Therefore, + the optional codec homing feature defined in later versions of GSM 06.10 spec + cannot be implemented in a wrapper around legacy libgsm, causing the resulting + FOSS implementation to be inferior to existing commercial implementations + (deployed in practice by incumbent nation-scale networks) which do implement + this feature. + +In response to the above issues, we now have a new library named libgsmfr2 that +provides all needed functions for GSM-FR codec "under one roof", harmonized +with our support for other GSM speech codecs. However, the modularity that is +inherent in the way this codec is defined in the specs (contrast with EFR) is +still retained in the design of our library, which exhibits the following 4 +functional divisions: + +1) Libgsmfr2 includes a "pure" GSM 06.10 encoder and decoder, directly + corresponding to old libgsm. This code is in fact lifted from libgsm, + ported into Themyscira gsm-codec-lib style in terms of C dialect, defined + types and naming conventions. The reset function that is missing in libgsm + is now provided, however, fixing that defect. + +2) The Rx DTX handler component is unchanged from our previous libgsmfrp. Per + the relevant specs (GSM 06.11, 06.12 and 06.31) this component is a modular + piece: it emits a standard 260-bit frame of GSM 06.10 parameters that can be + fed to anyone else's implementation of the latter standard. + +3) Full decoder: this component is a wrapper around an 06.10 decoder instance + and an Rx DTX instance, providing functionality equivalent to the standard + decoder function in other GSM speech codecs. + +4) Stateless utility functions for frame format conversions (packing and + unpacking) and for incoming SID classification. An application can freely + use just these functions, without pulling in any encoder or decoder or + stateful preprocessor functionality, making the present library very + convenient for debug utilities. + +The homing feature is available in both encoder and decoder directions, but it +is implemented differently between the two: + +* In the encoder direction, if the application wishes to enable the possibility + of in-band homing, it needs to call gsmfr_0610_encoder_homing() after the + regular call to gsmfr_0610_encode_frame() or gsmfr_0610_encode_params(). + +* In the decoder direction, the homing feature is always included if one uses + the "fulldec" (full decoder) wrapper (it is implemented in that layer), and + never included if one uses the "basic" GSM 06.10 decoder by itself, or the + Rx DTX handler block by itself. + +The only major feature of GSM-FR codec that is currently absent in libgsmfr2 is +application of DTX in the encoder direction: GSM 06.32 VAD followed by DTX +hangover logic and SID output. This omission is currently acceptable for +Themyscira Wireless: DTXd (DTX in the radio downlink direction) is not allowed +when each BTS operates at only one carrier frequency, which makes it pointless +to enable DTX for speech encoding at the network edge transcoder, and we are not +trying to replace TI Calypso DSP on the MS side of GSM. However, if this +situation changes and some need arises for a FOSS implementation of DTX-enabled +GSM-FR encoder, the architecture of Themyscira libgsmfr2 should make it possible +to integrate such addition.