# HG changeset patch # User Mychaela Falconia # Date 1716241991 0 # Node ID cd1f0fa936cc77678f9c915888b831310e78ca8c # Parent f35c8fd9ceba8ccea4bda357aa73849c0adcf4e6 doc/AMR-EFR-performance: new article diff -r f35c8fd9ceba -r cd1f0fa936cc doc/AMR-EFR-performance --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/doc/AMR-EFR-performance Mon May 20 21:53:11 2024 +0000 @@ -0,0 +1,102 @@ +Performance of libgsmefr and libtwamr, compared to competition +============================================================== + +Both libgsmefr and libtwamr are based on reference C code from ETSI/3GPP: +libgsmefr is a librified version of GSM 06.53 reference code, and libtwamr is a +librified version of TS 26.073 reference code. Both of those reference sources +were officially presented as simulations, not as production code for running on +general-purpose x86 servers implementing transcoding MGW functionality, and the +code is extremely inefficient. + +The problem of poor performance (taking too much CPU percentage per transcoded +call) would not be so acute if Themyscira GSM codec libraries existed in a +vacuum without competition - but we do have a competitor in the form of +libopencore-amrnb, part of the slightly larger opencore-amr package. Just like +libtwamr, that library is a derivative work based on the original AMR C code +from 3GPP, but by a more circuitous route: first PacketVideo created an +implementation of AMR for Android based on 3GPP reference code, then the +creators of opencore-amr took PV's Android implementation (named OpenCORE) and +ported it from Android to standard Unix/Linux userspace. + +Aside from its peculiar genealogy, there are practical problems with +libopencore-amrnb that made it unattractive for Themyscira Wireless, and thus +created the impetus for producing the present alternative: + +* That library implements only AMR and not EFR, whereas for ThemWi good support + for EFR is a higher priority than good (or any) support for AMR. + +* The idea of reshuffling bits in the manner of our gsm-amr2efr and gsm-efr2amr + hack-utilities (see AMR-EFR-conversion article) in order to implement "poor + man's EFR" via libopencore-amrnb was considered, but then rejected on + philosophical and aesthetic grounds: it feels philosophically wrong to + reshuffle bits in the application wrapper only to have internal code within + the library shuffle them back into the natural order of codec parameters. + +* Like any AMR-and-not-EFR library, libopencore-amrnb cannot grok EFR SID + frames. Therefore, an implementation of "poor man's EFR" using that library + would have to operate with DTXu disabled, needlessly burning battery capacity + in the MS. + +* Even for "pure" AMR and not EFR, libopencore-amrnb exhibits one serious + defect: it omits the in-band homing feature in both the encoder and the + decoder, even though this feature is mandatory per 3GPP specs. (It is not + clear if this removal of homing functions was the work of PacketVideo in the + world of Android, or if it happened at the time of subsequent opencore-amr + port to standard Unix/Linux userspace.) Furthermore, because the interface + wrapper module of libopencore-amrnb does not expose any state reset functions, + it is impossible to implement the homing feature externally to the library. + This lack of homing makes it impossible to test libopencore-amrnb against the + full set of 3GPP test sequences for AMR. + +In the present time, as of this release of Themyscira gsm-codec-lib, libgsmefr +is intended for production use while libtwamr is intended for lab use only, +matching test sequences and modeling bit-exact operation of other people's +network elements in reverse engineering workflows. Based on this difference in +applicability to production, more work has been put into improving performance +of libgsmefr compared to libtwamr. Libgsmefr version 1.1.0 exhibits +approximately 2x performance improvement over the original version, but it is +still significantly slower than libopencore-amrnb. + +The following performance numbers were obtained on Mother Mychaela's development +laptop, Intel Core2 Duo P8600 CPU @ 2.40 GHz, running 32-bit version of +Slackware, i.e., running only i686 code and not x86_64 despite the CPU having +the 'lm' feature. All tests involve encoding TEST4.INP sequence from GSM 06.54 +set (301 frames in total, 6.020 s audio duration), then decoding the output of +the encoder run. + +Implementation Encoding time Decoding time +--------------------------------------------- +libgsmefr 183 ms 23 ms +libtwamr 359 ms 52 ms +opencore-amr 53 ms 13 ms +libgsmfr2 (FR) 13 ms 6 ms + +FRv1 encoding and decoding of the same TEST4.INP was included in the test in +order to convince ourselves that FRv1 is not in any need of performance +optimization. In a network deployment that generally prefers EFR over FRv1, +almost all calls between a local GSM MS and the outside world (PSTN) will +involve the EFR transcoder rather than FRv1. If on rare occasion the network +selects FRv1 instead of EFR because the subscriber is using a super-old +non-EFR-capable handset (or has artificially restricted the speech version list +to FRv1 only), there will be some CPU cycles saved "for free" by using the much +less computationally expensive FRv1 codec - but CPU capacity planning for the +transcoding MGW has to be based on EFR performance numbers rather than FRv1. + +In terms of EFR or MR122 encoder and decoder CPU cycle demand, libtwamr is +currently the worst performer; the first version of libgsmefr exhibited the +same poor performance. The current version of libgsmefr exhibits somewhat +better performance because most saturation arithmetic "basic ops" have been +inlined (eliminating the use of a function call for every elementary operation), +and the implementation code of those newly-inlined functions has been +streamlined for performance, as much as possible with immediately obvious +programming methods. The result is about 2x improvement over the original - +but sadly, still significantly slower than libopencore-amrnb on the same +machine with the same 32-bit execution mode and ABI. + +Given that we (Themyscira Wireless) have a ton of other work tasks in our queue +besides the present problem of performance-optimizing libgsmefr to match +opencore-amr, this problem is currently shelved. We are releasing our libraries +as they currently stand, with the ~2x improvement in libgsmefr over the original +but still not matching opencore-amr, and leaving the problem of further +performance improvement to be addressed later, after other work of higher +priority.