--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/doc/RTP-BFI-extension	Sat Dec 10 23:54:11 2022 +0000
@@ -0,0 +1,124 @@
+We (Themyscira Wireless) have invented our own non-standard extension to the
+generally accepted standard for RTP-based transport of GSM FR and EFR traffic
+within a GSM RAN, on stretches running from a BTS to a TRAU-like component.
+
+The fundamental question is: when the radio subsystem of the BTS does not have
+any good traffic frame to send in a given 20 ms window, what should it do?  The
+generally accepted standard behavior is that no packet is sent, an intentional
+gap is created in the RTP stream (the next time an RTP packet does go out, the
+timestamp increments over the gap while the sequence number increments only by
+1, indicating an intentional gap rather than packet loss), and apparently the
+intent was/is that this gap in the RTP stream serves as the BFI (bad frame
+indication).
+
+The problem with this generally accepted gap-as-BFI approach is that it deprives
+the downstream transcoding MGW (a "soft TRAU" of sorts) of its timing source.
+If the TRAU-like entity on the receiving end of the RTP stream originating from
+the BTS were an RTP to TDM gateway, there would be no problem - such a gateway
+would have to buffer received RTP packets in order to synchronize to fixed TDM
+timing, and the absence of an RTP packet arriving in time would serve just fine
+as the BFI marker, signaling BFI condition to the Rx DTX handler.  But what if
+the G.711 interface on the 64 kbps side of the TRAU is also an RTP stream, this
+time going to a PSTN-via-SIP connectivity provider?  Now the TRAU-like component
+becomes a transcoding RTP forwarding MGW without any inherently fixed timing.
+
+If the desire is to implement a traditional TRAU in every way except for an
+RTP-based implementation instead of TDM-based, i.e., if the desire is to emit a
+fully continuous G.711 RTP stream from the MGW toward PSTN with comfort noise
+generation and in-band DTMF insertion happening inside the MGW, rather than
+emit gaps in the outgoing stream or punt CN generation (and DTMF) to VoIP
+network elements, this task becomes dramatically easier if the BTS can be
+forced to send an RTP packet in every 20 ms window, be it rain or shine,
+conveying either a good traffic frame or a BFI marker.
+
+Representing BFI markers in an RTP stream
+=========================================
+
+In the case of AMR codec, the existing standard RTP payload format already
+provides an obvious way to send a BFI marker: it is the NO_DATA frame type,
+i.e., FT=15 - see RFC 4867 section 4.3.2.  That same section also categorizes
+what we seek to do here as a "SHOULD NOT":
+
+   Note that packets containing only NO_DATA frames SHOULD NOT be
+   transmitted in any payload format configuration, [...]
+
+However, the just-quoted directive is a SHOULD NOT rather than a MUST NOT,
+and RFC 2119 states:
+
+   SHOULD NOT   This phrase, or the phrase "NOT RECOMMENDED" mean that
+   there may exist valid reasons in particular circumstances when the
+   particular behavior is acceptable or even useful, but the full
+   implications should be understood and the case carefully weighed
+   before implementing any behavior described with this label.
+
+Our situation is just that: in our particular circumstance (desire to implement
+a traditional GSM TRAU in an RTP-to-RTP environment with no TDM network to act
+as a rigid timing governor) a valid reason exists why this "SHOULD NOT" behavior
+is not only acceptable, but becomes necessary.  Thus in the case of AMR, we are
+good - there is no need to invent our own totally non-standard extensions to
+RTP payload format, it just needs to be a configurable option in the IP-based
+BTS or in OsmoMGW converting from an E1-based BTS to RTP.
+
+But what about the older FR and EFR codecs?  In the case of existing standard
+RTP payload formats for FR and EFR, there is no defined way to represent a BFI
+condition as distinct from any possible good traffic frame, and there lies our
+challenge.
+
+Inventing an RTP BFI marker for FR and EFR
+==========================================
+
+The existing code in osmo-bts-trx (but not in the osmo-bts-sysmo version of
+interest to us) already contains a partial implementation of what we seek to do
+here: it runs its own ECU instance in the case of a BFI from the channel
+decoding layer, and if there is still no luck, there is code present to send a
+BFI packet.  The implemented behavior is not useful for us because RTP output
+is still fully suppressed when the uplink is expected to be in DTX, and there
+is a higher-level check in common/l1sap.c (l1sap_tch_ind() function) that also
+suppresses RTP output, but still, the point is that someone did already write
+code for sending an RTP packet intended to serve as a BFI.  In the case of AMR,
+that code sends out the expected NO_DATA (aka AMR_BAD) frame type - but what
+about FR and EFR?
+
+The existing code in osmo-bts-trx sends its FR codec BFI as a valid-looking FR
+frame with all 260 content bits set to 0, and it sends its EFR codec BFI as a
+valid-looking EFR frame with all 244 content bits set to 0.  I (Mother Mychaela)
+have given consideration to using this all-zeros in-band BFI representation as
+our RTP BFI marker for ThemWi, but then rejected this idea and decided to
+implement our own non-standard extension to RTP payload format instead,
+described further below.
+
+The fundamental philosophical problem which I (Mother Mychaela) have with this
+in-band BFI representation is that in the world of ETSI and 3GPP standards, BFI
+has always been meant to be out-of-band, not in-band.  In the TRAU frame format
+defined in GSM 08.60 there is an explicit control bit that carries BFI - the
+condition is NOT to be derived from the 260 or 244 traffic frame bits carried
+in data bit positions.  Abusing one particular bit pattern within the regular
+260-bit or 244-bit frame, even if it happens to be all zeros, goes against the
+spirit of classic GSM and 3GPP.  Per the specs, an FR codec frame of all zeros
+would be a SID frame with all LAR coefficients set to 0, and standards-compliant
+FR decoders would accept it as a valid SID frame, not as BFI.  The situation is
+likely to be even worse with EFR, where a frame of all zeros would not be
+treated as SID (EFR SID code word is 95 ones instead of 95 zeros) and would
+probably produce garbage at the decoder output.
+
+Themyscira Wireless implemented solution
+========================================
+
+We have invented our own non-standard extension to RTP payload format for GSM
+FR and EFR codecs.  Our extension is as follows: wherever a BTS needs to send a
+BFI marker in the place of a traffic frame, instead of sending a 33-byte payload
+beginning with 0xD nibble or a 31-byte payload beginning with 0xC nibble, it
+needs to send a 2-byte payload formatted as follows:
+
+byte 0: 0xBF signature;
+byte 1: least-significant bit encoding TAF per GSM 06.31 or GSM 06.81,
+        section 6.1.1 in both documents; other bits are reserved.
+
+In the uplink direction, with an RTP stream going from a BTS to our "soft TRAU"
+MGW, our themwi-mgw recognizes these BFI packets and acts accordingly, feeding
+BFI and TAF to the spec-prescribed Rx DTX handler for FR or EFR.  However, if a
+BTS receives these BFI marker packets in the downlink direction as a result of
+TrFO (the RTP stream comes from the uplink of another GSM call), it simply
+discards them without any processing - because a BTS always runs on its own TDMA
+timing, there is no difference between receiving a BFI packet vs receiving no
+RTP packet at all for that 20 ms frame.