NOTE: This document describes the old BFI representation format that was used
by Themyscira Wireless for a few months from late 2022 into early 2023.  We no
longer use this format, instead we have a new, bolder extended payload format
for FR & EFR codecs, defined and described in the new RTP-TRAUlike-format
document.

Original document follows
=========================

We (Themyscira Wireless) have invented our own non-standard extension to the
generally accepted standard for RTP-based transport of GSM FR and EFR traffic
within a GSM RAN, on stretches running from a BTS to a TRAU-like component.

The fundamental question is: when the radio subsystem of the BTS does not have
any good traffic frame to send in a given 20 ms window, what should it do?  The
generally accepted standard behavior is that no packet is sent, an intentional
gap is created in the RTP stream (the next time an RTP packet does go out, the
timestamp increments over the gap while the sequence number increments only by
1, indicating an intentional gap rather than packet loss), and apparently the
intent was/is that this gap in the RTP stream serves as the BFI (bad frame
indication).

The problem with this generally accepted gap-as-BFI approach is that it deprives
the downstream transcoding MGW (a "soft TRAU" of sorts) of its timing source.
If the TRAU-like entity on the receiving end of the RTP stream originating from
the BTS were an RTP to TDM gateway, there would be no problem - such a gateway
would have to buffer received RTP packets in order to synchronize to fixed TDM
timing, and the absence of an RTP packet arriving in time would serve just fine
as the BFI marker, signaling BFI condition to the Rx DTX handler.  But what if
the G.711 interface on the 64 kbps side of the TRAU is also an RTP stream, this
time going to a PSTN-via-SIP connectivity provider?  Now the TRAU-like component
becomes a transcoding RTP forwarding MGW without any inherently fixed timing.

If the desire is to implement a traditional TRAU in every way except for an
RTP-based implementation instead of TDM-based, i.e., if the desire is to emit a
fully continuous G.711 RTP stream from the MGW toward PSTN with comfort noise
generation and in-band DTMF insertion happening inside the MGW, rather than
emit gaps in the outgoing stream or punt CN generation (and DTMF) to VoIP
network elements, this task becomes dramatically easier if the BTS can be
forced to send an RTP packet in every 20 ms window, be it rain or shine,
conveying either a good traffic frame or a BFI marker.

Representing BFI markers in an RTP stream
=========================================

In the case of AMR codec, the existing standard RTP payload format already
provides an obvious way to send a BFI marker: it is the NO_DATA frame type,
i.e., FT=15 - see RFC 4867 section 4.3.2.  That same section also categorizes
what we seek to do here as a "SHOULD NOT":

   Note that packets containing only NO_DATA frames SHOULD NOT be
   transmitted in any payload format configuration, [...]

However, the just-quoted directive is a SHOULD NOT rather than a MUST NOT,
and RFC 2119 states:

   SHOULD NOT   This phrase, or the phrase "NOT RECOMMENDED" mean that
   there may exist valid reasons in particular circumstances when the
   particular behavior is acceptable or even useful, but the full
   implications should be understood and the case carefully weighed
   before implementing any behavior described with this label.

Our situation is just that: in our particular circumstance (desire to implement
a traditional GSM TRAU in an RTP-to-RTP environment with no TDM network to act
as a rigid timing governor) a valid reason exists why this "SHOULD NOT" behavior
is not only acceptable, but becomes necessary.  Thus in the case of AMR, we are
good - there is no need to invent our own totally non-standard extensions to
RTP payload format, it just needs to be a configurable option in the IP-based
BTS or in OsmoMGW converting from an E1-based BTS to RTP.

The same situation holds for the rarely-used HR1 codec: RFC 5993 extends GSM-HR
RTP representation with a ToC byte modeled after the one defined for AMR in RFC
4867.  Just like in AMR, GSM-HR ToC byte allows the possibility of a No_Data
frame (FT=7 for GSM-HR), with exactly the same semantics - and exactly the same
argument as above applies for sending such No_Data frames against the general
SHOULD NOT.

But what about the older FR and EFR codecs?  In the case of existing standard
RTP payload formats for FR and EFR, there is no defined way to represent a BFI
condition as distinct from any possible good traffic frame, and there lies our
challenge.

Inventing an RTP BFI marker for FR and EFR
==========================================

The existing code in osmo-bts-trx (but not in the osmo-bts-sysmo version of
interest to us) already contains a partial implementation of what we seek to do
here: it runs its own ECU instance in the case of a BFI from the channel
decoding layer, and if there is still no luck, there is code present to send a
BFI packet.  The implemented behavior is not useful for us because RTP output
is still fully suppressed when the uplink is expected to be in DTX, and there
is a higher-level check in common/l1sap.c (l1sap_tch_ind() function) that also
suppresses RTP output, but still, the point is that someone did already write
code for sending an RTP packet intended to serve as a BFI.  In the case of AMR,
that code sends out the expected NO_DATA (aka AMR_BAD) frame type - but what
about FR and EFR?

The existing code in osmo-bts-trx sends its FR codec BFI as a valid-looking FR
frame with all 260 content bits set to 0, and it sends its EFR codec BFI as a
valid-looking EFR frame with all 244 content bits set to 0.  I (Mother Mychaela)
have given consideration to using this all-zeros in-band BFI representation as
our RTP BFI marker for ThemWi, but then rejected this idea and decided to
implement our own non-standard extension to RTP payload format instead,
described further below.

The fundamental philosophical problem which I (Mother Mychaela) have with this
in-band BFI representation is that in the world of ETSI and 3GPP standards, BFI
has always been meant to be out-of-band, not in-band.  In the TRAU frame format
defined in GSM 08.60 there is an explicit control bit that carries BFI - the
condition is NOT to be derived from the 260 or 244 traffic frame bits carried
in data bit positions.  Abusing one particular bit pattern within the regular
260-bit or 244-bit frame, even if it happens to be all zeros, goes against the
spirit of classic GSM and 3GPP.  Per the specs, an FR codec frame of all zeros
would be a SID frame with all LAR coefficients set to 0, and standards-compliant
FR decoders would accept it as a valid SID frame, not as BFI.  The situation is
likely to be even worse with EFR, where a frame of all zeros would not be
treated as SID (EFR SID code word is 95 ones instead of 95 zeros) and would
probably produce garbage at the decoder output.

Themyscira Wireless implemented solution
========================================

We have invented our own non-standard extension to RTP payload format for GSM
FR and EFR codecs.  Our extension is as follows: wherever a BTS needs to send a
BFI marker in the place of a traffic frame, instead of sending a 33-byte payload
beginning with 0xD nibble or a 31-byte payload beginning with 0xC nibble, it
needs to send a 2-byte payload formatted as follows:

byte 0: 0xBF signature;
byte 1: least-significant bit encoding TAF per GSM 06.31 or GSM 06.81,
        section 6.1.1 in both documents; other bits are reserved.

In the uplink direction, with an RTP stream going from a BTS to our "soft TRAU"
MGW, our themwi-mgw recognizes these BFI packets and acts accordingly, feeding
BFI and TAF to the spec-prescribed Rx DTX handler for FR or EFR.  However, if a
BTS receives these BFI marker packets in the downlink direction as a result of
TrFO (the RTP stream comes from the uplink of another GSM call), it simply
discards them without any processing - because a BTS always runs on its own TDMA
timing, there is no difference between receiving a BFI packet vs receiving no
RTP packet at all for that 20 ms frame.

Our patch to osmo-bts
=====================

Our current BTS is a sysmoBTS, as opposed to a T1/E1-based BTS interfaced via
OsmoBSC and OsmoMGW, hence the software component in need of patching is
osmo-bts.  The patch in osmo-patches/osmo-bts-rtp-bfi.patch in the present
repository applies to the common part of osmo-bts, hence it should theoretically
work with all models, but it has only been tested with osmo-bts-sysmo.  If
anyone wishes to play with this patch in an osmo-bts-trx environment, they will
probably need to remove TRX-specific ECU and BFI code from
osmo-bts-trx/sched_lchan_tchf.c, otherwise the two implementations will "fight":
the TRX-specific code will kick in on "unexpected" bad frames and times of FACCH
stealing, whereas the common code will fill in the new form of BFI during times
of uplink DTX.

Aside from the just-described incompatibility with osmo-bts-trx version, our
current patch has the following outstanding issues which will need to be fixed
before it can be proposed for merging:

* The code will need to be extended to support AMR, and possibly HR1 too - our
  current patch always emits our FR/EFR BFI, implicitly assuming that the speech
  codec is either FR or EFR, as that is how we currently run our network.

* A configuration option will need to be implemented, to be controlled from
  OsmoBSC or from the BTS' own vty, enabling or disabling the non-standard
  behavior of sending explicit BFI packets instead of gaps in the RTP stream.
  It would probably be best implemented as a tristate option, with the choices
  being no BFI packets at all (default), sending BFI packets only for AMR
  (where a standard encoding exists) or sending BFI packets for AMR and also
  for FR & EFR, using Themyscira encoding for the latter.