--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/doc/BFI-payload-fill	Sat May 25 23:07:13 2024 +0000
@@ -0,0 +1,161 @@
+The traditional TRAU-UL frame format of GSM 08.60 & 08.61 for FR/HR/EFR codecs
+always includes a full payload in every frame, even those marked as BFI=1.
+Hence a question naturally arises: what does a traditional T1/E1 BTS put in the
+payload bits of its TRAU-UL output when there are no "received, but erroneous"
+bits for this frame, i.e., when the speech frame position was stolen for FACCH,
+or when nothing at all was received (no burst midambles detected), with nothing
+to run the channel decoder on?  A satisfactory answer to this question will be
+needed by anyone who sets out to do either of the following:
+
+a) build a new (FOSS-based) T1/E1 BTS;
+
+b) implement GSM 08.62 TFO (in-band) in a soft transcoder that works with an
+   IP-based GSM RAN.
+
+Case (a) is probably only a theoretical thought exercise, but case (b) is quite
+real: I (Mother Mychaela) do very much desire to bring back in-band TFO, and do
+so under current economic conditions that call for IP transport within GSM RAN
+and an IP interface to G.711 PSTN.  For the latter problem, the enhancements of
+TW-TS-001 & TW-TS-002 help greatly (and are required for proper implementation
+of TFO), but the question still remains of what should be emitted in TFO frames
+when a no-data BFI comes in.  (Most BFIs on OsmoBTS are the no-data kind; there
+is currently little opportunity for BFI-with-data UL output.)
+
+In order to get better insight into this question, we (the small community of
+people who care about this topic) need to look at the Abis UL output from real
+historical E1 (or T1) BTS implementations, and see what they actually put out
+under these conditions.  I (Mother Mychaela) do not currently own any T1/E1 BTS
+hardware, but thankfully there are some published capture files in Osmocom which
+we can look at.  The two binary files in trau-files directory of this repository
+have been copied from libosmo-abis/contrib/trau2rtp, where they were committed
+some time in Anno Horribilis (2020).
+
+Analysis of Osmocom contrib/trau2rtp files
+==========================================
+
+The README file in libosmo-abis/contrib/trau2rtp describes the origin of
+e1_ts2_efr.bin and e1_ts2_fr.bin thusly:
+
+> The input data (*.log.bz2) was generated using strace on an osmo-nitb process
+> while a MO-to-MT call was running on two sub-slots of TS2.
+>
+> The strace log is converted to a binary stream of the raw 64bit E1 slot
+> using strace-write-parse.py
+
+Unfortunately there is no annotation as to what make/model of E1 BTS was used
+in the OsmoNITB test call setup that produced these logs.  The same README file
+later says:
+
+> The code has been tested against BS-11 and RBS6000/DUG20 in both modes
+> (loop vs. RTP) and for FR and EFR.
+
+but that note seems to refer to the workflow of the trau2rtp.c program operating
+directly on a "live" DAHDI timeslot, not to the provenance of the included
+capture files.
+
+However, despite not knowing the specific BTS model that produced the TRAU-UL
+frame streams that have been preserved for posterity in those Osmocom-published
+capture files, we can make some observations after parsing them with our
+trau-parse utility in trau-decode directory:
+
+* In both FR and EFR captures, sub-timeslot 1 comes to life first (the idle fill
+  pattern of 2'b01 changes to TRAU-UL frames), then sub-timeslot 2.  Thus we can
+  infer that subslot 1 is the MO leg of the test call, and subslot 2 is the MT
+  leg.
+
+* The FR capture looks like I expected: every TRAU-UL frame (right from the
+  point where the sub-timeslot comes to life) is of type FR UL, and various bits
+  are set as I expected from the reading of GSM 08.60 spec.
+
+* The EFR capture exhibits this strange oddity: each subslot starts out emitting
+  FR UL frames, then switches to emitting EFR frames some significant time
+  later.  (920 ms later on the MO leg and 740 ms later on the MT leg in this
+  test call capture.)  I have no clue why this oddity is occuring: is it perhaps
+  an artifact of OsmoNITB initially activating the GSM timeslot as FR and then
+  switching to EFR?  Or is it a quirk intrinsic to whatever BTS model these
+  captures came from?
+
+BFI output seen in these Osmocom-published captures
+===================================================
+
+Having done the preliminary analysis presented above, we can return to our
+original question: what did that model-unknown E1 BTS emit in its BFI frames,
+those marked with C12=1?  The evidence in the artifacts under examination
+indicates that at least this particular E1 BTS model uses the buffer method,
+similar to what is done on the mobile side of GSM Um in the well-studied
+Calypso DSP.
+
+What I mean by "the buffer method" is that the Rx Radio Subsystem (as defined
+in GSM 06.31, 06.41 and 06.81 specs) maintains a buffer, potentially persisting
+from one Rx frame position to the next, that holds the output of GSM 05.03
+channel decoder block and the associated SID detector.  Under good Rx
+conditions, those that lead to BFI=0 output, this buffer gets overwritten with
+new bits on every frame, and its potentially persistent nature is not apparent.
+However, when no new traffic bits were received, neither good nor bad, the
+buffer holds its previous content - and this buffer content gets emitted in
+subsequent BFI=1 frames.  How can we tell by black-box observation whether a
+given implementation uses this buffer method?  When the buffer method is used,
+the following output should be visible at the observed interface (TRAU-UL frames
+on the BTS side or Calypso DSP a_dd_0[] buffer on the MS side):
+
+* When a certain frame position gets stolen for FACCH in the middle of what is
+  otherwise a stream of good traffic frames (the transmitter is not cut off as
+  in DTX), the BFI=1 output corresponding to the FACCH position will have the
+  same bit content as the previous good traffic frame.
+
+* During times of prolonged absence of radio Rx (DTX pauses, or times during
+  channel activation or shutdown), there will be occasional appearances of new
+  bits (frame positions in which some radio noise was decoded as a frame),
+  followed by exact repetitions of previous "garbage" bit content.
+
+The behavior just described is indeed what we see in the TRAU-UL captures under
+examination, with one exception.  The exception is that in the case of plain FR
+(not in EFR), whenever this BTS emitted a repetition of previous buffer content
+(no new decoded bits, good or bad), 4 out of 260 bits got corrupted, or rather
+overwritten with a fixed pattern.  These 4 bits are those that appear at the
+very end of class 2 portion in GSM 05.03 bit order; in the state of "old buffer
+output" they get replaced with 4'b1001.  When reordered from GSM 05.03 into TRAU
+frame bit order (the natural bit order of codec parameters), these 4 bits end up
+in disparate places, hence when we look at the output of trau-parse, we see what
+looks like corruption in the lsb of 4 different speech parameters:
+
+* The lsb of 2nd LARc becomes 0;
+* The lsb of 6th LARc becomes 1;
+* The lsb of second-to-last Xmc becomes 1;
+* The lsb of the very last Xmc becomes 0.
+
+The presence of this corruption (can't really call it a bug, as the output from
+the BTS under these conditions is officially undefined garbage) does not
+invalidate the buffer hypothesis, but on the contrary, further confirms that
+this hypothesis is most likely correct.
+
+EFR CRC observations
+====================
+
+Taking advantage of EFR being more compact than the original FR (244 bits
+instead of 260), the TRAU frame format for EFR adds 5 CRC-3 fields, covering
+some of the "most important" bits of the payload.  I was wondering if perhaps
+some T1/E1 BTS may be transmitting intentionally bad CRC to indicate that they
+got no payload bits at all, i.e., BFI with no data - but at least the (unknown)
+BTS model from which these Osmocom-published captured were taken always emitted
+good CRC in every TRAU-UL frame, including buffer-output frame positions which
+were clearly "BFI with no new bits".
+
+Ideas for software implementation of TFO
+========================================
+
+The main motivator for seeking answers to the question of BFI payload fill is
+deciding what to put in those payload bits in the case of TFO frame output when
+the RTP-based source is BFI with no data.  From the standpoint of implementation
+costs (both the effort of implementation and CPU load in operation), the buffer
+method looks very attractive.  One implementation strategy would be to have an
+array of 320 bits (with each bit expanded to a byte at this stage) in the
+per-call state structure, holding the TFO frame to be transmitted.  This way
+bits that never change (sync pattern, frame type bits etc) would only be
+initialized once and then reused for the duration of the call; the frame output
+function executing every 20 ms would fill in the new Dn bits most of the time,
+set new C12-15, and do the final step of packing the content of this buffer
+into the two lsbs of outgoing PCM samples.  With this approach, the processing
+of BFI-no-data frames would entail simply skipping the step that writes the new
+Dn bits (and associated C13-14), leaving the old bits in place - apparently the
+exact same approach used by T1/E1 BTS and MS DSP implementors.