# HG changeset patch # User Mychaela Falconia # Date 1642061883 0 # Node ID a87d9ee278fbd3b8c0c5c9e4177a391750ad9347 # Parent e279a924f7a3e5ad7afda51571698ae8fb68c5b8 CMU200-maintenance-notes: new article diff -r e279a924f7a3 -r a87d9ee278fb CMU200-maintenance-notes --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/CMU200-maintenance-notes Thu Jan 13 08:18:03 2022 +0000 @@ -0,0 +1,341 @@ +Rohde & Schwarz CMU200 instrument is an absolutely essential piece of test +equipment for anyone in the business (or hobby) of designing and building his +or her own personal cellphones of 2G and/or 3G variety. I (Mother Mychaela) +currently only work with GSM, but depending on installed hw and sw options, +CMU200 instruments also support AMPS, IS-136, IS-95 (CDMA 2G) and both WCDMA +and CDMA2000 varieties of 3G. + +Over the course of owning and maintaining a CMU200 instrument since 2017 and +having had to repair it twice now (as of 2022-01), and having conversed with +another CMU200 owner who had to repair his instrument in the same way, I +started observing a pattern in that many of these instruments are now failing +in the field in exactly the same ways. All of these failures happen in the +RXTX board, and the purpose of this article is to educate other CMU instrument +owners about these failures and most importantly, how to repair them. + +Credit attribution +================== + +I sincerely thank Michael Katzmann, NV3Z / VK2BEA / G4NYV, for his invaluable +help in reverse-engineering the insides of the culprit RXTX board, identifying +various critical components on that board, including the ones that habitally +fail, and identifying Eccosorb-caused galvanic corrosion as the root cause of +these failures. Without his help, I would not have made it this far! + +What is this RXTX board +======================= + +This board is common among CMU200, CMU300 and CRTU-RU instruments from R&S - or +at least these are the ones I know - maybe there are others I don't know about. +This board encapsulates the instrument's main RF Rx and Tx chains: on the Rx +side it takes RF input from the front end and performs triple IF downconversion +to 10.7 MHz IF3, and on the Tx side it takes 13.85 MHz IF3 input and upconverts +it to RF output, going through IF2 and IF1 in the process - triple IF in both +directions. + +Every CMU200 instrument always has one RXTX board - it is an absolutely required +component irrespective of option configurations. The hardware architecture of +this instrument also has a place for an optional second RXTX board, providing a +complete second Rx and Tx channel - however, as far as I can tell, CMU200 +software won't do anything with it, i.e., there are no test modes or +applications in CMU200 software repertoire that can make use of a second RXTX +board. Instead it seems that configurations with two RXTX boards are better +supported on the CRTU-RU platform - but I know next to nothing about that one. + +Also note: if your CMU200 is equipped with Aux Tx model B96 (as opposed to B95), +there is an output from that B96 add-on that goes to the front end input that +was originally meant for second RXTX. + +RXTX board failures +=================== + +In terms of externally visible symptoms, almost all CMU200 units are now failing +in the same ways: + +1) If Tx side fails, the visible symptom is completely absent or extremely weak +output, and the internal loopback test fails with no signal detected at any of +the frequencies in the test sequence. A key point is that this failure mode is +independent of the selected output frequency. + +2) If Rx side fails, different frequency ranges are affected differently. As I +shall explain momentarily, there are two different IF1 Rx paths inside the RXTX +board: one handles the frequency range from > 1200 to <= 2200 MHz, and the other +handles lower (<= 1200 MHz) and higher (> 2200 MHz) input frequencies. When a +given RXTX board develops Rx path failure, this failure happens separately in +each of these two IF1 Rx paths. The resulting symptoms vary: if only one of +the two IF1 Rx paths fails, then only that frequency range will be affected, +or if both fail, the observed loss will typically be different between the two +frequency ranges. The failure symptom is unexpected large attenuation: +sometimes around 5 to 6 dB of loss, othertimes as much as 25 dB of loss. + +RXTX board architecture explained +================================= + +Unfortunately R&S' official service manual for CMU200 instruments is only a part +swapper guide: it tells you which boards do what in general terms and tells you +how to remove and replace each part, but no schematics, and no detailed +explanation of what happens inside each board. However, I draw the reader to +the block diagram on page 3.2 of this manual - this block diagram does provide +an important starting point for understanding what happens inside the RXTX +board. + +In the Tx direction, 13.85 MHz IF3 comes in from the digital board - or from +B68 board in WCDMA test modes. This Tx IF3 is mixed with Tx LO3 to produce +Tx IF2. This Tx IF2 is fixed at 487.52 MHz, thus one would think that Tx LO3 +frequency ought to be fixed as well - but it seems to be a synthesized variable +frequency. (Remember, all of this understanding is from reverse engineering, +hence we can only figure out so much.) + +Tx IF2 of 487.52 MHz is then passed through a pair of identical SAW filters, +Sawtek 855272 - two cascaded identical filters, with an amplifier in between. +This SAW filter has a center frequency of 479.75 MHz with 20 MHz bandwidth, +thus the passband spans from 469.75 to 489.75 MHz. Notice how Tx IF2 of +487.52 MHz stands just 2.23 MHz away from the edge of the passband - is it +intentional? What are they filtering? Without original design notes, we can +only guess. As I shall explain later in this article, one of these two Tx IF2 +SAW filters is a component prone to failure. + +After these cascaded SAW filters, Tx IF2 is mixed with LO2. Unlike LO1 and LO3, +there is only one LO2 for both Rx and Tx, and it is fixed at 1329.6 MHz. When +Tx IF2 at fixed 487.52 MHz is mixed with LO2 at fixed 1329.6 MHz, the output of +this mixer will always contain two frequencies: 842.08 MHz and 1817.12 MHz. +These are the two possible Tx IF1 frequencies, and there is a frequency- +selective filter for each of these two Tx IF1 modes. Based on the final output +frequency to be generated, instrument control software selects either low or +high Tx IF1, controlling switches before and/or after the filters. I have not +investigated to see if the frequency ranges for high vs. low Tx IF1 are the same +as on the Rx side or not - maybe they are the same, maybe they are different. + +After Tx IF1 output is combined or switched from the two filters, it is mixed +with Tx LO1 to produce the final RF output. The mixer that does this job is +MACOM SM4T, which is one of the larger, prominently visible components on the +board. There also seems to be a fourth mixer and LO stage that kicks in only +for frequencies above 2200 MHz, but I haven't really studied that one as my main +interest is in the classic cellular frequency bands, 1900 MHz and below. + +On the Rx side the same process happens in reverse, but the specific frequencies +used for IF1, IF2 and IF3 are slightly different. At first there is a stage +that only kicks in for frequencies above 2200 MHz (bypassed otherwise), and +then there is an SM4T mixer (identical to the one on Tx side) that takes in RF +and Rx LO1 to produce Rx IF1. High-side injection is used, i.e., Rx LO1 is +programmed to generate frequency equal to the external RF of interest PLUS the +desired Rx IF1 output. + +Rx LO1 is programmed as follows by the instrument control software: + +* Rx IF1 will be at 1816.115 MHz (call it high) if the listening frequency is + <= 1200 MHz or > 2200 MHz; + +* Rx IF1 will be at 843.085 MHz (call it low) if the listening frequency is in + the intermediate range, i.e., 1200 MHz < RF <= 2200 MHz. + +In addition to programming Rx LO1 to produce the desired IF1 per the logic +above, the software also controls switches that select one or the other IF1 +filter: either the filter that passes low IF1 or the one that passes high IF1. + +The filters used for low and high IF1 modes are the same on both Rx and Tx +sides. (The actual frequencies are slightly different, but in each case they +fit within the passband of the common filter parts.) The filter for low IF1 is +Murata DFC3R836P025HHD, package marking 836 CD, and the one for high IF1 is +DFC31R84P075HHA, package marking CR. The two filter packages are NOT the same +mechanically: the low IF1 filter is physically larger. Both parts are ceramic +monoblock filters from the same family, and it seems that these filter parts +were originally made for mobile phones, not for RF metrology instruments: the +"836 CD" filter is for AMPS uplink band, and the "CR" filter is for DCS downlink +band. + +On the Tx side of the board there are only two IF1 filters: one for low Tx IF1 +and one for high Tx IF1. However, on the Rx side there are 3 of these ceramic +filters in total: two for high IF1 (two cascaded identical filters with an +amplifier in between) and just one for low IF1. Why am I covering these filters +in so much detail? You probably guessed it: they are components that fail, as +will be covered shortly. + +After the selection of either low or high IF1 filter, Rx IF1 coming out of the +selected filter (either 843.085 MHz or 1816.115 MHz) is mixed with LO2, which is +shared between Rx and Tx sides and fixed at 1329.6 MHz. The output of this +mixer is Rx IF2 at 486.515 MHz. This Rx IF2 then passes through a pair of +cascaded Sawtek 855272 filters, two identical filters with an amplifier in +between, exactly the same as on the Tx side. Then there is Rx LO3 and the final +mixer, producing Rx IF3 at 10.7 MHz that goes to the digital board, to the rear +panel BNC output and to the WCDMA board (B68) if the latter is present. + +How these RXTX boards fail +========================== + +There are 3 specific components on this RXTX board that have been seen to fail +over and over in the field: + +* The second of the two cascaded IF2 SAW filters (Sawtek 855272) on the Tx side + often fails, breaking the Tx chain (output totally gone or extremely weak) + for all frequencies. Note that there are a total of 4 identical Sawtek 855272 + filters on this board (2 on Rx side, 2 on Tx side), and only one of the four + fails: Tx side, second filter in the cascade. + +* The "836 CD" filter on the Rx side is prone to failure. When it fails, the + visible symptom is severe attenuation in measured Rx signal levels for input + frequencies in the 1200 MHz < RF <= 2200 MHz range. Only the Rx side filter + fails, not the identical one on the Tx side! + +* One of the two cascaded "CR" filters on the Rx side likewise fails - this time + it is the first one in the cascade. The other two identical "CR" filters on + the same board (the second in cascade for Rx and the one for Tx) are likewise + NOT seen to fail. + +The root cause of all 3 component failures has been traced to galvanic corrosion +caused by direct contact between these components and Eccosorb RF absorber foam. +The complete RXTX board assembly consists of the traditional PCBA plus heavy +metal shields on both sides; the front and back metal shield pieces are custom- +made for this board, with individually shielded cavities matching different +sections of the board. Some (not all) of these cavities are filled with a +special black foam called Eccosorb - it is an RF absorber, presumably added to +prevent these cavities from acting as parasitic oscillators. Trouble occurs +when this Eccosorb foam comes into direct contact with metal surfaces of +components on the board: the result is galvanic corrosion, a process that takes +many years before it results in component failure. The reason why only 3 +particular filter components fail is because they got the bad luck of residing +in cavities with Eccosorb - the other identical components that don't fail +reside in cavities without Eccosorb. + +We don't know how R&S allowed this design flaw to escape and remain in their +sold and field-deployed products: there is the "innocent" explanation that they +simply didn't notice, and there is the conspiratorial view that this slow +failure mechanism is intentional as in planned obsolescense - pick your choice +of hypothesis. + +How to repair failed boards +=========================== + +All 3 of the failing filter components (one SAW filter part and two ceramic +monoblock filter parts) are now unobtainium. However, because so many of these +RXTX boards fail in exactly the same ways, our community at large is now +accumulating a very substantial "graveyard" of failed boards, and here is the +good news: we can make one good board out of every two failed ones. Suppose +that every RXTX board in our community's collective inventory has fully failed, +leaving no failure-free boards - what now? Here is the recipe for making one +good RXTX board out of two fully failed ones: + +1) Out of the two failed boards, choose one to be the part donor and the other + to be the part recipient. + +2) Take the part donor board and harvest 3 parts from it: one of the 3 Sawtek + 855272 filters that aren't subject to corrosion, and the two IF1 filters + (one 836 CD and one CR) from the Tx side. Tx side IF1 filters aren't in + contact with Eccosorb and thus don't corrode, and 3 out of the 4 SAW filters + are likewise safe - hence we expect that every "dead" RXTX board can still + serve as a donor of good parts in this manner. + +3) Take the part recipient board and transplant the donor parts onto it, + replacing all 3 corroded filters. + +4) Before putting the repaired board back into its metal casing, cover all + corrosion-prone components with Kapton tape, preventing direct galvanic + contact with Eccosorb - this way the newly transplanted uncorroded components + won't suffer the same fate. + +RXTX disassembly instructions +============================= + +Before you can start working on an individual RXTX board, you first need to pull +it out of your CMU. Disassembly instructions are provided in the official part +swapper guide from R&S (which they call "service manual"), but here is the gist: + +* Using a Torx T20 screwdriver, remove the 4 rear feet and lift the sleeve part + of the instrument case. + +* Remove two small Phillips screws that secure the cover over the main board + cage, and lift that cover off. + +* Unhook all MMCX little coax connections from the RXTX board: 3 on the top side + (IF3 interface) and one on the bottom (netclock input). + +* Loosen and remove the two semi-rigid coax pieces that connect RF between the + RXTX board and the front end. In this Mother's opinion, this step is the + least pleasant of all, but it is unavoidable. + +* After ensuring that nothing remains connected to the RXTX board on the bottom + side, pull the board out from the top. + +Once you got the complete RXTX board assembly out, how do you extract the actual +board out of the metal casing? The not-immediately-obvious answer is that you +don't need to remove all of the screws, instead there are shortcuts that will +save you a lot of pain: + +* There are two smooth thin metal plates, one on the front side of the board + (facing toward the front of the CMU when installed) and one on the back side. + Each is secured with a small Phillips screw. You only need to remove the one + on the front side. You don't need to remove the thin metal plate from the + back side of RXTX assembly - doing so will only add more clutter and loose + parts to your lab bench while the board is disassembled. + +* Once you remove the thin metal plate from the *front* side of your RXTX + assembly, you will see all of the many screws that hold together the sandwich + of two heavy metal pieces with the board in the middle. These screws are + Torx T8. + +* Put the board down on your bench so that the side that faces the front of the + CMU when installed (the side with the T8 screw heads) will become the top, + with the rear side becoming bottom. + +* Each of the T8 screws passes through thread in the top metal piece, a hole in + the PCB, and then thread in the bottom metal piece. As you loosen these + screws, you don't need to remove them all the way - instead loosen each screw + so that its far end comes out of the thread in the bottom metal piece, but + let it remain captive in the top metal piece. Letting the screws remain + captive in the top metal piece will reduce bench clutter while the board is + disassembled, and there is a lot less screwing and unscrewing work to be done, + as there is no need to work through the thread in the top metal piece. + +Once you loosen all of the T8 screws, the top metal piece should lift off, +leaving just the bottom metal piece and the PCBA. The bottom metal piece has +two thin metal pins sticking out of it; both the PCBA and the top metal piece +align on these two pins. + +When you lift the top metal piece (the one with the screws), the side of the +board that will be immediately exposed to you is the side that faces the front +of the CMU when the board is installed. It is the Rx side, and you can confirm +that you are looking at the Rx side by noting that there are two "CR" filters +for high IF1, as opposed to just one. And chances are, right here at this step +in the disassembly process you will see the galvanic corrosion or the lead-up +to it. + +As you lift the top metal piece from the board, look at its inside and note the +many individual cavities. Also note how some of these cavities are filled with +some black foam - that's the Eccosorb. And note how only some of the cavities +have Eccosorb in them, not all. + +Now look at the ceramic IF1 filters on the Rx side of the board. The one "CR" +filter that is NOT in contact with Eccosorb will be bright copper-colored (it +actually is copper), whereas the two filters that are in contact with Eccosorb +(one 836 CD, one CR) will often be green instead of copper-colored on their top +surface - that's patinated copper! Furthermore, there will typically be some +black Eccosorb material directly adhered to the corroding top surfaces of those +two unlucky filters. + +Now lift the PCBA off the two metal pins, separating it from the bottom metal +piece. Like you did with the top metal piece, observe the inside of the bottom +metal piece: note which cavities have Eccosorb in them and which don't. Then +flip the board over and look at its Tx side. You will see that there are only +two ceramic IF1 filters on this side (one 836 CD and one CR), and both should +be in pristine shape, bright copper-colored, no corrosion - these two are not +in contact with Eccosorb! + +Now look at the two Sawtek 855272 filters on the Tx side. The one closer to +the middle of the board will often appear in worse physical condition that the +other 3 - and the culprit is once again in contact with Eccosorb. + +MACOM SM4T mixer corrosion +========================== + +Neither I nor my collaborator on this project have seen an RXTX board on which +either the Rx SM4T mixer or the Tx one went bad - i.e., we haven't seen a +failure in this part *yet*. However, this mixer *is* in contact with Eccosorb, +and looking visually at the collection of RXTX boards in my possession, I +(Mother Mychaela) see definite signs of corrosion - the metal surface of this +SM4T mixer component is beginning to corrode. Therefore, as a preventative +measure, I recommend cleaning off any Eccosorb that is adhered to this component +and then covering the component with Kapton tape before putting the board back +into its metal casing. + +Unlike the failing filters, this MACOM SM4T mixer is still available new - but +it's an expensive component, so let's protect these mixers from corrosion.