FreeCalypso > hg > freecalypso-tools
view doc/Flash-programming @ 976:ff4ce8d5ece4
fc-loadtool flash: definitions for AMD sector lock architecture
author | Mychaela Falconia <falcon@freecalypso.org> |
---|---|
date | Fri, 01 Dec 2023 06:43:21 +0000 |
parents | a7496a1e0df7 |
children |
line wrap: on
line source
Our Calypso device flash programming tool fc-loadtool can be used in several different paradigms; this article is an attempt to clarify the many available modes of usage. You should also read the companion article Binary-file-formats for further background. Flashing firmware release images ================================ In conventional forward engineering environments where you develop or maintain firmware for hardware made by your own company (meaning no reverse eng, no "illicit" aftermarket tinkering on hw made by some other company aeons ago), you have a firmware build system that produces fw build images (some of which may subsequently be blessed as releases), and you have a tool that flashes these fw build images into your hardware, operating as efficiently as possible, automated as much as possible, requiring minimal user action for the boring repetitive task of flashing a new fw image every time you build one. And if you become lucky enough to produce your hardware in volume, the same objectives of maximal efficiency and automation carry over into the production line environment as well. In TI's environment the standardized format for firmware build images which are then flashable into hardware targets was a variant of Motorola SREC written in *.m0 files, a variant which we have named moko-style m0 after its most famous user. The special quirk of this particular SREC variant is its peculiar byte order. TI's firmware build system produces a *.m0 S-record image as its final build product, and TI's official Calypso flash programming tool (FLUID) takes these *.m0 files as its input. Since the beginning of FreeCalypso we have had two ways of flashing TI-built firmware images into suitable targets (initially OM GTA02 modem, then many others including our own FCDEV3B): 1) Our fc-loadtool has had a flash program-m0 command from the beginning, programming device flash with bits from an m0 file directly and natively. However, prior to fc-host-tools-r12 this command was poorly supported: it ran significantly slower than flash program-bin, had poorer progress indication and did not perform CRC-32 verification at the end, which is an important integrity check. Also this original flash program-m0 command (as opposed to flash e-program-m0 added in fc-host-tools-r13) does not include a built-in erase step, thus prior to fc-host-tools-r13 the user had to have outside knowledge of how many sectors to erase first with a separate flash erase command. The new flash e-program-m0 command added in fc-host-tools-r13 is m0 image flashing finally done right. It reads in the specified S-record image in moko-style m0 format, builds a map of potentially discontiguous flash regions into which the image deposits bits, erases the set of flash sectors which need to be erased before programming these regions, then programs the new image bits into flash, exactly like TI's own FLUID. 2) The alternative way is to first convert the *.m0 S-record image produced by TI's hex470 post-linker tool into straight binary (*.bin) with a FreeCalypso tool called mokosrec2bin, then program the binary fw image into flash with fc-loadtool command flash program-bin. This method is the one we've been using since 2017, and our FC Magnetite firmware build system is now set up to produce not only fwimage.m0, but also fwimage.bin (it runs mokosrec2bin), and it also generates an fc-loadtool command script (a text file named flash-script) with two commands in it: a flash erase command with a calculated sector address range and a flash program-bin command to program the accompanying fwimage.bin image. As of fc-host-tools-r13 both methods work equally well: if you have an official FreeCalypso firmware release (containing fwimage.m0, fwimage.bin and flash-script files) which you need to flash into a device such as our own FCDEV3B or OM GTA02 (but *not* Mot C1xx!), you can execute either 'exec flash-script' or 'flash e-program-m0 fwimage.m0' at the loadtool> prompt, and both ways will produce exactly the same result with equal performance and reliability. And if you need a more special operation such as erasing the entire flash (factory production lines) or erasing and reprogramming only a certain part of the normally affected sector range, that's what custom command scripting ability is for. For the sake of symmetry, we also have a flash e-program-bin command that is a binary image format counterpart to flash e-program-m0: it first erases the sectors into which new bits will be programmed, then programs the new bits. Thus a third equally good way to flash a new FreeCalypso fw release into a target such as FCDEV3B or GTA02 is to execute 'flash e-program-bin 0 fwimage.bin' - but don't *ever* do it on a Mot C1xx phone! Discontiguous m0 fw images with large gaps ========================================== One sometimes-advantage of S-record-based formats over straight binary is that they can have discontiguous filled regions with gaps in between them; all m0 fw images produced by TI's TCS211 build system have some gaps in them. In our current FC Magnetite firmwares these gaps have been reduced to less than 16384 bytes in total, reducing the wasted disk space and image transfer time to sufficiently insignificant if these m0 images are converted to straight binary with mokosrec2bin and then flashed as binary as we've been doing. For comparison, in Openmoko's classic mokoN firmwares (almost TI's original version of the memory map) the first flash sector of 65536 bytes had only 5072 bytes of image payload programmed into it, thus indirect programming via mokosrec2bin followed by flash program-bin meant sending just over 60k filler 0xFF bytes to be flashed - becomes noticeable at 115200 baud. But these gaps were even bigger in TI's classic firmwares (and many vendor firmwares) built for 8 MiB or 16 MiB flash chips with TI's classic memory map. They have two code regions: one region is exactly 4 MiB in size but is placed not quite at the beginning, thus spilling past the chip midpoint, and the other region starts where the first 4 MiB region ends, going to the end of the flash chip or to the beginning of the area reserved for FFS. The way TI's linker works results in the first of these two program regions being not quite full before bits start getting thrown into the second one, thus the resulting m0 image can easily have a gap of a megabyte or more between the two regions. TI's own FLUID handles such discontiguous images with no sweat, but our fc-loadtool has gained matching capability only as of fc-host-tools-r13 with our new e-program-m0 command. And yes, it is smart enough to avoid needlessly erasing sectors in the big gap, just like FLUID. Flash backup and restore ======================== A completely different paradigm takes place on alien targets such as Motorola C1xx and Pirelli DP-L10, made by alien manufacturers, meaning not FreeCalypso, not Openmoko and not TI. The most important flash operation on these alien targets is making a flash dump; these dumps can then be used for forensics, reverse engineering, or simply as a backup. When we subsequently write to flash on these alien targets (after having saved a backup first), we are not flashing an m0 fw image or a binary image made from one with mokosrec2bin, instead the most common operations are: * Flashing a backup image back into the same device it was originally made from (flash restore); * Changing a device from one firmware version to a different one by programming its flash with firmware bits that were originally read out from some fw-donor unit; * Surgical manipulations such as erasing FFS sectors or rewriting one specific part of the flash based on reverse-engineered understanding of its structure. This different paradigm leads to a different mode of usage for fc-loadtool: instead of needing a maximally-automated operation that flashes a firmware release image with as little user thought involvement as possible, our flash manipulations need to be of a more manual peek-n-poke manner. We provide a flash dump2bin command for making and saving flash dumps first and foremost, allowing any part of the flash to be dumped and saved selectively if desired, including the second flash bank on the Pirelli DP-L10 and likewise on our own FCDEV3B. When it comes to flash write operations, we provide a manual flash erase command that allows (and requires) the operator to specify exactly which sector range should be erased and a manual flash program-bin command that allows any range of 16-bit words to be programmed at any flash address, with the bits to be programmed coming from a binary file, either the whole file or any specified subrange. These manual flash erase and flash program-bin commands give full control to the operator, allowing every possible flash manipulation which the hardware itself allows, at the expense of requiring the operator to think about which flash addresses, offsets and lengths need to be operated on, and either enter long commands manually or write a command script. Given our historical origins (long before we got to the point of producing our own hardware, we started out by exploring the forbidden GSM realm of devices made by alien manufacturers who were hostile to our cause), our original flash manipulation support in fc-loadtool had been centered around the manual peek-n-poke paradigm, with elementary flash erase and flash program-bin commands as our main staple, and no thought had been given originally to producing functionality that would work like FLUID or like our current flash e-program-m0 and e-program-bin commands. But all actively maintained software evolves, and as our FreeCalypso family of projects has matured over the years, we now offer richer functionality covering a wider range of use cases. Binary vs. S-records ==================== (Please read the companion article Binary-file-formats for background, then come back here.) If you are exploring and manipulating the flash content of a GSM device in an aftermarket fashion, as opposed to flashing your own fw builds into your own hw design produced by your own company like Openmoko did in the late 2000s and like we do currently at FreeCalypso HQ, then binary is the generally preferred format: you make dumps with flash dump2bin, and when you selectively program these images back into devices, you use flash program-bin with the right offsets and length, along with appropriate flash erase commands. We also have flash dump2srec and flash program-srec commands in fc-loadtool, they were implemented back in the founding stage of FreeCalypso in 2013 for the sake of completeness and symmetry (it seemed right to support both binary and S-record formats), but they never got any practical use: if you are making a flash dump, you would normally want to examine it afterward, and any such examination almost always needs a straight binary image, not S-records. Furthermore, our flash program-bin command allows you to selectively program just a particular portion of a binary image file into flash, at any arbitrary flash address, but we don't have the same flexibility with flash program-srec - the latter command is really just a sibling of program-m0 with the opposite byte order. Thus the short summary is as follows: * If you are flashing an official firmware release image into your device, you need to use flash e-program-bin or flash e-program-m0 depending on whether the image is provided in *.bin or *.m0 format, or alternatively our older flash program-bin or flash program-m0 commands preceded by a separate flash erase command with the right sector range, possibly packaged in a supplied fc-loadtool command script. * If you are restoring a flash dump made with flash dump2bin or performing aftermarket flash manipulations on Mot C1xx or Pirelli DP-L10 phones or other such alien devices, you need to use binary-format-based flash manipulation commands; the specific commands will depend on exactly what you are seeking to do. * flash program-srec and e-program-srec commands do not currently have a valid use case. Special considerations for Compal phones ======================================== Motorola C1xx and Sony Ericsson J100 phones made by Compal have brickable flash: the right kind of flash-resident bootloader must always be present at the beginning of the flash, or else the phone is unrecoverably bricked. We have special support in fc-loadtool for minimizing the bricking vulnerability window when operating on these phones, but this special support requires user cooperation, meaning that you must limit your flash manipulations on these phones to a narrower subset: * flash program-m0, program-srec, e-program-m0 and e-program-srec commands are not appropriate for these brickable phones - do not use any of these commands on these targets. * Flash sector 0 must be manipulated only with the special flash erase-program-boot command, not any of the regular erase or program commands. * Regular flash erase, flash program-bin and flash e-program-bin commands can and should be used for the rest of the flash starting at offset 0x10000 - but you still need to understand what you are doing.