`intel_fw`

This is both a command-line interface (CLI) and a library for analyzing and editing firmware images for Intel platforms.

The architecture is based on knowledge from prior research.

Architecture and Design

The following considerations around specifics in the domain of Intel platforms and firmware are meant to offer a deeper understanding of design choices.

Goals

Separate detection from action

me_cleaner mixes both, and assessing the exact actions to happen is hard. Via its CLI, everything a one-shot operation, and the user needs to know the options beforehand. Note that it is very Unixesque to wrap CLIs in shell scripts as if they were intended as APIs. That is not meant here.

For programmatic and interactive use, provide an analysis result and let the programmer/user then choose what they want to do. The programmer has to deal with all possibilities and may choose strategies around them, whereas the user can benefit from seeing the options fitting their actual case.

E.g., in a GUI app such as Fiedka or a TUI that could be created with ratatui, setting a HAP bit vs legacy (Alt)MeDisable bit(s) can be elaborated on in a help message or tooltip. Removable partitions and files can be listed rather than expected to be provided by the user who, before the analysis, doesn’t know what their firmware contains.

Comprehensible and useful API

Provide an API that requires little knowledge of internal details. The job of an API is to simplify, after all. Otherwise, both caller and callee would duplicate efforts.

Encapsulate rather than exposing many raw structs directly. Make relationships and containment clear.

E.g., wrap partition entry structs in another struct that carries supporting metadata with it, such as offset and size, parsing result of what the partition entry points to, etc..

Follow the Rust API guidelines.

Approaches

Replies to a StackOverflow question on separating preparation from execution reference the Composite pattern and the Command pattern.

The Composite pattern helps in that there is a hierarchical structure, i.e., IFD, FIT and ME firmware on a high level, and then IFD has its underlying data, the ME firmware has its FPT and partitions therein, and the FIT points to many things. Underlying details are very different though, so there are no meaningful abstractions for common interfaces.

The Command pattern may make sense on a higher level, for example, to collect the actions resulting from command line switches or API options. At the same time, it creates more boilerplate code, which may be hard to follow.

Note that we need to check the applicability of operations on two levels: Some operations exclude others, and some only apply if their assumptions fit with findings from analyzing the given firmware image.

Fiano/utk implements the visitor pattern, which is meant for objects that are quite similar in some regard.

One issue with firmware images is that while many things are related to each other, they also have their very specific purposes and semantics, and there is a rather fixed structure. It would thus be simpler to deal with specific operations on a higher level that understands the domain and carries clear semantics with it rather than having a visitor/executor encoding each possible operation for each bit and requiring more understanding from the user and programmer. I.e., just have each entity offer its own unique set of clearly understandable helpers and assisting parameters, possibly following a loose pattern when multiple entities are similar.

For example, some partitions contain directories in the case of generation 2 and generation 3 ME: For both generations, certain directories can be partially cleared (whole ranges of data set to 0xff), and both can take a list of things to retain. That requires helpers which may implement a common trait (interface). Similarly, other partitions can be cleared, but entirely, so they do not need extra functionality themselves. Trying to force the same interface on them only results in unnecessary extra code.

On the other hand, the FPT itself is a small memory slice (found to be <2K), so it can take care of itself. It has a checksum in its header. Offer helpers for editing and returning a new slice to replace the original one. The IFD is similar in that it is also small, but has its own unique features. And the FIT is yet again very different.

Intel platforms

Over the years and decades, Intel has developed many hardware platforms¹.

Since the 4-bit 4004 in 1971, they have progressed over 8-bit up to 64-bit systems, retaining a lot of backwards compatibility. Starting with the ICH7 based platforms, Intel introduced their AMT (Active Management Technology)², an out-of-band management solution³ for remote provisioning and support.

AMT evolved with more features over time, carrying the vPro label for machines targeting the business market⁴ and finally converging with more security features such as Boot Guard⁵, Intel’s secure boot implementation, digital content protection (DRM), and more⁶.

Now running on a coprocessor called the (Converged Security and) Manageability Engine⁷, or (CS)ME for short, henceforth abbreviated as ME, a full second operating system of its own is backing the platform.

Boot flow

The ME has its own firmware and bootstraps an Intel platform⁸. The main x86 cores are held in reset until the ME releases them to boot with their own firmware.

Both the ME firmware and the main x86 firmware are stored in the same flash part on a mainboard, partitioned via the Intel Flash Descriptior (IFD).

The following diagram is based on knowledge from various sources, including the coreboot documentation on Intel.

The boot flow for trusted boot is documented publicly⁹ by Intel.

ME classification and security

Security researchers have analyzed the ME¹⁰ and divided hardware variants into 3 generations¹¹ thus far, each with their own multiple firmware versions, including security patch releases¹² ¹³. One core aspect in security research has been Boot Guard¹⁴ ¹⁵, which had been introduced with Haswell, Intel’s 4th generation Core series platforms¹⁶, and discussed in the coreboot community¹⁷.

Note that the ME generations roughly correspond with the overall platform, in that ranges of Intel platforms are expected to carry a certain ME hardware generation and specific platforms a certain firmware version range. For example, Lenovo ThinkPad X270 laptops came with 6th/7th gen Intel Core processors, which means 3rd generation ME hardware and version 11.x.x.x ME firmware.

Processor names

Intel publicly documents how to interpret procesor names¹⁸ and what their suffixes mean¹⁹.

Abbreviations

abbr.	expansion
ACM	Authenticated Code Module
AMT	Active Management Technology
CSME	Converged Security and Manageability Engine
DAL	Dynamic Application Loader
FIT	Firmware Interface Table
FPT	Firmware Partition Table
HAP	High-Assurance Platform
{I,M,P}CH	{I/O,Memory,Platform} Controller Hub²⁰
IFD	Intel Flash Descriptor
PTT	Platform Trust Technology
RBE	ROM Boot Extensions (part of ME firmware)
SPS	Server Platform Services
TXE	Trusted Execution Engine
TXT	Trusted Execution Technology

Ambiguities

There are colliding acronyms, even within this domain. The following abbreviations have a second meaning:

FIT: Flash Image Tool (sometimes also called FITC)
FPT: Flash Programming Tool

Knowledge on Firmware Images for Intel Platforms

The knowledge here focuses on firmware images, their partitioning and data structures.

Official resources

Intel publishes general information on firmware in their developer portal: https://www.intel.com/content/www/us/en/developer/topic-technology/firmware/overview.html Note that the public information mainly focuses on host processor firmware.

Other projects

Research

Many researchers have looked into the Intel Management Engine over the years and presented on their findings:

Forums

People keep asking in Intel’s community forum for tools and information around Intel platform firmware, resorting to third-party forums because Intel does not publish what they need or limits access to necessary resources to certain customers only, notably not end users. Here is a short list of places to find useful information and tools.

Extraction

For older FIT tools, use binwalk to extract resources and then grep for LayoutEntry to find descriptions of straps, which include the HAP bit, e.g.:

<LayoutEntry name="PCH_Strap_CSME_CSE_HAP_Mode" type="bitfield32"
    value="0x0" offset="0x68" bitfield_high="16" bitfield_low="16" />

Note that later generation XML files just call the HAP bit “reserved”. Educated guessing by looking at neighboring bits will help you to locate it.

Newer FIT tools (e.g., v18) contain Python code that can be extracted with pyinstxtractor. Relevant code is in the plugins/ directory:

pyinstxtractor.py ../mfit_18.exe
ls -l tools.exe_extracted/plugins/

Firmware images

There are multiple ways of obtaining firmware images:

read out from one’s own mainboard, e.g. using flashprog
download from a vendor; however, those are mostly upgrade images, not the same as what would be found on real hardware, usually in a proprietary format
download from an archive, obtain a full backup image, or similar

Full images

ChromeOS

The coreboot project offers a utility to download full recovery images for ChromeOS (Chromebooks) including a base system and firmware, and extract the firmware image: util/chromeos/crosfirmware.sh

To download all the images and extract the firmware, run: ./crosfirmware.sh all

As of the time of writing this, that will download: https://dl.google.com/dl/edgedl/chromeos/recovery/recovery.conf

Which looks somewhat like this:

recovery_tool_version=0.9.2
recovery_tool_linux_version=0.9.2
recovery_tool_update=


name=Dell Chromebook 13 (3380)
version=15393.58.0
desc=
channel=STABLE
hwidmatch=^ASUKA .*
hwid=
md5=9e3788b775f0c55f37682a6db6add00a
sha1=b54a3762e22d08dc3b67846e3ab16bd7333e519d
zipfilesize=1275561266
file=chromeos_15393.58.0_asuka_recovery_stable-channel_mp-v2.bin
filesize=2330960384
url=https://dl.google.com/dl/edgedl/chromeos/recovery/chromeos_15393.58.0_asuka_recovery_stable-channel_mp-v2.bin.zip

and so on. Each entry is about 12 lines. How much is in there?

wc recovery.conf
  9186  11369 297858 recovery.conf

More than 650 images?! … no, there are lots of duplicates. But not exactly. Some recovery images contain firmware for multiple devices!

grep url= recovery.conf | uniq | wc
    76     76   9310

A good bunch of downloads would fail, though. I reduced the script to fewer entries. Note that Chromebooks may be based on any platform, not only Intel.

How many images that you downloaded are for Intel?

for f in coreboot*.bin; intel_fw me scan $f; end 2>&1 | grep 'No ME' | wc
    19    133   1235

The remaining images can now be used to test intel_fw.

Win-Raid Forum

Many samples can be found via this forum post: https://winraid.level1techs.com/t/intel-cs-me-cs-txe-cs-sps-gsc-pmc-pchc-phy-orom-firmware-repositories/30869

Unpacking Firmware

Firmware images are typically packed and consist of various pieces henceforth called components. In different places, they may be called more specific terms, such as partitions, directories, modules, files, etc.

Many components are containers, which in turn are comprised of other things. As it happens over time, a firmware image for a platform of today may look very different from one meant for a platform from the past. It may be, however, that the target platform cannot be recognized right away, making analysis harder. Such is the case for Intel. We thus need an architecture that is able to distinguish at any given level and allows for extraction.

Partitions

The following diagram is a generic example of a partitioned firmware image with a container that may be of either one or another kind. And in turn, it would contain one or another kind of entries.

In Rust, we can use enum types to express this:

enum Container {
    AContainer(Vec<AEntry>),
    BContainer(Vec<BEntry>),
}

Intel ME Generation 3

With the third hardware generation of Intel ME based platforms, a new operating system was introduced, based on MINIX 3. It needs bootstrapping first.

There are multiple kinds of partitions, including Code Partition Directory (CPD) partitions. Those contain executables, their corresponding metadata files, and a manifest that holds a signature over the manifest.

The signed data in the manifest includes hashes of the metadata files and other things, so that the manifest suffices to verify the entire CPD’s integrity. Each metadata file contains the counterpart binary’s hash. The binaries themselves are mostly compressed, commonly using LZMA and a few via Huffman encoding.

Knowledge on CPDs, manifests, metadata and binaries can be found in PT Research utilities for unpacking:

Keyboard shortcuts

intel_fw