Progress Report Aug'21: pin referenced source (#16)

TophEvich · web-flow · commit 2167a13084aa · 2021-09-22T19:03:32.000+09:00
So that that referenced source code on github stays in sync
with the explanations of the 2021-08-14 progress report,
all links have been changed from "main" branch to
commit: c2c6da3df25c0605894244b4ea9387e882321efc which is
the current version of the day.

Signed-off-by: TophEvich &lt;84676511+TophEvich@users.noreply.github.com&gt;
diff --git a/content/blog/2021/08/14-progress-report.md b/content/blog/2021/08/14-progress-report.md
@@ -34,7 +34,7 @@ The hypervisor also includes standard debugging tools (like stopping execution,
 
 If you're intererested in the inner workings of the hypervisor, I did a [3-hour code recap stream](https://www.youtube.com/watch?v=igYgGH6PnOw) covering most of the implementation, as well as the general topics of ARMv8-A virtualization, M1-specific details and oddities, and more.
 
-On top of the hypervisor, we've built a flexible hardware I/O tracing framework that allows us to seamlessly load and upload tracers that understand how a particular piece of hardware works. For example, the [tracer](https://github.com/AsahiLinux/m1n1/blob/main/proxyclient/m1n1/trace/gpio.py) for the GPIO (General Purpose I/O) hardware can tell us when macOS toggles the state or configuration of each GPIO pin. This allows us to build up our understanding of the hardware, from raw register reads and writes to higher level functions. This was invaluable for the next bit of hardware we tackled: the DCP.
+On top of the hypervisor, we've built a flexible hardware I/O tracing framework that allows us to seamlessly load and upload tracers that understand how a particular piece of hardware works. For example, the [tracer](https://github.com/AsahiLinux/m1n1/blob/c2c6da3df25c0605894244b4ea9387e882321efc/proxyclient/m1n1/trace/gpio.py) for the GPIO (General Purpose I/O) hardware can tell us when macOS toggles the state or configuration of each GPIO pin. This allows us to build up our understanding of the hardware, from raw register reads and writes to higher level functions. This was invaluable for the next bit of hardware we tackled: the DCP.
 
 ## Reverse Engineering DCP
 
@@ -46,13 +46,13 @@ On most mobile SoCs, the display controller is just a piece of hardware with sim
 
 Reverse engineering this is a huge challenge, but thanks to the hypervisor, we can build up our understanding of how this all works layer by layer. At the lowest layer, DCP is an instance of what apple calls an "ASC", which is their term for these coprocessors (the M1 has about a dozen!). ASC processors run their own firmware and communicate with the main CPU through a mailbox interface, which is a simple message queue where each side can send 64-bit messages to the other, tagged with an "endpoint".
 
-Above this simple interface, Apple uses a shared set of endpoints for all ASC processors that run RTKit, Apple's bespoke RTOS. This interface provides features such as sending syslog messages and crash dumps from the ASC to the main CPU, and initializing endpoints (services). So we built a [tracer](https://github.com/AsahiLinux/m1n1/blob/main/proxyclient/m1n1/trace/asc.py) that can understand these messages, and do things like print the syslog messages directly to the hypervisor console.
+Above this simple interface, Apple uses a shared set of endpoints for all ASC processors that run RTKit, Apple's bespoke RTOS. This interface provides features such as sending syslog messages and crash dumps from the ASC to the main CPU, and initializing endpoints (services). So we built a [tracer](https://github.com/AsahiLinux/m1n1/blob/c2c6da3df25c0605894244b4ea9387e882321efc/proxyclient/m1n1/trace/asc.py) that can understand these messages, and do things like print the syslog messages directly to the hypervisor console.
 
 On top of this, the DCP implements multiple endpoints, one of which serves as the "main" interface. This interface itself supports making remote method calls in both directions. The DCP often issues synchronous callbacks to the main CPU after it receives a call, and the main CPU can in turn issue more synchronous DCP calls - effectively, the execution call stack extends across the CPU-to-DCP boundary! The interface even supports asynchronous reentrancy, having multiple "channels" so that, for example, the DCP can send asynchronous messages to the main CPU at any time, even during another operation.
 
-The method calls themselves send their arguments and return data via buffers in shared memory. These buffers encode simple types like integers; pointers that can pass data in the input, output, or both directions; more complex fixed structures; and even two *different* serialization formats for JSON-like higher-level data structures (blame IOKit)! Our [dcp tracer](https://github.com/AsahiLinux/m1n1/blob/main/proxyclient/hv/trace_dcp.py) takes these buffers and dumps them out to a trace file, so that they can be analyzed offline as we improve our understanding of the protocol.
+The method calls themselves send their arguments and return data via buffers in shared memory. These buffers encode simple types like integers; pointers that can pass data in the input, output, or both directions; more complex fixed structures; and even two *different* serialization formats for JSON-like higher-level data structures (blame IOKit)! Our [dcp tracer](https://github.com/AsahiLinux/m1n1/blob/c2c6da3df25c0605894244b4ea9387e882321efc/proxyclient/hv/trace_dcp.py) takes these buffers and dumps them out to a trace file, so that they can be analyzed offline as we improve our understanding of the protocol.
 
-We then started building a [Python implementation](https://github.com/AsahiLinux/m1n1/blob/main/proxyclient/m1n1/fw/dcp/ipc.py) of this RPC protocol and marshaling system. This implementation serves a triple purpose: it allows us to parse the DCP logs from the hypervisor to understand what macOS does, it allows us to build a [prototype DCP driver](https://github.com/AsahiLinux/m1n1/blob/main/proxyclient/m1n1/fw/dcp/manager.py) entirely in Python, and it will in the future be used to automatically generate marshaling code for the Linux kernel DCP driver.
+We then started building a [Python implementation](https://github.com/AsahiLinux/m1n1/blob/c2c6da3df25c0605894244b4ea9387e882321efc/proxyclient/m1n1/fw/dcp/ipc.py) of this RPC protocol and marshaling system. This implementation serves a triple purpose: it allows us to parse the DCP logs from the hypervisor to understand what macOS does, it allows us to build a [prototype DCP driver](https://github.com/AsahiLinux/m1n1/blob/c2c6da3df25c0605894244b4ea9387e882321efc/proxyclient/m1n1/fw/dcp/manager.py) entirely in Python, and it will in the future be used to automatically generate marshaling code for the Linux kernel DCP driver.
 
 Passing the hypervisor DCP traces through this decoder, we can get a trace of all the method calls that are exchanged between macOS and the DCP:
 
@@ -117,7 +117,7 @@ Passing the hypervisor DCP traces through this decoder, we can get a trace of al
 
 And armed with this knowledge of how everything fits together, we can implement enough of the interface to finally send display buffer swaps to the display. This allows us to implement tear-free graphics with double buffering, hardware mouse pointer acceleration, framebuffer scaling and compositing, and more!
         
-{{% figure src="/img/blog/2021/08/asahi_dcp_coming_soon.png" caption="The first frame presented using the <a href='https://github.com/AsahiLinux/m1n1/blob/main/proxyclient/experiments/dcp.py'>prototype DCP driver</a> (screenshot taken via HDMI capture)" %}}
+{{% figure src="/img/blog/2021/08/asahi_dcp_coming_soon.png" caption="The first frame presented using the <a href='https://github.com/AsahiLinux/m1n1/blob/c2c6da3df25c0605894244b4ea9387e882321efc/proxyclient/experiments/dcp.py'>prototype DCP driver</a> (screenshot taken via HDMI capture)" %}}
 
 As a further twist, the DCP interface is not stable and changes every macOS version! This finally answers a question for the Asahi Linux project: we will only support specific firmware versions. Unlike macOS, which can afford to support only its "paired" firmware and change with every release, Linux has to support all firmware versions going back to the initial supported one, in order to allow people to upgrade their kernel without upgrading their firmware in tandem. It would be too much of a maintenance nightmare to attempt to support every DCP firmware version that Apple releases, so instead we will pick certain "golden" firmware versions that are blessed to be supported by Linux. Don't fret: this doesn't mean you won't be able to upgrade macOS. This firmware is per-OS, not per-system, and thus Linux can use a different firmware bundle from any sibling macOS installations.
 
