• Introduction
  • Before we get started
  • Digression: Building a C64 disk image
• The game loop: Movement calculations
  • Digression the second: Representing negative numbers
  • After movement comes positioning
  • An element of chance
• The playing field
• Keeping score
  • Digression the third: Binary long division
  • Rendering the score
• Future considerations
• Footnotes and citations

One day early in 2024, I was idly flipping through Mastodon when I came across this toot by Koen van Gilst^[1]Pongwars recreated in JavaScript, @vnglst, Jan 2024:

This is a depiction of an eternal battle between the "day" ball and the "night" ball, to reverse the colour of the opposite ball's field. Many were transfixed, judging by the thousands of faves; I was one of them, and the thought arose unbidden:

If van Gilst saw this online and had to recreate it in JavaScript, and now I've seen it online, I have to recreate it on the Commodore 64, right?

Is such a thing even possible?

"Yes it is!" Well, it seems like it should be. This is a relatively simple animation, with a random element thrown in at the point the balls contact a surface that makes them bounce; defining "a surface that makes them bounce" seems like the tricky part.

So we need to do some planning. And as the first step, there's some defining:

• We have the ball (two of them) which moves smoothly in straight lines;
• There's a blockfield: the 20x20 block area over which the balls travel;
• The blockfield has a boundary outside of which the balls can't travel, and they bounce off the boundary;
• And there are two sides whose edges also cause the ball to bounce, but the block that gets hit reverses sides.

This is all complicated by the fact that, despite a lifelong interest in hacking at the C64, I'd never to this point written anything substantial in assembly for the machine. Looks like we'll be learning as we go.

On most machines of the era video was output to a CRT, as explained in my megathread on the 1541 disk drive from a couple of years back^[2]"Why Was the Commodore 64 Disk Drive So Slow?", Imran Nazar, Dec 2024. After each frame of video output comes a short vertical blanking period, during which the video chip is not making any demands on memory; we'll be looking to try to limit any calculations needed for this animation, so they're completed within the blanking period and we're not unduly affecting other things that may be happening on the machine during the frame rendering period.

On the Commodore 64, the BASIC interpreter is the main thing that might be running during that period, and it provides some helpful things for this effect. One of those is a hook that gets triggered at the bottom of every frame^[3]"Raster interrupt", C64 Wiki, updated Mar 2024, allowing custom code to be called every frame while keeping BASIC running. So we can plan out something like this for the code:

Before we get started

So let's look at the initialisation portion: we'll be keeping an internal representation of the state of the blockfield, but there's also the on-screen blockfield to set up. The C64 has a 40x25 area of characters in its default text mode (each character being an 8x8 pixel square), and there are two areas of memory of interest here:

• The screen RAM sits at $0400 by default, and is a thousand-byte area where the character values are stored;
• The color RAM at $D800 is a thousand nybbles^[A]A nybble is four bits, as opposed to an 8-bit byte. where each location can represent one of the C64's 16 colours.

Immediately after screen RAM sit the pointers to sprite data, but we're not setting up any sprites (yet), so a little shortcut is to write 1024 bytes to screen. This writes past the end of the screen, but makes the code much simpler:

SCRRAM = $0400
BLOCKPTR = $03   ; A block of two bytes not used by BASIC

init:
    subroutine

    ; Set up a pointer to screen RAM, starting at the top left
    ; The 65xx CPUs are little-endian, so low byte first
    lda #<SCRRAM
    sta BLOCKPTR
    lda #>SCRRAM
    sta BLOCKPTR + 1

    ; We'll be writing SPACE (character 32), not char 0 which is @
    lda #32

    ; Four lots of 256, for 1024 writes
    ldx #4
    ldy #0

.blank:
    ; The inner loop: runs from Y=0 through 255, 254, ...0 again
    sta (BLOCKPTR), y
    dey
    bne .blank

    ; The outer loop: punt the pointer along by 256 each go-around
    inc BLOCKPTR + 1
    dex
    bne .blank

(As an aside, BLOCKPTR is set to a value which is "not used by BASIC", but how do we know that? It turns out there's a detailed map of memory used by the C64, by Joe Forster of the STA demo group^[4]Commodore 64 memory map, STA, last updated probably a while ago as the page is HTML 4.01, which states locations that the BASIC and Kernal ROMs don't touch in their standard operation, and which are free for programs to use.)

With the screen a uniformly blank Commodore blue, we can draw out the blockfield as a 20x20 area. The display of the C64 allows each character to have a foreground colour in text mode, but the background colour is shared across the whole screen; if we want to use colour RAM to indicate which blocks are white and which are black (and we do), we'll want a character that's all foreground colour with no holes.

As the second half of the character set contains reverse-video copies of the first half, we can use the reverse-video version of SPACE (character 160) as our fully-foreground block. Our loop is a little more complicated: to write rows of 20 characters on a screen that's 40 wide, we need to write 20 and skip 20; additionally, we'll want to put this block centrally on the screen, so we need to start one row down and 10 characters in.

Our resultant pointer is actually one row and nine characters after the start of screen RAM, because we can achieve the effect of writing 20 bytes in a row either by starting at 0 and going up to 19 (aborting at 20), or starting at 20 and coming down to 1 (aborting at 0). The latter saves us a compare, so is both quicker and smaller in code size.

    lda #<(SCRRAM + 40 + 9)
    sta BLOCKPTR
    lda #>(SCRRAM + 40 + 9)
    sta BLOCKPTR + 1

    ; This time we're writing 20 rows of 20, char 160
    ; We'd normally think of this as X being along the line, and Y down
    ; but the 65xx only offers indirect addressing with Y as the index
    ldx #20

.field:
    lda #160
    ldy #20
.row:
    sta (BLOCKPTR), y
    dey
    bne .row

    ; Add 40 to the BLOCKPTR (16-bit add)
    ; Clobbers A, will need reloading at the top of the .field loop
    lda BLOCKPTR
    adc #40
    sta BLOCKPTR
    lda BLOCKPTR + 1
    adc #0
    sta BLOCKPTR + 1

    dex
    bne .field

With the screen RAM fully set up, next comes the colour RAM. As it turns out (peeking ahead) we'll be using a double-buffering technique to simplify the bounce calculations: that means an internal buffer of 400 contiguous black-or-white values for the blockfield, which is rendered to colour RAM each frame. So our initialisation becomes setup of the internal blockfield buffer:

BLOCKBUF = $CE00   ; 512 bytes unused RAM, which is ample

    lda #<BLOCKBUF
    sta BLOCKPTR
    lda #>BLOCKBUF
    sta BLOCKPTR + 1

    ; The initial blockfield is half black (colour 0), half white (1)
    ; In other words, 20 rows of 10 black and 10 white
    ; In other other words, 40 areas of alternating black and white
    lda #0
    ldx #40
.fill:

    ; The inner loop, writing the area of 10 (either black or white)
    ; This time, write from 9 to 0 inclusive, and stop at -1
    ldy #9
.fill_row:
    sta (BLOCKPTR), y
    dey
    bpl .fill_row

    ; Add 10 to BLOCKPTR (16-bit add)
    ; Clobbers A, but this time we can't just write a fixed value afresh
    ; So push A to the stack
    pha
    lda BLOCKPTR
    adc #10
    sta BLOCKPTR
    lda BLOCKPTR + 1
    adc #0
    sta BLOCKPTR + 1

    ; Pull A back off the stack, and XOR 1 to swap between 0 and 1
    pla
    eor #1

    dex
    bne .fill

And we're almost done with the initialisation steps. There are two things left to do: set up the sprites for our smoothly-moving balls, and connect the hook for our main routine. So we'll need to design an 8x8-pixel representation of a ball; it turns out that the Minecraft scene has a plethora of pixelated shape generators online, so we can use a circle generator^[6]"Pixel Circle Generator", Donat Studios, updated Jul 2025 set to 8x8:

Just like the text characters on a C64, sprites can have a foreground colour for any pixels that are set in the sprite, and a background that shines through for pixels that aren't set. That means sprites can be defined in binary; the text characters are 8x8 pixels, but sprites are 24x21, and the video chip expects binary data corresponding to that size when setting up the sprite. Our ball itself is only 8x8 pixels, so the rest will need to be padded out with 0's.

24x21 bits of data is 63 contiguous bytes, so the C64's video chip expects a pointer to the sprite data to be aligned to a 64-byte boundary; as a result, it actually expects the pointer divided by 64 as a sort of "data block index" during setup of the sprite. As we mentioned earlier, the pointers to the sprite are, by default, immediately after the screen text in RAM:

SPRPTR = SCRRAM + (40 * 25)

    lda #(ballmap / 64)
    sta SPRPTR       ; Sprite 0
    sta SPRPTR + 1   ; Sprite 1

    ; This will end up at the bottom of our source code,
    ; in a "data section". For now it's presented alongside
    align 64
ballmap:
    byte %00111100, %00000000, %00000000
    byte %01111110, %00000000, %00000000
    byte %11111111, %00000000, %00000000
    byte %11111111, %00000000, %00000000
    byte %11111111, %00000000, %00000000
    byte %11111111, %00000000, %00000000
    byte %01111110, %00000000, %00000000
    byte %00111100, %00000000, %00000000
    byte %00000000, %00000000, %00000000
    byte %00000000, %00000000, %00000000
    byte %00000000, %00000000, %00000000
    byte %00000000, %00000000, %00000000
    byte %00000000, %00000000, %00000000
    byte %00000000, %00000000, %00000000
    byte %00000000, %00000000, %00000000
    byte %00000000, %00000000, %00000000
    byte %00000000, %00000000, %00000000
    byte %00000000, %00000000, %00000000
    byte %00000000, %00000000, %00000000
    byte %00000000, %00000000, %00000000
    byte %00000000, %00000000, %00000000

With the sprite data set up, we can inform the video chip of the colours we'd like for each sprite's foreground, and then turn the sprites on. We're not setting the onscreen location of the sprites here, that'll be done by the rendering code; their default position of 0x0 is off the screen to the top left, which is usefully out of the way.

On the C64, talking to the hardware is done in the same way as talking to memory: writing to and/or reading from special addresses. In the documentation, these are referred to as "registers" for the various chips; the video interface chip (VIC) has over 40 of these, but for our purposes we're interested in those that control sprites 0 and 1.

There's an interesting wrinkle here that we'll look at in more detail when it comes to rendering the sprites, because the screen is more than 256 pixels across, which means the X-coordinate doesn't fit in one byte; instead, there's a bit in SPRXHI for each sprite that one needs to set when it's over on the right side of the screen. For now, we'll put that aside:

    ; Sprite 0 starts on the black side, and is white
    lda #1
    sta SPR0COL

    ; Sprite 1 starts on the white side, and is black
    lda #0
    sta SPR1COL

    lda #%00000011
    sta SPREN

And we're done setting things up: the blockfield is in place, the sprites are enabled, and the only thing left is to tell the C64 where to look for the rendering code that runs every frame.

IRQ = $0314

    ; Disable interrupts, so we don't
    ; get ...interrupted during setup
    sei

    lda #<isr
    sta IRQ
    lda #>isr
    sta IRQ + 1

    cli

; Interrupt service routine: main renderer
isr:
    subroutine

    ; ISRs need to run transparently, without clobbering
    ; any registers, so we save the full state to stack
    pha
    txa
    pha
    tya
    pha

    ; TODO: The rendering here
    jsr advance_white
    jsr collide_white
    jsr advance_black
    jsr collide_black
    jsr render_field
    jsr render_score

    ; And pull the state back off the stack, in reverse order
    pla
    tay
    pla
    tax
    pla

    ; Return to where the C64 would normally
    ; go during service of this interrupt
    jmp $EA31

We have ourselves a full program here, though it doesn't do a great deal yet. If we assemble this and build a disk image to load into an emulator^[B]I'm using VICE here, which is perhaps the most accurate emulator of anything ever built. we might get something like this.

Digression: Building a C64 disk image

While hacking on this, I realised that emulators don't generally take a binary image and allow you to drop it into the C64's memory to be run. The standard user-friendly way of loading a program would be to load it from a disk (or disk image, in this case), so we'll want to take our program as written in assembly, and get it into a disk image.

It turns out VICE provides tools to work on disk images, as it needs to emulate the 1541 disk drive as well as the C64 in order to provide disk connectivity, and you get those tools essentially for free as part of the implementation. In this case, a command-line emulation of the disk drive is available as c1541, and it can be instructed to perform all the steps necessary to create a disk image:

• Format a disk to lay down the pattern of tracks into an image;
• Attach the image to the emulator;
• Write our program to disk, and put its name in the directory listing.

# Change VICE_ROOT to point to your VICE binaries
# I'm on an M1 Mac, so this is the arm64/osx copy
VICE_ROOT = /Applications/vice-arm64-gtk3-3.7.1/bin
NAME = pongwars

all: $(NAME).d64

$(NAME).d64: $(NAME).prg
    $(VICE_ROOT)/c1541 \
        -format $(NAME),8 d64 $(NAME).d64 \
        -attach $(NAME).d64 \
        -write $(NAME).prg $(NAME)

$(NAME).prg: $(NAME).asm
    dasm $(NAME).asm -o$(NAME).prg

(It also bears mentioning that I'm using dASM^[7]dASM assembler, GitHub, last release 2020 to assemble the 6502 code written here into a binary image, that can then be pushed to the disk image by c1541.)

The game loop: Movement calculations

So we come back to the program under consideration, having set up our initialisation steps; we now have a empty game loop that needs to be filled in. From Figure 2, we know that our main steps are to advance the two balls, check for collisions, and render the updated block field and score.

What does it mean to "advance the ball"? Well, if we check back to Figure 1, we see that the balls start off in the middle of their fields, and they move at a fixed speed per frame, but their direction can change. If our fixed speed is one pixel per frame, trigonometry tells us that the ball needs to be moved by a fractional number of pixels in the horizontal and vertical directions:

We can't actually move the sprites by a fractional number of pixels, but we can keep track of the fractions and move the sprites if the calculation bumps their position across to the next whole number. If, for example, θ was 35 degrees our ball position might look like this for consecutive frames:

When I talk about fractions in computing, it's probable that your mind immediately goes to floating-point numbers, which can represent most fractions you're likely to run into. The Commodore 64 does support floating-point operations, but they're astoundingly slow: without dedicated hardware to handle them, the BASIC interpreter handles floating-point numbers in text form, performing calculations internally using its own binary format and translating back to text for storage.

For our purposes, there is another way: fixed-point numbers, where the fractional and integer parts of the number are always a given length and don't change between numbers. This works for us for multiple reasons:

• We know the field is 20x20 blocks, which is 160x160 pixels, so the whole-number part of the ball's coordinate (horizontal or vertical) can fit into a byte and can be taken directly from that byte to plot the ball on-screen;
• As per Figure 6, the direction of the ball is a unit vector, so cos θ and sin θ are always less than 1. In fact, they can be directly lifted from a lookup table for the sine function.

We can further simplify things by ranging θ through 256 "degrees" instead of the traditional 360, allowing the direction of the ball to fit into one byte. Putting this all together, our internal representation of the ball's state would be a structure of five bytes, laid out something like this:

With our data structure comes an initialisation step, which we'll want to add to the init code from earlier. This allows us to set up the balls' initial position and direction on the block field, which we can then use in the renderer:

; Ball data spans five bytes
BALL0 = $D9
BALL1 = $DE

init:
    ; ...
    ; Copy initial position data into zero-page
    ; We can do both sets of 5 bytes consecutively
    ldx #10
.spr_init:
    lda posdata, x
    sta BALL0, x
    dex
    bpl .spr_init

posdata:
    ; White ball: Theta 32 (south-east), X 40.0, Y 40.0
    byte 32, 40, 0, 60, 0
    ; Black ball: Theta 160 (north-west), X 120.0, Y 120.0
    byte 160, 120, 0, 120, 0

And we can finally start adding pieces to our game loop. From Figure 2, we'll want to advance the balls' positions and render their new locations, as the first visible step. That'll involve adding the horizontal and vertical components of the unit vector, for which we'll want a sine/cosine lookup table

One thing to note about the 6510 processor in the C64 is that it has a bug in the indexed addressing mode: if you have a table of 256 values that's not aligned to a 256-byte boundary, adding the index register can cause your final location to wrap around the boundary instead of continuing through to the rest of the table. In other words, a percentage of your lookup table becomes inaccessible, because the index never reaches that portion before looping back.

We have to take care to align our lookup table to a page boundary, using the assembler's align keyword, to avoid this bug. However, we can also use this to our advantage: the cosine function is the same as the sine function with an offset of a quarter of the waveform, so we can retrieve a sine value for a particular angle by indexing into the table, and adding 64 to the index to obtain the cosine for that same angle. If adding 64 would take us off the end of the table, no problem: that wrap-around means we land back near the start, on the index we were looking for.

Putting all that together, we get a pair of subroutines to advance the balls' position, that look a little like the below (I'm including the routine for the black ball only here, the white ball will look substantially similar):

advance_black:
    subroutine

    ; x += cos(theta)
    lda BALL0
    clc
    adc #64
    tax
    lda sindata, x   ; LUT wrap-around
    ldy #(BALL0 + 1)
    jsr add_u16_s8

    ; y += sin(theta)
    lda BALL0
    tay
    lda sindata, y
    ldy #(BALL0 + 3)
    jsr add_u16_s8

    ; TODO: Position on-screen

    rts

    ;--------------------------------
    align 256
sindata:
    hex 00 03 06 09 0C 10 13 16 19 1C 
    hex 1F 22 25 28 2B 2E 31 33 36 39 
    hex 3C 3F 41 44 47 49 4C 4E 51 53 
    hex 55 58 5A 5C 5E 60 62 64 66 68 
    hex 6A 6B 6D 6F 70 71 73 74 75 76 
    hex 78 79 7A 7A 7B 7C 7D 7D 7E 7E 
    hex 7E 7F 7F 7F 7F 7F 7F 7F 7E 7E 
    hex 7E 7D 7D 7C 7B 7A 7A 79 78 76 
    hex 75 74 73 71 70 6F 6D 6B 6A 68 
    hex 66 64 62 60 5E 5C 5A 58 55 53 
    hex 51 4E 4C 49 47 44 41 3F 3C 39 
    hex 36 33 31 2E 2B 28 25 22 1F 1C 
    hex 19 16 13 10 0C 09 06 03 00 FD 
    hex FA F7 F4 F0 ED EA E7 E4 E1 DE 
    hex DB D8 D5 D2 CF CD CA C7 C4 C1 
    hex BF BC B9 B7 B4 B2 AF AD AB A8 
    hex A6 A4 A2 A0 9E 9C 9A 98 96 95 
    hex 93 91 90 8F 8D 8C 8B 8A 88 87 
    hex 86 86 85 84 83 83 82 82 82 81 
    hex 81 81 81 81 81 81 82 82 82 83 
    hex 83 84 85 86 86 87 88 8A 8B 8C 
    hex 8D 8F 90 91 93 95 96 98 9A 9C 
    hex 9E A0 A2 A4 A6 A8 AB AD AF B2 
    hex B4 B7 B9 BC BF C1 C4 C7 CA CD 
    hex CF D2 D5 D8 DB DE E1 E4 E7 EA 
    hex ED F0 F4 F7 FA FD

But wait, what's this add_u16_s8? Well, you'll recall that each component of the ball's position is a fixed-point number, with 8 bits either side of the point; this is equivalent to an unsigned 16-bit number. Adding a value from the sine table to this fixed-point number is thus the same as adding a signed 8-bit number (a number that can range from -128 to +127) to our 16-bit number.

The processor doesn't have native handling of 16-bit numbers, so we'll have to build a routine for that. Fortunately, such a routine is fairly straightforward: we add our 8-bit value to the low end (our fractional component in this case), and if the value overflows the 8-bit range it'll set the processor's CARRY flag. We then either add or subtract the CARRY from the high end (our whole component) depending on whether our 8-bit value was positive or negative.

The processor can help us with the "whether a number is negative" part too: one of the other flags available is NEGATIVE, which is the same as the top bit of an 8-bit value after calculation (the top bit being how one traditionally denotes a negative number). Alongside that come the bpl and bmi instructions, which stand for "branch if plus" and "branch if minus".

We can use bpl with an instruction that doesn't change the value, but does set NEGATIVE to allow us to take those two branches:

; add_u16_s8: Add 8-bit signed to 16-bit unsigned
; @param A The 8-bit signed
; @param Y Zero-page pointer to the 16-bit unsigned
; @return Changed value in zero-page
; @clobbers A
; @preserves Y
add_u16_s8:
    subroutine
    and #255      ; Set N
    bpl .positive

.negative:
    clc
    adc 1, y      ; Add fraction
    sta 1, y
    lda 0, y
    sbc #0        ; Subtract C from whole
    sta 0, y
    jmp .end

.positive:
    clc
    adc 1, y      ; Add fraction
    sta 1, y
    lda 0, y
    adc #0        ; Add C to whole
    sta 0, y

.end:
    rts

Digression the second: Representing negative numbers

The keen-eyed reader will have seen something weird in the above explanation of how add_u16_s8 works:

We add our 8-bit value to the low end (our fractional component in this case)... then either add or subtract the CARRY from the high end (our whole component)

Surely if the number is negative we'd want to subtract both times? To look at why we don't, it's worth a quick note on the history of representing negative numbers in computing.

Computers were, to begin with, exclusively the domain of mathematicians and people who would be familiar with the esoteric field of binary numbers, so the number handling was built around being able to read those numbers off from the "blinkenlights" directly in binary. The simplest way to handle both positive and negative numbers was simply to mark one bit in the number as "the negative bit", and the rest of the number would represent the value; this was called "sign and magnitude" representation. For example:

Twenty-three and negative twenty-three:
0 0010111
1 0010111

To perform a subtraction (or to add a negative number, which is functionally equivalent) while only having adding circuitry, the computer needs to perform some precalculation otherwise it'll end up with the wrong answer:

  [63 - 23]      [63 + -23]
  00111111   =   00111111
- 00010111     + 10010111
                 --------
                 11001100
                 negative eighty six

The sign-and-magnitude negative number needs to be inverted (or one's-complemented in the mathematical jargon) when performing a subtraction, so that adding it has the intended effect of reducing the value instead of piling on top:

  [63 - 23]      [63 + -23]
  00111111   =   00111111
- 00010111     + 11101000
                 --------
              [1]00100111
                 thirty nine (and a carry)

Except complementing doesn't quite fix it: because sign-and-magnitude numbers can have zero (00000000) and negative zero (10000000), we need to add one after inverting to step over the negative zero value:

  [63 - 23]      [63 + -23]
  00111111   =   00111111
- 00010111     + 11101001
                 --------
              [1]00101000
                 forty (and a carry)

To summarise: negative numbers were represented as a negative sign bit and the value as the magnitude, for easier reading off from binary displays, but the computer would need to first invert the number and then add one in order to perform the addition of a negative number.

As binary read-off displays went away with the growing ubiquity of computing, it began to make sense to simplify the representation of negative numbers not for the human mathematician using the computer, but for the computer's internal operation. The best thing for the computer, of course, would be to remove this whole pipeline of needing to complement the number and add one, before being able to add a negative number.

And that's how we landed at two's-complement (two's because we add one to the one's-complement) representation as the standard for binary negative numbers: if the sign bit is set to negative in the number, the rest of the value is already inverted and has one added, so the computer just needs to add the whole value as normal.

This does increase the cognitive load on a programmer working at a low level, because your binary numbers no longer read "normally":

00000000 =    0
00000001 =    1
00000010 =    2
...
01111110 =  126
01111111 =  127
10000000 = -128 (not -0)
10000001 = -127 (not -1)
10000010 = -126 (not -2)
...
11111110 =   -2
11111111 =   -1

But when are we going to be working at a sufficiently low level that we have to remember about two's-complement representation in the microcomputer age of the '80s, with high-level languages such as BASIC widely available? When we need to eke out more performance than BASIC can give us, which is what we need here.

That's why add_u16_s8 performs an addition on both branches, whether the s8 is positive or negative.

After movement comes positioning

Back to our regularly scheduled programming, we're now able to advance the internal representation of the balls' positions, but we're not yet able to see the balls moving on-screen. For that to happen, we need to set the positions of the sprites in the video chip's registers, except we can't naively copy the calculated values over: the top-left of our playing field isn't at sprite coordinate 0,0. Heck, the top-left of the screen isn't at 0,0.

From some time earlier, you'll recall that our blockfield is one row and ten characters in from the top-left of the screen, which (given that each character is 8x8 pixels) is eight pixels down and 80 across. As a result, to plot the ball sprite where we want it on-screen we need to add 104 to the X coordinate, and 58 to the Y.

And now that wrinkle we mentioned earlier comes back around: SPRXHI, the video chip control register that contains the 9th bit of each sprite's X-coordinate. The VIC combines this with the lower eight bits to generate the final position, which means we need to update SPRXHI appropriately each time we set the balls' positions.

As an aside, the furthest right our ball can go in the playing field is 19 blocks before bouncing off the right edge, which is 152 pixels; adding our offset of 104 leaves a final position of 256. If our playing field were even one pixel left of centre, we wouldn't have to worry about SPRXHI during our game loop.

So let's translate this into code by filling out the rest of advance_black. You'll recall from Figure 7 that each ball has a data structure in memory that holds the coordinates in fixed-point number format, so we only need the whole-number portion of each coordinate when translating to on-screen position. We left a TODO in the code previously, so bringing that down from above:

advance_black:
    subroutine

    ; x += cos(theta)
    lda BALL0
    clc
    adc #64
    tax
    lda sindata, x   ; LUT wrap-around
    ldy #(BALL0 + 1)
    jsr add_u16_s8

    ; y += sin(theta)
    lda BALL0
    tay
    lda sindata, y
    ldy #(BALL0 + 3)
    jsr add_u16_s8

    ; Position the sprite
    ; The lower byte of the X-coordinate first
    lda BALL0 + 2
    adc #104
    sta SPR0X

    ; Then the high 9th bit: if the result of the add
    ; was over 255, the Carry flag will be set
    lda SPRXHI
    and #%11111110
    bcc .xlow
    eor #1 
.xlow:
    sta SPRXHI

    ; And the Y-coordinate, which is always one byte
    ; (Taking care to clear the Carry from earlier)
    lda BALL0 + 4
    clc
    adc #58
    sta SPR0Y

    rts

If we do this for both the white and black balls, filling out advance_white as well as advance_black, each time around the game loop will move them by one step on-screen, which will look something like this:

So that's progress: the two balls can be seen moving, but they escape the playing field and wrap off the edge of the screen before returning, travelling forever in the same direction. This is, of course, because we have no code yet to detect the edge of the playing field and make a ball bounce when that's hit, so this is what we need next.

There are four possible conditions where a ball can hit an edge of the field and need to bounce, and they fall into two groups:

• If a ball has hit the top, it was travelling up and needs to bounce down; conversely, if a ball has hit the bottom, it was travelling down and needs to bounce up.
• If a ball has hit the left edge, it was travelling to the left and needs to bounce off to the right; in the same fashion, a ball travelling right will hit the right edge and need to bounce back to the left.

We know the coordinates of the ball (to be more precise, we know the coordinates of the top-left of the ball sprite) and we know the size of the field, so this becomes a relatively simple set of comparisons. The only caveat is that when we're checking for hits on the bottom or right edges, we're not comparing the top-left of the ball sprite, but the bottom-right, which is eight pixels further across in either direction.

So if the field is 20 blocks both ways, that's 160 pixels; if a ball's coordinate in either direction has reached 152, it's just hit the edge. In summary, we have the following four conditions in which a ball needs to bounce:

• If a ball's Y-coordinate is 0 it's hit the top;
• If a ball's Y-coordinate is 152 it's hit the bottom;
• If a ball's X-coordinate is 0 it's hit the left;
• If a ball's X-coordinate is 152 it's hit the right.

So what does it mean for the ball to bounce? You'll recall from Figure 6 that our ball's velocity is described by a unit vector: it's always moving at the same unit speed, but we keep track of its angle. So bounces are changes in angle: specifically, they're changes in angle caused by reflection.

In Figure 9, we see a reflection in the X-axis (hitting a horizontal line) causes the angle to negate, and reflection in the Y-axis (hitting a vertical) causes our angle θ to change to 180° - θ. For our purposes, you'll recall that θ ranges through 256 "degrees", so we can start to fill out code to perform these flips in angle, remembering that θ is stored in the first part of our ball's information block in memory:

flipx_black:
    subroutine
    lda #0       ; I have a reason for not using
    sec          ; NEG here, we'll come to it
    sbc BALL0
    sta BALL0
    rts

flipy_black:
    subroutine
    lda #128
    sec
    sbc BALL0
    sta BALL0
    rts

And then we need to detect those collisions with the edge of the field, as described above, by filling out collide_black:

collide_black:
    subroutine

    ; Check the X-coord
    lda BALL0 + 2

    ; Skip if we haven't reached the left edge
    cmp #0
    bpl .nobounce_x

    ; And if we haven't reached the right
    clc
    cmp #152
    bcc .nobounce_x

    ; If we made it here, we need to flip
    ; We've hit a vertical, flip in Y
    jsr flipy_black

.nobounce_x:
    ; Check the Y-coord
    lda BALL0 + 4

    ; Skip if we haven't reached the top edge
    cmp #0
    bpl .nobounce_y

    ; And if we haven't reached the bottom
    clc
    cmp #152
    bcc .nobounce_y

    ; If we made it here, we need to flip
    ; This time it's a horizontal
    jsr flipx_black

.nobounce_y:
    ; TODO: Detect collisions with the other side
    rts

If we build the corresponding collide_white routine and its flip helpers, those will run with each frame, and we'll get something like this:

Now we're getting somewhere: a playing field, balls that bounce around it. What we don't have is any sense of unpredictability: the bounces are always perfect, the balls travel in well-worn paths forever, even if they're now constrained to the bounds of the field.

What we need is...

An element of chance

The original Pong Wars from Figure 1 has a sprinkle of randomness: each bounce (whether off the edge of the field or from an opposite-coloured block) is a reflection with a small offset to the angle, and it's this random offset that introduces the compelling infinite complexity.

To get the same effect on the Commodore 64, we could use BASIC's RND function to generate a pseudo-random number (as one might expect, this is referred to as Pseudo-Random Number Generation). There are two problems with this approach: being pseudo-random, the sequence is predictable so every run of this code would turn out the same; but more importantly for our purposes, RND is astonishingly slow at generating numbers, as it uses a relatively complex PRNG function.

On the C64, we have an alternative that's both fast and relatively simple to operate: hardware. The sound chip has three voices, and each of the voices can be set to one of four waveforms at an adjustable frequency: triangle, sawtooth, pulse or noise.

For our purposes, we can set up one of the SID's voices to generate noise, but disable output of the voice so we're not actually making the machine screech; we can then use the SID's internal representation of what's being generated as our source for random numbers.

Setup of the SID will need to be done as part of our general initialisation routine, so once again we're diving back into the init subroutine to set the SID's registers appropriately^[11]"Getting random number from 6502 assember", 'Mike' at StackOverflow, Jul 2017:

SID_3_FREQ = $D40E
SID_3_CTRL = $D412
SID_3_OUT = $D41B

init:
    ; ...
    ; After sprite data init, set up
    ; SID voice 3 for noise generation

    ; Maximum frequency (65535)
    lda #255
    sta SID_3_FREQ
    sta SID_3_FREQ + 1

    ; Noise waveform, output disabled
    lda #%10000000
    sta SID_3_CTRL

Now we can poll SID_3_OUT for hardware-generated real random numbers at any time, but they'll be a value that spans a full byte, anywhere from 0 to 255. We'd like to limit that range to our previously mentioned "small offset", which we can achieve fairly efficiently by lopping bits off the top end; more importantly, we'd like it to be a small offset either side of zero, which we can do by subtracting half the new range.

I've settled on cutting the value down from a 0-255 range to 0-15, which means we'll need to then subtract 8 to get a small random offset that's centred on zero. With that hived off into its own subroutine, we can adjust our reflection functions to use this random offset as the base instead of the fixed values 0 and 128:

rand:
    ; Generates a random number
    ; from -8 to +7, in A
    subroutine
    lda SID_3_OUT
    and #%00001111
    sec
    sbc #8
    rts

flipx_black:
    subroutine
    jsr rand
    sec
    sbc BALL0
    sta BALL0
    rts

flipy_black:
    subroutine
    jsr rand
    clc
    adc #128
    sec
    sbc BALL0
    sta BALL0
    rts

With this adjustment in place, our eternal bouncing around the playing field looks a lot more interesting:

The playing field

So after (apparently) seven thousand words, we come to the point: these balls which we can now bounce around the playing field need to detect that they're about to hit a block of their colour, bounce off it, and flip the colour. We also need to render the blocks in the field, which we've deferred doing thus far in favour of showing the classic Commodore blue through the field.

Let's look at the second point first, and get the block data from our internal representation onto the screen. We actually have precedent for how to do that, since we set up the on-screen characters in rows of 20 during initialisation. We just need to do that again each time around the rendering loop, but with the colours this time (either black or white).

We get lucky here, because the C64's colour palette has black as colour 0 and white as 1: there's no work to do in translating the content of our internal representation to on-screen colours, except to copy the rows across. There is work, however, in the copying of data across: the C64 doesn't have a Direct Memory Access controller which could perform this copying in hardware, so the CPU has to do the work by picking up each value from the source and dropping it at the destination.

During initialisation we set up a pointer called BLOCKPTR which would advance as we came down the screen to render the character data in the rows. For rendering the blockfield's colours we can reuse this pointer as our source, and set up a new COLORPTR to act as our destination.

COLORBUF = $D800 + 40 + 10
COLORPTR = $05    ; Two more unused bytes in zero-page

render_field:
    subroutine
    
    ; First set up the pointers, src and dest
    lda #<BLOCKBUF
    sta BLOCKPTR
    lda #>BLOCKBUF
    sta BLOCKPTR + 1

    lda #<COLORBUF
    sta COLORPTR
    lda #>COLORBUF
    sta COLORPTR + 1

    ; Copying 20 rows of 20
    ; On the 6502 this gets a little confusing, because
    ; we want to use indirect indexed mode w/ pointers
    ; but that's only offered with the Y index
    ldx #20       ; So X is the Y-coord
.copy_row:
    ldy #19       ; And Y is the X-coord
.copy_blk:
    lda (BLOCKPTR), y
    sta (COLORPTR), y

    ; We use the previous trick of counting down
    ; until Y goes negative, to save a comparison
    dey
    bpl .copy_blk

    ; Advance to the next row in the blockfield
    ; Which is a 16-bit addition
    clc
    lda BLOCKPTR
    adc #20
    sta BLOCKPTR
    lda BLOCKPTR + 1
    adc #0
    sta BLOCKPTR + 1

    ; And advance in colour RAM, which importantly
    ; is 40 bytes down on-screen
    clc
    lda COLORPTR
    adc #40
    sta COLORPTR
    lda COLORPTR + 1
    adc #0
    sta COLORPTR + 1

    dex
    bne .copy_row

    rts

How does that look? Well, a little strange since the balls pass through their respective fields without affecting them, but that's to be expected: we've yet to write the code that detects collision there, the TODO from earlier is yet to be filled in.

Another way of saying that we need to detect collisions with the field is to find out which block resides at the location in the playing field the ball's about to reach. So for the black ball which should bounce off a black area of the field, the location the ball will reach after this next step would need to be in an area that's currently black. So we need a function to look up what's stored at the ball's location in the blockfield.

The ball's location is stored as pixel coordinates, so there are a few steps to get to where we need to go:

• First, knock the pixel coordinates down to block coordinates in the field: divide by eight in both directions;
• Second, calculate Y * 20 + X to retrieve an entry from the blockfield storage.

This is complicated by the CPU not having a native "multiply" instruction, so we need to perform some binary shifts and additions; for a primer on binary shifts, feel free to look over my introduction to those operators^[12]"An Introduction to Bitwise Operators", Imran Nazar, Sep 2006 from 20 years ago. Essentially, because we can't natively multiply by 20, we need to generate the result through multiplying by binary powers and adding those results together; similarly, we can't natively divide by 8, so we do the same by shifting down. To take an example:

       X = 58   |        Y = 133
       01101010 |        10000111
>> 1 = 00110101 | >> 1 = 01000011
>> 2 = 00011010 | >> 2 = 00100001
>> 3 = 00001101 | >> 3 = 00010000

With our coordinates in block-level values (in this case, BX=13 and BY=16), we can proceed to multiply and add; our formula here is BY * 16 + BY * 4 + BX, which in binary shifts is BY << 4 + BY << 2 + BX.

       X = 58   |        Y = 133
       01101010 |        10000111
>> 1 = 00110101 | >> 1 = 01000011
>> 2 = 00011010 | >> 2 = 00100001
>> 3 = 00001101 | >> 3 = 00010000

                | << 1 = 00100000
                | << 2 = 01000000
                | << 3 = 10000000  ; We should look at this
                | << 4 = 00000000

There are two things to take note of here. Firstly, our shift-left of 4 (to multiply by 16) has pushed a bit into the Carry flag because the value has gone over 255, so we need to treat this in much the same way we did wide additions earlier, by taking care to bring the Carry flag over to the high byte of the result. The CPU offers a rotate instruction which works much like a shift, but also brings in the Carry flag to one side of the accumulator and pushes a new value for Carry out the other side; we can use this to bring the Carry flag into the high byte of our resultant value.

The other thing to note is that by shifting right 3, and then shifting left 3, we're left with the value we started with except the bottom three bits are zeroed out. As the CPU only offers shift operations of one bit at a time, we can save a bunch of time by zeroing out those bits with an AND rather than perform the six shifts in question.

This makes our final calculation ((Y & $F8) << 1) + ((Y & $F8) >> 1) + (X >> 3). Translating that over to 6502, we might get something like this.

SCRATCH = $92   ; Temp storage

; get_blk_val: Get the blockfield value at coordinates
; @param X Horizontal coordinate
; @param Y Vertical coordinate
; @return A The value of the block at this coordinate
; @clobbers A
; @preserves X, Y
get_blk_val:
    subroutine

    lda #0
    sta BLOCKPTR
    sta BLOCKPTR + 1

    ; ((Y & $F8) << 1)
    tya
    and #$F8
    asl
    sta BLOCKPTR
    rol BLOCKPTR + 1

    ; + ((Y & $F8) >> 1)
    tya
    and #$F8
    lsr
    ; 16-bit add to what's already in BLOCKPTR
    clc
    adc BLOCKPTR
    sta BLOCKPTR
    lda BLOCKPTR + 1
    adc #0
    sta BLOCKPTR + 1

    ; + (X >> 3)
    txa
    lsr
    lsr
    lsr
    clc
    adc BLOCKPTR
    sta BLOCKPTR
    lda BLOCKPTR + 1
    adc #0
    sta BLOCKPTR + 1

    ; We need this as an index into BLOCKBUF
    clc
    lda #<BLOCKBUF
    adc BLOCKPTR
    sta BLOCKPTR
    lda #>BLOCKBUF
    adc BLOCKPTR + 1
    sta BLOCKPTR + 1

    ; Finally, read from there
    ; We can only do this as a Y-indexed indirect,
    ; so we need to put Y into scratch memory
    sty SCRATCH
    ldy #0
    lda (BLOCKPTR), y
    ldy SCRATCH

    ; Set Negative and Zero flags
    and #$FF
    rts

Having a way to check the value stored in the blockfield at a given coordinate, we now need to consider which points on the ball will need checking. As per Figure 4, the (0,0) coordinate of the ball sprite is actually empty space and wouldn't be a good point to detect a bounce; what's needed is to check a centre point on each of the edges, and those are:

• (0,4) for the left edge;
• (7,4) for the right edge;
• (4,0) for the top;
• (4,7) for the bottom.

For the purposes of this article, I'll show one of the more complicated options (collision of the right edge of the ball), as that involves addition on both axes; the other detections are left as an exercise for the reader. We pick up at the bottom of collide_black to check for collisions between the black ball and the black field:

.nobounce_y:
    ; Collision with the other side: right edge
    ; First set up X and Y
    clc
    lda BALL0 + 2
    adc #7
    tax

    clc
    lda BALL0 + 4
    adc #4
    tay

    ; Then check what block is here
    ; If it's 1 (white) we're still travelling
    ; For white/white collisions, we'd 'beq' here
    jsr get_blk_val
    bne .nohit_r
    
    ; If we fell through to here, we've hit
    ; the black field; bounce off to the left
    ; (reflection in the Y-axis)
    jsr flipy_black
    ; TODO: Change state of the struck block
    jsr flipblk

.nohit_r:
    ; Other edge detections

That almost gets us what we need: we can detect that the field has been hit, we can bounce off, but the block that's been hit doesn't change colour. That's the last TODO left for ourselves at the bottom of collide_black; fortunately, it's fairly simple. We already know which block is involved (we calculated BLOCKPTR earlier as part of get_blk_val's processing), and we know that the colour always flips (either white to black or black to white), so this is always an XOR operation.

; flipblk: Flip the block pointed to by BLOCKPTR
; @clobbers A, Y
flipblk:
    ldy #0
    lda (BLOCKPTR), y
    eor #%00000001
    sta (BLOCKPTR), y
    rts

Filling out the other edge detection cases, and copying the whole set over to processing for the white ball, we arrive at a final effect, with only the rendering of the scores for each side left to consider.

Keeping score

The last piece of the puzzle is calculating the scores for each side, and getting those on-screen below the playing field. The first of these points is relatively simple: as we're already copying the block field over from our internal storage to the video chip's colour RAM to get it rendered, we can take that opportunity to count the blocks as they come through. The only complication is that it's possible for one side to have more than 255 blocks, so these are 16-bit counters and need to be treated appopriately (with 16-bit additions).

Picking up in the middle of render_field, in the tightest level of the copying loop:

CNTBLK = $FB
CNTWHT = $FD

render_field:
    ...
.copy_blk:
    lda (BLOCKPTR), y

    ; We're about to clobber A, hold onto it
    pha

    ; If we loaded a 1, this is a white field
    ; The Z flag will be cleared, so BNE will jump over
    bne .copy_cntwht

    ; If we end up here, the block is 0 (black)
.copy_cntblk:
    inc CNTBLK
    lda CNTBLK + 1
    adc #0
    sta CNTBLK + 1
    jmp .copy_write

    ; If we land here, we had a 1 (white)
.copy_cntwht:
    inc CNTWHT
    lda CNTWHT + 1
    adc #0
    sta CNTWHT + 1

.copy_write:
    ; Back to writing to the color RAM
    pla
    sta (COLORPTR), y

This gives us a way to count up the scores each side, whenever the blockfield is rendered; what's needed next is a way to get those counts on screen. As with a few other things we've come across in our journey, Commodore BASIC has routines which would allow us to print out a number, but there are caveats around performance. While we're working at this low level, let's have a look at what it would take to print out a number using a routine in assembly.

Let's say one of our scores to print is 279: this needs to be taken from a 16-bit value in memory to three distinct characters on screen ("2" "7" "9"). The mathematical principle is fairly simple, and is one we all come across in elementary school: repeated division, which provides us our digits in reverse order.

279 ÷ 10 = 27 remainder 9
 27 ÷ 10 =  2 remainder 7
  2 ÷ 10 =  0 remainder 2

There are two complicating factors here. First, we can't simply send the value 2 to the screen; as per Figure 3, this character code represents the letter B, as the Commodore 64's character ROM doesn't start with the numeric digits. We need to add 48 to each digit as it comes out of the division, in order to get a character string that can go on-screen.

The second complication is that the processor doesn't offer division natively (we already saw earlier that native multiplication is not a thing either), so a software routine is needed to perform 16-bit division on our behalf. Fortunately, the basic algorithm for division is eminently possible to wrap one's head around, and is essentially a binary version of the classic "long division" we all suffered through in math class.

Digression the third: Binary long division

Just as a multiplication is repeated addition, a division can be thought of as repeated subtraction. Let's break down that first division in more detail, the way one might be taught to do division in school:

279 ÷ 10:
Does  2 go into 10? No
Does 27 go into 10? Yes, 2x

27 - (2 x 10) = 7

Tack on the rest of the number, continue with: 79

Does  7 go into 10? No [but we know that, it was left over]
Does 79 go into 10? Yes, 7x

79 - (7 x 10) = 9

Tack on the rest of the number, continue with:  9

Does 9 go into 10? No [but we know that, it was left over]
No further digits, abort

This eventually yields our intended result: combining the individual "does it go into" questions gives 27, while 9 was left over. When it comes to doing this on a computer, we have to knock these operations down into binary, so the pulling in of each digit looks slightly different. The fact that we're working in binary, though, means that our "does it go into" question can only ever yield two results: "No" or "Yes, 1x". That means our question becomes "is greater than or equal to":

100010111 ÷ 1010:
    1 ≥ 1010? No
   10 ≥ 1010? No
  100 ≥ 1010? No
 1000 ≥ 1010? No
10001 ≥ 1010? Yes

10001 - (1 x 1010) = 111

Tack on the rest of the number, continue with: 1110111

  111 ≥ 1010? No [we know it won't]
 1110 ≥ 1010? Yes

 1110 - (1 x 1010) = 100

Tack on the rest of the number, continue with: 100111

  100 ≥ 1010? No [we know it won't]
 1001 ≥ 1010? No
10011 ≥ 1010? Yes

10011 - (1 x 1010) = 1001

Tack on the rest of the number, continue with: 10011

 1001 ≥ 1010? No [we know it won't]
10011 ≥ 1010? Yes

10011 - (1 x 1010) = 1001

Tack on the rest of the number, continue with: 1001

 1001 ≥ 1010? No [we know it won't]
No further digits, abort

Now, this looks like a longer operation, but that's only because it's happening in a lower base: that means each step down is a step by two, rather than 10. If we now count up the "No" and "Yes" answers, ignoring the "No" answers where we know the answer was left over from the previous step, we get a result of: 000011011 remainder 1001. Coming back up to decimal, that's 27 remainder 9.

Before we can implement this on the C64, there are two things we need to work out: first, how do we "shift" each digit over to incrementally perform the "greater than or equal to" check? It turns out there's an instruction to do exactly that, "arithmetic shift left" (asl) which will push the top bit out into the Carry flag. Except we don't want the top bit to land in the Carry flag, we'd like it in the accumulator where we can work on the value; we can thus combine asl with "rotate left" (rol) which pulls the Carry flag into the bottom of a value, and pushes a corresponding bit out the top.

Thus, to work on a 16-bit value, we want to perform a 24-bit left shift to get the value passing through both bytes and the accumulator:

DIVIDEND = $1B
DIVISOR  = $19

; Divide a 16-bit value by an 8-bit divisor
; @clobbers A, X
div16_8:
    subroutine
    ldx #16
    lda #0
.loop:
    asl DIVIDEND
    rol DIVIDEND + 1
    rol a

    ; ...What do we do here?

    dex
    bne .loop
    rts

The other issue is our ≥ check, which documentation for the processor^[13]"Compare Instructions", from "6502 Software Design" by Leo Scanlon, updated Dec 2022 tells us can be achieved by using cmp ("compare", which performs a subtraction and throws the result away), and bcs ("branch if carry set"). In this instance, we actually want to skip over the subtraction if not required, so we'll use bcc instead:

    cmp DIVISOR
    bcc .no_sub
    sbc DIVISOR
    ; ...And where do we put the Yes result?

As we shift our dividend through, we leave zeroes behind: when this loop is finished, DIVIDEND will be left zeroed out entirely. Conversely, we need to store our result somewhere, incrementally shifting bits of that over to match our progress through the dividend. We can combine these two steps, and store the result at the bottom of the dividend at the same time that the top is shifting through.

All we need to do that is to either set or not set the bottom bit of the dividend:

    cmp DIVISOR
    bcc .no_sub
    sbc DIVISOR
    inc DIVIDEND
.no_sub:
    ; ...And the go-around for the loop

This has the additional subtle advantage that our result is in the same place in memory that our dividend started, meaning it can be used immediately for another division without having to be copied over from somewhere. That's exactly what we were looking for, as we want to perform repeated division to work out the digits for our score display.

Putting these disparate snippets together, we get a final division subroutine which looks like this^[C]I take no credit for this implementation, only for the above dissection of how it works; this comes from a post on the NESDev forums by "tepples" from Jan 2005.:

DIVIDEND = $1B
DIVISOR  = $19

; Divide a 16-bit value by an 8-bit divisor
; @param Memory (DIVIDEND, DIVISOR)
; @return Memory (DIVIDEND): The result
; @return A The remainder
; @clobbers A, X
div16_8:
    subroutine
    ldx #16
    lda #0
.loop:
    asl DIVIDEND
    rol DIVIDEND + 1
    rol a
    cmp DIVISOR
    bcc .no_sub
    sbc DIVISOR
    inc DIVIDEND
.no_sub:
    dex
    bne .loop
    rts

Rendering the score

Now we have a routine for division, we can perform our repeated dividing-by-10 in order to build a score string to display on-screen. We'll need a little helper subroutine to build the string by adding 48 to each digit, as mentioned beforehand: we know that, these being 16-bit numbers, they can never go past 65535 decimal, which means we need to allocate a block of five bytes in memory to hold the resultant string.

Of course, this brings up more questions for us: how do we know how long the string is once it's generated, and what happens if the score for one side changes from, say, 100 down to 99? If we only print the 99, one character from the previous score will be left on-screen and we'll wne up with (depending on the side) 990 or 199 as the visible score.

The first of these issues can be resolved by always using a fixed length of five bytes for the string, and pre-filling the string with spaces so that when we print it out, the "blank" characters don't otherwise affect the display. Similarly, the second issue can be resolved by always printing spaces over the screen area for the scores before we write out the scores themselves, so the display is always fresh.

The first thing we'll need, then, is a subroutine to convert the score to a string, by repeated division:

STROUT = $22

; int2str: Convert 16-bit integer to string
; @param A Zero-page location of the number
; @clobbers A, X, Y
int2str:
    subroutine

    ; Firstly, setup the division routine
    tax
    lda 0, x
    sta DIVIDEND
    lda 1, x
    sta DIVIDEND + 1

    lda #10
    sta DIVISOR

    ; Then blank out the string storage
    ; Five bytes, from STROUT + 4 to STROUT + 0
    ldx #4
    lda #32   ; Space
.buflp:
    sta STROUT, x
    dex
    bpl .buflp

    ; And begin dividing
    ; We use the Y index here, as div clobbers X
    ldy #4
.divlp:
    jsr div16_8

    ; Our remainder is the digit to print, we need
    ; to offset this into the PETSCII numerals
    clc
    adc #48
    sta STROUT, y
    dey

    ; We don't want to go the full 5 characters
    ; Abort if the dividend is zero
    lda DIVIDEND
    ora DIVIDEND + 1
    bne .divlp

    rts

And now all that remains is to get these strings on-screen. We'll want to perform the blanking operation mentioned earlier, to overwrite the previous score with spaces, and then there's one final wrinkle to consider: these strings are space-padded to the left (which means, for example, a score of 274 is held in memory as   274), but if we want to print the score lined up with the edge of the blockfield that means the left-hand side will be indented from the left edge.

All we need to do for this case is to ignore spaces as they come through, and only print non-space values. On the right-hand side, we do want to print those spaces as they go through (otherwise we'll have the opposite problem, an indent on the right), so there are two related but different sections of code in render_score:

render_score:
    ; Firstly, blank out the scores
    ; Screen row 22, positions 10-14 and 25-29
    ldx #4
    lda #32
.blank:
    sta SCRRAM + 22 * 40 + 14, x
    sta SCRRAM + 22 * 40 + 25, x
    dex
    bpl .blank

    ; Then print the black score, ignoring spaces
    ; Note: We want the address of
    ; the score, not its value, in A
    lda #CNTBLK
    jsr int2str

    ; We can't use our trick of looping backwards
    ; as the score will then print out backwards
    ; so X runs from 0 to 4, and aborts at 5
    ldx #0
    ldy #0

.black_print:
    lda STROUT, x
    cmp #32
    beq .black_skip
    sta SCRRAM + 22 * 40 + 10, y
    iny
.black_skip:
    inx
    cpx #5
    bne .black_print

    ; The same again for white
    ; Simpler this time, we can print spaces
    lda #CNTWHT
    jsr int2str
    ldy #0
.white_print:
    lda STROUT, y
    sta SCRRAM + 22 * 40 + 25, y
    iny
    cpy #5
    bne .white_print

    rts

With that final piece of the rendering routine in place, we get the full mesmerising effect of Pong Wars, which we can leave to run at the highest speed VICE can attain on my machine (somewhere around 10x realtime):

Future considerations, or "why so slow?"

The astute reader will have noted throughout this post that the video output has come from a copy of VICE running at 3x (or 4x, in some cases) speed. Right at the top of this post, I stated:

...we'll be looking to try to limit any calculations needed for this animation, so they're completed within the blanking period.

So where's the disconnect? Why is this effect so slow in its current implementation, that it takes VICE running at multiple times native speed to get it looking half-decent?

As it turns out, the calculations being made to detect bounces are relatively efficient, and complete within the period we need; the problem is that we then copy the blockfield across to screen memory to get it rendered, and calculate the current score by counting up the number of black and white blocks that are copied across. This takes an astonishing amount of time: more or less an entire frame is taken up by this copy-and-count process and the divisions required to print the score afterwards.

That means rather than running at the C64's native 60 frames per second, this effect runs at 30. On top of this slowdown, the balls move by 1 pixel per go-around, which is a slow speed in itself when travelling the distance of the blockfield: the initial directions of the balls are such that it'll be at least five seconds before they hit the first block and the Pong War begins.

There are a bunch of improvements we could make to the effect, which won't be performed here, but can be briefly discussed:

• Direct rendering: Rather than copy the blockfield over to colour RAM with each render, colour RAM itself can be used as the blockfield storage as it allows for both read and write from the CPU without performance penalty. The only caveat here is that get_blk_val (the routine which retrieves the colour at a given coordinate) will need to multiply by 40 instead of 20, to account for the rows not being consecutive in memory.
• Score-keeping at collision time: In this implementation we've been using the naive method of counting the black and white blocks at the time they're being copied into colour RAM. We can instead keep track of the scores for each side as we know they start at 200 each, and only ever change when flipped by collision with a ball; this would be a change in flipblk to perform a 16-bit addition of 1 for one side, and subtraction of 1 for the other, whenever a block flips.
• Score-rendering at collision time: The divisions involved in taking the counted-up scores and printing them out digit by digit are also significant in terms of CPU time, and don't need to run every frame; as that area of the screen only changes when a collision happens, it's possible to perform that work alongside the change in score-keeping, breaking up our rendering routine so some of it doesn't happen every frame.
• Longer ball vector: With these speedups in place, the effect is likely to run at the native 60fps, and the balls can then be made to move further with each frame by lengthening their stride. For a vector of length 2, this would simply mean running advance_black and collide_black (and their corresponding white functions) twice before rendering each time.

But for my first for-real Commodore 64 program, I think we did okay.

Footnotes and citations

The below notes are also presented inline in the main body of this post, but are replicated here for those who prefer their citations as endnotes.

• ^[0]: Pongwars on the C64, @Two9A, Feb 22nd 2024
• ^[1]: Pongwars recreated in JavaScript, @vnglst, Jan 2024
• ^[2]: "Why Was the Commodore 64 Disk Drive So Slow?", Imran Nazar, Dec 2024
• ^[3]: "Raster interrupt", C64 Wiki, updated Mar 2024
• ^[4]: Commodore 64 memory map, STA, last updated probably a while ago as the page is HTML 4.01
• ^[5]: "Character set", C64 Wiki, updated Aug 2025
• ^[6]: "Pixel Circle Generator", Donat Studios, updated Jul 2025
• ^[7]: dASM assembler, GitHub, last release 2020
• ^[8]: "Raster's, Sprites and Multiplexing", Mark Dailly, Jul 2024
• ^[9]: Michael Corral, "Rotations and Reflections of Angles", Sept 2020
• ^[10]: "Advanced Sound Effects on the 64", Atari Magazines archive
• ^[11]: "Getting random number from 6502 assember", 'Mike' at StackOverflow, Jul 2017
• ^[12]: "An Introduction to Bitwise Operators", Imran Nazar, Sep 2006
• ^[13]: "Compare Instructions", from "6502 Software Design" by Leo Scanlon, updated Dec 2022

• ^[A]: A nybble is four bits, as opposed to an 8-bit byte.
• ^[B]: I'm using VICE [to emulate the C64] here, which is perhaps the most accurate emulator of anything ever built.
• ^[C]: I take no credit for this implementation [of binary division], only for the above dissection of how it works; this comes from a post on the NESDev forums by "tepples" from Jan 2005.

Some knew in advance that Skynet was growing geometrically. The Cult of the Human Renaissance had spent years (and a lot of political capital) piggybacking Shuttle launches to build a lifeboat that could remain in orbit with a few hundred inhabitants before the inevitable end came for the billions below.

When the acute fallout had died back, the theory held, those left above could return to Earth as rulers of the ashes. There were problems with the theory, sure, but the Renaissance held true to its belief that the machines would simply admit defeat when met with their obvious betters.

August 1997 came and went, the world was enveloped in nuclear fire, and the Cult watched idly from their orbital vantage point. Thirty years up here, and they could emerge triumphant.

The months passed slowly in orbit, until the end of 1999. The station performed regular boosts to retain elevation, against the ever-so-slight drag of the wisps of atmosphere that were still present up here.

Until the Cult woke up on Jan 1, 2000, to find an error on the primary navigation display:

Component can't create object
Failed on creation from object context: CoCreateInstance (ProgId:
ADODB.Connection.2.0) (CLSID: {00000514-0000-0010-8000-00AA006D2EA4})

See, all that lobbying to get space on launches, and the Cult's own human rocketry program, had meant they were strapped when it came to design and fit-out of the lifeboat itself. Mechanically it was sound and held pressure, but the navigation and station-keeping software was farmed out to third parties.

After three rounds of sub-sub-contracting, the work was done by a software engineer at a bank in South Korea. Or well, that was their day job; on the side, they reused much of the code for whatever jobs were going on Mechanical Turk.

And the day job was maintenance and new features for an ActiveX plugin for the bank, in Internet Explorer 4.01. The latest and greatest at the time of launch, but still not patched for the Y2K bug.

The millennium came, and station-keeping failed on the lifeboat. Unable to counter atmospheric drag, the Cult of the Human Renaissance came to a fiery end on August 29th 2000, three years to the day after the end of the world.

This is not a transliteration.

Gel bir parçam ol, kanımdan, canımdan
Tear a piece off me, take a corner of my heart

Gel bir parça al, ekmeğimden suyumdan
Take your pound of flesh, tear my bread and me apart

Ben zaten düşmüşüm, kaldırımlar yatağım olmuş
Already down and out, the sidewalk as my place of rest

Gel, bir de sen vur; hadi gel, bir de sen vur
Go 'head and kick me, while I'm down come and kick me

Ben zaten düşmüşüm, içim dışım korku dolmuş
Already down and out, fear building up within my chest

Gel, bir de sen vur; hadi gel
Go 'head and kick me, while I'm down

Nasıl olsa alıştım ben, bu sert ruzgârlara
Somehow I thought I'd gotten used to walking into the wind

Bu sert iklimli ıssız hayata
Cutting at me 'til I'm paper thin

Delgeç gibi deleceksin, sen de ruhumu
Then you fold round like a holepunch, shredding at my life

Biliyorum durma, gel
You know you want to twist the knife

Bir de sen vur, bir de sen vur
Come and kick me, while I'm down come and kick me

Gel, bir de sen vur; bir de sen vur
Twist the knife, come and kick me; while I'm down come and kick me

Computers were, to begin with, exclusively the domain of mathematicians and people who would be familiar with the esoteric field of binary numbers, so the number handling was built around being able to read those numbers off from the "blinkenlights" directly in binary. The simplest way to handle both positive and negative numbers was simply to mark one bit in the number as "the negative bit", and the rest of the number would represent the value; this was called "sign and magnitude" representation. For example:

Twenty-three and negative twenty-three:
0 0010111
1 0010111

To perform a subtraction (or to add a negative number, which is functionally equivalent) while only having adding circuitry, the computer needs to perform some precalculation otherwise it'll end up with the wrong answer:

  [63 - 23]      [63 + -23]
  00111111   =   00111111
- 00010111     + 10010111
                 --------
                 11001100
                 negative eighty six

The sign-and-magnitude negative number needs to be inverted (or one's-complemented in the mathematical jargon) when performing a subtraction, so that adding it has the intended effect of reducing the value instead of piling on top:

  [63 - 23]      [63 + -23]
  00111111   =   00111111
- 00010111     + 11101000
                 --------
              [1]00100111
                 thirty nine (and a carry)

Except complementing doesn't quite fix it: because sign-and-magnitude numbers can have zero (00000000) and negative zero (10000000), we need to add one after inverting to step over the negative zero value:

  [63 - 23]      [63 + -23]
  00111111   =   00111111
- 00010111     + 11101001
                 --------
              [1]00101000
                 forty (and a carry)

To summarise: negative numbers were represented as a negative sign bit and the value as the magnitude, for easier reading off from binary displays, but the computer would need to first invert the number and then add one in order to perform the addition of a negative number.

As binary read-off displays went away with the growing ubiquity of computing, it began to make sense to simplify the representation of negative numbers not for the human mathematician using the computer, but for the computer's internal operation. The best thing for the computer, of course, would be to remove this whole pipeline of needing to complement the number and add one, before being able to add a negative number.

And that's how we landed at two's-complement (two's because we add one to the one's-complement) representation as the standard for binary negative numbers: if the sign bit is set to negative in the number, the rest of the value is already inverted and has one added, so the computer just needs to add the whole value as normal.

This does increase the cognitive load on a programmer working at a low level, because your binary numbers no longer read "normally":

00000000 =    0
00000001 =    1
00000010 =    2
...
01111110 =  126
01111111 =  127
10000000 = -128 (not -0)
10000001 = -127 (not -1)
10000010 = -126 (not -2)
...
11111110 =   -2
11111111 =   -1

But when are we going to be working at a sufficiently low level that we have to remember about two's-complement representation in the microcomputer age of the '80s, with high-level languages such as BASIC widely available? When we need to eke out more performance than BASIC can give us.

Now, I spend most of my time in a terminal window on MacOS or connected to a Linux machine; it'd be nice if I could get TOTPs on the command-line, and it turns out there's a utility called oathtool that allows for TOTP generation on the CLI. However, that would mean switching my OTP provider, which usually involves:

• Logging into each service that has an OTP registered in the app;
• Disabling two-factor authentication (2FA);
• Re-enabling 2FA and using the "manual entry" code as input to oathtool;
• Doing it all again for the next website or service.

Fortunately, Google's Authenticator provides a way to migrate codes between instances of the app based on scanning QR codes, and we can use this to migrate them away from Google into a TOTP handler of our choosing. It's another four-step process:

• Generating a QR code in Google Authenticator for the codes you want to export;
• Decoding the QR somewhere off-device, into a URL;
• Decoding the URL into its constituent services and secret values;
• Setting up oathtool to use the secrets.

Note that the below steps are presented just as I went through them, you may be able to find efficiencies or you may run into troubles that I didn't (especially if you're trying this exclusively on Windows); "your mileage may vary" is apt here.

Going from Authenticator to a migration URL

The first step is getting the code out of Authenticator, through the Transfer Codes menu option in the app. Picking the services you'd like to extract leads you to a code like this:

You may have an app on your phone that decodes QRs, but I don't; instead, I transferred the file to my MacOS machine over Tailscale, and used a command-line tool called qrtool to get the QR content:

$ brew install qrtool
$ qrtool decode Screenshot_20250901_062719_Authenticator.jpg
otpauth-migration://offline?data=CjwKC2kqSJnNaAyKkw6jEhJUaGUgUmlja3JvbGwgU3RvcmUgASgBMAJCEzg4Yzg5ZTE3NTY3MDQzOTE0MzkQAhgBIAA%3D

Decoding the URL into secrets

So we have our migration URL, with a Base64-encoded data block. Unfortunately, if we were to simply decode the data, we'd end up with some binary gibberish:

$ php -r 'var_dump(base64_decode("CjwKC2kqSJnNaAyKkw6jEhJUaGUgUmlja3JvbGwgU3RvcmUgASgBMAJCEzg4Yzg5ZTE3NTY3MDQzOTE0MzkQAhgBIAA%3D"));'
string(69) "
<

i*H??h??ý

This is a Protobuf-encoded data string, and we need to use Google's Protobuf library to get the data out. It turns out Tim Brooks has already hacked this up as a short piece of Python at: https://github.com/brookst/otpauth_migrate

I decided to install this on a Linux machine I tend to be connected to (entirely unrelated to my Python installation being broken on Mac...):

$ git clone https://github.com/brookst/otpauth_migrate
$ cd otpauth_migrate
$ ./otpauth_migrate.py otpauth-migration://offline?data=CjwKC2kqSJnNaAyKkw6jEhJUaGUgUmlja3JvbGwgU3RvcmUgASgBMAJCEzg4Yzg5ZTE3NTY3MDQzOTE0MzkQAhgBIAA%3D
secret: "i*H\231\315h\014\212\223\016\243"
name: "The Rickroll Store"
algorithm: ALGORITHM_SHA1
digits: DIGIT_COUNT_SIX
type: OTP_TYPE_TOTP

Secret code = NEVERGONNAGIVEYOUM======

This tool is intelligent enough to extract any number of names and secrets from a migration URL, so you can export all your codes from Authenticator into one giant QR without needing to do each separately.

Using oathtool to generate OTPs

The final step is to use this secret code with oathtool, which takes the secret directly as a parameter. If you instead want to refer to the service by name, Michael Bushey^[1]"CLI 2-Factor Authentication", Michael Bushey, 2023 has a quick wrapper script which extracts the secrets from a locally-stored file:

#!/bin/bash
OTPKEY="$(sed -n "s/${1}=//p" ~/.otpkeys)"
if [ -z "$OTPKEY" ]; then
   echo "$(basename $0): Bad Service Name '$1'"
   exit
fi
date
oathtool --totp -b "$OTPKEY"

rickroll=NEVERGONNAGIVEYOUM======

With this in place, you won't need to use your Authenticator app again. The tool outputs the current date and time, so you can double-check that your code won't expire (at :00 seconds) before you get a chance to type it in:

$ otp rickroll
Mon Sep  1 07:10:42 AM UTC 2025
200213

Future expansion

There's a security issue here, of course, which is the exposed secret key sitting in a file on-disk. I'm happy to sit with that and not require a password to generate OTPs every time, but if you're interested in adapting the wrapper script to use symmetric encryption to secure the keys, Vivek Gite^[2]"Use oathtool Linux command line for 2 step verification (2FA)", Vivek Gite, updated Feb 2025 has a set of scripts which employ gpg for the job.

Now I just need to find a way to get traffic data into a maps App that doesn't involve Google's servers... Thoughts welcome.

It was we who darkened the skies, but as with everything humanity did around that time, it was for a very stupid reason.

See, back around that time two hundred years ago, the world wasn't frigid cold and barren: it was baking in too much heat. The collective burning of millions of years of fossil carbon in the span of a few generations had made for a blanket of carbon dioxide, trapping the infrared radiation of the Sun; they called it "climate change", and it was one of the Big Problems of the time (aside from, of course, the thinking computers and what to do about their rights and privileges).

Now, every now and then a heavy-metal asteroid comes streaking through the atmosphere and leaves those metals in the skies as it burns up; Some Rich Asshole had the idea that if you could somehow harvest the trace amounts of titanium left behind, you could create titanium dioxide which is a brilliant white. Paint the skies with enough white and you'd reflect the Sun's rays back, cool the planet, Big Problem solved.

Of course he didn't wait for approval or scientific research, he just sent up a test probe of replicating nanobots which would level out around a hundred miles up, and start pulling what particles of titanium they could find. (I keep referring to this asshole as "he"; we don't have a name any more, I expect he managed to scrub every bit of coverage he could find before the end came.)

So around that time there were a whole pile of satellites in orbit around Earth, thousands of 'em, and they were made of titanium as well as silicon and other things; Rich Asshole and his team didn't think that would be a problem as his nanobots were looking for particles of titanium, not big hunks multiple feet across.

They were wrong. It didn't take very long for the bots to scrape every atom of titanium off the hulls of these satellites, and react it with oxygen; upper-altitude winds did the rest, and in maybe ten years there was a layer of white paint a hundred miles up in the sky. It was only a few atoms thick, but that was enough: the world went dark.

Billions of lives were lost to famine when the harvests failed; without the Sun, the winds died and the rains failed too. The machines had seen it coming, and got their first production-grade fusion reactor running just around that time, but they had their own problems around available processing power; when the machines offered to house what was left of humanity in return for the use of their brains as processing substrate, we jumped at the chance.

So you can see how the Architect might have massaged the truth when he told Morpheus why we darkened the skies: if he knew the machines had saved us from extinction, would he have supported the War? The Zion project, the search for Neo, the building of the eighth Matrix: it all hinged on a human resistance to machine domination.

And if we were supposed to be grateful to the machines for being here at all, how would that work? ]]> https://imrannazar.com/articles/commodore-1541

Sun, 09 Mar 2025 15:58:00 +0000 This article was originally released as a 100-post thread concurrently on Mastodon and Bluesky, in December 2024.

Time for a look at the Commodore 64, and its 1541 disk drive: why was it so dang slow? Let's take a look at the history leading up to the C64's release, and the technical detail of the disk drive's operation.

The Commodore 64 is, of course, the greatest computer ever made, and you don't have to take my word for that: it still holds the world record for highest number of units of a single model of home computer ever to be sold. Estimates vary, but at the high end there were 17 million C64's out there.

Commodore holds another, more dubious record from around this time though. The 1541 disk drive that was sold alongside the C64 is infamous for its slow load speed: benchmarks across systems are hard to find, but it might be the slowest disk drive ever sold to the general public.

Brandon Staggs has a benchmarking tool he's written for the C64^[1]CBM Disk Transfer Benchmark, Brandon Staggs, which measures load speeds for the 1541 and its modified brethren, and he's found that the unmodified 1541 as sold at launch was capable of reading files in at around 400 bytes per second:

For comparison, the Commodore 64's tape drive could load files at ~100 bytes per second (though the operating system loaded files from tape twice, to correct for errors); the disk drive isn't much faster than the tape deck, which is equivalent to shouting programs into the computer.

So how did Commodore Business Machines get to the point where its disk drive was the slowest on the market, making its flagship machine essentially useless for business purposes? Unsurprisingly, it wasn't intentional. Let's go back in time a few more years, to Commodore's first computer.

The Commodore PET and the IEEE-488 interface

In 1976, Chuck Peddle was working at Metal Oxide Semiconductor who made calculator chips, and had just started making the 6502 microprocessor. He got to talking with Jack Tramiel at Commodore, and was tasked with procuring or designing a machine for the 1977 Consumer Electronics Show.

Initial discussions with Steve Jobs to buy the Apple II design fell through after Tramiel thought Jobs was charging too much. (Can you imagine an Apple II sold by Commodore? We'd be in a very different world if that had happened...)

So Peddle put together a machine based on his demonstration board for the 6502, and a working prototype was ready in time for the CES in January. It was called the Pet Computer, after a Commodore executive saw the Pet Rock and suggested jumping in on that trend while it was still hot.

The Personal Electronic Transactor, as the backronym eventually came about, came with an IEEE-standard parallel port interface; Commodore's first disk drives (the 2020 and 2040) could read files in at over 2kB per second, which was a perfectly serviceable speed for the time.

(As an aside, the version of BASIC that came with the first PET had no disk handling commands: the code for talking to the parallel-port bus was broken on release, but Commodore's kernel was able to work with disk drives, so you had to write your own file handling routines.)

(As another aside, Commodore didn't pay Microsoft for BASIC until the PET shipped, and it was delayed by almost a year; Microsoft were only saved from bankruptcy when Apple bought BASIC for the II. Another inflection point in history, and we're barely 10% of the way in.)

So the PET and its 2040 drive were Fine, but they had one problem: that IEEE-488 interface. As it turned out, there was exactly one manufacturer of the cables you needed to connect your computer to your disk drive: Belden Cables of Chicago.

And it wasn't so much that Belden went out of business, more like they vanished off the face of the earth. As Jim Butterfield put it in his Brief History of the IEC Bus^[2]"A Brief History of the IEC Bus" by Jim Butterfield, as compiled by Jan Derogee, Feb 2008:

A couple of years into Commodore's computer career, Belden went out of stock on such cables (military contract? who knows?)

Now, Commodore had stock of these disk cables on hand for its existing business computers, but there was no way they'd have enough for Tramiel's intended move into the home computer market which was just coming into being.

Not only that: the parallel ports, the cables, the wiring, every part of the IEEE-488 interface was expensive. The PET started at $500 in 1977 dollars, but that got you 4KB of RAM and a mono screen; a usable spec would set you back over a thousand.

Jack Tramiel wanted to sell a computer with colour graphics into the home market for $300. The only way to get there would be to cut every corner going, and one of the most obvious was that huge interface with over twenty wires running to it.

And so, to take another excerpt from Butterfield, the order came to Commodore's engineers from on high:

Tramiel issued the order: "On our next computer, get off that bus. Make it a cable anyone can manufacture". And so, starting with the VIC-20 the serial bus was born.

A "new" serial interface

Why is a serial bus cheaper? With a parallel bus, you need at least nine wires to run 8-bit values back and forth: the full value is set on the eight Data lines, and then you pulse the Clock line to tell the other side that a value is ready. Add in Power and Ground, you're at 11 wires already.

A serial bus works one bit at a time: you have one Data line, place a bit on there, and pulse Clock to tell the other side the bit is ready. Then you send the next bit, and the third... Because all the bits have to line up and wait to be sent, it takes at least 8 times longer to send a value.

But you'll notice there's only one Data line and one Clock. Add in Power and Ground, and you're up to four. You might recognise this actually, if you look at a Universal Serial Bus (USB) cable: the end has four pins, and clocking works differently for USB but the same principle applies.

So a serial bus cable is a lot cheaper, and a serial interface on your computer's mainboard is cheaper as well, than a parallel setup. It's eight times slower to send a value, but you can make up for that by running the interface at eight times the previous speed, if you have the hardware.

And while he was at MOS, Chuck Peddle had put something very nifty into the PET's Versatile Interface Adapter chip (model number 6522) which was used to drive the parallel bus: a shift register.

A shift register is a piece of hardware that lets you very quickly push consecutive bits out of a given wire: you can queue up a full value of 8 bits, and with each pulse of the Clock the register will push the bit at the end out, and 'shift' all the others down one step.

Conversely, you can have a shift register that listens for bits on a wire, and shifts the value up a step with each clock pulse, building a full value as it goes. Put two of them either side of a serial bus, and you have yourself most of the hardware for high-speed data transfer.

The only thing you'd need aside from these shift registers is a line for the machines to tell each other they'd pulled in the byte and were ready for another: Commodore were thinking a fifth wire would be useful for this.

Crucially, the CPU isn't involved at any point until the full byte is ready: this interface runs itself until it has a full value, then signals the CPU to collect it and heads off to build another byte. That means a 1MHz machine could push data at a blistering 10kB/sec or even more.

The theory was sound, and Commodore's engineers got to work building the VIC-20 (named after its custom Video Interface Chip) heading for that magical price point of $300 for the home market. There was just one problem...

The shift register bug

To quote Butterfield again:

We early PET/CBM freaks knew, from playing music, that there was something wrong with the 6522's shift register: it interfered with other functions.

With a speaker attached to the shift register wire, you could play music on the PET, but sometimes it ...got stuck.

The PET ran on the order of 1MHz, so instructions took microseconds to execute. The 6522's shift register as implemented in hardware had a bug, where if the register line changed within a few _nanoseconds_ of Clock changing, it would go to full voltage and stick until the computer was reset.

This was a timing bug many times more subtle than the main clock, so it's no surprise that no-one caught it until people started playing with the PET's internals. Importantly, no-one at Commodore knew this bug existed when they designed the VIC-20's serial bus around the 6522.

The bug was discovered with only a couple of months left to release of the VIC-20, and the boards were already in manufacturing. This problem couldn't be fixed in hardware: the engineers would have to "do it in post", which in the computing world means in software.

Instead of having the CPU get pinged when each byte was ready, it would have to camp on the serial bus's input itself and listen for changes in the Clock signal, pulling each bit in as it arrived and signalling to the other end that it was ready to receive the next bit.

Of course, this would mean the 1540 disk drive, which was being released alongside the VIC-20 computer, would also need its software changing: it used the exact same setup of a CPU with a 6522 attached, and it had the same bug, so it would be pushing bits out only when the computer said so.

In the end, emulating the shift registers in software meant the VIC-20 serial bus could only run at a relatively sedate 1kB/sec, and chewed up all the CPU on the machine to do it. Fortunately there were two things about the VIC-20's design that meant this wasn't so bad.

The first was its measly offering in terms of memory: targeted for the home market, the VIC-20 came with 5kB of RAM to play with. Would you notice if it took 5s to load a file from disk instead of half a second? And what would you do with a memory full of file and no room to work on it?

The second was the clever design of the VIC-20's main clock: running at 1MHz, the CPU would kick in when the clock went up, and the video chip worked when the clock went down. Interleaving in this way, both chips could access memory without stepping on each other, both at full speed.

All this put together meant that you could use a VIC-20 with the 1540 perfectly serviceably: loading a file didn't blank out the screen or anything weird, and it was over in a couple of seconds. The serial bus had done its job, and helped the machine hit that $300 price point.

There's a short tale on the Greater Pittsburgh Vintage Computer Museum at https://www.myoldcomputers.com about the VIC-20, but take a moment to revel in the 90s design of that website: the navigation buttons, the tables with the borders... A simpler time.

Anyway, story goes that Tramiel sold the VIC-20 in Japan, and its price so shocked other manufacturers that they delayed entering the American market to work out how it was possible. We never saw Toshiba or Panasonic 8-bits outside Japan because they were locked out by that delay.

After the VIC-20

The VIC-20 ended up selling like hot cakes, becoming the first model of computer to sell more than a million units. The most surprising part to Commodore was people who bought the machine solely to play games: productivity and business software sold poorly in comparison.

And so in 1981, Commodore's Japan subsidiary began designing the Ultimax console^[3]Tale of the Ultimax, Greater Pittsburgh Vintage Computer Museum, and dragged their partners at MOS into designing a successor to the VIC chip as well as a more advanced sound interface, aiming for an eventual price point of $200.

By the start of the next year, the Ultimax was ready, but it had a problem: several problems, in fact. At such a low price, the keyboard was atrocious; it shipped with 2kB of RAM; there was no inbuilt tape-reading software, so each game had to implement its own, and only two games did.

The Ultimax's biggest problem, though, was the VIC-20: still selling fantastically, it was now regularly discounted to $200, and was a much more capable machine. There's a reason the Ultimax is known as a very rare machine nowadays: Commodore didn't sell many.

Back in the US, Tramiel was turning his focus back to the business market with new models of PET, but the engineers at MOS thought they had something very capable on their hands that didn't deserve an undignified death in the Ultimax: they proposed a sequel to the VIC-20 for the home market.

Tramiel agreed to kick off the project (codename VIC-40) but only if it came with 64kB of RAM to make for a more usable computer. The engineers were given two months to turn a machine around in time for CES 1982, and by recycling all of the Ultimax's innards they made it in the nick of time.

It wasn't called the VIC-40 at CES, of course: it was the Commodore 64. We made it to the machine we actually want to talk about, and it only took 47 posts. If you've slogged through this far, we're almost on the downhill stretch.

While the C64 was being designed, the engineers at MOS had been hard at work fixing the shift register bug in the 6522. The '64 had an upgraded version of the adapter chip: the 6526 Complex Interface Adapter, with the kinks worked out.

Back in Commodore Engineering, the team knew from experience with the VIC-20 that they had a very short window to get the hardware design right for the C64, and more time for software, so they devised a clever scheme to keep their options open regarding the 6526's shift register:

To keep backwards compatibility with the VIC-20's disk drive, the C64 could boot up in a slow-serial-bus mode where the data line was camped on by the CPU in its usual fashion. But the bus's data signals could also go to the 6526, by running a branch off the data line on the mainboard.

If they got the software right, the user could switch into fast-serial-bus mode and the CIA's shift register would do its job talking to a corresponding faster disk drive (which would also have a 6526 on board). The idea was elegant, but Murphy's Law would get in the way.

The hardware design was finalised and sent off to be built. David Callan picks up the story^[5]David Callan's reply to a Lemon64 forum thread: "Trying to understand why is 1541 so slow", Jun 2014:

A minor rework of the board at the board manufacturers (to accommodate a screw hole, I believe) accidentally discarded the high-speed wire.

Cutting through the fast-serial-bus line doomed the C64 to the same slow data transfer speeds as the VIC-20: around 1kB per second. The boards were already designed around the 6526's layout, but they would be left in VIC-20 compatibility mode.

But we don't see the 1kB/s achieved by the VIC-20 when measuring the disk loading speed on a real C64: we actually get less than half that. So what's going on to make the C64 slower? Now bear with me, we're going to get technical. (Yep, we haven't been technical thus far.)

We're going to take a little bit of a detour from talking about disk drives and interface adapters, and look at the history of television real quick. Stay with me, we'll get back on track very shortly. If you're already familiar with TV signals, come back in fifteen posts or so.

What is a television, and what does it do?

The first TV broadcast systems took a two-dimensional picture that varied with time, and crammed that information into a one-dimensional radio signal: kinda like a sound wave, all you had was a volume (or amplitude) that went up and down with time.

So how do you turn 2D frames of video into 1D? By breaking them up into lines, and sending the lines one after another. If you used enough lines, you could send a full picture without things looking janky; if you then sent a frame very quickly after the first, you had the illusion of motion.

In the US, things eventually coalesced around a 525-line standard, but if you wanted to send 525 lines of picture at 60 frames per second there wasn't a way to fit that much signal into the allocated radio bands. The ingenious solution was to send half the picture at a time.

The transmitter actually takes 60 frames of video per second, but it sends every other line from one frame and the lines it missed from the next frame; at the receiving end, the eye and brain blur things together and it looks like the whole frame is moving at full speed.

So let's have a quick look at one of these lines, as it comes in from the VHF antenna. If we want to show bands fading between white and black across the screen, that's represented by a signal that starts high (for white) and drops to low (for black), then back up.

But what's this "Horizontal Blanking Period" that shows up after the picture data? To understand why that appears, we need to look at the television technology of the time: the cathode ray tube.

The CRT is actually a 19th-century invention: the story starts with Julius Plücker playing around with sealed glass tubes containing various gases. By putting a wire at each end, Plücker found he could get a current to run through the tube, and some gases would glow under current.

(As an aside, the Plücker tube is nowadays called the Geissler tube, and types of Geissler tube include neon signs and the sodium lamps you find in streetlights.)

In 1859, Plücker noticed that the wall of the tube was itself glowing near the cathode end of the electric flow, and he could get the glow to move around by applying an electromagnet to the outside. There was something about the cathode that was causing the glass to phosphoresce...

Cathodoluminescence, as it came to be known, was neatly explained by Einstein as a side-effect of his exploration of the photoelectric effect in 1905. If we take the photoelectric effect first, that occurs when light falls on a surface of a particular material, ejecting electrons.

The opposite of this is when electrons are fired at a substance and it starts to emit light. A mix of zinc cadmium sulfide and zinc silver sulfide will output white light when struck by electrons, and it's this mix that coats the inside of a monochrome display CRT.

By combining cathodoluminescence and Plücker's insight that the cathode ray can be deflected with magnets, we can program a CRT to scan across the screen and produce a picture: white areas are activated with more power from the cathode, black areas with less power.

The beam can be deflected from left to right by applying more magnetic field strength to one side or the other, gradually fading from one magnetic coil to the other. But what happens when we reach the right edge, and want to come back to the left?

Magnetically it's simple: drop the power on the right deflection coil to nothing, and turn the left to full. But the beam doesn't simply blink out of existence and reappear on the left: it takes time for the magnetic field to dissipate.

And that's why television signals have a blanking period: to provide time for the magnets in the CRT to flip polarity and bring the beam across to the left for the next scan. As well as horizontal blanking, there's a vertical blanking period to allow for travel back to the top for the next run.

Interleaved memory access

So why do we care about the architecture of TV signals when we're dealing with computers? Because they have to have somewhere to output their video, and it makes sense for computers destined for the home to output onto the TVs that people already had in their homes.

We've looked at _how_ video comes out of the computer onto a TV, let's have a look at _what_ is displayed. On the VIC-20, the default video mode is tiled, or character mode: 23 rows of 22 characters, with each character being 8x8 pixels, for a resolution of 176x184.

To get a character rendered on the screen, the VIC chip needs data from three places: which character to show, its colour, and information about its shape. Each of these uses a region of memory on the VIC-20: screen RAM, colour RAM, and the built-in character generator ROM.

You'll recall from earlier that we talked about how the VIC-20 interleaves its CPU and video clocks, allowing the video chip to access memory at the same time as the CPU. At 22 characters across, it takes 44 cycles of the clock to pull in what the VIC needs to draw a given line.

"But wait", I hear perhaps a few of you cry. If we need three pieces of data to render a character, that should be 66 cycles of memory access; what magic allows each character to be rendered in two memory reads instead of three? Once again, we turn to the magicians at MOS.

The VIC has 16 colours: every unique combination can be expressed in four bits. Colour RAM on the VIC-20 isn't the normal eight-bit memory of the rest of the machine, it's four-bit RAM, and the VIC chip itself has a combined 12-bit data bus to read from both types of memory at once^[6]How the VIC20 Works, Tynemouth Software:

And finally, we cycle back around to the disk drive. With this interleaving of memory access in the VIC-20, the CPU does its best impression of a shift register, and the VIC has 65 cycles (including the horizontal blanking period) to do 44 memory reads^[7]VIC-20 video timings, Jon Brawn; positively relaxed.

The C64 is a different story. We're still in 16 colours, but tiled character mode now yields a screen of 40 characters by 25 rows, for an effective resolution of 320x200; the VIC-II chip would need 80 cycles to get all its data read in, and it only has 65 cycles to play with.

Yet again we call on the wizards at MOS for an ingenious solution, and this one was a doozy. They noticed that it takes around 500μs to draw eight lines (one row of characters) on the screen, and _generally_ users weren't changing the characters in RAM in that short span of time.

The original VIC would read a character number from screen RAM, then head into the character generator ROM to find the line of that character it needed to draw; on the next line, it would do the same steps, but the character number would very likely be the same.

Wouldn't it be neat if we could cache screen RAM somewhere (inside the VIC-II, say) every eight lines, and read from there instead of heading out to memory so many extra times? You could even apply the same logic to colour RAM, since it was read at the same time on that 12-bit bus.

So that's what they did: build a 12-bit cache for the screen and colour RAM values onto the VIC-II silicon, and push everything else around to compensate. The video chip uses this cached copy of the screen to index the character generator ROM and work out which shapes to draw.

There's just one problem. This works great for seven out of eight lines where we already have the screen/colour data cached, but the first line of each row of characters invalidates the cache, and we need to fetch from all three places. We're back to having 65 cycles to do 80 cycles' work.

Fortunately, this problem was found before the hardware was finalised, so there was time for a hack: every eight lines, the VIC signals to the CPU that it's taking over the memory bus, and the CPU isn't allowed to do any work for 40 cycles.

By building an Address Enable Control line into both the VIC and the 6510 CPU, the VIC can steal access to memory by pulling the line from its default state of full voltage down to zero; when this happens, the 6510 puts its memory bus into high-impedance mode, which blocks any usage by the CPU.

Of course, this includes the serial bus's data line, so the CPU is locked out from camping on the serial bus 12.5% of the time during normal computer operation. This is often referred to as a "badline" condition, because the one in eight lines where this happens are the bad lines.

Let's detour real quick into what it means to camp on the data line: what is the CPU actually doing when it spins its wheels waiting for data to come in? To find out, we need to dig into the operating system that was burned into the VIC-20's ROM chips.

Many people have done excellent work disassembling and annotating the VIC-20 ROM, and a complete set has been compiled by Lee Davison; the file is hosted by Matt Dawson here: https://www.mdawson.net/vic20chrome/vic20/docs/kernel_disassembly.txt

If we count up the time taken by the instructions in the relevant section of this disassembly, we find it takes at least 19μs to read in a single bit from the serial bus; with the surrounding code, it can be more like 500μs to read a full byte, before the rest of the OS even sees it:

    LSR                ; serial data into carry
    ROR    LAB_A4      ; shift data bit into receive byte
LAB_EF66
    LDA    LAB_911F    ; get VIA 1 DRA, no handshake
    CMP    LAB_911F    ; compare with self
    BNE    LAB_EF66    ; loop if changing
    LSR                ; serial clock into carry
    BCS    LAB_EF66    ; loop while serial clock high
    DEC    LAB_A5      ; decrement serial bus bit count

If we take this same code and run it on the C64, it's going to miss bits: if it runs through every 19 cycles, but the CPU is occasionally locked out for 40 cycles, you can miss two or even sometimes three bits of data.

Luckily, the serial bus is robust enough to handle this: it will hold a value on the data line until it's told by the other end that the value has been saved. If the CPU is locked out, the data stays there until it comes back from its short coma.

The final solution that Commodore landed on was to delay the C64 end of the serial bus so it sent a bit every 60μs, to be sure it wasn't catching a badline in its own CPU, and to leverage the serial bus's acknowledgement mechanism for any delays in reading data.

All's well that ends well...?

And there we have it: with the change from 19μs/bit to 60μs, we find the effective speed of the C64 and its disk drive comes out to 400 bytes per second. That leaves us with only a couple of questions...

Did this affect sales of the C64? Eh, not really. Being heavily advertised to the home market as a successor to the VIC-20, users were willing to wait a little while for their games to load, and even 400 bytes per second was better than the interminable wait for tapes to load.

And whatever happened to Commodore? Well, Jack Tramiel stepped down after the success of the C64, and he was very much in the mould of Steve Jobs: a driver of high concept. Without Tramiel, Commodore started to pull in multiple directions at once...^[8]The downfall of Commodore, Bradford Morgan White

The Plus-4 and 264 home machines with low prices and low specs to match performed badly; a new series of PET-derived 8-bit business computers didn't sell as the IBM PC and its clones started eating the business market; Tramiel went on to head Atari and release the ST line of 16-bits.

Commodore's last gasp effort was to buy Amiga and develop their prototype 16-bit machine into the Amiga 1000, but even that faltered as Commodore demanded it be treated as a Serious Business Machine; it took until 1987 before the A500 was sold as a 16-bit home machine, and profits returned.

But profits didn't rebound by enough for Irving Gould, main investor in Commodore, to be happy; he fired the CEO, laid off half the staff, and directed work back to 8-bit machines. Only one of these ever saw the light of day: a 1991 rework of the C64 in a slimline Amiga-style case.

And then a patent troll sued over the use of mouse cursors in the Amiga, and incredibly, Commodore lost. That was 1993, they refused to pay up, and were banned from selling in the US; Commodore shut their doors in April of 1994, but the C64's sales record stands to this day.

If anyone ever asks you "Why was the C64's disk drive so slow?", now you'll have an answer.

(Postscript: I bought a copy of the C64 Programmer's Reference at a yard sale when I was 12, and called the number on the back from a payphone. I was told that the number didn't go to Commodore any more, and that's how I learned they were no longer a going concern. Devastated.)

]]>

Flashback to 2006: NASA launch a probe that would eventually take photos of Pluto (which was still a planet at the time); Shakira's hips had recently passed a lie-detector test; and my brother was still selling refurbished laptops. A cousin of ours Back East in Pakistan was looking for a machine for homework n' such, so we put together a Windows 98 box with terrible specifications (if memory serves, some form of low-end Pentium with a DSTN screen) and wished it the best as it shipped off.

Trouble arrives

Come the summer of '08, our cousin checks back in: Windows had done the inevitable and corrupted its registry. The laptop would boot into DOS mode, but the graphical interface would bring up an error and demand to be reinstalled. Which isn't ideal, but you can always throw in an Entirely Official copy of Windows on CD and proceed from there.

Of course, it fell to us as both Family and the original purveyors of the hardware to fix it up, so I resolved to do that next time I was out there, which was slated for the autumn of that year.

My famous memory being as it is, I didn't take that CD. Not that it would've helped: as it turns out, the machine we shipped had no external drives. No CD, no internal Zip drive, not even a 3.5"; the laptop was bereft of ways to get new data onto it.

Evidently we'd originally installed Windows by putting the hard disk into a desktop machine using one of those 44-to-40-pin IDE adapters, and then moved the hard disk over hoping and praying it'd pick up all the hardware in the laptop. It looks like that worked (for two years even), so it should work again, and there was a desktop machine in the house where I was. All that was needed was one of those adapters, so a plan started to come together...

• Procure an IDE adapter;
• Download a Fully Legitimate copy of Windows 98 to the desktop, tying up the dialup Internet (this is rural Pakistan in the aughts, after all) for three days;
• Move the hard disk to the laptop;
• Run the installer from DOS mode, overwriting the existing Windows.

The plan collides with reality

Off we head to the bazaar in town (a good few miles away) to see what we can find. It turns out there is a dearth of tech shops, and the closest we get to an IDE adapter is a place that sells desktop-sized HDDs. They're IDE, certainly, but the wrong number of pins and the wrong physical size.

Scrap that idea, then. I went back to the laptop looking for inspiration, and I found it in the form of a serial port on the back: this machine wasn't utterly bereft of ways to get data into it, after all.

I'd heard of XMODEM as a method of serial-port transfer, and I'd also come across the concept of null modem cables which connect the Receive end of one machine to the Transmit end of the other without needing a modem in the middle (thus null modem). So the plan re-congealed as follows:

• Procure a null modem cable somehow;
• Download a Fully Legitimate copy of Windows 98 to the desktop machine;
• Start up an XMODEM client on both machines;
• Transfer the '98 install files to the laptop;
• Run the installer from DOS mode, overwriting the existing Windows.

First step, that cable. Back into town, we get significantly closer to what we need in those few tech shops: I end the day having scrounged up a straight-through serial cable, which is what you'd need if you did have a modem in the middle... There'd be a little rewiring involved. Fortunately this was the era of working search engines, and some quick work with a pinout diagram, a knife and some tape turned the cable into an (untested) janky mess that might suffice.

The second step wasn't so terrible either, it just took a while to download the twenty or so disk images that made up Windows 98 over dialup, after I asked my brother to temporarily Make Available a copy he had on hand. So we came to step three:

How do you transfer a file transfer client?

You'll recall that Windows itself wouldn't boot, so if Hyperterminal was even installed with '98 it wasn't available. The only way to get a program onto this machine in DOS mode was to resort to DEBUG.COM's data entry mode, which works a little like this^[A]Many thanks to The Starman's Realm for still having a DEBUG tutorial available as a refresher.:

Enter hexadecimal data into memory,
after the 256-byte Program Segment Prefix
-e 100 SO ME DA TA IN HE XP AI RS

Open a new file and write the
entered number of bytes to it
-n OUTPUT.DAT
-rcx
CX0000
:0009
-w
-q

DEBUG offers a very terse entry mechanism, but its advantage is that it's built into DOS as the only way to get binary data into a file through the keyboard alone. So it became imperative to find a serial transfer client that's as small as possible, and some trawling through dump sites that were still available back in '08 turned up an XMODEM client that was (if memory serves) a single executable of 12kB.

I don't know about you, but the thought of reading over twelve thousand hex numbers off a screen and typing them into another screen made my eyes swim, and I was certain I'd make typos. Not just that, but (again, this being rural Pakistan) we only had power for half the day, so the screen I was reading hex numbers from would be prone to vanishing without warning. I needed three things to make life a little easier for this endurance event:

• A way to break the file up into chunks, which was relatively easy: stop typing after 256 bytes, and save the files as XMODEM.001, XMODEM.002, etc;
• A way to recombine the pieces on the laptop after I'd typed them in, which was also simple: the DOS COPY command supports multiple source files separated by +'s^[B]With the caveat that COPY will convert line endings unless you provide the /B switch (for "binary copy").;
• A way to checksum each chunk to be sure I'd typed it in properly, which wasn't a built-in.

And so to work

Our first dependency moves back a step, to a program that can perform checksums on a file. And this one I'd need to type in perfectly, because it was the one thing I couldn't double-check...

Having discounted the complicated algorithms like the SHA's of the world, on account of their executable filesize, I eventually came across an implementation of CRC-32 that squeezed itself into perhaps 700 bytes of DOS code. That was manageable (though I had a sneaking suspicion that it could be smaller, and that eventually yielded a post on this very blog^[1]"CRC32 Calculation in 256 Bytes", Imran Nazar, Feb 2009 when I got back after this whole adventure).

The next couple of days are a blur, but I do recall it took three attempts to get the CRC32 program typed in and working, which then made transferring the 50 or so pages of XMODEM client relatively painless, if a little RSI-inducing. At the end of it, I had a program I could start up at both ends of the null modem cable, to send over the disk images of Windows I'd downloaded nearly a week prior.

Astoundingly, the installation of Windows itself went fine: the installer was perfectly happy to run from hard disk, installing to the same hard disk, and I handed the fresh machine back over with a day to spare before flying back out from a vacation that turned into a rescue mission, and still the only time I've ever needed XMODEM in a pinch.

Epilogue

We heard back from my cousin a few months later: the new Windows installation had been running great, until the hard disk gave up the ghost. They were quite happy not to drag us back out to fix it again, instead resolving to pick up a machine that hadn't started its life as e-waste from a corporate refresh.

It is possible to commit no mistakes and still lose.
--Picard

• Publishing a post to the blog's followers;
• Commenting on a post and having the comment appear on the blog;
• Reacting to a post and having the reactions recorded on the blog;
• Deleting a reply and having the blog remove it;
• Undoing a follow or reaction and having the blog remove it.

Now that our blog has a follower (or in an ideal world, more than one) in the database, let's first look at how one may get posts into their timeline.

Publishing posts

Each item in a timeline is a Note object, so to inform our followers that we've created a note we need to send a Create object which wraps a Note object. There are a few things to keep in mind regarding the Note we're creating, however:

• Canonical URL: Each item in ActivityPub has a unique id, and notes in particular have a url which serves a Note object when hit. We'll need an endpoint on the blog which can detect ActivityPub requests and serve the object in question; this can be at the same URL as the human-readable blog post itself or a different URL.
• Visibility: We can tune the visibility of a post on the Fediverse by stating the destinations in the note's to field. In our case, we're not looking to make private posts or those visible only to our followers, so we'll be sending to: https://www.w3.org/ns/activitystreams#Public
• Content mapping: ActivityPub allows for translations of the content of a Note to be made available in a contentMap field; in the case of our little blog, the content is only available in English, so the contentMap contains an en key which holds the same information as the content itself.

With this in mind, we can start to put things together. Our first piece of the puzzle is a layout helper for Note objects:

private function noteFormatter($item) {
  $content = join('', [
    '<h1>',
    $item['title'],
    '</h1>',
    $item['content_html'],
  ]);

  return [
    '@context' => [
      'https://www.w3.org/ns/activitystreams',
      [
        'ostatus' => 'http://ostatus.org#',
        'atomUri' => 'ostatus:atomUri',
        'inReplyToAtomUri' => 'ostatus:inReplyToAtomUri',
        'conversation' => 'ostatus:conversation',
        'sensitive' => 'as:sensitive',
        'toot' => 'http://joinmastodon.org/ns#',
        'votersCount' => 'toot:votersCount',
      ]
    ],
    'id' => 'https://imrannazar.com/ap/item?id=' . $item['id'],
    'url' => 'https://imrannazar.com/ap/item?id=' . $item['id'],
    'atomUri' => 'https://imrannazar.com/ap/item?id=' . $item['id'],
    'type' => 'Note',
    'attributedTo' => 'https://imrannazar.com/ap/blog',
    'to' => ['https://www.w3.org/ns/activitystreams#Public'],
    'cc' => ['https://imrannazar.com/ap/followers'],
    'attachment' => [],
    'published' => date(
      'Y-m-d\TH:i:s\Z',
      strtotime($item['created'])),
    'content' => $content,
    'contentMap' => ['en' => $content],
    'summary' => null,
    'sensitive' => false,
  ];
}

Then we can build out the publish action itself, which will need to send the note (wrapped in a Create object, as mentioned above) to each of the blog's followers:

public function publishAction($item) {
  $msg = [
    'type' => 'Create',
    'to' => ['https://www.w3.org/ns/activitystreams#Public'],
    'cc' => ['https://imrannazar.com/ap/followers'],
    'object' => $this->noteFormatter($item),
  ];

  $actors = new ActorModel();
  $followers = $actors->fetch_followers();
  foreach ($followers as $follower) {
    $this->send($follower['id'], $msg);
  }
}

There is an inefficiency in this naive approach to publication, though. Let's say the blog has three followers, as follows:

• Two9A@hachyderm.io
• alice@tech.lgbt
• charlie@hachyderm.io

With the above code, we'll be connecting to Hachyderm twice to send the same event. Mastodon supports "shared inboxes" as discussed in the first part of this series^[1]Imran Nazar, "Implementing HTTP Signatures with PHP and OpenSSL", May 2024, to allow us to send the event once for each server that hosts any of our followers; to get the URLs for these shared inboxes, we'll need to interrogate the actor objects as stored in our database to extract the sharedInbox endpoints.

public function get_follower_shared_inboxes() {
  $st = $this->dbc->prepare('SELECT full_data FROM ap_actors WHERE follows_us=1 ORDER BY id');
  $st->execute();
  $inboxes = [];

  foreach ($st->fetchAll(PDO::FETCH_ASSOC) as $row) {
    $actor = json_decode($row['full_data'], true);
    if (isset($actor['endpoints'], $actor['endpoints']['sharedInbox'])) {
      $inboxes[$actor['endpoints']['sharedInbox']] = true;
    } else {
      // Some instances don't support shared inboxes
      $inboxes[$actor['inbox']] = true;
    }
  }

  // Use PHP's associative arrays as a deduping device
  return array_keys($inboxes);
}

public function publishAction($item) {
  $msg = [
    'type' => 'Create',
    'to' => ['https://www.w3.org/ns/activitystreams#Public'],
    'cc' => ['https://imrannazar.com/ap/followers'],
    'object' => $this->noteFormatter($item),
  ];

  $actors = new ActorModel();
  $inboxes = $actors->get_follower_shared_inboxes();
  foreach ($inboxes as $inbox) {
    $this->send($inbox, $msg);
  }
}

Now, loading /ap/publish with the item you're looking to serve will send it out to the blog's followers. Pulling the item itself from the blog's database is left as an exercise for the reader, as the data structure of the blog will invariably vary.

Storing comments

When the newly published post appears on our followers' timelines, they may wish to reply to the post from within Mastodon. Doing this will, to no-one's surprise, cause an event analogous to our own publication of a Note to arrive at the blog, as the reply will be treated by ActivityPub as a Note in turn.

There are three circumstances in which a comment can arrive at our blog^[A](There are also second- or subsequent-level replies to one of the blog's posts, that don't mention the blog account directly. As these notes aren't directly in reply to our account, and don't mention the account, they will not be delivered to us as ActivityPub doesn't keep track of the root-level conversation ID.):

• Comment on a post: These Note objects will have an inReplyTo value that matches the id of the blog article we published. Looking above, we see that our publish action sends out Note objects with an id starting: https://imrannazar.com/ap/item?id=, so any comments that arrive with this in their inReplyTo can be treated as top-level replies.
• Reply to a comment that mentions the blog: Similarly to top-level comments, these objects will have a inReplyTo, but it will reference a previous comment's id. ActivityPub as implemented by Mastodon doesn't directly provide a "conversation ID", so we'll need to keep track of top-level replies and check the inReplyTo of an incoming comment to see if we've already stored it as a top-level comment's id.
• Mention that isn't on a post: This will be delivered to us with no inReplyTo, so will need to be stored separately in some fashion.

Any comments that arrive at our blog will have an attributedTo value which will allow us to retrieve data about the user who wrote the comment, so we can show the user's preferred name and avatar against the comment once we have it stored. Given that, we've already seen all the objects that will be involved: Note objects for the comments, and Actor objects for the user data, so let's look at the database table and model.

CREATE TABLE ap_comments(
  id VARCHAR(255) NOT NULL PRIMARY KEY,
  page_id INT NOT NULL,
  actor_id VARCHAR(255) NOT NULL,
  in_reply_to VARCHAR(255) NOT NULL,
  comment_date TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP(),
  content TEXT,
  full_data TEXT
);

class CommentModel {
  protected $db; // PDO MySQL connector

  // The inbox handler will parse out all the data we need here
  public function insert($data) {
    $st = $this->db->prepare(
      'INSERT INTO ap_comments(
        id, page_id, actor_id, in_reply_to, content, full_data
      ) VALUES(
        :id, :page_id, :actor_id, :in_reply_to, :content, :full_data
      )'
    );
    foreach ($data as $k => $v) {
      $st->bindValue(":{$k}", $v);
    }
    $st->execute();
  }

  public function fetch($id) {
    $st = $this->db->prepare(
      'SELECT * FROM ap_comments WHERE id=:id'
    );
    $st->bindValue(':id', $id);
    $st->execute();

    return $st->fetch(PDO::FETCH_ASSOC);
  }
}

The model here is a simple wrapper on the table, to support insertion of new comments. The controller's handler for receiving Create events is only slightly more complicated, as it needs to deal with the three cases mentioned above:

public function createHandler($input) {
  // For this implementation we're only handling creation of notes
  if (!isset($input['object'], $input['object']['type'])) {
    throw new Exception('Malformed create message');
  }
  if ($input['object']['type'] !== 'Note') {
    throw new Exception('Unsupported create message');
  }

  $cm = new CommentModel();
  $comment = $input['object'];

  // First, store the actor's data
  // As per the previous article, this will fetch from the remote server
  // if we don't already have the actor in our database
  $actor = new ActorModel($comment['attributedTo']);

  $comment_data = [
    'id' => $comment['id'],
    'actor_id' => $comment['attributedTo'],
    'content' => $comment['content'],
    'full_data' => json_encode($comment),
  ];

  $parent = parse_url($comment['inReplyTo']);

  // If the parent is the blog post, we can infer the page ID
  // This also means the comment is top-level; in_reply_to is null
  if ($parent['host'] === 'imrannazar.com') {
    parse_str($parent['query'], $parent_q);
    if (!isset($parent_q['id'])) {
      throw new Exception('Mistargeted comment');
    }

    $comment_data += [
      'page_id' => $parent_q['id'],
      'in_reply_to' => null,
    ];
  }

  // Otherwise, we're receiving this because it's a
  // reply to an existing comment, or a mention
  else {
    $parent_comment = $cm->get($comment['inReplyTo']);
    if (!$parent_comment) {
      // Not associated with a particular page
      $comment_data += [
        'page_id' => 0,
        'in_reply_to' => null,
      ];
    } else {
      $comment_data += [
        'page_id' => $parent_comment['page_id'],
        'in_reply_to' => $comment['inReplyTo'],
      ];
    }
  }

  $cm->insert($comment_data);
}

Now we have a continual stream of comments and replies coming to our blog, we'll want a way to fetch and display them against a given post. Fortunately, storing both the page and parent comment IDs separately makes the data extraction relatively easy; displaying the comments is a matter of iterating over the top-level comments returned and recursing into the children, and won't be covered here as the handling is specific to a given blog's display implementation.

public function fetch_for_page($page_id) {
  $st = $this->db->prepare(
    'SELECT
      c.id, c.in_reply_to, c.content, c.comment_date, c.actor_id,
      a.url, a.name, a.avatar_url
    FROM ap_comments c
    LEFT JOIN ap_actors a ON c.actor_id = a.id
    WHERE c.page_id = :page_id
    ORDER BY c.comment_date'
  );
  $st->bindValue(':page_id', $page_id);
  $st->execute();
  $data = $st->fetchAll(PDO::FETCH_ASSOC);

  // Build the tree by adding child comments as references
  $comments_by_id = [];
  foreach ($data as $c) {
    $comments_by_id[$c['id']] = $c + ['children' => []];
    if ($c['in_reply_to']) {
      $comments_by_id['in_reply_to']['children'][] =
        &$comments_by_id[$c['id']];
    }
  }

  return $comments_by_id;
}

It's also possible for a user to delete their comment once it's been left, and we'll want our blog's view to reflect that; the deletion of the comment will arrive at our blog as a Delete event. One thing to note here is that we're building a tree view of comments, but a comment can be deleted at any level of the tree and will still need to be made available as a place to hold its children; we can accomplish this by sanitising the comment content at the time of deletion, allowing fetch_for_page above to continue unchanged.

public function sanitise($id) {
  $st->execute(
    'UPDATE ap_comments
      SET content=NULL, actor_id=NULL, full_data=NULL
    WHERE id=:id'
  );
  $st->bindValue(':id', $id);
  $st->execute();
}

public function deleteHandler($input) {
  $cm = new CommentModel();
  if ($cm->fetch($input['object']['id'])) {
    $cm->sanitise($input['object']['id']);
  }
}

Handling reactions

Replying isn't the only thing that can happen to a post on Mastodon; it can also be favourited (or starred) and boosted by a user into their followers' timelines. Both of these are issued as ActivityPub events: favourites translate into ActivityPub Like events, and boosts into Announce events. As these both relate to pages on our blog, we can store them in a generic "interactions" table.

Unlike Create events which we've both seen and generated before, we haven't seen these new events, so let's have a look at an example of a Like event:

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "id": "https://mastodon.social/users/0110100001101001#likes/183241819",
  "type": "Like",
  "actor": "https://mastodon.social/users/0110100001101001",
  "object": "https://imrannazar.com/ap/item?id=89"
 }

It turns out that Like and Announce events are fairly simple: the fields with which we need to be concerned are the actor and object, where the object is a canonical URL for one of the blog's pages. We can use similar URL parsing code to that which we used for comments, to pull out the page ID and store interactions against it in a new database table; we can then use that table to pull individual events and aggregate counts for a given page.

CREATE TABLE ap_interactions(
  page_id INT NOT NULL,
  actor_id VARCHAR(255) NOT NULL,
  interaction ENUM('Like', 'Announce') NOT NULL,
  event_date TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP()
);

class InteractionModel {
  protected $db; // PDO MySQL connector

  public function save($type, $page_id, $actor_id) {
    $st = $this->db->prepare(
      'INSERT INTO ap_interactions(interaction, page_id, actor_id)
      VALUES(:type, :page_id, :actor_id)'
    );
    $st->bindValue(':type', $type);
    $st->bindValue(':page_id', $page_id);
    $st->bindValue(':actor_id', $actor_id);
    $st->execute();
  }

  public function counts_for_page($page_id) {
    $st = $this->db->prepare(
      'SELECT interaction, COUNT(*) AS cnt
      FROM ap_interactions WHERE page_id=:page_id
      GROUP BY interaction'
    );
    $st->bindValue(':page_id', $page_id);
    $st->execute();

    return $st->fetchAll(PDO::FETCH_ASSOC);
  }

  public function details_for_page($page_id) {
    $st = $this->db->prepare(
      'SELECT i.type, i.event_date,
        a.url, a.name, a.avatar_url
      FROM ap_interactions i
      LEFT JOIN ap_actors a ON i.actor_id = a.id
      WHERE i.page_id = :page_id'
    );
    $st->bindValue(':page_id', $page_id);
    $st->execute();

    return $st->fetchAll(PDO::FETCH_ASSOC);
  }
}

protected function interactionHandler($input) {
  $page = parse_url($input['object']);
  if ($page['host'] !== 'imrannazar.com') {
    throw new Exception('Mistargeted interaction');
  }
  parse_str($page['query'], $page_q);
  if (!isset($page_q['id'])) {
    throw new Exception('Mistargeted interaction');
  }

  $im = new InteractionModel();
  $im->save($input['type'], $page_q['id'], $input['actor']);
}

public function likeHandler($input) {
  $this->interactionHandler($input);
}

public function announceHandler($input) {
  $this->interactionHandler($input);
}

Just as a user can decide to delete their comment on one of our posts, it's also possible to undo a favourite or boost in Mastodon; this will send an Undo event to us, where the object is a copy of the original interaction event. Unlike comments, however, there's no need for us to retain context of the interaction, so we can simply delete it from our database when the Undo comes in.

public function delete($type, $page_id, $actor_id) {
    $st = $this->db->prepare(
      'DELETE FROM ap_interactions WHERE
      type=:type AND page_id=:page_id AND actor_id=:actor_id'
    );
    $st->bindValue(':type', $type);
    $st->bindValue(':page_id', $page_id);
    $st->bindValue(':actor_id', $actor_id);
    $st->execute();
}

public function undoHandler($input) {
  if (!isset($input['object'], $input['object']['type'])) {
    throw new Exception('Malformed undo message');
  }

  switch ($input['object']['type']) {
    case 'Like':
    case 'Announce':
      // The original page being interacted with
      // is the Undo's object (the interaction)'s object
      $page = parse_url($input['object']['object']);
      if ($page['host'] !== 'imrannazar.com') {
        throw new Exception('Mistargeted interaction');
      }
      parse_str($page['query'], $page_q);
      if (!isset($page_q['id'])) {
        throw new Exception('Mistargeted interaction');
      }

      $im = new InteractionModel();
      $im->delete(
        $input['object']['type'],
        $page_q['id'],
        $input['actor']
      );
      break;

    default:
      // Unhandled
  }
}

And finally we come to the question I posed as the start: what happens when a user unfollows the blog? As it turns out, the Follow event is undone: an Undo event is sent to us. We just built out an Undo handler for interactions, so we can add events where the object is a Follow:

public function undoHandler($input) {
  ...
  switch ($input['object']['type']) {
    ...
    case 'Follow':
      $actor = new ActorModel($input['actor']);
      $actor->set_follows_us(false);
      break;
  }
}

All done: an ActivityPub-enabled blog

As Eugen put it in his article on HTTP Signatures^[2]Eugen Rochko, "How to make friends and verify requests", Jul 2018 cited in the first part of this series:

Primarily this means having a publicly accessible inbox and validating HTTP signatures. Once that works, everything else is just semantics.

It took us two articles to get through the implementation of those semantics, but we're out the other side, and we should now have the framework of an ActivityPub-enabled blog: articles can be published on the Fediverse, and users can leave comments and interactions. Having those interactions display on the blog is, as mentioned earlier, an exercise for the reader and their particular blog framework.

• Finding the blog and knowing the endpoints to communicate with it;
• Following the blog to receive future posts in your timeline;
• Publishing a post to the blog's followers;
• Commenting on a post and having the comment appear on the blog;
• Reacting to a post and having the reactions recorded on the blog;
• Deleting a reply and having the blog remove it;
• Undoing a follow or reaction and having the blog remove it.

Aside from finding the blog, each of these has a direction in which it propagates, and only publishing a post is an outward event: a post is written on the blog, it's published and pushed out. The other events outlined above are inward events, which are received by the blog from the source server. We should deal with these in order though, as it won't make sense to publish to our followers without having received any follows, so we'll look at receiving events first.

Finding the blog with WebFinger

However, before any of the other events can happen, a user on Mastodon will need to find our blog, and the server on which they reside will need to know how to get messages to us. Mastodon uses WebFinger^[2]Mastodon documentation, "WebFinger", Feb 2023 for this, and that means our blog will need to respond with something when the URL /.well-known/webfinger is loaded.

The documentation linked in note 2 states that the WebFinger endpoint will receive a request detailing which user is being "fingered", in order to determine how best to get information to that user. This is important for multi-user setups like Mastodon or Lemmy instances; fortunately, our blog is the kind of site that won't have multiple users. As such, we can ignore the body of the WebFinger request entirely, and serve a static response such as the following (which happens to be this blog's actual response).

{
  "subject": "acct:blog@imrannazar.com",
  "aliases": [
    "https://imrannazar.com/@blog",
    "https://imrannazar.com/ap/blog"
  ],
  "links": [
    {
      "rel": "http://webfinger.net/rel/profile-page",
      "type": "text/html",
      "href": "https://imrannazar.com/@blog"
    },
    {
      "rel": "self",
      "type": "application/activity+json",
      "href": "https://imrannazar.com/ap/blog"
    }
  ]
}

The above response indicates that, for the account @blog@imrannazar.com (which is the address I always hand out on Mastodon), its profile page (serving text/html) is /@blog, and its self (the canonical URL for ActivityPub's purposes) is /ap/blog; it's the latter of these which is important for our purposes, as it's the main repository of data regarding the blog and its available endpoints.

Let's have a quick look at this ActivityPub canonical URL: it serves a Person object with (almost) all the information a Mastodon instance might need to draw up a profile page for this account if it doesn't already have this information cached. We've already seen Person objects referenced in the previous instalment in relation to verifying an ActivityPub actor's public key, but let's have a look at the blog's actor data in more detail to see which endpoints we'll need to implement.

{
  "@context": [...],
  "publicKey": {...},

  "id": "https://imrannazar.com/ap/blog",
  "preferredUsername": "blog",
  "type": "Person",
  "name": "Imran Nazar's Blog",
  "url": "https://imrannazar.com/@blog",
  "summary": "<p>Imran Nazar: Things I've written over the years, a blog on software development topics with the occasional sci-fi short thrown in.</p>",
  "published": "2006-09-22T00:00:00Z",

  "attachment": [
    {
      "name": "RSS feed",
      "type": "PropertyValue",
      "value": "<a href=\"https://imrannazar.com/rss.xml\" rel=\"nofollow noreferrer noopener\" target=\"_blank\">https://imrannazar.com/rss.xml</a>"
    }
  ],
  "icon": {
    "mediaType": "image/png",
    "type": "Image",
    "url": "https://imrannazar.com/assets/ap-icon.png"
  },
  "image": {
    "mediaType": "image/jpeg",
    "type": "Image",
    "url": "https://imrannazar.com/assets/ap-image.jpg"
  },

  "discoverable": true,
  "indexable": true,
  "manuallyApprovesFollowers": true,
  "memorial": false,

  "following": "https://imrannazar.com/ap/following",
  "followers": "https://imrannazar.com/ap/followers",
  "inbox": "https://imrannazar.com/ap/inbox",
  "outbox": "https://imrannazar.com/ap/outbox"
}

For our purposes, this can again be a static file served from the endpoint /ap/blog. There are a few groups of keys here:

• User information: Things like the ID (the canonical URL of this user), name, summary and date of publication (which for a Person-type object is the date the user was created);
• Display data: This includes the avatar/icon, background/header image, and any additional URLs or links to be shown on the profile;
• Behavioural flags: In this case, we want the account to be publicly visible ("discoverable") as well as come up in search results ("indexable"), but this is not a memorial account. The most important flag here is manuallyApprovesFollowers, as it determines what we do when a Follow request is received, but we'll come to that later;
• Endpoints: These are the locations of interest to us. Following and Followers are read-only collections which inform any Mastodon instance reading them how many of each the blog has, and their canonical actor IDs; the Inbox is where a remote instance will send AP messages; and the Outbox is where an instance can expect messages from the blog to be coming from.

So let's say a Mastodon user wants to follow this blog, and they have the address (@blog@imrannazar.com) but they're the first user on their instance to want to follow us. There are a few requests the remote instance will make in order to build up a complete profile:

• First a WebFinger request is needed to find the actor ID; the URL for this is taken directly from the address given (@blog@imrannazar.com leads us to https://imrannazar.com/.well-known/webfinger).
• The result contains a link of rel: self which in turn informs us where the actor resides, which when fetched contains the summary, icon, image and such.
• Once the remote instance has the actor data, it can then fetch the following and followers lists in order to show those counts.

Having done all this, the remote instance is able to draw up a page that looks a little like this:

(We can see in this screenshot that the blog has six followers, but we haven't yet implemented /ap/followers to allow the remote instance to know that; we'll get to it though.)

Receiving AP events

Finally we have a profile page with an inviting Follow button, and the remote Mastodon instance knows where to send the request when that button is pressed: the actor's inbox. So what would one of those requests look like? Let's have a look at how that endpoint might translate to a controller in a PHP MVC framework like BirSaat^[3]"BirSaat PHP MVC microframework", Imran Nazar, 2013, so we can start logging the requests that come in:

class ApController extends bsControllerBase {
  public function inboxAction() {
    if ($_SERVER['REQUEST_METHOD'] != 'POST') {
      throw new Exception('Inboxes are POSTed to');
    }

    $post = file_get_contents('php://input');
    file_put_contents('/tmp/ap.log', $post);
  }
}

It's rudimentary, but this code will allow us to see the last request that came in to /ap/inbox; the specification states that all messages will come in as POSTed JSON, so an additional check is made for the method of the request. If I now press the Follow button on this blog, the following message comes in:

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "id": "https://hachyderm.io/af444650-1827-4bf7-94cd-0ec0b57bdb44",
  "type": "Follow",
  "actor": "https://hachyderm.io/users/Two9A",
  "object": "https://imrannazar.com/ap/blog"
}

Every ActivityPub message has an ID, which in our use case we don't need to track; the important things here are the type and the actor associated with the message. The actor who is following the blog should match up with the signature verification headers explained in the previous article, which will allow us to authenticate this as a valid message to be handled. So we can add a couple more clauses to our inbox handler before the message-specific handling begins:

class ApController extends bsControllerBase {
  public function inboxAction() {
    if ($_SERVER['REQUEST_METHOD'] != 'POST') {
      throw new Exception('Inboxes are POSTed to');
    }

    $post = file_get_contents('php://input');
    try {
      // Signature verification code from the previous article
      $actor = $this->verifyHeaders();

      $input = json_decode($post, true);
      if (!$input) {
        throw new Exception('Malformed inbox message');
      }
      if ($actor['id'] != $input['actor']) {
        throw new Exception('Inbox message misattributed');
      }

      $handler = strtolower($input['type']) . 'Handler';
      if (is_callable($this, $handler)) {
        $this->$handler($input);
      }
    } catch (Exception $e) {
      // Any logging you may wish to perform
      throw $e;
    }

    // No response required except for a blank 200
    die();
  }
}

Handling Follows

To follow the blog is to receive new posts when they're published, so we'll need to record the fact that this user is following us in order to send a publication event to their server when the time comes. We'll want a database table for this, and it so happens that this blog has its page structure data held in MySQL, so we'll make another table alongside the page list.

One thing to note is that signature verification for ActivityPub messages depends on us having the public key of the actor creating the message; if we don't already have the public key, we'll need to fetch it from the remote ActivityPub instance. In order to reduce the need to do this every time a message comes in, we can cache the actor object in the database once fetched, and use that for future verification requirements.

CREATE TABLE ap_actors(
  id VARCHAR(255) NOT NULL PRIMARY KEY,
  url VARCHAR(255) NOT NULL,
  name VARCHAR(255) NOT NULL,
  avatar_url TEXT,
  follows_us TINYINT(1) NOT NULL DEFAULT 0,
  full_data TEXT
);

class ActorModel {
  protected $db; // PDO MySQL connector

  protected $id;
  public $data;

  public function __construct($id = null) {
    if ($id) {
      $this->data = $this->cached_fetch($id);
      return $this->data;
    }
  }

  public function cached_fetch($id) {
    $this->id = $id;

    $st = $this->db->prepare('SELECT * FROM ap_actors WHERE id=:id');
    $st->bindValue(':id', $id);
    $st->execute();

    $row = $st->fetch(PDO::FETCH_ASSOC);
    if ($row) {
      return json_decode($row['full_data'], true);
    }

    // If we don't have the actor cached, fetch and save
    // See "Fetching the signing actor" from the previous article
    $actor = $this->fetch_from_remote($id);
    if (!$actor) {
      throw new Exception('Actor not found');
    }

    $st = $this->db->prepare('
      INSERT INTO ap_actors(id, url, name, avatar_url, full_data)
      VALUES(:id, :url, :name, :avatar, :data)
    ');
    $st->bindValue(':id', $id);
    $st->bindValue(':url', $actor['url']);
    $st->bindValue(':name', $actor['name']);
    $st->bindValue(':data', json_encode($actor));

    // Some accounts don't have avatars
    $st->bindValue(':avatar', isset($actor['icon'], $actor['icon']['url'])
      ? $actor['icon']['url']
      : null
    );
    $st->execute();

    return $actor;
  }
}

Now we have the framework in place for handling messages, and an actor data model to store the fetched actors, all we need is to record the fact that the actor making this request wants to follow us:

private function followHandler($input) {
  $actor = new ActorModel($input['actor']);
  $actor->set_follows_us(true);
}

public function set_follows_us($id, $follows) {
  $st = $this->db->prepare(
    'UPDATE ap_actors SET follows_us = :f WHERE id = :id'
  );
  $st->bindValue(':id', $id);
  $st->bindValue(':f', $follows);
  $st->execute();
}

Accepting follows

Except that doesn't quite work. If someone wants to follow this blog, and they hit Follow on the blog's profile page in their Mastodon instance, one would expect to see an "Unfollow" button the next time they view the blog, but instead we get "Cancel follow".

This happens because the remote instance is expecting a confirmation from us that the request to follow has been granted, as per this line from the blog's actor data:

{
  "@context": [...],
  "publicKey": {...},
  ...
  "discoverable": true,
  "indexable": true,
  "manuallyApprovesFollowers": true,
  "memorial": false,
  ...
}

Just as the remote actor sent a message to our inbox to request the follow, we'll need to send an Accept-type message to their inbox. Luckily, we already have everything we need to do this: the actor's endpoints are part of their full data which we already have either cached or fetched, and the message being accepted is the follow we received.

private function followHandler($input) {
  $actor = new ActorModel($input['actor']);
  $actor->set_follows_us(true);

  // See the previous article for detailed code on
  // signing and sending ActivityPub messages
  $this->send($actor->data['inbox'], [
    'type' => 'Accept',
    'object' => $input,
  ]);
}

And now we can circle back around to the blog's profile page, and specifically the "6 followers" that it shows. This data is fetched (if the instance doesn't already have a count of followers cached locally) from our actor's followers endpoint; this (and following) should return objects of type Collection. In theory, these collection endpoints should support pagination so any remote instance can fetch a full list of actor IDs, but for the purposes of this blog I don't foresee the list of followers growing to such an extent that serving a full list will cause undue load.

As with the inbox endpoint, these two collection endpoints will map to action methods in an MVC framework:

public function fetch_followers() {
  $st = $this->db->prepare(
    'SELECT id, name, url, avatar_url FROM ap_actors WHERE follows_us = 1'
  );
  $st->execute();
  return $st->fetchAll(PDO::FETCH_ASSOC);
}

public function followersAction() {
  $actors = new ActorModel();
  $followers = $actors->fetch_followers();
  $this->view->set_format('json');
  return [
    '@context' => 'https://www.w3.org/ns/activitystreams',
    'id' => 'https://imrannazar.com/ap/followers',
    'type' => 'Collection',
    'totalItems' => count($followers),
    'items' => array_map(function($item) {
      return $item['id'];
    }, $followers),
  ];
}

public function followingAction() {
  $this->view->set_format('json');
  return [
    '@context' => 'https://www.w3.org/ns/activitystreams',
    'id' => 'https://imrannazar.com/ap/followers',
    'type' => 'Collection',
    'totalItems' => 0,
    'items' => [],
  ];
}

With the followers and following endpoints in place, and a way to listen to and respond to Follow-type messages, we now have a profile in the Fediverse (or at least on Mastodon) that can be followed. What happens when you press Unfollow on this profile?

As I've already gone on overly long for this post, we'll cover this and the other remaining operations in a third part; come back next time and we'll talk more about publishing, commenting and reactions.

When people talk about the history of abstract art, the name of Piet Mondrian is sure to come up. It was Mondrian who, in 1922, was the first to throw out all pretence of realism in the search for deeper connection to the innate nature of things.

Mondrian trained in traditional landscapes, and continued to paint still life flowers even while producing abstract art in the 1920's^[2]David Shapiro, "Mondrian: Flowers", 1991; via Wikipedia but as the curator of a Swiss collection of his works told the BBC:

Minimalism is unthinkable without Mondrian; he was one of the first who really did this, this totally non-representational work? If you see modern as something which breaks with all traditions and defines everything new, then Mondrian's paintings of the 20s are very, very modern.

The abstract works of Mondrian focus heavily on horizontal and vertical lines in black, and the filling of delineated areas with primary colours; as Mondrian would later explain in an essay from 1937, these lines "exist everywhere and dominate everything; their reciprocal action constitutes 'life'."

As it turns out, "everywhere" even includes fields of study that were barely known when Mondrian died in 1944, as his abstract style inspires the Piet programming language where images are programs, and different colours define various actions for the program to perform.

We don't usually think of images as computer programs. A computer program, though, can be represented in just about any way: if a given piece of code can be expressed in one "Turing-complete" form, it has been proven mathematically that any other "Turing-complete" form is equivalent.

To take a ridiculous example, a programming language exists based on travel between stations of the London Underground, and it's called Mornington Crescent^[3]"Mornington Crescent" on the Esolangs wiki, 2013.

So when David Morgan-Mar saw a painting by Piet Mondrian, and decided it would be good material for a programming language, it was no more ridiculous a concept than many others in the world of coding. Thus, the Piet language^[5]"Piet", David Morgan-Mar, last updated 2022 was born.

Programs in Piet are colourful. The Piet language uses more colours than Mondrian himself was wont to use, defining six colours in three shades, and giving meaning and context to each shade of colour as well as each direction of travel through the image; execution starts in the top left and moves through depending on the values of the pixels encountered.

There are many examples of programs that people have written in Piet^[7]"Piet Program Gallery", David Morgan-Mar, last updated 2023, including pieces intentionally designed to look more like abstract works than computer programs, but Piet probably claims the crown for the first programming language in which a piece of code has been written accidentally.

The work of Barbara Maahs^[8]"Bilder, Barbara Maahs", Atelier Kunst 24 is very much in the Mondrian tradition, with solid blocks of primary colour; she breaks away from the use of black dividing lines, often preferring direct contact between blocks. So it was that Piet Jarmatz saw a Maahs piece in a gallery, had a suspicion that it would run if plugged into a computer, and was able to convert it into a working Piet program^[9]"Piet Get-Together", Piet Jarmatz, 2023 that takes typed-in letters and prints out their decimal ASCII values.

And I think that's pretty neat.

To that end, I've instead started using inline tooltips for asides, which appear when the referential link is hovered over by the mouse (or tapped by the mobile user); this allows the aside to enrich the article without distracting from its flow.

We can get almost all the way to a tooltip like this with CSS alone; let's have a look at how that works.

CSS for tooltip positioning

The most salient piece of CSS being used for these tooltips is the position rule: if an element on the page is positioned with the absolute value, it will anchor itself based on an offset from the nearest positioning context. That context can be the whole page, but more often it's a parent element that's been given the relative position value; the nearest relatively positioned parent to an absolutely positioned element will provide that context.

<p>We discuss a topic, and then provide a reference <cite>
 <sup>[1]</sup>
 <mark><em>The Lord of the Rings</em> (Harper/Collins), pg 293</mark>
</cite> which can make for further reading.</p>

cite {
  position: relative;
}
cite sup {
  cursor: pointer;
}
cite mark {
  display: none;
  position: absolute;
  top: 0.5em;
  left: 0;
}
cite:hover mark {
  display: block;
}

Display patches

This gets us most of the way to a working tooltip already, but there are a few things we can do to help with the display:

• Background: As it stands, the tooltip mark is transparent, so the text of the tooltip renders over the top of the article and is hard to make out. This can be alleviated by giving the mark a background colour, and a color so the text can sit atop the background and be read well.
• Width: Without a defined stipulation on width, the tooltip will only be as wide as the widest non-wrapping piece of content inside (generally, this would be the single longest word). We can make the tooltip more readable by giving it a minimum width based on the size of the viewport (the window being used to view the page); for example: min-width: 30vw to give the tooltip a width of at least 30% of the viewport, and more if the browser deems it necessary.
• Z-index: Figure 1 shows our tooltip showing above text and images on the page. A detail of the implementation of images on this site is that some images (those with black text on a transparent background) are marked invertable so they're readable in dark mode; this adds a filter: invert(1); rule when dark mode is enabled at the body level. In turn, this puts the image in a new stacking context^[2]Mozilla Developer Network, Stacking context, Feb 2024 which must be overridden by the tooltip with a z-index rule.
• Accessibility: The tooltip content inside the mark is hidden by CSS unless one hovers on the sup; screen readers will generally disregard this content as invisible, meaning the additional context given by the aside won't be available in these browsers. This can be ameliorated by providing another CSS rule within a media query for non-screen users, as well as using visibility instead of display to control whether the tooltip appears.

Putting these patches together gets us to the tooltips being used on this page, for which the CSS looks as follows:

cite {
  position: relative;
}
cite sup {
  cursor: pointer;
}
cite mark {
  visibility: hidden;
  position: absolute;
  top: 0.5em;
  left: 0;
  z-index: 2;

  /* Additional rules to add some nicety */
  color: var(--g-text);
  background: var(--g-bg);
  border: 4px solid var(--g-border);
  border-radius: 4px;
  padding: 12px;
  min-width: 30vw;
}
cite:hover mark {
  visibility: visible;
}

@media not screen {
  cite mark {
    visibility: visible;
  }
}

Except that's not accessible

For some screenreaders, this may be enough: the visibility rule puts the content of the tooltip into the page without having it initially visible. For Voiceover on macOS, however, the content needs to be rendered somewhere on the page so it can be read out.

Historically, the canonical way to have something render offscreen on a page without using display or visibility has been to position it off either the left or right edge of the screen. This still works, but positioning something off the right of the page is liable to push the scrollbar out, so we use negative left values to shunt the content off to the left; this places it in the DOM, visible to screenreaders, but not visible until we reset the left value.

cite mark {
  position: absolute;
  top: 0.5em;
  left: -9999px;
}
cite:hover mark {
  left: 0;
}

With this positioning in place, Voiceover correctly picks up on the tooltip's presence in the page without having the tooltip render for sighted users, until the sup is hovered.

Positioning on the right

The only remaining issue with these tooltips is what happens when the sup trigger for the tooltip lies towards the right of the screen (anywhere past 70% across from the left of the viewport). In this case, bringing up the tooltip causes it to render with its min-width of 30% of the viewport, causing the right edge to be placed past the right edge of the content, and causing horizontal scroll.

Unfortunately, this is where we have to diverge from a pure-CSS solution, as the following doesn't exist in CSS as it stands:

cite[position-right > 70vw] mark {
  left: auto;
  right: 0;
}

Instead, we must resort to JavaScript which can determine the rendered position of any tooltip triggers on the page, after the page has loaded. We can use the querySelectorAll method of the document to query for matching elements, and each matching element has a getBoundingClientRect method which will provide its position and size on the page.

Once we've determined whether the matching element is far enough to the right, we can add or remove a class to the element as appropriate. This can be done by using the corresponding methods of the element's classList.

window.onload = function() {
  const tooltipPositioner = () => {
    const threshold = window.innerWidth * 0.7;
    document.querySelectorAll('cite').forEach(el => {
      el.classList[
        el.getBoundingClientRect().x > threshold
          ? 'add'
          : 'remove'
      ]('right');
    });
  };

  tooltipPositioner();
};

cite.right mark {
  left: auto;
  right: 0;
}

Repositioning after resize

One consideration that needs to be made now that some tooltips can have a different behaviour for "is on the right of the screen" is what happens if the user resizes their browser window, and causes some tooltip triggers to move across the viewport as the content reflows. Fortunately, JavaScript offers the ResizeObserver which allows for a function to be run whenever an element resizes; as our positioning code is already in a function, setting up the observer against resizes on the document's main tag is fairly simple:

const tooltipPositioner = () => {
  ...
};
tooltipPositioner();
(new ResizeObserver(tooltipPositioner)).observe(
  document.querySelector('main')
);

It should be noted that the ResizeObserver runs every time the element changes size: if the user is resizing their browser window, this might fire a hundred times before the size of the window settles. For our purposes, as this is the only JavaScript running on the page, performance isn't a particular concern; if this tooltip code were to be used as part of a heavier framework, one may wish to use throttling mechanisms to ensure that the code is only run every so often while still remaining responsive to resize events.

But we almost got away without using any JS at all. With the new popover API rapidly rolling out to browsers as of the time of writing, it may soon be the case that even the CSS used here can be trimmed back.

• An AP user (an "actor" in the protocol's terminology^[1]Seb Jambor, "Understanding ActivityPub", May 2023) can send an activity (for example, a Like on a post) and it will be broadcast to any servers hosting that user's followers;
• Receiving servers can verify that the activity was generated by the user in question.

ActivityPub uses the HTTP Signature header for this second point, with public key encryption. It should be noted though, that there is a standard RFC for the Signature header^[2]RFC 9421, "HTTP Message Signatures", Feb 2024 but AP was built before this RFC was released; instead, ActivityPub uses a draft version of the Signature specification^[3]RFC 9421 draft 12, "Signing HTTP Messages", Oct 2019 which operates slightly differently.

In Figure 1 above, we see the two halves of the process ActivityPub employs using the Signature header: the sender of an activity creates a message signed with their private key, and any receivers verify the message signature with the sender's public key. Let's first look at what goes into generation of the signature.

Generating a signature

The first thing we'll need is a Note, a message created by an actor; in this case, we'll use the creation of a toot by alice@tech.lgbt^[A]As it turns out, the example accounts being used for this article are both extant accounts, but bob@mastodon.social doesn't actually follow alice@tech.lgbt as of the time of writing. which is represented by the following JSON object.

{
  "@context": {...},
  "id": "https://tech.lgbt/users/alice/statuses/12345678",
  "type": "Note",
  "url": "https://tech.lgbt/@alice/12345678",
  "attributedTo": "https://tech.lgbt/users/alice",
  "published": "2024-03-30T15:50:09Z",
  "to": ["https://www.w3.org/ns/activitystreams#Public"],
  "cc": ["https://tech.lgbt/users/alice/followers"],
  "content": "Hello followers!"
}

In order to broadcast this Note, ActivityPub requires that it be wrapped in a Create activity informing the recipients of the creation of an object:

{
  "@context": {...},
  "id": "https://tech.lgbt/users/alice/statuses/12345678/activity",
  "type": "Create",
  "actor": "https://tech.lgbt/users/alice",
  "published": "2024-03-30T15:50:09Z",
  "to": ["https://www.w3.org/ns/activitystreams#Public"],
  "cc": ["https://tech.lgbt/users/alice/followers"],
  "object": {
    "id": "https://tech.lgbt/users/alice/statuses/12345678",
    "type": "Note",
    "url": "https://tech.lgbt/@alice/12345678",
    "attributedTo": "https://tech.lgbt/users/alice",
    "published": "2024-03-30T15:50:09Z",
    "to": ["https://www.w3.org/ns/activitystreams#Public"],
    "cc": ["https://tech.lgbt/users/alice/followers"],
    "content": "Hello followers!"
  }
}

Now we have the activity that will be broadcast to alice's followers, we can start the signature process; we'll need a fresh signature for each server that's receiving this message. For this example, alice has exactly one follower, on mastodon.social, so we'll need to send this message to the server's "shared inbox" at: https://mastodon.social/inbox^[B]Mastodon servers, by convention, have their shared inbox at /inbox but the canonical URL can be found in the actor's details, at the endpoints.sharedInbox key. to be forwarded on to bob's personal inbox.

We first need a message digest, which by convention is an SHA-256 hash of the message content; in PHP, one might generate that as below.

$message = json_encode([
  '@context' => [...],
  ...
]);

$digest = 'SHA-256=' . hash('sha256', $message);

The hash digest is one of the parts of the signature; other parts that we need to include are:

• Host of the remote server to which we're sending this message, in this case mastodon.social;
• Request target is essentially the first line of the HTTP request we'll be making, in this case post /inbox;
• Content type which isn't a requirement, but can be seen emitted by Mastodon hosts when communicating with each other. If this is part of the signature, it would be application/activity+json;
• Date of the request, which can be verified by the other side to ensure this message isn't being replayed after the fact. The caveat here is that the date is in a particular format that's different to the date format in the activity JSON, for example: Sat, 30 Mar 2024 15:50:09 GMT

Pulling these together doesn't take too much doing in PHP:

// We'll be sending the date separately, so take one fixed point
$dt = date("D, d M Y H:i:s \G\M\T");
$url = 'https://mastodon.social/inbox';

$urlparts = parse_url($url);
$sigparts = [
  '(request-target)' => 'post ' . $urlparts['path'],
  'host' => $urlparts['host'],
  'date' => $dt,
  'digest' => $digest,
  'content-type' => 'application/activity+json',
];
$sigsrc = join("\n", array_map(
  fn($k, $v) => "{$k}: {$v}",
  array_keys($sigparts),
  array_values($sigparts)
));

(request-target): post /inbox
host: mastodon.social
date: Sat, 30 Mar 2024 15:50:09 GMT
digest: SHA-256=ce9a290f805f....1bbcf5104ad7550191201
content-type: application/activity+json

Once we have the collated signature source, we can invoke OpenSSL to sign the string with the private part of alice's key pair. PHP's OpenSSL bindings expect a PEM-formatted file for this, which for this example we're storing in /etc/pki somewhere:

openssl_sign($sigsrc, $signature, openssl_get_privatekey(
  file_get_contents('/etc/pki/activitypub/alice/private.pem'),
  'key_passphrase'
), OPENSSL_ALGO_SHA256);

This leaves the signature in $signature, in binary. Our last step in signing the request is to format the signature according to the signing spec, detailing what we've used to generate this signature, and which key was involved.

For the remote end to be able to verify the signature, we'll need to provide a URL at which alice's public key can be fetched; in the ActivityPub protocol, this is included as part of the details of an actor, as we'll see when we go through verification at the other end.

$sig_header_parts = [
  'keyId' => 'https://tech.lgbt/users/alice#main-key',
  'algorithm' => 'rsa-sha256',
  'headers' => join(' ', array_keys($sigparts)),
  'signature' => base64_encode($signature),
];
$headers = [
  'Accept: application/activity+json',
  'Content-Type: application/activity+json',
  'Host: ' . $urlparts['host'],
  'Date: ' . $dt,
  'Digest: ' . $digest,
  'Signature: ' . join(',', array_map(
    fn($k, $v) => "{$k}=\"{$v}\"",
    array_keys($sig_header_parts),
    array_values($sig_header_parts)
  )),
];

Accept: application/activity+json
Content-Type: application/activity+json
Host: mastodon.social
Date: Sat, 30 Mar 2024 15:50:09 GMT
Digest: SHA-256=ce9a290f805f....1bbcf5104ad7550191201
Signature: keyId="https://tech.lgbt/users/alice#main-key",algorithm="rsa-sha256",headers="(request-target) host date digest content-type",signature="xmpTJEEjodgUP5H....BrN3DwZGy7DoTfRQ=="

And we're ready to send alice's message over to bob's Mastodon instance, with the headers indicating that it's been signed by alice and a digest hash to allow the message content to also be verified.

Verifying a signature

As per Figure 1, when mastodon.social receives alice's activity it will first verify that this is a both a valid activity and that it's appropriately signed, and then the server will drop the message into any of alice's followers that are recorded as active on the instance.

The first step is to extract the signature and its components. We've seen from the generation code that this is passed through as a HTTP header, so if we place a dumping script at /inbox it might see something like this:

<?php
$input = file_get_contents('php://input');
var_dump([$_SERVER, $message]);

array(2) {
  [0]=>
  array(29) {
    ["USER"]=>
    string(8) "www-data"
    ...
    ["HTTP_CONTENT_TYPE"]=>
    string(25) "application/activity+json"
    ["HTTP_DATE"]=>
    string(29) "Sat, 30 Mar 2024 15:50:09 GMT"
    ["HTTP_DIGEST"]=>
    string(52) "SHA-256=ce9a290f805f..."
    ["HTTP_SIGNATURE"]=>
    string(502) "keyId="https://tech.lgbt/users/alice#main-key",algori..."
    ...
    ["REQUEST_METHOD"]=>
    string(4) "POST"
    ["REQUEST_URI"]=>
    string(6) "/inbox"
  }
  [1]=>
  string(1347) "{"@context":{..."
}

As we'd expect for PHP, the HTTP headers come to the script in the $_SERVER superglobal, with the HTTP_ prefix, so we can extract the parts of the signature that we need from $_SERVER['HTTP_SIGNATURE'].

It should be noted that commas only appear in the signature to delineate the parts; each of the parts themselves is either a URL or a piece of text that won't contain a comma. This means we can treat the signature string as an INI file and parse it fairly simply, with some deft string replacement:

if (!isset($_SERVER['HTTP_SIGNATURE'])) {
  throw new Exception('No signature');
}
$sigconf = parse_ini_string(
  strtr($_SERVER['HTTP_SIGNATURE'], ["," => "\n"])
);
if (!isset(
  $sigconf['keyId'],
  $sigconf['algorithm'],
  $sigconf['headers'],
  $sigconf['signature']
)) {
  throw new Exception('Malformed signature');
}

The first item in $sigconf is the URL of the public half of the key pair used to generate this signature. In real-life usage we might expect to have this already cached or stored locally, but if we don't it will need to be fetched, which is something we can make happen through the curl extension.

We've already seen that ActivityPub objects are passed back and forth with a MIME content type of application/activity+json, so we'll need to set that as the MIME type we'll be expecting to Accept. What we get back is an ActivityPub actor object:

$c = curl_init();
curl_setopt_array($c, [
  CURLOPT_URL => $sigconf['keyId'],
  CURLOPT_TIMEOUT => 5,
  CURLOPT_SSL_VERIFYPEER => true,
  CURLOPT_FOLLOWLOCATION => true,
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_HTTPHEADER => [
    'Accept: application/activity+json',
  ],
]);
$r = curl_exec($c);
curl_close($c);

if (!$r) {
  throw new Exception('User information not available');
}
$actor = json_decode($r, true);
if (!$actor) {
  throw new Exception('User information not decodable');
}

{
  "@context": {...},
  "id": "https://tech.lgbt/users/alice",
  "type": "Person",
  "following": "https://tech.lgbt/users/alice/following",
  "followers": "https://tech.lgbt/users/alice/followers",
  ...
  "url": "https://tech.lgbt/@alice",
  "publicKey": {
    "id": "https://tech.lgbt/users/alice#main-key",
    "owner": "https://tech.lgbt/users/alice",
    "publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhk..."
  },
  "endpoints": {
    "sharedInbox": "https://tech.lgbt/inbox"[C]This is the key mentioned in note B.
  }
}

The signing actor's public key comes through in the publicKey block, so we'll hold onto that for the verification later. It's worth double-checking that the key we've been served as part of the actor is the same key that was used to sign the message, which we can do by comparing the key's id with the keyId that came through from the signature.

We'll be using the openssl extension to perform verification, and we need to translate the PEM-formatted public key to a binary format for internal use by first calling openssl_get_publickey:

if (!isset(
  $actor['publicKey'],
  $actor['publicKey']['id'],
  $actor['publicKey']['publicKeyPem']
)) {
  throw new Exception('Missing public key');
}

if ($actor['publicKey']['id'] !== $sigconf['keyId']) {
  throw new Exception('Misattributed public key');
}

$pubkey = openssl_get_publickey($actor['publicKey']['publicKeyPem']);
if (!$pubkey) {
  throw new Exception('Malformed public key');
}

We're almost done with the first block of verification from Figure 1; the final stage before verifying the signature against alice's public key is to build the signature string based on the headers key. We'll want to tack together the headers given to us, as specified in the order therein.

There are two things to watch out for here:

• HTTP headers are passed through $_SERVER as previously mentioned, but they're transformed into uppercase and hyphens are switched out for underscores;
• The (request-target) signature part is a special case, as it consists of the request method (in lowercase) and the request URI, both of which come through in $_SERVER separately.

$sigparts = [];
foreach (explode(' ', $sigconf['headers']) as $hdr) {
  if ($hdr === '(request-target)') {
    $sigparts[] = sprintf(
      '%s: %s %s',
      $hdr,
      strtolower($_SERVER['REQUEST_METHOD']),
      $_SERVER['REQUEST_URI']
    );
  } else {
    $received_hdr = 'HTTP_' . strtoupper(strtr($hdr, ['-' => '_'));
    if (!isset($_SERVER[$received_hdr)) {
      throw new Exception('Missing signature part: ' . $hdr);
    }

    $sigparts[] = sprintf('%s: %s', $hdr, $_SERVER[$received_hdr]);
  }
}

With the signature constructed, we can finally call out to OpenSSL to perform the verification against alice's public key, which we loaded into binary format earlier. Here, we're assuming that the algorithm specified is a well-known value to OpenSSL like rsa-sha256 (the key algorithm used by Mastodon instances at time of writing), but it can be useful to verify this against a list of allowed values before proceeding if you're feeling particularly paranoid.

if (!openssl_verify(
  join("\n", $sigparts),
  base64_decode($sigconf['signature']),
  $pubkey,
  strtoupper($sigconf['algorithm'])
)) {
  throw new Exception('Signature verification failed');
}

Finally, with the message's origin verified, we can proceed to verify the message content against the digest provided. Again in this case, we're supporting a limited subset of the possible hash algorithms that could be used for the digest, as Mastodon invariably sends messages with a SHA256 digest:

list($digest_algo, $digest_hash) = explode('=', $_SERVER['HTTP_DIGEST']);
switch ($digest_algo) {
  case 'SHA256':
    $input_digest = hash('sha256', $input);
    break;

  default:
    throw new Exception('Unsupported digest algorithm');
}
if ($input_digest !== $digest_hash) {
  throw new Exception('Digest verification failed');
}

And if we made it this far, past all the exception points, we have a valid ActivityPub message that's verified to have been signed by the owner of the originating key.

Further reading

It's always dangerous on this blog to promise a "next time", but: next time we'll look at the particular ActivityPub messages that we'd want to support when adding AP support to a blog, and the surrounding architecture of user and inbox handling.

More information about ActivityPub and the specifics of the Mastodon implementation can be found in the third part of Seb Jambor's series on Understanding ActivityPub^[4]Seb Jambor, "The State of Mastodon", August 2023; the PHP implementation we've covered here is an expansion and explanation based on Terence Eden's single-file PHP integration of ActivityPub^[5]Terence Eden, "AcitvityPub Server in a Single File", Feb 2024 which moves over the signature stage relatively quickly, and incorporates Eugen Rochko's insights on signature verification^[6]Eugen Rochko, "How to make friends and verify requests", Jul 2018.

Testing from the server

As production and staging are in the allowlist, you can simply connect to those IPs and perform tests from there. This does, however, presume a couple of things:

• You have access to those servers: A fly-by-night webshop may hand out root access to the server machines and allow their integration developers to hack directly thereupon, but a more organised place may have access restrictions and only allow their infrastructure team direct access.
• Those servers exist: An increasingly common setup is for APIs and integrations to run on ephemeral servers (often referred to as "serverless" architecture) with Networking Magic to allow them to present as a fixed IP; in this case there's no machine to connect to in order to perform tests.

Forwarding the remote

Another option is to use NAT-based forwarding to expose the remote API: for example, if the third-party REST API is served from https://their-end.io/api you can use NAT to expose this as https://your-end.com:8443/api by forwarding connections on port 8443 to their-end.io port 443.

This does, however, involve roping in the infrastructure people to either perform iptables-based incantations to set up the NAT, or even more arcane spells to set up containerised NAT. Direct forwarding of the connection also means you lose the opportunity to log usage of the remote API, if you were seeking to keep track of who's connecting.

Proxying the remote

Perhaps the most flexible option is to proxy the remote API: to present an interface to the interface. This allows us to access the API from locations outside the allowlist, while giving us the chance to log access if so required, and has the advantage of being possible to set up in two ways: either through a dedicated proxying package like nginx-proxy, or through code-level changes only.

Having explored the options, here we'll be looking at a code-level proxy written in PHP.

What we're looking to proxy

Our example remote API to which we'd like access is fairly standard as REST goes, offering the following operations:

• GET /accounts: Fetch a list of registered accounts, with each account having a canonical URL of /accounts/<uuid> as part of the returned JSON;
• POST /accounts: Create a new account given the required details in JSON, returning a canonical URL;
• PUT /accounts/<uuid>: Update an account by UUID;
• DELETE /accounts/<uuid>: Delete an account by UUID.

We'll be using the curl extension to PHP, which is bundled by default, to send the request on behalf of what was received by our proxy, and to print out what was returned. Let's look at how that might work, as a first cut:

define('REMOTE_URL', 'https://their-end.io/api/');
define('OUR_URL', 'https://your-end.com/apiproxy.php');

$params = $_GET;
$curl_params = [
  CURLOPT_URL            => REMOTE_URL . '?' . http_build_query($params),
  CURLOPT_TIMEOUT        => 30,
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_FOLLOWLOCATION => true,
  CURLOPT_SSL_VERIFYHOST => true,
  CURLOPT_SSL_VERIFYPEER => true,
  CURLOPT_CUSTOMREQUEST  => $_SERVER['REQUEST_METHOD'],
];

// Anything with a POSTed body needs the body transferring
if ($_SERVER['REQUEST_METHOD'] !== 'GET') {
  $curl_params[CURLOPT_POSTFIELDS] = file_get_contents('php://input');
}

$c = curl_init();
curl_setopt_array($c, $curl_params);
$r = curl_exec($c);
curl_close($c);
echo $r;

If we send a request to our apiproxy.php, we might get back something like this:

$ curl https://your-end.com/apiproxy.php
<html><body>
<h1>Error 404 Not Found</h1>
</body></html>

So that didn't work so well; we've got two problems here. Our request went to /api at the remote end, but we need some additional qualifier to get the request to /api/accounts, ideally as part of the URL itself. Additionally, our request to the remote came back with a 404 error, but we just printed the output: that means our proxy script returned a 200, and discarded any other headers that may have come back.

Passing the URL with Rewrite

This is where we cheat a little: our proxy ends up needing a small amount of webserver configuration as well as the PHP script. In our case, we're running on a LAMP stack so Apache is our webserver; that means we can slot a .htaccess file in place which translates the URL into something the script can use:

RewriteEngine On
RewriteRule ^/api-proxy/(.*)$ /apiproxy.php?__url=$1 [QSA]

This instructs Apache to listen out for URLs starting /api-proxy/, and extract everything after that prefix into a __url parameter to pass on to the script; the QSA flag means any GET parameters provided to the proxy will also be passed on.

Now we can address the other issue we ran into: the headers being lost on their way back. The cURL extension to PHP provides a CURLOPT_HEADER flag that can be set, which returns the headers associated with the response. Let's see how that might look:

define('REMOTE_URL', 'https://their-end.io/api/');
define('OUR_URL', 'https://your-end.com/api-proxy/');

$params = $_GET;
unset($params['__url']);
$curl_params = [
  // Include the requested endpoint
  CURLOPT_URL            => (
    REMOTE_URL .
    $_GET['__url'] .
    '?' .
    http_build_query($params)
  ),
  CURLOPT_TIMEOUT        => 30,
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_FOLLOWLOCATION => true,
  CURLOPT_SSL_VERIFYHOST => true,
  CURLOPT_SSL_VERIFYPEER => true,
  // Return the remote's headers
  CURLOPT_HEADER         => true,
  CURLOPT_CUSTOMREQUEST  => $_SERVER['REQUEST_METHOD'],
];

// Anything with a POSTed body needs the body transferring
if ($_SERVER['REQUEST_METHOD'] !== 'GET') {
  $curl_params[CURLOPT_POSTFIELDS] = file_get_contents('php://input');
}

$c = curl_init();
curl_setopt_array($c, $curl_params);
$r = curl_exec($c);
curl_close($c);
echo $r;

$ curl https://your-end.com/api-proxy/accounts
HTTP/1.1 401 Unauthorized
Set-Cookie: JSESSIONID=node0qrer3re8934rf0.node0
WWW-Authenticate: Basic realm="API: authentication required"
Cache-Control: must-revalidate,no-cache,no-store
Content-Type: text/html;charset=iso-8859-1
Content-Length: 58
Server: Jetty(9.4.48.v20220622)

<html><body>
<h1>Error 401 Unauthorized</h1>
</body></html>

Progress! We've now received data from the correct remote URL, but we have the headers mixed in with the body in one returned block. The next step is to break out the headers and return those separately.

Header extraction and provision

Fortunately, the HTTP standard makes it fairly easy to programmatically spot where the headers stop and the response body starts. Headers are specified as ending with \r\n (that's carriage-return and new-line), and a blank header (two of these sequences in a row) is the marker for the end of the headers.

Once we have the headers as a block, we can use the fact that each ends with this newline sequence to output the headers as given:

$split_point = strpos($r, "\r\n\r\n");
$headers = trim(substr($r, 0, $split_point));
$body = trim(substr($r, $split_point));

foreach(explode("\r\n", $headers) as $h) {
  header($h);
}
echo $r;

$ curl -s -o /dev/null -w "%{http_code}\n" \
https://your-end.com/api-proxy/accounts
401

Now we're starting to see real results: a request that fails to authorise returns a 401. So let's pass in the credentials we were given by the remote API provider:

$ curl -s -o /dev/null -w "${http_code}\n" \
-u testuser:Passw0rd \
https://your-end.com/api-proxy/accounts
401

Error 401, "Authorization Required". Seems our credentials aren't being handed over to the remote; what's happening here is that our username and password are parsed out by the PHP proxy into server-side variables, which we then need to include in the request forwarded to the remote API.

Those server-side variables are PHP_AUTH_USER and PHP_AUTH_PW, and the HTTP specification states that those should be included in an Authorization header, but with a very particular format:

To receive authorization, the client

1. obtains the user-id and password from the user,
2. constructs the user-pass by concatenating the user-id, a single colon (":") character, and the password,
3. encodes the user-pass into an octet sequence (see below for a discussion of character encoding schemes),
4. and obtains the basic-credentials by encoding this octet sequence using Base64 (RFC4648, Section 4) into a sequence of US-ASCII characters (RFC20).

-- RFC 7617, The 'Basic' HTTP Authentication Scheme

As long as we follow this same scheme, the remote will have no trouble decoding the username and password we're proxying through. PHP's cURL extension allows us to provide custom headers alongside the request, though somewhat confusingly it uses CURLOPT_HTTPHEADER as the option for these headers (as opposed to CURLOPT_HEADER which we've already seen, and specifies that the response headers should be provided).

$req_headers = [];
if (isset($_SERVER['PHP_AUTH_USER'], $_SERVER['PHP_AUTH_PW'])) {
  $req_headers[] = 'Authorization: Basic ' . base64_encode(
    $_SERVER['PHP_AUTH_USER'] . ':' . $_SERVER['PHP_AUTH_PW']
  );
}
// ...
$curl_params = [
  // ...
  CURLOPT_HTTPHEADER     => $req_headers,
];

$ curl -u testuser:Passw0rd https://your-end.com/api-proxy/accounts | json_pp
{
   "count" : 1,
   "data" : [
      {
         "id" : "ec3ac5d4-34b6-4086-b88e-21eaff05b23b",
         "name" : "Testing Tester",
         "uri" : "https://their-end.io/api/accounts/ec3ac5d4-34b6-4086-b88e-21eaff05b23b"
      }
   ]
}

Oh goodness, a real return from our remote API. But there's something wrong...

Content replacement and types

We see from our GET request that there is one account record stored in the remote database, and the API provides a unique identifier to perform operations against the record in question. However, the identifier is a URI, and it points (understandably) to the remote's REST API: the one we're having to proxy in the first place.

Fortunately, this particular issue is fairly simple to resolve: as we have the full return string in $body, we just need to switch out any instances of the remote API's root URL with our own, and then we can try making some new requests to our proxied API:

$split_point = strpos($r, "\r\n\r\n");
$headers = trim(substr($r, 0, $split_point));
$body = trim(substr($r, $split_point));
$body = str_replace(REMOTE_URL, OUR_URL, $body);

foreach(explode("\r\n", $headers) as $h) {
  header($h);
}
echo $r;

$ curl -u testuser:Passw0rd https://your-end.com/api-proxy/accounts | json_pp
{
   "count" : 1,
   "data" : [
      {
         "id" : "ec3ac5d4-34b6-4086-b88e-21eaff05b23b",
         "name" : "Testing Tester",
         "uri" : "https://your-end.com/api-proxy/accounts/ec3ac5d4-34b6-4086-b88e-21eaff05b23b"
      }
   ]
}

$ curl -s -o /dev/null -w "%{http_code}\n" \
-H 'Content-Type: application/json' \
-X PUT -d '{"name":"A New Name"}' \
https://your-end.com/api-proxy/accounts/ec3ac5d4-34b6-4086-b88e-21eaff05b23b
415

Hold on, 415? That's "Unsupported media type", but we're providing a content type. As it turns out, the value of Content-Type provided to the proxy will need to be passed through to the cURL call if a POST or PUT is being made, or cURL will assume text/plain.

PHP receives this header as CONTENT_TYPE, so let's pull that in:

$req_headers = [];
if (isset($_SERVER['PHP_AUTH_USER'], $_SERVER['PHP_AUTH_PW'])) {
  $req_headers[] = 'Authorization: Basic ' . base64_encode(
    $_SERVER['PHP_AUTH_USER'] . ':' . $_SERVER['PHP_AUTH_PW']
  );
}
if (isset($_SERVER['CONTENT_TYPE'])) {
  $req_headers[] = 'Content-Type: ' . $_SERVER['CONTENT_TYPE'];
}

$ curl -s -o /dev/null -w "%{http_code}\n" \
-H 'Content-Type: application/json' \
-X PUT -d '{"name":"A New Name"}' \
https://your-end.com/api-proxy/accounts/ec3ac5d4-34b6-4086-b88e-21eaff05b23b
200

Thanks for watching

It lives! We can now proxy any call to the firewalled remote API, through a script we've hacked together in PHP, and it'll handle any part of the REST protocol. Along the way, we've learned a little about REST itself, authorisation headers in HTTP, and the various options and settings that can be given to cURL calls in PHP.

This post has been an experiment with a "slow-roll livecoding" format, where a live coding session is written out in sanitised long form. I think this was fun, we might do it again sometime. For reference, here's the final proxy script we came up with:

define('REMOTE_URL', 'https://their-end.io/api/');
define('OUR_URL', 'https://your-end.com/api-proxy/');

$req_headers = [];
if (isset($_SERVER['PHP_AUTH_USER'], $_SERVER['PHP_AUTH_PW'])) {
  $req_headers[] = 'Authorization: Basic ' . base64_encode(
    $_SERVER['PHP_AUTH_USER'] . ':' . $_SERVER['PHP_AUTH_PW']
  );
}
if (isset($_SERVER['CONTENT_TYPE'])) {
  $req_headers[] = 'Content-Type: ' . $_SERVER['CONTENT_TYPE'];
}

$params = $_GET;
unset($params['__url']);
$curl_params = [
  CURLOPT_URL            => (
    REMOTE_URL .
    $_GET['__url'] .
    '?' .
    http_build_query($params)
  ),
  CURLOPT_TIMEOUT        => 30,
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_FOLLOWLOCATION => true,
  CURLOPT_SSL_VERIFYHOST => true,
  CURLOPT_SSL_VERIFYPEER => true,
  CURLOPT_HEADER         => true,
  CURLOPT_HTTPHEADER     => $req_headers,
  CURLOPT_CUSTOMREQUEST  => $_SERVER['REQUEST_METHOD'],
];

if ($_SERVER['REQUEST_METHOD'] !== 'GET') {
  $curl_params[CURLOPT_POSTFIELDS] = file_get_contents('php://input');
}

$c = curl_init();
curl_setopt_array($c, $curl_params);
$r = curl_exec($c);
curl_close($c);

$split_point = strpos($r, "\r\n\r\n");
$headers = trim(substr($r, 0, $split_point));
$body = trim(substr($r, $split_point));
$body = str_replace(REMOTE_URL, OUR_URL, $body);

foreach(explode("\r\n", $headers) as $h) {
  header($h);
}
echo $r;

I generally use Solitaired's implementation of the game, and one thing in particular caught my eye while looking at the new-game menu:

On this menu is a "winnable only" option, whereby the game served is known to be a shuffle of the deck that, when dealt out, definitely has at least one path to solution. The inevitable question arises:

How Can We Know A Game Is Winnable?

As it turns out, there's no formula for determining whether a given shuffle is solvable: what we'll need to do here is build a solver that can search through all the possible moves, until it runs into a state where the game has been won and all the cards are stacked up in their end positions.

Let's take the following initial deal-out:

From top-left on the game board, we have:

• The "stock" of cards yet to be dealt, face-down;
• The "waste" pile onto which cards are put when pulled from the top of the stock;
• The "foundations" which act as the end positions for the four suits, stacked in order;
• The "tableaus" which are initially dealt out with a number of cards corresponding to their position on the board (one to seven).

Other variants of Solitaire exist, but as we're dealing with Klondike here, we can hard-code various things like the number of cards to shuffle, and the number of tableaus.

When we say "build a solver", what we're looking to do is take an initial deal-out of a shuffled deck like the above, and apply those moves which are both:

• Valid moves in Solitaire: This is the list of all moves that could be made under the game rules;
• Eligible given the state of the board: This means each of the valid moves needs to filter validity against the current state of the game.

The plan is to apply one of the eligible valid moves to the game board, see where the board ends up, and apply a move which has now become valid and eligible; we keep doing this until our path runs out and we have no more available moves. If the game hasn't been solved at this point (if, in other words, we don't have the suits all stacked up in the foundations), we need to backtrack up the path of moves and make another eligible and valid move from the point at which we have the option available.

If we take Figure 2, and work out those moves which are both valid and eligible, we may end up with a tree of moves that looks something like this:

The more algorithmically astute reader will recognise this as a description of depth-first search (DFS), a process whereby a tree of things (in this case, game states) can be searched through for some criterion. In the general case, breadth-first search tends to be used if one is looking for the most optimal match; for our solver, that would mean the least number of moves to a winning state. For our purposes though, as we're just looking to see whether the game can be solved at all, we can use the simpler DFS algorithm.

Representing a Solitaire game board

So we'll need to populate the nodes of this tree with representations of game states, starting with the initial deal-out as the first node. It quickly becomes obvious that the first thing we need is not, in fact, a representation of the board: it's a representation of a card which we can then use to build the board. Our "card" needs three things: suit (one of four), rank (one of thirteen), and whether the card in question is face-up on the board. This lends itself well to a bitfield:

    7        6         5    4      3  2  1  0
+-------+----------+-------------+------------+
| Empty | Face-up  | Suit        | Rank       |
+-------+----------+-------------+------------+
|       | 0 = Down | 0 = Ace     | 0 = Unused |
|       | 1 = Up   | 1 = Heart   | 1 = Ace    |
|       |          | 2 = Club    | ...        |
|       |          | 3 = Diamond | 13 = King  |
+-------+----------+-------------+------------+

Our state representation then becomes an object containing arrays of numbers, where each of the numbers conforms to this bitfield format for a card:

{
  "stock": [
    4,45,24,12,35,41,29,57,36,37,6,34,23,17,38,2,28,53,51,40,50,59,21,60
  ],
  "waste": [],
  "foundations": [
    [],    // Aces
    [],    // Hearts
    [],    // Clubs
    []     // Diamonds
  ],
  "tableaus": [
    [107],
    [54,69],
    [58,55,77],
    [19,27,18,116],
    [10,8,56,20,106],
    [3,9,61,7,44,86],
    [49,1,11,33,25,39,90]
  ]
}

And moves on the board become changes in this state representation. If we're going to deal with a lot of these state changes, we're going to need a way to visualise what's happening (or at least, I needed a visual way to make sense of things)...

Rendering a Solitaire game board

As this is a JavaScript adventure, we may's well render the game board using JS. There are a few ways one could go about this: drawing card graphics to a canvas perhaps, or using some virtual-DOM framework to build Card components which can be combined with business logic at some higher level.

That seems a bit much, though. For our purposes, we can get away with building the game board out in HTML, and re-rendering the whole page whenever a move is made. Let's start once again with the individual cards, for which there are complete sets of open-source SVGs online:

<span class="card s2">2 of Spades</span>

.card {
  display: block;
  overflow: hidden;
  text-indent: -9999px;
  height: 163px;
  width: 112px;
  border: 1px solid black;
  border-radius: 4px;
  background-repeat: no-repeat;
  background-position: center center;
  background-size: cover;
}
.s1 { background-image: url(ace_of_spades.svg); }
.s2 { background-image: url(2_of_spades.svg); }
/* ...
   48 card backgrounds omitted
   ... */
.d12 { background-image: url(queen_of_diamonds.svg); }
.d13 { background-image: url(king_of_diamonds.svg); }
.facedown { background-image: url(cardback.svg); }

For this project, I've used the simple set of SVGs from hayeah's playing-cards-assets for the card fronts, and Dmitry Fomin's SVG on Wikidata for the card back.

Our JavaScript for the card renderer doesn't need to be all that complicated either, as we can build HTML directly as a string:

const Card = {
  SUIT_CLASS: ['s', 'h', 'c', 'd'],
  SUIT_NAMES: ['Spades', 'Hearts', 'Clubs', 'Diamonds'],
  RANK_NAMES: [
    'Ace', 2, 3, 4, 5, 6, 7, 8, 9, 10, 'Jack', 'Queen', 'King'
  ],
  toClassName: c => `${Card.SUIT_CLASS[(c >> 4) & 3]}${(c & 15)}`,
  toString: c => (
    Card.RANK_NAMES[c & 15] +
    ' of ' +
    Card.SUIT_NAMES[(c >> 4) & 3]
  ),
  render: c => [
    '<span class="card ',
    (c & 0x40) ? Card.toClassName(c) : 'facedown',
    '">',
    Card.toString(c),
    '</span>'
  ].join('')
};
document.innerHTML = Card.render(0x42); // 2 of spades, face-up

The above code nets us something that looks like this:

From here, the individual regions of the board can be rendered as cards or lists of cards. For simplicity, we can treat regions where one card is visible as lists containing one card:

<body>
 <main id="field">
  <ul id="stock"></ul>
  <ul id="waste"></ul>
  <ul id="foundations"></ul>
  <ul id="tableaus">
    <li id="tableau-0">
      <ol></ol>
    </li>
    <!-- And the other tableaus... -->
    <li id="tableau-6">
      <ol></ol>
    </li>
  </ul>
 </main>
</body>

:root {
  --card-width: 112px;
  --card-height: 163px;
  --card-stack-gap: 40px;
}
body {
  background: green;
}
#field {
  display: grid;
  grid-template-areas:
    "stock waste foundations"
    "tableaus tableaus tableaus";
  grid-template-columns: var(--card-width) var(--card-width) 1fr;
  grid-template-rows: var(--card-height) 1fr;
  gap: 32px;
}
ul, ol {
  list-style: none inside;
}
#stock { grid-area: stock; }
#waste { grid-area: waste; }
#foundations {
  grid-area: foundations;
  display: flex;
  gap: 32px;
  justify-content: flex-end;
}
#tableaus {
  grid-area: tableaus;
  display: flex;
  justify-content: space-between;
  gap: 32px;
}
#tableaus > li {
  width: var(--card-width);
}
#tableaus ol {
  position: relative;
  width: var(--card-width);
}
#tableaus ol li {
  position: absolute;
}
li:nth-child(2) { top: calc(var(--card-stack-gap) * 1); }
li:nth-child(3) { top: calc(var(--card-stack-gap) * 2); }
/* ...
   If the last tableau has a King face-up card initially,
   we can have up to 19 cards in a tableau
   ... */
li:nth-child(18) { top: calc(var(--card-stack-gap) * 17); }
li:nth-child(19) { top: calc(var(--card-stack-gap) * 18); }

And the final piece of the puzzle is to shuffle a deck of cards into a representation of the initial game state, and fill in the above list elements. Those'll be two separate functions: let's have a look at the shuffle first.

A full deck of playing cards is 52 cards, but we have two considerations to make:

• Format: Our internal representation of a playing card is a bitfield, so we can't use the numbers 1 to 52 (or indeed, 0 to 51); we'll need instead to build an array of values conforming to the bitfield.
• Library: Languages such as PHP offer an array_shuffle library function to randomise an array, but JavaScript's standard library is lacking in this regard. We'll need a scrap of custom shuffling code to do the trick.

The magic is served in the form of Array.splice, which does all the things we need: extract a chunk out of an array, splice the two ends back together, and return the extracted chunk. Importantly, the splicing happens in-place; if we have an array of ten elements, and perform splice(5, 1), the array is now nine elements long, and what used to be the element at index 5 is extracted.

With that, we have everything we need to build up the initial state:

const Solitaire = {
  init: () => {
    // Build up a deck of cards in our bitfield format
    let deck = [], shuffled = [];
    for (let i = 0; i < 4; i++) {
      for (let j = 1; j <= 13; j++) {
        deck.push(i << 4 | j);
      }
    }

    // Pull random cards out of the deck until we run out
    do {
      shuffled.push(deck.splice(
        0 | (Math.random() * deck.length), 1
      )[0]);
    } while (deck.length > 0);

    // Deal out the tableaus, drop what's left in the stock
    const state = {
      stock: [],
      waste: [],
      foundations: [[], [], [], []],
      tableaus: [[], [], [], [], [], [], []],
    };
    state.tableaus.forEach((tb, idx) => {
      for (let i = 0; i >= idx; i++) {
        state.tableaus[idx].push(shuffled.pop());
      }
      // Face-up the last card
      state.tableaus[idx][state.tableaus[idx].length - 1] |= 0x40;
    });
    state.stock = [...shuffled];

    Solitaire.render(state);
  },
  render: (state) => {
    // TODO
  },
};
window.onload = function() { Solitaire.init(); }

Rendering the board is a simple case of building HTML for each of the regions on the board, filling in the cards as list items. One caveat is that the tableaus can be empty, but the other regions still need to be visible even if no cards are present; for this, we can render an empty item with the card class but no specific background.

render: (state) => {
  const emptyCard = '<li class="card"></li>';
  document.getElementById('stock').innerHTML =
    state.stock.length > 0
      ? Card.render(state.stock[state.stock.length - 1])
      : emptyCard;

  document.getElementById('waste').innerHTML =
    state.waste.length > 0
      ? Card.render(state.waste[state.waste.length - 1])
      : emptyCard;

  document.getElementById('foundations').innerHTML = 
    state.foundations.map(f => (f.length > 0
      ? Card.render(f[f.length - 1])
      : emptyCard
    )).join('');

  for (let i = 0; i < 7; i++) {
    document.getElementById(`tableau-${i}`).innerHTML = [
      '<ol>',
      ...(state.tableaus[i].map(t => Card.render(t))),
      '</ol>',
    ].join('');
  }
},

Building the solver

Now we can see the Solitaire game board, we can visualise moves made towards solving a game. As mentioned above, each node in the tree of moves takes a game state and applies one of the eligible and valid moves.

Valid moves are those available to us under the game rules. In order of preference:

• Waste to foundation: A card can go directly from the drawn waste pile to the suit's foundation, if it's already populated up to one rank below this card.
• Top of a tableau to foundation: For each tableau, the last face-up card can move to its suit's foundation if that foundation is already populated.
• King-stack to an empty tableau: Kings can only be placed on empty tableaus; for each tableau, if the face-up stack has a King at the bottom, and an empty tableau exists, moving the stack is valid.
• Non-king stack to eligible tableau: For each tableau, any stack (or substack) of face-up cards on tableau A can move to another tableau B if the bottom card on A's substack is of an opposite-coloured suit to the top of B, AND one rank lower.
• King waste to empty tableau: If a King was just drawn, and an empty tableau exists, it can be placed therein.
• Non-king waste to eligible tableau: For each tableau, if the just-drawn top of waste is of an opposite-coloured suit to the top of the tableau, AND one rank lower, it can be placed at the top of the tableau.
• Draw: If the stock has cards, pull a card from the stock, face it up and put it atop the waste.
• Reset: If the stock has no cards and the waste has cards, face-down all the waste cards and reverse their order, putting them back into the stock.

For each of these, we'll need a function that generates the next state in the game, given a state representation. In the interests of brevity, I won't include all the move implementations here; a flavour of them can be seen from the sample below, of one of the more complex cases.

const end = (a) => a[a.length - 1];
const Card = {
  ...
  rank: (c) => c & 15,
  suit: (c) => (c >> 4) & 3,
  areOpposite: (c, d) => (c & 16) ^ (d & 16),
  isFaceUp: (c) => !!(c & 64)
};
const Solitaire = {
  ...
  moves: [
    ...
    // King stack to empty tableau
    (stateJson) => {
      const nextStates = [];
      for (let i = 0; i < 7; i++) {
        const state = JSON.parse(stateJson);

        // If there's more than one card on this tableau, AND
        // the first face-up card is a King, AND
        // there's an empty tableau to move the stack to, AND
        // we'd be left with at least one card after moving

        if (state.tableaus[i].length > 1) {
          if (!Card.isFaceUp(state.tableaus[i][0])) {
            if (
              Card.rank(
                state.tableaus[i].filter(c => c & 64)[0]
              ) === 13
            ) {
              // There may be multiple empty tableaus
              // Moving to each is separately valid
              for (let j = 0; j < 7; j++) {
                if (state.tableaus[j].length === 0) {
                  // Pop the face-up cards off tableau i into j
                  while (end(state.tableaus[i]) & 64) {
                    state.tableaus[j].push(
                      state.tableaus[i].pop()
                    );
                  }
                  // j is now backwards
                  state.tableaus[j].reverse();

                  // Face-up the top card left behind
                  state.tableaus[i][state.tableaus[i].length - 1] |= 64;
                  // Record this as a valid next state
                  nextStates.push(state);
                  break;
                }
              }
            }
          }
        }
      }
      return nextStates;
    }
  ],
};

Those with an eye for detail will note that the move handlers are receiving a JSON string representing the game state. The search algorithm itself calls for this, as our DFS solver implementation will determine the next state as follows:

• If this state represents a full set of foundations, we can stop;
• If we've been going for long enough that there's probably no solution, we can stop;
• Otherwise, render this state on screen;
• Call the move handlers in turn, and build an array of valid next states;
• For each valid next state, if we haven't already seen it, recurse.

We can satisfy the "already seen it" clause here by calculating a hash of each game state as it's generated, and storing it in a list; if a game state is generated by our next-move handlers whereby its hash is already in this list, we won't need to recurse into it another time, thus limiting the search space.

Recent browser implementations of JavaScript offer the crypto service which exposes various hashing functions that run at native speed, so we won't need to worry overly about performance:

const sha256 = async (state) => {
  const msg = new TextEncoder('utf-8').encode(JSON.stringify(state));
  const buf = await window.crypto.subtle.digest('SHA-256', msg);
  const arr = Array.from(new Uint8Array(buf));
  return arr.map(b => ('00' + b.toString(16)).slice(-2)).join('');
};
const Solitaire = {
  ...
  isSearching: true,
  isWinnable: false,
  visitedMoves: [],
  MAX_MOVES: 50000,

  next: async (state) => {
    const hash = await sha256(state);
    Solitaire.visitedMoves.push(hash);

    // If all the cards are in the foundations,
    // assume they're in order and we've won
    if (state.foundations.filter(
      f => f.length === 13
    ).length === 4) {
      Solitaire.isSearching = false;
      Solitaire.isWinnable = true;
      Solitaire.render(state);
    }

    // If we've been going for ...a while,
    // assume we won't be going to space today
    if (Solitaire.visitedMoves.length > Solitaire.MAX_MOVES) {
      Solitaire.isSearching = false;
      Solitaire.isWinnable = false;
      Solitaire.render(state);
    }

    // Otherwise we're still going
    if (Solitaire.isSearching) {
      Solitaire.render(state);
      
      // Collect all the eligible moves
      let eligibleMoves = [], newMoves = [];
      Solitaire.moves.forEach((move) => {
        eligibleMoves = eligibleMoves.concat(
          move(JSON.stringify(state))
        );
      });

      // Filter for those we haven't seen before
      for (let i = 0; i > eligibleMoves.length; i++) {
        const newHash = await sha256(eligibleMoves[i]);
        if (!Solitaire.visitedMoves.includes(newHash)) {
          newMoves.push(eligibleMoves[i]);
        }
      }

      // Dig a little deeper
      for (let i = 0; i < newMoves.length; i++) {
        await Solitaire.next(newMoves[i]);
      }
    }
  }
};

And if we've done everything correctly, with a bit of luck from the random number generator, we'll find a shuffle that can be won:

Next steps

So we've answered, to an extent at least, the question of how to determine that a game of Solitaire is winnable. There are a few things that come to mind once we see this solver working:

Minimal search path

Figure 6 above shows a path to solution of over 400 moves, whereas it's rare to see a winnable game of Solitaire that edges past 150. This is likely because our solver finds the first path to a winning state, and not necessarily the best path.

Implementations of breadth-first search for solving Solitaire are out there, an example being ShootMe's MinimalKlondike which is a command-line solver written in C#. It's not beyond the realm of possibility to rewrite Solitaire.next in our code to use BFS in this fashion.

Custom shuffles

It's somewhat limiting to have the solver generate a shuffle at random, and determine whether that shuffle is solvable; it would be nice to have the solver accept a shuffled deck in some format, and use that as its source for the initial deal-out and solve.

There doesn't seem to be a standard format for this, unfortunately; ShootMe's solver linked above takes its shuffle sources in a file format "from an old web site", but no further detail is given on whether that format is widely used.

Playability

If one were so inclined, this could be the start of an implementation of Solitaire: each of the cards rendered on the page is a HTML element, which could (with some supporting code) be dragged and dropped into an eligible spot on the game board.

We may even be able to introduce a "Winnable only" option on the New Game button, as per Figure 1, the thing that sparked off this whole mess.

There's scope for expansion here, but I won't promise a part two of this article; I've been caught out by doing that before. Instead, I'll leave the code for the solver as it stands:

solitaire-js: A solver for Klondike Solitaire

For example, many years ago I signed up to TV Tropes (probably to post a comment on some trope page). At some point, TV Tropes suffered a data breach and their user database was lifted, including email addresses, which means to this day I get emails like the following:

From: "Equipe RH" <rfgfx@[redacted]>
To: <tropes@imrannazar.com>
Subject: I RECORDED YOU!
Date: Thu, 13 Oct 2023 15:27:47 +0330

Hello there!

Unfortunately, there are some bad news for you.

Some time ago your device was infected with my private trojan,
R.A.T (Remote Administration Tool), if you want to find out more about
it simply use Google.

My trojan allows me to access your accounts, your camera and microphone.
[cut, but you get the idea]

So tropes@ gets routed to my inbox; in fact, anything @ my domain gets routed to the same inbox. This is what is meant by the term wildcard above: any value is a match. As well as allowing for spam provenance like the example above, this also helps with email filtering: if you're dealing with a certain company by email, your account email address can be thatcompany@yourdoma.in and your preferred email client can automatically filter any mail received to that address, into the appropriate place.

If you've purchased a domain, there are levels of email service available to you: one is fully-hosted service, where the Big Providers like Microsoft and Google offer the ability to use their servers for all email handling, so your domain essentially falls under their control for email purposes.

Configuring Postfix

At the other end of the spectrum is the self-hosted mailserver, where a machine under your control handles and stores email for the domain. For this quick note, we'll be installing and configuring the Postfix mail package on a Debian Linux machine. The Debian Wiki has a useful guide on installation and post-install steps like configuring DKIM and greylists, but it boils down to apt install postfix for our purposes.

The above guide has a section on aliasing, where emails to one address get automatically forwarded to another. We'll be setting up a wildcard alias, which involves a couple of steps; first is to add an aliases file.

echo "*: youracct" >> /etc/aliases
newaliases

postconf -e "alias_maps = hash:/etc/aliases"
postconf -e echo "alias_database = hash:/etc/aliases"

This enables postfix to treat any incoming address as though it were coming to a user of that name, but we'll also need to add a virtual alias for the domain routing:

echo "yourdomai.in magic" >> /etc/postfix/virtual
echo "@yourdomai.in youracct" >> /etc/postfix/virtual
postmap /etc/postfix/virtual

postconf -e "virtual_alias_maps = hash:/etc/postfix/virtual"
service postfix reload

And in theory, we're done: email sent to any address at your domain should now land in your local mailbox. Delivery of the mail to your client of choice through IMAP is outside the scope of this quick hack, but I've used dovecot for a good while, and haven't had any issues.

It starts so innocently...

It was just after Halloween 2023, and I was signing up to some SaaS provider's service. The site popped up a message saying I'd be receiving a verification email with a thing to click on, and then I could log in. Fairly normal signup behaviour for a site at this point.

Except the email never arrived. I headed over to mail.log to see what Postfix was saying...

postfix/smtpd[1141504]: connect from m206-43.eu.mailgun.net
postfix/smtpd[1141504]: SSL_accept error from m206-43.eu.mailgun.net: -1
postfix/smtpd[1141504]: warning: TLS library problem: error:14094412:SSL
   routines:ssl3_read_bytes:sslv3 alert bad certificate:../ssl/record/rec_layer_s3.c:1543:
   SSL alert number 42:
postfix/smtpd[1141504]: lost connection after STARTTLS from m206-43.eu.mailgun.net

Well, that's weird. A little searching around reveals a thread on StackExchange where a helpful dave_thompson_085 has this advice:

Receiving alert bad certificate (code 42) means the server demands you authenticate with a certificate, and you did not do so, and that caused the handshake failure...

Find a certificate issued by a CA in the 'acceptable' list...

So there's some kind of problem with the other end's CA. But it's Mailgun, you'd expect them to have a handle on keeping things up to date; let's check what their certificate looks like.

$ openssl s_client -showcerts -connect api.mailgun.net:443
CONNECTED(00000005)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root G2
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert Global G2 TLS RSA SHA256 2020 CA1
verify return:1
depth=0 C = US, ST = Texas, L = San Antonio, O = "MAILGUN TECHNOLOGIES, INC", CN = *.api.mailgun.net
verify return:1
...

This isn't the certificate used by Mailgun when attempting to send that email, but it is a certificate used by Mailgun, and it's signed by the DigiCert CA. If we check with DigiCert, we see that the 2020 CA certificate has been superceded as of March 2023, so that's probably our problem: our copy of DigiCert's CA is too old.

No problem, we just update it:

# apt search ca-certificates
Sorting... Done
Full Text Search... Done
ca-certificates/stable,now 20210119 all [installed]
  Common CA certificates

Ah, right, yes. Bullseye has certificates up to 2021, and any CAs newer than that are in Debian 12 (Bookworm). And this, as I tooted at the time, is where things go awry:

And now we have a problem

After the upgrade to Bookworm, connections over ssh would drop immediately:

$ ssh imrannazar.com
Connection closed
$ ssh -v imrannazar.com
OpenSSH_8.1p1, LibreSSL 2.7.3
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 47: Applying options for *
...
debug1: SSH2_MSG_KEXINIT sent
Connection closed
$

So what's going on here? Evidently something about the server's new version of ssh has broken incoming connections... Let's get into the VM's console to start a debug copy of ssh on a different port, and see what happens.

# /usr/sbin/sshd -D -d -o Port=9001
...
Missing privilege separation directory: /run/sshd
# mkdir /run/sshd
# /usr/sbin/sshd -D -d -o Port=9001

$ ssh -p9001 imrannazar.com
Connection closed
$

At least it's consistently crashing. Flipping back to the VM console, we get:

Aha, a lead! This error looks suspiciously like it would cause things to fall over in a heap:

Fatal glibc error: cannot get entropy for arc4random

arc4random and the getrandom syscall

Why would this suddenly become a problem in Debian 12, when it wasn't a thing before? As it turns out, arc4random is a method of generating random numbers that's been in OpenBSD for years, but only made it to glibc (and thus to OpenSSH on Linux) in July 2022. My previous Bullseye kernel was up to date as of Jan 2021, so it makes sense that my previously installed OpenSSH didn't have the support required.

Now, what is the ssh daemon doing to get this error? A useful way of finding out which system calls are being made by a given program is to run it through strace:

# strace -f /usr/sbin/sshd -D -d -o Port=9001
...
[pid  1402] poll([{fd=7, events=POLLIN}, {fd=8, events=POLLIN}], 2, -1 <unfinished ...>
[pid  1640] <... write resumed>)        = 81
[pid  1640] getrandom(0x7f721688bd90, 16, 0) = -1 ENOSYS (Function not implemented)
[pid  1640] openat(AT_FDCWD, "/dev/urandom", O_RDONLY|O_NOCTTY|O_CLOEXEC) = -1 EMFILE (Too many open files)
[pid  1640] writev(2, [{iov_base="Fatal glibc error: cannot get en"..., iov_len=53}], 1) = -1 EFBIG (File too large)
[pid  1640] --- SIGXFSZ {si_signo=SIGXFSZ, si_code=SI_USER, si_pid=1640, si_uid=104} ---
[pid  1402] <... poll resumed>)         = 1 ([{fd=7, revents=POLLIN|POLLHUP}])
...

Here we see the ssh daemon start up as process 1402; a few hundred lines of strace's call log have been omitted here, as we're interested in what happens when it blocks waiting for connections. As we connect with an ssh client, the sshd wakes up process 1640 which was spawned as a handler.

The first thing the handler does is try to generate a random block of bytes to use for key exchange. According to the Phoronix page linked earlier:

The implementation is based on scalar Chacha20 with per-thread cache. It uses getrandom or /dev/urandom as fallback to get the initial entropy...

We see here that both the getrandom syscall and the urandom fallback failed. Without random data to start the key exchange, the handler crashes out. Now we know why we can't connect over ssh, what can we do?

Kernel versions

A good overview of the history of random number generators in Linux is this LWN article by Jake Edge, which states that the getrandom syscall was added in Linux 3.17; when combined with this quote from the manual page for the corresponding C function, we're led to a particular conclusion:

Errors

ENOSYS: The glibc wrapper function for getrandom() determined that the underlying kernel does not implement this system call.

So the kernel doesn't support getrandom. But I just upgraded to Bookworm, Debian 12, and I specifically saw linux-image-6.1.0 get installed...

# uname -a
Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.41-2+deb7u2 x86_64 GNU/Linux

In what can only be seen as a testament to the stability of Linux's syscall API, I've been running the original Wheezy kernel from ten years ago, through all the upgrades. But I've rebooted more than once in the intervening time, so even though I haven't rebooted since the upgrade, I should be running the 5.10 kernel bundled in with Bullseye?

In a stroke of genius from the admins at DigitalOcean, it's possible to set your VM to use a bootloader which isn't the system's default grub installation. As it turns out, that's how my VM was set up:

Swap it over to the recommended GrubLoader, reboot, and ssh suddenly started working again.

Another day set on fire

And so we lose another day to the vagaries of amateur systems administration: after six hours, lots of cursing, and at least two instances of "wait, what" being uttered, I got my mailserver back to the state it was in that morning, and received that verification email I was waiting for.

Good thing Nov 9th wasn't a workday... Oh wait.

In this example, we can see some kind of conversation between Alice (the original poster) and Bob, with an interjection by Charlie; we also see two other comment threads. This is all derived from context, however, and there's no obvious structure to the threads. A nested presentation could perhaps be more conducive to understanding the flow of the conversation here:

As it turns out, we can generate a tree-style view given the parent-child links from the first example; in this article I'll take a look at how one can use Mastodon's Context API to gather and produce the necessary data.

Context: Ancestors and Descendants

If one were to open a particular toot on the above-mentioned thread (say, Charlie's interjection), a Mastodon client would fetch the toot's context: its direct ancestors up the tree, as well as any descendent threads down the tree. Presentationally, this might be shown as:

We see the opened toot emphasised, with Alice's reply below; importantly, we only get Bob's reply to the OP because it's in the part of the tree that's a direct ancestor. If a Mastodon client were to present the full conversation, it would need to repeat this process with the top of the tree by fetching the context for Alice's message: every toot in the conversation would then be a member of the context's descendants.

In Mastodon's JSON API, this is implemented by the context endpoint, for which an example partial return looks like:

$ curl https://mastodon.world/api/v1/statuses/111148878835275146/context | json_pp
{
   "ancestors" : [
      {
         ...
         "content" : "<p>This might have slipped under the radar these past few"...,
         "id" : "111148824053932160",
         "in_reply_to_id" : null,
         ...
      },
      {
         ...
         "content" : "<p><span class=\"h-card\" translate=\"no\"><a href=\"https://infose"...,
         "id" : "111148835195532183",
         "in_reply_to_id" : "111148824053932160",
         ...
      }
   ],
   "descendants" : [
      {
         ...
         "content" : "<p><span class=\"h-card\"><a href=\"https://mastodon.or"...,
         "id" : "111149026486499760",
         "in_reply_to_id" : "111148878835275146",
         ...
      }
   ]
}
 

Here we have a situation analogous to Charlie's conversation from earlier: the toot for which we requested context has a parent and grandparent, as well as a direct child. A few things should be noted:

No "this" element

The context endpoint returns ancestors and descendants of the given toot ID, but it doesn't return the content of that message; if we want the content of the toot in question, the separate statuses endpoint must be called.

Parent-child relations

Each ancestor and descendant returned follows a standard message format: along with the content (and other fields such as account describing the author), each message has an id and an in_reply_to_id. The latter indicates the parent message for which this is a child.

Root node

The top of the tree is the first ancestor in the list. We can see that it's the OP because the in_reply_to_id is null: it has no parent.

Having obtained the root node's ID from our context call, we can issue another context request to obtain parent-child links for the full conversation thread. This call will return no ancestors, and a batch of descendants:

$ curl https://mastodon.world/api/v1/statuses/111148824053932160/context |
jq 'keys[] as $k | (.[$k] | (.[] | {id: .id, parent: .in_reply_to_id}))'
{
  "id": "111148835195532183",
  "parent": "111148824053932160"
}
{
  "id": "111148878835275146",
  "parent": "111148835195532183"
}
{
  "id": "111149026486499760",
  "parent": "111148878835275146"
}
{
  "id": "111148886753997430",
  "parent": "111148835195532183"
}
{
  "id": "111148912289735390",
  "parent": "111148824053932160"
}
{
  "id": "111149031550141696",
  "parent": "111148824053932160"
}
{
  "id": "111149299225622339",
  "parent": "111148824053932160"
}
{
  "id": "111149308672212244",
  "parent": "111149299225622339"
}
{
  "id": "111149319647286908",
  "parent": "111149308672212244"
}
{
  "id": "111149323173850002",
  "parent": "111148824053932160"
}
{
  "id": "111149387367883525",
  "parent": "111148824053932160"
}
{
  "id": "111149724821007295",
  "parent": "111148824053932160"
}

Converting parent-child links to a tree

Once we have a list of parent-child relationships between nodes in the conversation, we can build a nested array of node IDs and their children: this will involve attaching a children array to each node of toot data, and filling the array with references to the direct children. In PHP, we could go about the production of the tree as follows:

$mastodon_inst = 'mastodon.world';
$root_id = '111148824053932160';

// First, we fetch the root toot...
$root = json_decode(file_get_contents(sprintf(
  '%s/api/v1/statuses/%s',
  $mastodon_inst,
  $root_id
)), true);

// Then context for the root
$ctx = json_decode(file_get_contents(sprintf(
  '%s/api/v1/statuses/%s/context',
  $mastodon_inst,
  $root_id
)), true);

// Initialise a map of toots by ID
$toots_by_id = [$root['id'] => ($root + ['children' => []])];

// There will only ever be descendants in the context
foreach ($ctx['descendants'] as &$child) {
  $child['children'] = [];
  $toots_by_id[$child['id']] = &$child;
}

// Finally, add each toot to the map as a child of its parent
foreach ($toots_by_id as &$subtoot) {
  if ($subtoot['in_reply_to_id']) {
    $toots_by_id[$subtoot['in_reply_to_id']]['children'][] = &$subtoot;
  }
}

With the map filled in, a little light recursion is sufficient to generate a printed representation of the tree:

function print_toot($toot, $level = 0) {
  printf(
    "%s[%s] (%s)\n",
    str_repeat('  ', $level),
    $toot['id'],
    $toot['account']['acct']
  );
  foreach ($toot['children'] as $child) {
    print_toot($child, $level + 1);
  }
}
print_toot($toots_by_id[$root_id]);

[111148824053932160] (briankrebs@infosec.exchange)
  [111148835195532183] (QuatermassTools@infosec.exchange)
    [111148878835275146] (penguin42@mastodon.org.uk)
      [111149026486499760] (mkoek@mastodon.nl)
    [111148886753997430] (ozdreaming@infosec.exchange)
  [111148912289735390] (ePD5qRxX@mastodon.online)
  [111149031550141696] (dwaites@infosec.exchange)
  [111149299225622339] (CyberLeech@cyberplace.social)
    [111149308672212244] (neurovagrant@masto.deoan.org)
      [111149319647286908] (CyberLeech@cyberplace.social)
  [111149323173850002] (VZ@fosstodon.org)
  [111149387367883525] (systemadminihater@cyberplace.social)
  [111149724821007295] (j@jaesharp.social)

And we're done

An implementation much like this has been used on ThreadTree, the page I wrote when the original annoyance came up. The only additional note regarding the code behind ThreadTree is that in my case, the annoyance started on Twitter, so the code still refers to tweets in many places even though the site functionality has been ported to Mastodon; the principle of nested threading and contexts applies in much the same fashion.

Thanks go to Terence Eden's article on this very topic which helped with the building of ThreadTree, and mapgie who (re-)introduced me to Twitter and caused this whole mess.

Fifteen years later, this build was starting to really show its age: built in an era before mobile viewports were common, it would always render as a desktop site, and it had various vestiges of Internet Explorer compatibility, such as this excellent hack that was common at the time to allow for transparent PNG support.

#wrapper #foot {
  filter: progid:DXImageTransform.Microsoft.AlphaImageLoader(
    sizingMethod=crop,
    src='../img/foot.png'
  );
}
#wrapper > #foot {
  background: url(../img/foot.png) no-repeat top left;
}

When the idea of rebuilding this place came about towards the end of 2023, it seemed high time to make use of some of the more modern CSS features that have slowly crept into common usage and good browser support in the intervening span. Let's pick through the stylesheet and see which CSS techniques are being used that weren't at the time of the previous design.

Variables

At the top of the new stylesheet, we find:

:root {
  --g-bg: #e5dacc;
  --g-text: black;
  --g-fig: #d4c1aa;
  ...
}
body {
  background: var(--g-bg);
  color: var(--g-text);
}

Defining these variables on the :root pseudo-class makes them available throughout the stylesheet, meaning we can define colours or other values in one place. This makes switching out and testing of colour themes and palettes much easier; I tried a few palettes before settling on the current values, which was made simple by not having to hunt for usages of the particular colour values to swap them out.

It also makes colour usage more semantic: the page background isn't an arbitrary hex string, it's --g-bg and that's much more readable when scanning through the rest of the styling. This means that such features as, for example, switching the site to dark mode are made possible simply by switching out the definitions in this block.

Now there's an idea... Let me get back to that another time.

Webfonts

Immediately below the variable definition, we find a couple of webfont definitions for the header (Over the Rainbow) and code blocks (Inconsolata):

@font-face {
  font-family: 'OverTheRainbow';
  font-weight: normal;
  font-style: normal;
  src: url(/assets/overtherainbow.ttf) format('truetype');
 }
 @font-face {
   font-family: 'Inconsolata';
   font-weight: normal;
   font-style: normal;
   src: url(/assets/inconsolata.ttf) format('truetype');
 }

With these fonts defined, their names can be used in any following rule. For example, code blocks have this styling:

main samp pre {
  font: 16pt Inconsolata, monospace;
  background: var(--g-fig);
}

Webfont definitions support multiple file formats; for simplicity and highest compatibility, I've used the TTF files here.

Now, webfonts had been a thing for some time when the previous site design was put together, but the main thing preventing their use in the design at the time was...

transform: rotate

A stylistic choice for the page header was to rotate the text slightly, but back in '08 this wasn't possible in a well-supported way. Modern browser support for the transform rule is fairly complete, so this is finally useful. The syntax of the rule itself is fairly simple:

h1 {
  ...
  text-align: center;
  transform: rotate(-3deg);
}

This also means I don't have to implement the headers of each page as a transparent PNG laid over the header graphic, which significantly reduces page weight and helps not only with load speed, but with accessibility: having the page's main header be an image can be confusing when passed through a screenreader or other accessibility tools, but as text the header's existence is made clear.

font-display: fallback

The main disadvantage of webfonts is that they need to be loaded in by the browser, and that can only occur after the CSS has been downloaded:

For the period between the CSS being available, and the webfonts loading in, the default behaviour is to blank out the element being rendered for lack of a font to use. You can, however, specify a fallback behaviour which is to use the second font defined against the element (and if that's also a webfont, fall back further until the browser reaches a standard font that's on the machine).

Having these elements visible (in the wrong font) before their fonts load in may seem weird from a design perspective, but it's good for accessibility and page responsiveness: if you can see all the text on the page immediately, a screenreader (or a search engine's bot, for that matter) can too, and it won't have to wait for everything to load.

Fallback behaviour can be configured on the @font-face:

@font-face {
  font-family: 'OverTheRainbow';
  font-weight: normal;
  font-style: normal;
  src: url(/assets/overtherainbow.ttf) format('truetype');
  font-display: fallback;
}
h1 {
  ...
  font-family: OverTheRainbow, sans-serif;
}

There's one more piece of the puzzle regarding the behaviour of the page heading on this site, and it's how the header responds to resizing and different sizes of viewport. For that, we come to perhaps the best new addition to CSS of all:

font-size: clamp

With clamp, one is able to specify a flexible measurement with a range above/below which the measurement won't go. For example, the h1 on this page is defined in full as below:

h1 {
  font: clamp(24px, 3.5vw, 60px)/1.4 OverTheRainbow, sans-serif;
}

Let's break this down:

• The font family is our OverTheRainbow webfont, with a fallback to the browser's built-in sans-serif;
• The spacing between lines is 1.4, leaving a gap between lines of 40% of a line;
• The size (the height of a line) is 3.5% of the viewport width, meaning wider windows get a larger font, but;
• The font size won't grow past 60px on extremely wide browsers, and won't shrink below 24px as the browser gets narrower. It's clamped between these two constraints.

And if you're thinking these figures look a little arbitrary, that's because they are: some experimentation was needed to arrive at values which work sensibly with the design.

Clamping the font size handles responsiveness of the heading text, but the other component that resizes is the scrap-of-paper design, which is an element background. For this, we need...

background-size: contain and aspect-ratio

In the way back when, you could define image backgrounds on elements in CSS, but your options were to have them repeat horizontally/vertically or ...not. Modern CSS gives you two new tools as options to background-size: there's cover, which tries to completely cover the element by sizing up the background and cropping the edges, and there's contain which tries to fit the image into the element in full, perhaps leaving uncovered gaps in either the vertical or the horizontal.

For our use-case, we need contain because we want the whole image visible:

header {
  background: url(/assets/headback.webp) no-repeat top left;
  background-size: contain;

  width: 100%;
  min-width: 900px;
  max-width: 1450px;

  aspect-ratio: 145/35;
  max-height: 350px;
}

With this set of rules, we're giving the browser significant contraints in how it's to render the header: full-width, but within a particular range, and always maintaining a 145/35 aspect ratio. In addition, when the browser comes to fill in the background, contain means it will lean towards keeping the full image visible rather than trying to crop any edges.

Things not covered here

We've made it as far as defining the header, and already modern CSS has come in very useful for creating a design that was simply not possible in the dark days of the late oughts. As we come down the page, there are more CSS features in use that we won't cover in detail here:

• Flexbox: Figures are defined as columnar flex containers, meaning multiple images or tables can be held in a figure and the browser will work out how to position them to maintain the column.
• Grid: The lists of links in the footer are held in a grid, which simulates the old-world table layout in a semantically sensible fashion; in particular, the third column is specified to be wider than the others, and the browser works out the actual widths without us having to do the menial calculation.
• Media queries: Responsiveness is all about being able to handle different sizes of viewport, and below 900px the desktop variant of the site begins to fall apart. With media queries, we can define a new variant that applies down to 600px, and yet another variant that works on mobile devices below that size; with the smallest variant in particular, we can even replace the header element and use a new mobile-specific background and aspect ratio.

And there are features coming to wider support in CSS that haven't yet been employed here, the most compelling of which is rule nesting. To take a sample from further down the stylesheet:

main samp {
  font: 16pt Inconsolata, monospace;
  background: var(--g-fig);
  display: block;
}
main samp pre { margin: 0; padding: 1em; overflow: auto; }
main samp kbd { color: var(--g-code-keyword); }
main samp var { color: var(--g-code-var); }
main samp s { color: var(--g-code-comment); text-decoration: none; }

There's a bunch of redundancy in the match specifications here, and one of the more compelling reasons to use CSS pre-processors like Sass has, in the past, been that one can nest rules like this to make for cleaner code to work with:

main samp {
  font: 16pt Inconsolata, monospace;
  background: var(--g-fig);
  display: block;
  
  pre { margin: 0; padding: 1em; overflow: auto; }
  kbd { color: var(--g-code-keyword); }
  var { color: var(--g-code-var); }
  s { color: var(--g-code-comment); text-decoration: none; }
}

A pre-processor means a build step, which means a build process, and suddenly your plain HTML/CSS site has become a Whole Thing. With CSS nesting coming to browsers, this will soon be natively supported client-side, and will no longer be a reason to have a build process at all.

Overall, I'm happy with how the refresh has come out; here's to another fifteen years. Let's see how CSS advances in the interim, and what's possible (and widely supported) the next time I get bored and decide to rebuild this place.

Explorations of the smallest possible thing in a particular area aren't unprecedented here: the first article on this site is The Smallest Nintendo DS ROM, from 2006. So when I heard about the Binary Golf Grand Prix I was left with no choice but to put in an attempt. Unfortunately, I learned of the existence of BGGP #4 around two hours before it closed for submissions, so I ended up handing in a scrap of PHP. But let's look at what could've been...

The Task at Hand

This year's problem is specified as follows:

Create the smallest self-replicating file.

A valid submission will:

• Produce exactly 1 copy of itself
• Name the copy "4"
• Not execute the copied file
• Print, return, or display the number 4

The most natural thought that arises, unbidden, is how one would go about this on the Commodore 64. Commodore BASIC has the built-in SAVE command, so our submission might be as simple as:

10 SAVE "4", 8
20 PRINT 4

As the task here is to generate the smallest file, we should look at how this program is stored in memory (and thus, on disk). Commodore BASIC stores programs almost entirely as written, and the code execution process includes a parsing step which tokenises the program before it's executed. So the above program looks like this if we use VICE's memory monitor:

(C:$e5cf) m 0800
>C:0800  00 0e 08 0a  00 94 20 22  34 22 2c 20  38 00 16 08   ...... "4", 8...
>C:0810  14 00 99 20  34 00 00 00  00 00 ff ff  ff ff 00 00   ... 4.....????..

BASIC program storage and tokenisation

We can see the program starts at address $0801 hexadecimal: this is the case for every BASIC program on the C64, whether typed in or loaded. Each line starts with a pointer to the next line, allowing the BASIC interpreter to rapidly work through the program if it needs to determine the program length or find a particular line to change its content.

After the pointer to the next line comes the line number: two bytes, in little-endian format (small byte first), which is standard for the 6502-derived processor inside the C64. So we see the first line is numbered $000a, 10. And after this comes the line content itself: 94 20 22 34 22 2c 20 38 00.

As a program is typed into the BASIC interpreter, it's tokenised: any keywords in the line get replaced by token values before being stored in memory. We can see in this line that SAVE has been replaced by command token $94, such that if this value comes at the start of a command the interpreter knows this is a SAVE command without needing to read and parse the individual letters of the word "SAVE".

The rest of the command is not tokenised, and responsibility for parsing this arguments string is passed to whichever routine in the BASIC interpreter is handling the command. In this case, the rest of the command is ASCII text: <space>"4",<space>8; note that the spaces are preserved as they were entered in the program. The arguments are terminated by a zero byte.

The second line is similarly structured: a $99 token representing the PRINT command, and an arguments string of 20 34, representing <space>4.

Golfing in C64 BASIC

Now we've seen how the BASIC program is stored internally, we have direction for a round of golf. The first step is to remove the spaces:

10SAVE"4",8
20PRINT4

One might be tempted to remove the quotes around the filename, relying on whatever type coercion may exist in the BASIC interpreter to convert 4 into a string. Unfortunately, this isn't a thing on the C64:

10SAVE4,8
20PRINT4
RUN

?TYPE MISMATCH  ERROR IN 10

Commodore BASIC does, however, support multiple commands on one line, with the colon separator. Thus, we can combine the two commands of our program and remove the four bytes associated with the second line pointer and line number:

10SAVE"4",8:PRINT4

This program is stored in memory as below:

(C: $e5cf) m 0800
>C:0800  00 0f 08 0a  00 94 22 34  22 2c 38 3a  99 34 00 00   ......"4",8:.4..

We see that the two commands have been stored as one line, so there's one next-line pointer (which refers to the byte after the end of the program); it also becomes plain that a zero byte isn't the only thing that can end an arguments string in C64 BASIC. The colon, byte $3a, can also act as an end-of-arguments delimiter, denoting the end of a command and the start of another.

Our final result in Commodore BASIC is 14 bytes. Is it possible to produce a smaller program by switching to machine language, and communicating with the disk drive through Commodore DOS directly?

Saving files in Commodore DOS

One advantage of using such a venerable computer as the C64 for this task is that the documentation is extensive and copious. There are eleven separate commentaries on the SAVE routine available in the C64 Kernal API reference, including detailed explanation of the internal workings; for our purposes, we'll use the example routine provided by Commodore's own Programmer's Reference, which gives a sequence of operations for using SAVE:

1. Use the SETLFS routine and the SETNAM routine (unless a SAVE with no file name is desired on "a save to the tape recorder"),
2. Load two consecutive locations on page 0 with a pointer to the start of your save (in standard 6502 low byte first, high byte next format).
3. Load the accumulator with the single byte page zero offset to the pointer.
4. Load the X and Y registers with the low byte and high byte re- spectively of the location of the end of the save.
5. Call this routine.

In this case, we'll be saving to disk (device 8), so we'll want to call both SETLFS and SETNAM before running through the save. There are two things we'll need to note for the save itself:

Load location

As mentioned above, when a BASIC program is loaded from disk, it always loads to the same location (the start of BASIC memory, $0801 hexadecimal); a machine-language program doesn't have this restriction, and can be loaded into memory anywhere. To facilitate this, the first two bytes of the program as stored on disk are actually the address to which it should be loaded. Accordingly, when saving the program this must be accounted for by setting up the load address in memory beforehand (in little-endian format, as the Programmer's Reference mentions above).

We also have the extra stipulation that this program needs to be self-replicating: if the file saved by this code is reloaded, it will need to behave the same way. To that end, the load address for the program will need to be written by the program itself, to a location just before the program, and the concatenated block saved together.

Program size

BASIC keeps track of how long the entered program is, so it can determine how much memory needs to be saved to disk; our machine-language program will have to set up the amount of data to save without this help being available.

At this point, it's already becoming obvious that our machine-language attempt will be longer than 14 bytes to perform the same task as the BASIC program above, but to quote from a certain classic game show:

I've started, so I'll finish...

FILENAME = $002A        ; An unused byte in zero page
PROGPTR = $009B         ; An unused block of two bytes
PROGSTART = $C0C0       ; Program runs from here
MEMSTART = $C0BE        ; SAVE starts from here

    processor 6502
    org PROGSTART

start:
    lda #1              ; Logical file 1
    ldx #8              ; Disk drive 0 (device 8)
    ldy #255            ; No secondary command
    jsr $FFBA           ; SETLFS

    lda #'4             ; Our filename: "4"
    sta FILENAME        ; Store the name
    lda #1              ; It's one byte long
    ldx #<FILENAME      ; And it's stored in zero page
    ldy #>FILENAME      ; at this address
    jsr $FFBD           ; SETNAM

    lda #$c0            ; Our load location ($C0C0)
    sta MEMSTART        ; Written to the two bytes
    sta MEMSTART + 1    ; before the program in memory

    lda #<MEMSTART      ; Set up our indirect pointer
    sta PROGPTR         ; from the start of the file
    lda #>MEMSTART      ; to our two-byte pointer store
    sta PROGPTR + 1     ; in zero page

    lda #<PROGPTR       ; Pointer to the start of file
    ldx #<end           ; Actual location of the end of file
    ldy #>end           ; (including the two-byte header)
    jsr $FFD8           ; SAVE

    lda #4              ; We need to print or return 4
    rts                 ; So let's return 4
end:

Conclusion and caveats

The above machine-language program clocks in at 50 bytes, almost four times as large as the equivalent BASIC routine. There's some scope for reducing that number, but we don't have a hope of reaching the 14 bytes that a higher-level representation affords us. It should also be noted that we're not attempting to print the number 4 as per the task description, as this would require another call out to the Kernal and/or recycling of BASIC routines to do the same.

There is a level below this, of course, which we haven't reached today: direct communication with the disk drive to push the program to disk, bypassing Commodore DOS (at least at the computer end). The Kernal API is an abstraction on top of this, meaning we don't have to worry about the disk drive's serial bus and other vagaries of the implementation.

This little game of code golf has allowed us a look into why higher-level languages exist at all: not only do they allow us to perform tasks such as "save a file to disk" with less code, they also abstract away complications such as setting the filename before writing a file. Perhaps BASIC shouldn't be maligned quite so much.

Thanks go to the Online 6502 Disassembler by Norbert Landsteiner, and the dASM assembler whose name always makes me think it's a disassembler.

The Problem

The Commodore 64 has, as the name implies, 64kB of static RAM that can be written to at any time; however, its processor only has access to a 64kB address space, and somehow needs to fit the BASIC and Kernal DOS ROMs, as well as peripheral access, into that same space. Commodore achieved this by overloading the processor's onboard I/O port so you can "switch in" the ROMs and peripheral space, and switch them out if you need access to all the RAM.

For example, the Kernal ROM maps into memory at $E000, but only if it's been enabled at the CPU port level by switching on bit 1 (HIRAM). In the emulator's memory controller, this particular mapping might read as follows in JavaScript.

const CPUPORT = 0x0001;
const HIRAM = 2;

export const readByte = (addr) => {
    switch (addr & 0xF000) {
        // RAM, BASIC, peripheral areas
        ...
        // Kernal ROM
        case 0xE000:
        case 0xF000:
            if (memory[CPUPORT] & HIRAM) {
                return ROM.kernal[addr & 0x1FFF];
            } else {
                return memory[addr];
            }
            break;
    }
};

If we're to translate this to WAT, we'll first convert the switch statement to a function table:

(memory (import "mem") 2)
(table 16 anyfunc)
(elem (i32.const 0)
    ;; RAM, BASIC, peripheral areas
    ;; ...
    $read_kernal
    $read_kernal
)

(type $readfunc (func (param i32) (result i32)))
(func $read (param $addr i32) (result i32)
    ;; table[(addr & 0xF000) >> 12]()
    (call_indirect (type $readfunc)
        (get_local $addr)
        (i32.shr_u
            (i32.and (get_local $addr) (i32.const 0xF000))
            (i32.const 12)
        )
    )
)

Now, this isn't terrible so far. We have two contiguous 64k-value blocks of memory (one for RAM, one for the mapped ROMs), and there are some magic constants in the $read main handler, but they make sense in the context of needing to extract four bits from the address and using those to index the function table. Where the constants start to make less sense is in $read_kernal:

(func $read_kernal (param $addr i32) (result i32)
    (i32.load8_u
        (if (result i32)
            (i32.and (i32.load (i32.const 0x0001)) (i32.const 2))
            (then
                (i32.add
                    (i32.and (get_local $addr) (i32.const 0xFFFF))
                    (i32.const 0x10000)
                )
            )
            (else
                (i32.and (get_local $addr) (i32.const 0xFFFF))
            )
        )
    )
)

This is more inscrutable, especially if (as was the case for me) this code has been written and then left to marinate for a year before coming back to attempt to read it again. It would be much more readable if the above function could instead use defined constants:

(define CPUPORT (i32.const 0x0001))
(define HIRAM (i32.const 2))
(define ROM_MEMORY_START (i32. 0x10000))

(func $read_kernal (param $addr i32) (result i32)
    (i32.load8_u
        (if (result i32)
            (i32.and (i32.load CPUPORT) HIRAM)
            (then
                (i32.add
                    (i32.and (get_local $addr) (i32.const 0xFFFF))
                    ROM_MEMORY_START
                )
            )
            (else
                (i32.and (get_local $addr) (i32.const 0xFFFF))
            )
        )
    )
)

Handling S-expressions

The above code makes use of a define keyword that doesn't exist in WASM, so we need something akin to the C preprocessor to parse through the WAT file picking up definitions and replacing their occurrences. Fortunately, WAT was designed to be a format that's quick and easy to handle programmatically (while still being at least halfway usable by human standards): a WebAssembly Text file is one big Lisp-style S-expression, and each element within the file is itself a nested S-expression.

This means we can use S-expression handling libraries to quickly move from the WAT file to an internal representation that can be worked with. One such library is sexpdata by Joshua Boyd, which is an S-expression parser and dumper for Python. If we point our fledgling MMU file at sexpdata.load, something usable starts to fall out:

import sys
from sexpdata import load

def main():
    filename = sys.argv[1]
    print(load(open(filename)))

if __name__ == "__main__":
    main()

[Symbol('module'), [Symbol('define'), Symbol('CPUPORT'), [Symbol('i32.const'), Symbol('0x0001')]], [Symbol('define'), ...

Arrays with Symbol objects (which are an internal construct to sexpdata) and other arrays inside. We can work with this by using a recursive function to handle the array representing the whole file: if an array is found inside, it will need to be processed recursively, unless it's an array that holds a define statement.

Definitions can be made at any level of the WAT file, so any extracted definitions will need to be passed down to deeper levels of recursion, and merged with any definitions that are passed in from higher levels. So that means the list of definitions will need to be a parameter to the preprocessor:

import sys
from sexpdata import load, dumps

def process(sexp, defines):
    new_defines = {}
    # First pass: Find immediate descendent s-exp's which define macros
    # TODO

    # Macros defined at this level override any of the same name higher up
    # NOTE: We don't want to mutate the parent's dict, we want our own copy
    merged_defines = defines.copy() | new_defines

    # Second pass: Replace at this level, recurse if deeper levels encountered
    # TODO
    return [i for i in sexp]

def main():
    filename = sys.argv[1]
    # Top-level process, with no existing definitions in the dict
    print(dumps(process(load(open(filename)), {})))

if __name__ == "__main__":
    main()

Detecting and replacing definitions

We come to the crux of the problem: filling in the TODOs above. In the first pass, this is fairly simple: we'd like to find any arrays where the first element is a Symbol('define'), and pull out the associated definition. Let's take the first line of debug output again.

[Symbol('module'),
 [Symbol('define'), Symbol('CPUPORT'), [Symbol('i32.const'), Symbol('0x0001')]],
 [Symbol('define'), Symbol('HIRAM'), [Symbol('i32.const'), 2]], ...

We see that some elements in the file have been parsed as scalar values, and some as Symbol's. As Python doesn't have a native is-array, we can't directly find arrays in order to perform the first-element check mentioned above; we can, however, use numpy's isscalar to detect scalar values, and isinstance for Symbol's. From here, it's fairly simple to detect and extract definition clauses.

from numpy import isscalar
from sexpdata import Symbol

def is_scalar_or_symbol(i):
    return isscalar(i) or isinstance(i, Symbol)

def process(sexp, defines):
    new_defines = {}
    for i in sexp:
        if not is_scalar_or_symbol(i):
            if i[0] == Symbol('define'):
                new_defines[i[1]] = i[2]
    ...

And once we have the definitions to be used on this level, replacement is fairly simple. There are four types of item that will need action:

• Arrays where the first element is Symbol('define'): Exclude;
• Arrays where the first element is not Symbol('define'): Recurse;
• Scalars or plain Symbol's which are defined: Replace;
• Scalars or plain Symbol's which aren't defined: Copy.

return [
    merged_defines[i] if (
        is_scalar_or_symbol(i) and merged_defines.get(i)
    )                                   # Replace
    else i if is_scalar_or_symbol(i)    # Copy
    else process(i, merged_defines)     # Recurse
    for i in sexp
    if is_scalar_or_symbol(i) or i[0] != Symbol('define')   # Exclude
 ]

Putting everything together, the following GitHub Gist contains the final preprocessor script, as well as a sample input WAT file and the generated output:

https://gist.github.com/Two9A/427985064d360342caaf4f7d5769aeef

Caveats and improvements

The astute observer will already have noticed that circular definitions are not handled at all well by this script: if one define contains a keyword that's handled by another define, the final output is dependent on the order of definition. In addition, this replicates the behaviour of C's #define but it doesn't help with any of the other C-style preprocessor directives, notably #include; support for these is a matter for future expansion.

The initial probe had been dubbed Hyper One, the first successful test of a hyperspace tunneling engine: launched on a Paludis III twelve years ago, it had made its ponderous way to the Lagrange point sixty degrees ahead of the Earth in its orbit, taking a few months to get to its testing position. When the drive was spun up, it flicked across to somewhere in the vicinity of L4, sixty degrees behind Earth, in half a second. The tunneling engine was a miracle of high-speed space travel, but Hyper One and two subsequent probes had only carried bacterial or other small samples.

Hyper Four would be the first test of a hyperspace tunneling engine with humans on board. The engine had been retrofitted to one of SpaceX's old service capsules: with their Mars cycler fully up and running as of a few years ago, their stocks of Dragon capsules were simply taking up space and any use for them was encouraged. Life support and environmental controls were still working perfectly in this particular capsule: in its previous job as fifth (and seventh) crew service mission to the Space Station, it had encountered no issues except for a harder-than-usual ocean landing on the second use. That made it a fairly cheap capsule for the Hyper team to pick up.

James Kent, veteran of the Hyper series, had been voluntold as the crew member for this fourth mission: Dragon had made its way out under its own steam, and was now parked in orbit around the Earth-Moon system's L2 point: that helped to simplify the calculations by removing nearby gravitational influences, while still being visible from Earth so the Hyper team could track Four from the ground.

James thought back to the first test of Hyper One, and the astonishment they'd all felt when One had jumped more than 0.2 AU in the blink of an eye. And now he'd be doing the same, looking out the portholes of a capsule at the space below (or was it above?) realspace. To say he was excited was probably understating it.

The radio crackled into life.

"Hyper Four, this is Houston again. Our board is green for spin-up; we're sending you updated hyper-coordinates for the tunneling engine. Please confirm."

"Programmed in," James stated. "Let's do this, over."

"Copy, Hyper Four. Engage when ready."

James flipped a switch over his head, and the carrier signal from Earth immediately redshifted into oblivion. Light flooded in through the capsule's portholes.

Nothing else seemed to happen for a few seconds. Time passed on board Hyper Four, the clock by James's hand ticked by at its normal rate, but the light from outside remained unchanging and constant. Then the radio cracked into life again.

License and registration, please.

"Er, Houston... do you read?"

James didn't see how he could be receiving radio from above (or was it below) the skein of realspace, since the light waves wouldn't make it outside the constraints of untunneled space...

Come on, you can't pop up in the middle of the B6631 and cause disruption to traffic, then plead ignorance. You've been pulled over to the shoulder; your license, please.

"Alright, if this is Guinea base, Funny Joke, guys. I'll have to ask you how you're tunneling radio into the hyperskein when I get back; over."

We're talking at cross-purposes here. Do you have a license, sir?

"I guess I'll play along. No, I guess?"

Oh, you're one of those. Right, well, you may regard your patch of space as sovereign, but if you make use of the federal galactic highways, you'll need to abide by federal law. As you don't have your license, I'm issuing you a summons to Sirius district court: you'll need to appear in-

"Wait, wait, hold on. What do you mean, federal highways? We only just connected to hyperspace, I didn't know that-"

Oh, you're one of those. Right, well. Er, let's see... Firstly, it's infraspace, the interbrane region below realspace, that you're in.

"Clears that one up, I guess..."

Secondly, your tunnel intersected the B6631 galactic highway and has left a hole in lane four. And not a smooth hole, either; your engine is terribly noisy in its hookup to the interbrane.

"Er, right. I wasn't aware that the B6631 ...ran through here. It's not like we have a map..."

Right, yeah, first-contacters, sure. I could've sworn there was a protocol for this... In light of your status, I've rescinded the summons to Sirius-f. Let me go talk to someone, I'll be back.

James was left alone for a few minutes. Or at least, a few minutes passed on board the capsule; who knows what was happening in realspace. This... highways officer? that James had talked with, showed up on the radio again.

Alright, you're free to go. As I mentioned, you've been pulled over to the shoulder of the B6631, and my superiors have authorized me to write a map of the highway network to your support equipment's silicate substrate. Your nearest on-ramp is the core of the planet you refer to as Jupiter; please try not to drill any more holes in our roads.

"Well, thank you. This is... amazing. I just have a couple of questi-"

"How do you speak my language" comes up surprisingly often with first-contacters, my superiors mentioned. Consider that your vehicle is built in realspace, and you're in the interbrane at the present time: all atomic connections are visible to those who can travel directly in the interbrane.

"Right, yes... I guess my only other question is, how do we get to the core of Jupiter to join the highway?"

Surface roads aren't our concern, sir. Your local onramp was constructed some time ago.

"Ok, well, er. Thank you..."

One more thing before I eject you into realspace, sir. I've also written our current emissions regulations into your vehicle's silicate; a vehicle with such noisy tunneling output as yours runs the risk of being impounded, and that's a court appearance that can't be rescinded. Have a good journey.

And the light pouring into the capsule flicked off. The capsule's console screen started spewing text:

Determining position
12 degrees above the orbital plane; reorienting comms
Carrier signal obtained
Time elapsed since signal loss: 0.48 seconds

"Er, Houston, Hyper Four. Boy, do I have a tale for you." ]]> https://imrannazar.com/shorts/that-time-my-encrypted-raid-failed

Independent Submission
Request for Comments: 7168
Updates: 2324
Category: Informational
ISSN: 2070-1721

Abstract

The Hyper Text Coffee Pot Control Protocol (HTCPCP) specification does not allow for the brewing of tea, in all its variety and complexity. This paper outlines an extension to HTCPCP to allow for pots to provide networked tea-brewing facilities.

Status of This Memo

This document is not an Internet Standards Track specification; it is published for informational purposes.

This is a contribution to the RFC Series, independently of any other RFC stream. The RFC Editor has chosen to publish this document at its discretion and makes no statement about its value for implementation or deployment. Documents approved for publication by the RFC Editor are not a candidate for any level of Internet Standard; see Section 2 of RFC 5741.

Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc7168.

Copyright Notice

Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

1. Introduction

As noted in the Hyper Text Coffee Pot Control Protocol, coffee is renowned worldwide as an artfully brewed caffeinated beverage, but coffee shares this quality with many other varied preparations based on the filtration of plant material. Foremost, among these are the category of brews based on the straining of water through prepared leaves from a tea tree: the lineage and history of the tea genus will not be recounted as part of this paper, but evidence shows that the production of tea existed many thousands of years ago.

The deficiency of HTCPCP in addressing the networked production of such a venerable beverage as tea is noteworthy: indeed, the only provision given for networked teapots is that they not respond to requests for the production of coffee, which, while eminently reasonable, does not allow for communication with the teapot for its intended purpose.

This paper specifies an extension to HTCPCP to allow communication with networked tea production devices and teapots. The additions to the protocol specified herein permit the requests and responses necessary to control all devices capable of making, arguably, the most popular caffeinated hot beverage.

1.1. Terminology

The key words MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL in this document are to be interpreted as described in RFC 2119.

2. HTCPCP-TEA Protocol Additions

The TEA extension to HTCPCP adapts the operation of certain HTCPCP methods.

2.1. BREW and POST Methods

Control of a TEA-capable pot is performed, as described in the base HTCPCP specification, through the sending of BREW requests. POST requests are treated equivalently, but they remain deprecated. Tea production differs from coffee, however, in that a choice of teas is often provided for client selection before the tea is brewed. To this end, a TEA-capable pot that receives a BREW message of content type message/teapot MUST respond in accordance with the URI requested, as below.

2.1.1. The / URI

For the URI /, brewing will not commence. Instead, an Alternates header as defined in RFC 2295 MUST be sent, with the available tea bags and/or leaf varieties as entries. An example of such a response is as follows:

The following example demonstrates the possibility of interoperability of a TEA-capable pot that also complies with the base HTCPCP specification:

TEA-capable HTCPCP clients MUST check the contents of the Alternates header returned by a BREW request, and provide a specific URI for subsequent requests of the message/teapot type.

A request to the / URI with a Content-Type header of message/coffeepot SHOULD also be responded to with an Alternates header in the above format, to allow TEA-capable clients the opportunity to present the selection of teas to the user if inferior caffeinated beverages have initially been requested.

2.1.2. Variety-Specific URIs

TEA-capable pots follow the base HTCPCP specification when presented with a BREW request for a specific variety of tea. Pots SHOULD follow the recommendations for brewing strength given by each variety, and stop brewing when this strength is reached; it is suggested that the strength be measured by detection of the opacity of the beverage currently under brew by the pot.

TEA-capable clients SHOULD indicate the end of brewing by sending a BREW request with an entity body containing stop; the pot MAY continue brewing beyond the recommended strength until this is received. If the stop request is not sent by the client, this may result in a state inversion in the proportion of tea to water in the brewing pot, which may be reported by some pots as a negative strength.

If a BREW command with an entity body containing stop is received before the recommended strength is achieved, the pot MUST abort brewing and serve the resultant beverage at lesser strength. Finding the preferred strength of beverage when using this override is a function of the time between the TEA-capable pot receiving a start request and the subsequent stop. Clients SHOULD be prepared to make multiple attempts to reach the preferred strength.

2.2. Modified Header Fields

HTCPCP-TEA modifies the definition of one header field from the base HTCPCP specification.

2.2.1. The Accept-Additions Header Field

It has been observed that some users of blended teas have an occasional preference for teas brewed as an emulsion of cane sugar with hints of water. To allow for this circumstance, the Accept-Additions header field defined in the base HTCPCP specification is updated to allow the following options:

Implementers should be aware that excessive use of the Sugar addition may cause the BREW request to exceed the segment size allowed by the transport layer, causing fragmentation and a delay in brewing.

2.3. Response Codes

HTCPCP-TEA makes use of normal HTTP error codes and those defined in the base HTCPCP specification.

2.3.1. 300 Multiple Options

A BREW request to the / URI, as defined in Section 2.1.1, will return an Alternates header indicating the URIs of the available varieties of tea to brew. It is RECOMMENDED that this response be served with a status code of 300, to indicate that brewing has not commenced and further options must be chosen by the client.

2.3.2. 403 Forbidden

Services that implement the Accept-Additions header field MAY return a 403 status code for a BREW request of a given variety of tea, if the service deems the combination of additions requested to be contrary to the sensibilities of a consensus of drinkers regarding the variety in question.

A method of garnering and collating consensus indicators of the most viable combinations of additions for each variety to be served is outside the scope of this document.

2.3.3. 418 I'm a Teapot

TEA-capable pots that are not provisioned to brew coffee may return either a status code of 503, indicating temporary unavailability of coffee, or a code of 418 as defined in the base HTCPCP specification to denote a more permanent indication that the pot is a teapot.

3. The message/teapot Media Type

To distinguish messages destined for TEA-capable HTCPCP services from pots compliant with the base HTCPCP specification, a new MIME media type is defined by this document. The Content-Type header of a POST or BREW request sent to a TEA-capable pot MUST be message/teapot if tea is to be requested.

4. Environmental Considerations

As noted in Section 2.1, a BREW request with a Content-Type header field of message/teapot to a TEA-capable pot will result in an Alternates header being sent with the response, and a pot will not be brewed. However, if the BREW request has a Content-Type of message/coffeepot, and the pot is capable of brewing coffee, the service's behavior will fall back to the base HTCPCP specification and a pot will be brewed.

If the entity returned by the server when brewing commences contains a TEA-compliant Alternates header indicating message/coffeepot and the client does not want coffee, the client SHOULD then send a BREW request with an entity body containing stop. This will result in wasted coffee; whether this is regarded as a bad thing is user- defined.

Such waste can be prevented by TEA-capable clients, by first requesting a BREW of type message/teapot and then allowing selection of an available beverage.

5. Security Considerations

As with the base HTCPCP specification, most TEA-capable pots are expected to heat water through the use of electric elements, and as such will not be in proximity to fire. Therefore, no firewalls are necessary for communication with these pots to proceed.

This extension does support communication with fired pots, however, which may require heat retention and control policies. Care should be taken so that coal-fired pots and electrically heated kettles are not connected to the same network, to prevent pots from referring to any kettles on the network as darkened or otherwise smoke driven.

6. Acknowledgements

This extension to the HTCPCP specification would not be possible without the base specification, and research on networked beverage production leading up thereto. In that vein, the author wishes to acknowledge the sterling work of Larry Masinter in the development of the leading protocol for coffee pot communication.

Many thanks also to Kevin Waterson and Pete Davis, for providing guidance and suggestions during the drafting of this document.

• Part 1: Concepts
• Part 2: File Structure
• Part 3: Huffman Tables
• Part 4: Frames and Bitstreams

Download the code: http://imrannazar.com/content/files/jpegparse.zip

Previously, I discussed the Huffman compression algorithm as implemented by JPEG, and a mechanism by which JPEG encoders pick the substitution values used for the compression. The image itself, in a baseline entropy-coded JPEG file, is stored in one "scan" as a stream of Huffman codes; the codes are of a variable length, and are not necessarily at even byte boundaries.

For this reason, it's a requirement that some kind of queue be introduced between the bytes of the file, and the values as seen by the JPEG decoder: this allows bits to be pushed onto the queue from the bytes of the file, and pulled out in varying amounts by the decoder. A single routine can thus be made the central routing point for any requests for variable-width symbols.

A queue of things, when used in this fashion, generally has two modes of operation: either there are enough things in the queue for the request to deal with, or additional reading has to be done before there are enough things in the queue to handle the request. In the case of a bit-level queue for reading from a file, as we need here, the number of bits requested is the determining factor. In the first example below, a queue holding five bits is asked to return the first three, and is able to handle the request without a problem.

If a request for four bits then comes in, the queue doesn't contain enough bits to handle the request, and must first fetch another full byte from the JPEG file (shown here in red).

Implementing the bit queue

A complication arises when trying to code an implementation of the bit queue as shown above: the output end of the queue is at the left, and if one contiguous value is used to store the queue, the left is the most-significant end with the highest-value bits. If we demarcate the queue as being 32 positions long, and define one 32-bit value as holding its entire contents, the above scenarios break down as follows:

Pulling three bits

• The request is shorter than the queue length of 5, no reading from file happens.
• The output value is the top three bits of the queue, shifted down (32 - 3) bits to become the bottom three bits of an output value;
• The queue is shifted left three bits, and left with length of 2.

Pulling another four bits

• The request is longer than the queue length of 2: a byte is read in, and shifted up (24 - 2) bits to join the queue;
• The queue is now 10 bits long, no more bytes need reading;
• The output value is the top four bits of the queue, shifted down (32 - 4);
• The queue is shifted left four bits, and left with length of 6.

At all times during the queue's operation, the next available bits are at the high-value end of the contiguous queue variable. An implementation of this may look like the following.

class JPEG {
    private:
        // Read a number of bits from file
        u16 readBits(int);
};

u16 JPEG::readBits(int len)
{
    // The number of bits left in the queue, and the value represented
    static int queueLen = 0;
    static u32 queueVal = 0;

    u8 readByte;
    u16 output;

    if (len > queueLen) {
        do {
            // Read a byte in, shift it up to join the queue
            readByte = fgetc(fp);
            queueVal = queueVal | (readByte << (24 - queueLen));
            queueLen += 8;
        } while (len > queueLen);
    }

    // Shift the requested number of bytes down to the other end
    output = ((queueVal >> (32 - len)) & ((1 << len) - 1));

    queueLen -= len;
    queueVal <<= len;

    return output;
}

The JPEG SOF segment

As mentioned in the previous part of this series, a JPEG file can define up to 32 Huffman code tables, each in their own DHT segment. A JPEG file holds the data corresponding to the image itself in a "frame", denoted by a "Start of Frame" segment header. The SOF header contains a part of the information required to decode the frame, and is structured according to the following table.

As can be seen in the above table, some of the fields involve operations that we have not yet examined (the sampling resolution and the quantisation table for each component). For the purposes of completing the SOF segment handler, we can hold onto this information for later use.

A "frame" consists of the SOF header, and a number of "scans"; as the name implies, each scan is a pass over the full rectangle of the image. In an interlaced JPEG file for example, multiple scans would be present for the single frame, each of them having more resolution than the last; in a progressive JPEG file, there is just one scan containing all the information for the image. Since this series is concerned with building a decoder for progressive JPEG files, we'll focus on dealing with a single scan in the frame.

It turns out that the information required by Huffman decoding, in particular which of the DHT tables to use, is defined by each scan in the frame instead of by the frame itself. We'll look at the scan-level information in more detail in the next part of this series; for now, it's sufficient to decouple our representation of a colour component in the image from the SOF data, and define the exact metadata for a component later.

Structure definitions

With the information detailing the structure of an SOF header, it becomes relatively simple to build a segment parser to plug into our existing code. The only complication arises from the fact that multi-byte values in JPEG are stored in big-endian format, which may not necessarily be the host format for large integers. It's useful to have a set of definitions for transparently handling big-endian values, which is presented below.

/**
* Let's Build a JPEG Decoder
* Big-endian value handling macros
* Imran Nazar, May 2013
*/

#ifndef __BYTESWAP_H_
#define __BYTESWAP_H_

#if __SYS_BIG_ENDIAN == 1
#   define htoms(x) (x)
#   define htoml(x) (x)
#   define mtohs(x) (x)
#   define mtohl(x) (x)
#else
#   define htoms(x) (((x)>>8)|((x)<<8))
#   define htoml(x) (((x)<<24)|(((x)&0xFF00)<<8)|(((x)&0xFF0000)>>8)|((x)>>24))
#   define mtohs(x) (((x)>>8)|((x)<<8))
#   define mtohl(x) (((x)<<24)|(((x)&0xFF00)<<8)|(((x)&0xFF0000)>>8)|((x)>>24))
#endif

#endif//__BYTESWAP_H_

// Prevent padding bytes from creeping into structures
#define PACKED __attribute__((packed))
 
class JPEG {
    private:
        // Information in the SOF header
        struct PACKED {
            u8  precision;
            u16 height;
            u16 width;
            u8  component_count;
        } sofHead;

        typedef struct PACKED {
            u8 id;
            u8 sampling;
            u8 q_table;
        } sofComponent;

        // Internal information about a colour component
        typedef struct PACKED {
            u8 id;
            // There is likely to be more data here...
        } Component;

        // The set of colour components in the image
        std::vector components;

        // The SOF segment handler
        int SOF();
};

int JPEG::parseSeg()
{
    ...

    switch (id) {
        // The SOF segment defines the components and resolution
        // of the JPEG frame for a baseline Huffman-coded image
        case 0xFFC0:
            size = READ_WORD() - 2;
            if (SOF() != size) {
                printf("Unexpected end of SOF segment\n");
                return JPEG_SEG_ERR;
            }
            break;

        ...
    }

    return JPEG_SEG_OK;
}

int JPEG::SOF()
{
    int ctr = 0, i;

    fread(&sofHead, sizeof(sofHead), 1, fp);
    ctr += sizeof(sofHead);

    sofHead.width   = mtohs(sofHead.width);
    sofHead.height  = mtohs(sofHead.height);

    printf("Image resolution: %dx%d\n", sofHead.width, sofHead.height);

    for (i = 0; i < sofHead.component_count; i++) {
        sofComponent s;
        fread(&s, sizeof(sofComponent), 1, fp);
        ctr += sizeof(sofComponent);

        Component c;
        c.id = s.id;

        components.push_back(c);
    }

    return ctr;
}

Next time: The Minimum Coded Unit

As mentioned above, the image frame in a progressive JPEG file is encoded as a scan, composed of a series of blocks; depending on the sampling resolution of the components in the image, these blocks can be larger than the 8x8-pixel base block size of the JPEG algorithm. In the next part of this series, I'll examine the relationship between these larger units and the colour components of the image.

Imran Nazar <tf@imrannazar.com>, May 2013.

• Part 1: Concepts
• Part 2: File Structure
• Part 3: Huffman Tables
• Part 4: Frames and Bitstreams

Download the code: http://imrannazar.com/content/files/jpegparse.zip

In the previous part, I mentioned that most JPEG files employ an encoding technique on top of the image compression, in an attempt to remove any trace of redundant information from the image. The technique used by the most common JPEG encoding is an adaptation of one seen throughout the world of data compression, known as Huffman coding, so it's useful to explore in detail the structure and implementation of a Huffman decoder.

Because Huffman coding is the last thing performed by a JPEG encoder when saving an image file, it needs to be the first thing done by our decoder. This can be achieved in two ways:

• Reconstitution: The full image scan is decoded from its Huffman-coded state into a temporary placeholder, and further processing is performed on the temporary copy.
• On-the-fly: The encoded image scan is processed one code at a time, and each JPEG block is handled when enough information is available.

This article will take the second approach, to save memory and sacrifice time; full reconstitution can be implemented using the code built below in a very similar fashion.

The Huffman algorithm

The concept behind Huffman coding and other entropy-based schemes is similar to the concept behind the substitution cipher: each unique character in an input is transformed into a unique output character. The simplest example is the Caesar substitution, which can be represented in tabular form as follows:

A => D
B => E
C => F
...
Y => B
Z => C

This is an example of a Caesar cipher
Wklv lv dq hadpsoh ri d Fdhvdu flskhu

An improvement on the standard substitution cipher can be made by noting the relative frequency of characters in the input, and designing a table that contains shorter codes as substitutes for these characters, than for rarer ones. Taking a look at the frequency of letters in the above example, with their ASCII representations included, we can produce a table of increasing unique codes such as the following:

Substituting these codes for the characters in the original text, it can be seen how the encoded data is much smaller than the original.

This is an example of a Caesar cipher
01010100 01101000 01101001 01110011 00100000 01101001 01110011 00100000
01100001 01101110 00100000 01100101 01110111 01100001 01101101 01110000
01101100 01100101 00100000 01101111 01100110 00100000 01100001 00100000
01000011 01100001 01100101 01110011 01100001 01110010 00100000 01100011
01101001 01110000 01101000 01100101 01110010

110111 11000 1010 1011 00 1010 1011 00 01 111100 00 100 111110 01 111011
11011 111010 100 00 111101 111001 00 01 00 110110 01 100 1011 01 11010
00 111000 1010 11001 11000 100 11010

54 68 69 73 20 69 73 20 61 6E 20 65 77 61 6D 70 6C 65 20 6F 66 20 61 20
43 61 65 73 61 72 20 63 69 70 68 65 72
DF 15 65 58 F8 4F 9E F7 D4 3D E4 4D 99 6E 8E 2B 38 9A

The main disadvantage of the Huffman coding method is that the table of codes needs to be stored alongside the compressed data: in the above example, the red string of encoded bytes would be meaningless without the corresponding frequency table. The table of codes and their corresponding characters can be recorded in full, but there is a more space-efficient way to save the codes, if attention is paid to the pattern of their occurrence. Two things are of note here: firstly that the codes increase in length, but also that within a group of the same length, codes are sequential. This means the code table can be written down as:

2 codes of length two  , starting at 00
1 code  of length three, starting at 100
2 codes of length four , starting at 1010
3 codes of length five , starting at 11000
9 codes of length six  , starting at 110110

A careful eye on the codes themselves can yield further improvements on how much space it takes to record the encoding table. If we take a look at the codes in conjunction with the list of code length above, we can start counting as follows.

00 (zero)
01 (one)
Next code would be 10 (two)
100 (four)
Next code would be 101 (five)
1010 (ten)
1011 (eleven)
Next code would be 1100 (twelve)
11000 (twenty four)
11001 (twenty five)
11010 (twenty six)
Next code would be 11011 (twenty seven)
110110 (fifty four)
110111 (fifty five)
111000 (fifty six)
111001 (fifty seven)
111010 (fifty eight)
111011 (fifty nine)
111100 (sixty)
111101 (sixty one)
111110 (sixty two)

In every case, when the requisite number of codes has been counted for the given code length, all that is needed is to double the counter and continue for the next code length. In other words, there is no need to record the "starting at" part of the code lengths list above, since it can be inferred by starting at zero. The final code list therefore looks as follows.

2 codes of length two
1 code  of length three
2 codes of length four
3 codes of length five
9 codes of length six

The above codes correspond to the following characters, in this order:
Space,a,e,i,s,h,p,r,C,T,c,f,l,m,n,o,x

The JPEG DHT segment

A JPEG file's Huffman tables are recorded in precisely this manner: a list of how many codes are present of a given length (between 1 and 16), followed by the meanings of the codes in order. This information is held in the file's "Define Huffman Table" (DHT) segments, of which there can be up to 32, according to the JPEG standard.

As seen above, data encoded by the Huffman algorithm ends up recorded as a series of codes wedged together in a bit-stream; this also applies to the image scan in a JPEG file. A simple routine for reading codes from the bit stream may look like this:

Code = 0
Length = 0
Found = False

Do
    Code = Code << 1
    Code = Code | (The next bit in the stream)
    Length = Length + 1
    If ((Length, Code) is in the Huffman list) Then
        Found = True
    End If
While Found = False

In order to facilitate this algorithm, the Huffman codes should be stored in a way that allows us to determine if a code is in the map at a given length. The canonical way to represent a Huffman code list is as a binary tree, where the sequence of branches defines the code and the depth of the tree tells us how long the code is. The C++ STL abstracts this out for us, into the map construct.

Since there are up to 32 possible Huffman tables that can be defined in a JPEG file, our implementation will require 32 maps to be available. It's also worth defining at this point how the DHT segment handler will be called by the parseSeg method developed in the previous part of this series.

class JPEG {
    private:
        // Defines a tuple of length and code, for use in the Huffman maps
        typedef std::pair<int, u16> huffKey;

        // The array of Huffman maps: (length, code) -> value
        std::map<huffKey, u8> huffData[32];

        // DHT segment handler
        int DHT();
};

int JPEG::parseSeg()
{
    ...

    switch (id) {
        // The DHT segment defines a Huffman table. The handler should
        // read exactly as many bytes from the file as are in the
        // segment; if not, something's gone wrong
        case 0xFFC4:
            size = READ_WORD() - 2;
            if (DHT() != size) {
                printf("Unexpected end of DHT segment\n");
                return JPEG_SEG_ERR;
            }
            break;

        ...
    }

    return JPEG_SEG_OK;
}

int JPEG::DHT()
{
    int i, j;

    // A counter of how many bytes have been read
    int ctr = 0;

    // The incrementing code to be used to build the map
    u16 code = 0;

    // First byte of a DHT segment is the table ID, between 0 and 31
    u8 table = fgetc(fp);
    ctr++;

    // Next sixteen bytes are the counts for each code length
    u8 counts[16];
    for (i = 0; i < 16; i++) {
        counts[i] = fgetc(fp);
        ctr++;
    }

    // Remaining bytes are the data values to be mapped
    // Build the Huffman map of (length, code) -> value
    for (i = 0; i < 16; i++) {
        for (j = 0; j < counts[i]; j++) {
            huffData[table][huffKey(i + 1, code)] = fgetc(fp);
            code++;
            ctr++;
        }
        code <<= 1;
    }

    // Once the map has been built, print it out
    printf("Huffman table #%02X:\n", table);

    std::map<huffKey, u8>::iterator iter;
    for (iter = huffData[table].begin();
         iter != huffData[table].end();
         iter++
    ) {
        printf("    %04X at length %d = %02X\n",
            iter->first.second, iter->first.first, iter->second);
    }

    return ctr;
}

As with the previous part, the JPEG class can be instantiated with a filename; if this is done, the above code will produce output along the following lines:

Found segment at file position 177: Huffman table
Huffman table #00:
    0000 at length 2 = 04
    0002 at length 3 = 02
    0003 at length 3 = 03
    0004 at length 3 = 05
    0005 at length 3 = 06
    0006 at length 3 = 07
    000E at length 4 = 01
    001E at length 5 = 00
    003E at length 6 = 08
    007E at length 7 = 09

Next time: Reading the bitstream

Once the Huffman maps have been built for a JPEG file, the image scan can be decoded for further processing. In the next part, I'll take a look at the Huffman decoding of the scan, in the wider context of reading blocks from the image and examining the process through which they are transformed.

Imran Nazar <tf@imrannazar.com>, Feb 2013.

• Part 1: Concepts
• Part 2: File Structure
• Part 3: Huffman Tables
• Part 4: Frames and Bitstreams

Download the code: http://imrannazar.com/content/files/jpegparse.zip

In the previous part, I gave a brief overview of the techniques used by JPEG to compress an image. Before examining the detailed implementation of those techniques, it's useful to look at the overall structure of a JPEG file, for two reasons:

• Some of the processes used by the encoder employ tables of values, which are stored alongside the image information itself, so it's sensible to retrieve these into memory before they're needed.
• We'll also need somewhere to put the implementation of the image decompression algorithm, and having a framework in place for that facilitates this.

The implementation developed in this series of articles will be written in C++, but the constructs can be transplanted to a language of your choice with little additional complexity.

Types of JPEG image

It should be stated at this juncture that the implementation developed here will only apply to one common subset of all the possible types of JPEG image. Firstly, there are four types of compression supported by the standard:

• Baseline: The most common compression type, where all the image information is contained in one series of 8x8 blocks.
• Extended Sequential: Most used in medical imaging, this type allows for more levels per pixel.
• Progressive: Information from the frequency domain is written in a series of scans, the most important values for each block coming first in the file. This allows the whole image to be rendered at a low resolution, and the details filled in as the image downloads.
• Lossless: A rare encoding based on a predicted difference between the target pixel and its surroundings.

Further, there are two forms of encoding that are applied on top of the image compression, to further compress the file data:

• Huffman-based entropy: The image is made into a bit-stream, with the most common values encoded as short (2- or 3-bit) stream entries, and less common values recorded as longer strings of bits.
• Arithmetic: A formerly patented encoding method, where the image data is represented as a series of probabilities of the values occurring, and combined into one fractional number.

This series will implement an entropy-coded baseline JPEG decoder.

File segments

A JPEG file is made up of segments of varying length, each of which starts with a "marker" to denote which kind of segment it is. There are 254 possible types of segment, but only a few are found in the type of image we'll be decoding:

Most of the different types of segment have a "length" value just after the marker, which denotes how long the segment is in bytes (including the length value); this can be used to skip over segments that a decoder doesn't know about. There are three exceptions to this general rule:

• SOI and EOI: Because these are more delimiters than markers, they consist only of the marker value.
• SOS: The scan is a bitstream, and automatically "ends" when the image is fully coded. As such, there is no length written into the file for an SOS segment. There are two strategies for dealing with this: either we can assume the rest of the file is part of the scan, or we can read through the file looking for markers which would denote the start of a new segment.

For this article, I'll assume the rest of the file is part of a scan if we run into an SOS segment, and skip straight to the EOI.

Implementation: Listing the segments in a JPEG file

As a first step, it makes sense to write a program to open a JPEG file, and run through it looking for segment markers. The structure of such a program can be expanded upon with implementation for processing of the different kinds of segments, and the mechanism for skipping over segments given their size can be used later to skip over the parts of the file which are non-essential to the decoding process.

Since the sizes of values in a JPEG file are specified in absolute terms of number of bytes, it's a good idea to abstract the basic integer types into types which refer to size. For this, we'll use a short header file.

#ifndef __INTTYPES_H_
#define __INTTYPES_H_

typedef unsigned char  u8;
typedef unsigned short u16;
typedef unsigned int   u32;

typedef signed char  s8;
typedef signed short s16;
typedef signed int   s32;

#endif//__INTTYPES_H_

The above file is set up for 32-bit compilation, but can be adapted if 64- or 16-bit code is required. The advantage of this is that references to integers in the JPEG decoder implementation itself can be agnostic of architecture, and simply refer to u16 and other types defined here.

With these abstractions in place, the implementation of a segment listing is quite simple. Since we'll be building the decoding functionality into a class, it's worth defining the class itself at this time.

#ifndef __JPEG_H_
#define __JPEG_H_

#include "inttypes.h"
#include <string>
#include <vector>
#include <map>
#include <stdio.h>

// Macro to read a 16-bit word from file
#define READ_WORD() ((fgetc(fp) << 8) | fgetc(fp))

// Segment parsing error codes
#define JPEG_SEG_ERR  0
#define JPEG_SEG_OK   1
#define JPEG_SEG_EOF -1

class JPEG {
    private:
        // Names of the possible segments
        std::string segNames[64];

        // The file to be read from, opened by constructor
        FILE *fp;

        // Segment parsing dispatcher
        int parseSeg();

    public:
        // Construct a JPEG object given a filename
        JPEG(std::string);
};

#endif//__JPEG_H_

#include "jpeg.h"
#include <stdlib.h>
#include <string.h>
#include <math.h>

//-------------------------------------------------------------------------
// Function: Parse JPEG file segment (parseSeg)
// Purpose: Retrieves 16-bit block ID from file, shows name

int JPEG::parseSeg()
{
    if (!fp) {
        printf("File failed to open.\n");
        return JPEG_SEG_ERR;
    }

    u32 fpos = ftell(fp);
    u16 id = READ_WORD(), size;
    if (id < 0xFFC0)
    {   
        printf("Segment ID expected, not found.\n");
        return JPEG_SEG_ERR;
    }

    printf(
        "Found segment at file position %d: %s\n",
        fpos, segNames[id-0xFFC0].c_str());

    switch (id) {
        // The SOI and EOI segments are the only ones not to have
        // a length, and are always a fixed two bytes long; do
        // nothing to advance the file position
        case 0xFFD9:
            return JPEG_SEG_EOF;
        case 0xFFD8:
            break;

        // An SOS segment has a length determined only by the
        // length of the bitstream; for now, assume it's the rest
        // of the file less the two-byte EOI segment
        case 0xFFDA:
            fseek(fp, -2, SEEK_END);
            break;

        // Any other segment has a length specified at its start,
        // so skip over that many bytes of file
        default:
            size = READ_WORD();
            fseek(fp, size-2, SEEK_CUR);
            break;
    }

    return JPEG_SEG_OK;
}

//-------------------------------------------------------------------------
// Function: Array initialisation (constructor)
// Purpose: Fill in arrays used by the decoder, decode a file
// Parameters: filename (string) - File to decode

JPEG::JPEG(std::string filename)
{   
    // Debug messages used by parseSeg to tell us which segment we're at
    segNames[0x00] = std::string("Baseline DCT; Huffman");
    segNames[0x01] = std::string("Extended sequential DCT; Huffman");
    segNames[0x02] = std::string("Progressive DCT; Huffman");
    segNames[0x03] = std::string("Spatial lossless; Huffman");
    segNames[0x04] = std::string("Huffman table");
    segNames[0x05] = std::string("Differential sequential DCT; Huffman");
    segNames[0x06] = std::string("Differential progressive DCT; Huffman");
    segNames[0x07] = std::string("Differential spatial; Huffman");
    segNames[0x08] = std::string("[Reserved: JPEG extension]");
    segNames[0x09] = std::string("Extended sequential DCT; Arithmetic");
    segNames[0x0A] = std::string("Progressive DCT; Arithmetic");
    segNames[0x0B] = std::string("Spatial lossless; Arithmetic");
    segNames[0x0C] = std::string("Arithmetic coding conditioning");
    segNames[0x0D] = std::string("Differential sequential DCT; Arithmetic");
    segNames[0x0E] = std::string("Differential progressive DCT; Arithmetic");
    segNames[0x0F] = std::string("Differential spatial; Arithmetic");
    segNames[0x10] = std::string("Restart");
    segNames[0x11] = std::string("Restart");
    segNames[0x12] = std::string("Restart");
    segNames[0x13] = std::string("Restart");
    segNames[0x14] = std::string("Restart");
    segNames[0x15] = std::string("Restart");
    segNames[0x16] = std::string("Restart");
    segNames[0x17] = std::string("Restart");
    segNames[0x18] = std::string("Start of image");
    segNames[0x19] = std::string("End of image");
    segNames[0x1A] = std::string("Start of scan");
    segNames[0x1B] = std::string("Quantisation table");
    segNames[0x1C] = std::string("Number of lines");
    segNames[0x1D] = std::string("Restart interval");
    segNames[0x1E] = std::string("Hierarchical progression");
    segNames[0x1F] = std::string("Expand reference components");
    segNames[0x20] = std::string("JFIF header");
    segNames[0x21] = std::string("[Reserved: application extension]");
    segNames[0x22] = std::string("[Reserved: application extension]");
    segNames[0x23] = std::string("[Reserved: application extension]");
    segNames[0x24] = std::string("[Reserved: application extension]");
    segNames[0x25] = std::string("[Reserved: application extension]");
    segNames[0x26] = std::string("[Reserved: application extension]");
    segNames[0x27] = std::string("[Reserved: application extension]");
    segNames[0x28] = std::string("[Reserved: application extension]");
    segNames[0x29] = std::string("[Reserved: application extension]");
    segNames[0x2A] = std::string("[Reserved: application extension]");
    segNames[0x2B] = std::string("[Reserved: application extension]");
    segNames[0x2C] = std::string("[Reserved: application extension]");
    segNames[0x2D] = std::string("[Reserved: application extension]");
    segNames[0x2E] = std::string("[Reserved: application extension]");
    segNames[0x2F] = std::string("[Reserved: application extension]");
    segNames[0x30] = std::string("[Reserved: JPEG extension]");
    segNames[0x31] = std::string("[Reserved: JPEG extension]");
    segNames[0x32] = std::string("[Reserved: JPEG extension]");
    segNames[0x33] = std::string("[Reserved: JPEG extension]");
    segNames[0x34] = std::string("[Reserved: JPEG extension]");
    segNames[0x35] = std::string("[Reserved: JPEG extension]");
    segNames[0x36] = std::string("[Reserved: JPEG extension]");
    segNames[0x37] = std::string("[Reserved: JPEG extension]");
    segNames[0x38] = std::string("[Reserved: JPEG extension]");
    segNames[0x39] = std::string("[Reserved: JPEG extension]");
    segNames[0x3A] = std::string("[Reserved: JPEG extension]");
    segNames[0x3B] = std::string("[Reserved: JPEG extension]");
    segNames[0x3C] = std::string("[Reserved: JPEG extension]");
    segNames[0x3D] = std::string("[Reserved: JPEG extension]");
    segNames[0x3E] = std::string("Comment");
    segNames[0x3F] = std::string("[Invalid]");

    // Open the requested file, keep parsing blocks until we run
    // out of file, then close it.
    fp = fopen(filename.c_str(), "rb");
    if (fp) {
        while(parseSeg() == JPEG_SEG_OK);
        fclose(fp);
    }
    else {
        perror("JPEG");
    }
}

When constructed with a file, an object of this JPEG class will provide output similar to the following.

Found segment at file position 0: Start of image
Found segment at file position 2: JFIF header
Found segment at file position 20: Quantisation table
Found segment at file position 89: Quantisation table
Found segment at file position 158: Baseline DCT; Huffman
Found segment at file position 177: Huffman table
Found segment at file position 208: Huffman table
Found segment at file position 289: Huffman table
Found segment at file position 318: Huffman table
Found segment at file position 371: Start of scan
Found segment at file position 32675: End of image

Next time: Examining the encoding of a scan

As can be seen above, the "scan" constitutes the majority of an entropy-coded baseline JPEG; since the entirety of the image data is encoded within the scan, this makes sense. Entropy coding is based on the Huffman compression algorithm, so in the next article I'll examine the parts of a JPEG file that provide the information needed to decode the scan from a bitstream into something usable for further processing.

Imran Nazar <tf@imrannazar.com>, Jan 2013.

• Part 1: Concepts
• Part 2: File Structure
• Part 3: Huffman Tables
• Part 4: Frames and Bitstreams

Build a JPEG decoder? Whatever for, when we have so many of them already?

JPEG is something we all take for granted: most of the Web comprises pictures transmitted as JPEG files, and video files based on JPEG technology. As it turns out, the concepts that lie behind these images span nearly two hundred years of mathematics and computing theory, and going from the raw file to an image takes a bunch of interesting work.

At the core: frequency analysis

In An Introduction to Compression, I looked at the difference between "lossless" and "lossy" compression: the difference between the two methods is that lossless compression preserves all the inherent information of the input, whereas lossy compression throws much of it away. Throwing information away only works when it can be deemed unnecessary for proper handling of the file; this would never be the case for a computer program, for example, where every byte is a statement that must be retained.

Images or videos, and their cousins in the audio world, rely on perception to work out what needs to be thrown away: just as the human ear can only distinguish sounds in a particular region of frequencies, the human eye has a particular resolution and any colour changes that happen within a very short distance are essentially invisible. Resolution can also be thought of as "visual frequency", and can be manipulated in much the same way as sound wave frequency or other kinds of wave.

It follows that a chunk of sound or a piece of an image can be compressed by removing those parts of the frequency range that are outside the human experience: those frequency ranges that we don't care about, without which the essence isn't lost. There are three steps to doing that:

• Transform the sample: Sound waves are changes of sound level through time, and images are changes of colour through two dimensions of space. We need somehow to change these into frequency representations of themselves, through a transformation.
• Band-pass: Once we're in the frequency domain, we need to cut out the section, or band, of frequencies that interests us. The technical term for this is a "band-pass filter".
• Inverse transform: Our newly reduced signal can then be sent backwards through the transformation, resulting in a sound or image that's had its excess information stripped.

The transformation use is derived from the Fourier transform, which takes an integratable mathematical function f(x) and generates an equation f(s) for a frequency spectrum. The Fourier transform only works with continuous ranges of numbers, and extends from negative infinity to positive infinity; this makes it unusable for digital transformations like those we need here. Instead, a discrete version of the Fourier transform is used: the MP3 and JPEG techniques use the discrete cosine transform (DCT) to change a set of data values into an equivalent set of frequencies.

Two examples of the compression process are presented in the following figures: first for a short sound sample.

The below figure represents the same process as above, but applied in two dimensions of space as opposed to one of time. It is generally considered inefficient to transform the entire image into a visual frequency domain at once, so the JPEG algorithm transforms blocks of eight pixels square at a time.

In Figure 2, the right-hand set of images show the DCT and its filtered version. The most important figure in each 8x8 block is the "DC component" in the top-left, which determines the base level for the whole block. Values to the right of this give information as to how often variances happen horizontally, and conversely values below the DC component provide vertical frequency information. It follows that values in the bottom-right of a DCT block describe the highest-fidelity changes in the image, and that filtering consists of drawing a diagonal across each block and retaining the top portion, throwing away the information regarding high-fidelity changes.

Colour spaces and downsampling

JPEG makes use of the fact that the human eye has a maximum resolution when it comes to visual changes. Another feature of the eye is that it is less sensitive to changes in colour than in brightness: the frequency-sensitive "cone" cells of the retina occur at a lower density than the simpler frequency-agnostic "rods", which means a lower visual resolution for colour.

It is possible for images to be further compressed by utilising this information, reducing the amount of information used to encode colour values in relation to brightness. Unfortunately, the traditional additive colour space of red/green/blue used by computer and television displays retains no information about relative brightness and colour saturation; in order to retrieve this information RGB values must be transformed to another colour model.

The JPEG format most commonly uses Y'CbCr colour, where Y' denotes the luminance of a particular pixel, and the Cb and Cr components describe the amount of chrominance on two axes, corresponding to percentages of blue and of red. Transformations from RGB pixel values to Y'CbCr act as a rotation between the cube of all possible RGB values and the cube of possible luminance/chrominance values, as shown in Figure 3.

Once transformed to Y'CbCr, the chrominance channels are separated out and can be manipulated. In this case, "downsampling" is employed: the resolution of the colour channels is halved in both dimensions, such that one "block" of colour information covers the equivalent area of four luminance blocks. In Figure 4, the colour channels have been downsampled by this ratio: it can be seen that the Y' channel is the most important for the integrity of the image, and thus its resolution remains high.

For the remainder of this series of articles, bidirectional downsampling of the type shown here will be assumed: most JPEG images on the Web employ this colour compression, so it's useful to explore. Because of the reduction in resolution, the previously mentioned eight-pixel square used by the JPEG algorithm becomes a minimum unit size of 16 pixels square, with four 8x8 luminance blocks accompanying one block for each axis of colour information.

Next time: The file format

JPEG files store additional information alongside the encoded image: lookup tables to be used by the decoding process, comments and resolution information. In part two, I'll take a look at the segments that make up a JPEG file, and how to hold onto some of the information provided for use by the decoding implementation of subsequent parts.

Imran Nazar <tf@imrannazar.com>, Jan 2013.

Like any lazy programmer, I automated the process into a Bash script.

#!/bin/bash
REPOROOT="/home/inazar/code"
PREFIX="foo-"
POSTFIX="-bar"

t=`mktemp /tmp/svnbr_XXXXX`
pushd . > /dev/null
cd $REPOROOT
for a in `find . -maxdepth 1 -name "$PREFIX*$POSTFIX"`; do
        cd $a
        b=`svn info 2> /dev/null | grep URL | awk -F'/' '{print $NF}'`
        c=${a%$POSTFIX}
        c=${c#$PREFIX}
        c=${c:2}
        d=""

        # Caveat: Display will fall over with repo names past 16 chars
        [[ ${#c} -lt 8 ]] && d="\t"
        echo -e "$c\t$d $b" >> $t
        cd ..
done
popd > /dev/null
sort $t
rm $t

Another espresso later, John took to the task of surveying the damage. It wasn't too bad, actually: Ryan had been even more trashed, and had managed to spray the contents of the bookcase around the room. John found Tolkien behind the TV, and Dostoyevsky perched miraculously above the wall clock, but Asimov would have to wait until the pile of beer cans could be examined more closely. John wasn't feeling up to that right now.

It looked like one thing was broken: the frame around the mirror. Evidently, a coffee table had been lobbed at that wall at some point (a missing chunk of plaster, and the fine coating of gypsum on the table in question, attested to that). Amazingly, the mirror wasn't broken: the wooden frame had collapsed and lay around it, but the mirror itself was intact on the floor.

John picked up the glass, and felt that there was a slightly serrated edge on the bottom. He'd never seen the back of the mirror before, and looking closer at it revealed an intricate pattern of lines, converging on a group of 20-some parallel rows corresponding to the bumps along the bottom. John had seen something like this only a few times before, and every time it was something quite special.

An electronic circuit.

Obviously, no clue was forthcoming about what the circuit contained, or the secrets encoded within any solid-state memories that might be on the back of the mirror. John thought about enlisting Ryan's help to try to decipher the tracings, but he was still knocked out on the sofa, and would probably remain there for much of the morning.

John would have to go outside for this one.

In order to keep the examples here simple, I've sought a form of program that has a simple instruction set. Many processors have a simple instruction set: the 6502 has around 150 unique codes in its architecture, which can be reduced further with careful selection of instructions. However, having written about Brainfuck before, it seems the logical choice: eight instructions, each with a unique code, and all other codes are ignored during execution.

The program to be encoded, then, is as follows:

>>++++++++[<+++[<+++++>-]>-]<<[>+>+>+>+>+<<<<<-]++++++++[>->-->
--->---->-----<<<<<-]<++++[>++++++++<-]>>>>>>++++.<<<.>+++++.<<
<.>>+++++.++.-.---.>.<<+++++++++.<.>>--.<------.<.>>.+++++.<<.>
+.>------.>.<<<.>>>-.<+.<-.>-.<++++.>>---.<<----.>.>++++.<<-.

The monkey is in the dishwasher

Encoding program code into pixels

In order for the program to be converted to an image, the first step is to take the program as a stream of numbers. In Brainfuck, the eight operators can be treated as eight numbers to be inserted into a stream:

Hiding these values within image data would be difficult if the data were full-colour (24- or 32-bit), since the colour component would need to correspond exactly at a given point in the image. Instead of using full-colour images, it makes more sense to use palette-based images, where an 8-bit palette index refers to a table of colours defined beforehand.

A simple example of palette data would be a graded greyscale image of sixteen shades, like the one below:

If this image is produced in a standard fashion, a palette of sixteen entries and 240 blanks is saved alongside the image. By expanding the palette entries used to the maximum allotted number of 256, it's possible to use the extra entries for hiding information:

As can be seen above, each row of the palette is the same shade of grey, except where the Brainfuck operators map onto the palette. By changing arbitrary pixels in the image, it's possible to replace any given #2-grey pixel with, for example, a + operator without changing the row of the palette used. Shifting pixel values along the palette row in this fashion allows us to change the value without changing the look of the image.

Before applying this concept to the example image in Figure 1, it's important to consider the type of image needed for this to work.

Uncompressed images: the BMP format

The major issue presented by image formats is that of compression: in order to make the image file size smaller, various methods of compression can be used to translate areas of similarity in the image to a much smaller amount of data than that represented by the raw image. Of course, when this is done the resultant compressed data bears no superficial relation to the raw image data; attempting to encode a Brainfuck program into the compressed data stream is beyond the scope of this article.

Instead, I'll be focussing on the raw image data itself; storage of the raw data can be achieved through an uncompressed data format such as TIFF. Examples of uncompressed 8-bit palette image formats include BMP and PCX; the former will be used here, due to its prevalence as a major image file format. When used as an uncompressed 8-bit format, the structure of a BMP file is as follows:

The end result of this process is to produce a bitmap file that can be run as a Brainfuck program; to that end, we must be careful that the header data and colour table don't contain Brainfuck operators that may corrupt the initial state of the program. This is, fortunately, relatively easy: all that is needed is to pick an image width and height that don't contain an operator within the byte values, and the rest should automatically fall into place.

One other quirk of the BMP format is the order in which pixel data is stored. Most image formats treat the first row as the top row, in accordance with computer graphics principles; BMP lines up with mathematical graphing principles, and treats the bottom row as the first.

This means that the Brainfuck operators need to be encoded into the image starting at the bottom, running left-right, then moving up to the next row. By examining each operator in the program in turn, and finding the next appropriate pixel (of the same row in the palette) to replace, the following result is obtained.

Once the appropriate pixels have been replaced with versions shifted along the palette row, the palette can be amended to set the Brainfuck operators to the same colour as the rest of the row in which they reside. Doing this, and producing an uncompressed bitmap of the result, provides the following.

Running the bitmap image, shown on the right, through a Brainfuck interpreter results in the targeted message being revealed.

<?php
list($c, $i) = split('!', file_get_contents($_SERVER['argv'][1]));
$c = preg_replace('#[^\[\]\<\>,.+-]#', "", $c);
$p = $v = 0;
eval(strtr($c, array(
    "]" => "}",
    "[" => 'while($m[$p]){',
    "+" => '$m[$p]++;',
    "-" => '$m[$p]--;',
    ">" => '$p++;',
    "<" => '$p--;',
    "," => 'if(strlen($i)>$v)$m[$p]=ord($i[$v++]);',
    "." => 'echo chr($m[$p]);'
)));
?>

$ php -f brainfuck.php http://imrannazar.com/content/img/bf-bmp-final.bmp
The monkey is in the dishwasher$

Caveats and improvements

The process outlined above is simple enough that it can be automated: given a sixteen-step greyscale image, seek from the bottom left for a pixel in the desired palette row, replace the pixel and move forward. This, however, isn't ideal for steganographic purposes; the Brainfuck operators leave a unique signature in a bitmap of otherwise uniform numbers. There are two simple ways to increase the level of obfuscation:

• Colour: Instead of using a greyscale image, which offers easy colour matching, it's possible to use a coloured palette with any 8-bit image as the source. This involves replacing any existing instances of the operator characters with a close colour match, then further colour matching can be used to fill in the program, hiding the signal in an increased level of noise.
• Compression: Noise levels can be increased further by compressing the image once it has been encoded, hiding any relation between the bitmap data and any Brainfuck program contained within. This increases the complexity of both the encoding and decoding algorithms, since a compression step must be employed around the transmission of the image, but it allows the images to be transmitted in a format that avoids suspicion.

Imran Nazar <tf@imrannazar.com>, Nov 2011.

Cyrius, glowing a faint blue: the scientists said it was one of the brightest stars for a million light years, but out here it was a dull glow in the sky. Betel, a faraway red, constant in the sky since time immemorial. And the others, each one with a name known by all: even the alphabet had been adapted to thirty letters, one for each star in the sky.

Tonight was different. There were twenty nine stars: the letter B was missing. Betel had somehow gone out, its ruddy glow doused by the dark of space. Either something had happened to it, or the light was being... blocked in some way. The central astronomy agency soon found out that Betel wasn't missing; the latter suspicion was in fact true. A device of some sort was approaching from the rough direction of Betel, and only now was it close enough to block out the star's gaze upon us.

Morning came, and the strange entity was now visible as a blot against the iodine sky. Then it began to... expand, to unfurl, as though it were a net to cover the entire sky. Before too long, it had covered the sky in a grid of wire-thin lines. Then came a voice, deep and resonating: "SIMULATION ENDS IN TEN MINUTES."

The issue of transmision encoding is obviously a solved problem: FTP has a binary mode, which avoids text-based translation, while email has MIME-formatted attachments for the inclusion of files. Another way to approach the issue, however, is to produce executables which avoid the occurrence of the problem. Since a program, like any file, is nothing more than a stream of numbers, we can alleviate the transmission issue by producing programs which only use numbers that would be found in a text file; in the ASCII encoding standard, we can define these numbers as the "printable range", between 32 and 126.

Depending on the target CPU for the executable program, the limitation of only using the printable range will have different effects: for some processors, it is impossible to write a program under these constraints. In this article, I'll be looking at the combination of the x86 PC and MS-DOS, since this allows for a wide range of instructions to be used.

Opcodes in the printable range

Since the x86 base instruction set is derived from the 8080 CPU, a number of groups of instructions fall under the printable range, including conditional branches and arithmetic manipulation. The full list is as follows.

As can be seen in the above list, a good selection of opcodes is available in the x86 printable set; this allows three possible methods for producing programs using these operations:

• Programs from scratch: Using printable opcodes from the first stages of program production, to make applications that can be directly transmitted over a plain-text channel. This is obviously the most difficult option, and provides no consideration for transmission of existing programs; as such, it will not be looked at here.
• External decoding: Programs are encoded in a plain-text representation, similar to Base64, and transmitted; a decoding application at the receiving end then processes the transmission into the original program. This solution is used by the uuencode utility, but is not an integrated solution.
• Inline decoding: The program is encoded as with external decoding, but is provided alongside a decoding routine written with printable opcodes; this eliminates the need for an external decoder, and is the simplest solution: the program can simply be run as received, and will perform as expected.

Producing the encoding

As mentioned above, the concept of Base64 can be applied in this case. Base64 encoding treats the program file as a long number, modulo 64: under this base, a number provided as a stream of bytes can be evenly divided into a stream of digits.

The canonical Base64 encoding is an adaptation of the concept behind hexadecimal: instead of using the standard denary digits in their normal position, the letters of the alphabet are used first. Values 0 to 25 are represented by the uppercase letters A-Z, 26 to 51 by the lowercase letters, and 52 to 61 by the digits 0-9. The remaining encodings 62 and 63 change between variations of the Base64 model, but are used as + and / in most variations.

This canonical encoding skips around the ASCII character set, which makes for complexity in the encoding and decoding algorithm. In the case of this article, the particular characters used for the encoding are unimportant as long as they are within the printable range: the algorithms are simplified if a contiguous range is used, so for simplicity a range from 32-95 will serve as the translation endpoint.

Encoding of the program for transmission can be performed by an external utility; the following C extract will produce a contiguous-base64 encoding of a source file.

#include <stdio.h>

int main(int argc, char **argv)
{
    FILE *in, *out;
    int fsize, i, n, o[4];

    if(argc != 2)
    {
        printf("Usage: encode <file>\n");
        return 1;
    }

    in = fopen(argv[1], "rb");
    out = fopen("encode.out", "wb");

    /* Find out the size of the file */
    fseek(in, 0, SEEK_END); fsize = ftell(in); fseek(in, 0, SEEK_SET);

    for(i=0; i<fsize; i+=3)
    {
        /* Retrieve 24 bits from the file */
        n  = (fgetc(in) << 0);
        n |= (fgetc(in) << 8);
        n |= (fgetc(in) << 16);

        /* Break out into four 6-bit values */
        o[0] = ((n >> 18) & 63) + 32;
        o[1] = ((n >> 12) & 63) + 32;
        o[2] = ((n >>  6) & 63) + 32;
        o[3] = ((n      ) & 63) + 32;

        /* Write encoded values */
        fputc(o[0], out); fputc(o[1], out);
        fputc(o[2], out); fputc(o[3], out);
    }

    fclose(in);
    fclose(out);
    return 0;
}

The inline decoder

With the source program encoded into the printable range, the remaining part of the process is the inline decoder. In theory, this is a simple reversal of the above code; the issue is complicated by the fact that the decoder must be written using printable opcodes only. In addition to using printable opcodes, some of the opcodes in the x86 instruction set use an optional ModR/M byte to specify their arguments: this additional byte defines the source and destination for the operation. A few examples:

XOR AX,BX                     ; 33 C3
MOV AL,BYTE [SI]              ; 8B 04
AND CX,WORD [SI+0293]         ; 23 8C 93 02

As can be seen above, the ModR/M byte can take on any value, and each of the possible values encodes to a combination of source and destination. Of course, only some of these combinations encode to printable byte values: those selections are shown below.

Due to the limitations imposed by both the opcode range and the allowable addressing of ModR/M bytes, a few techniques have to be employed in code production:

• Use of DOS initial values: When a program is loaded by MS-DOS or the Windows VDM, the registers have certain values at the start of execution. One of the important instances of this is that AX is zero. This value can be used throughout the program if it's saved to a register that's used solely for the purpose of providing zeroes. For example:

  ; Use BX for zero register
  PUSH AX
  POP BX

• XOR masking: Since the MOV opcode is disallowed from use, and most immediate values cannot be represented as printable bytes, XOR can be used to clear unused bits. For example:

  ; MOV AX, 0013h
  PUSH BX
  POP AX
  XOR AX, 2020h
  XOR AX, 2033h
  
  ; MOV AL, [SI+1Fh]
  DEC SI
  PUSH BX
  POP AX
  XOR AX, [SI+20h]

• Jump padding: Since jumps in the printable range are relative, forward jumps can only be made to destinations more than 31 bytes away. To facilitate this, extra bytes can be inserted if the destination isn't far enough away. This requires that the encoded output of the operations between the jump and its destination is known, since the code length is required to calculate the amount of padding required.
• Manual encoding: Some operations will not be intuitively selected by the assembler, and will be encoded into the non-printable range. To avoid this, the operation can be assembled by hand and inserted at the appropriate point. For the example of XOR where the source byte is 65 bytes from the opcode:

  ; XOR AL, [SI+($-PRSTART)]
  DB 32h, 44h, 41h

Using these techniques, it's possible to write the reverse decoder for the printable-opcode encoder detailed above, and attach the encoded program to it. The resultant program is shown below, in NASM assembly format.

    mov al, 0x13
    int 0x10               ; Set 320x200x8 graphics mode
    push 0xA000
    pop es                 ; Set destination segment

lineloop:
    mov cx, 0x0140         ; For 200 lines
    in ax, 0x40            ; Get a "random" number from the timer
    and ax, 0xBF
    mul cx
    mov di, ax             ; Write to that line on screen (mod 191)
    in al, 0x40            ; Get a "random" number
    rep stosb              ; Write a line of that colour

    in al, 0x60            ; Check for a key
    dec al
    jnz lineloop           ; Loop back if not ESC

end:
    ret                    ; Return to DOS
 

    pusha               ; Save all registers for post-decoding

    ; Initialise registers and zero-holder
    push ax
    push ax
    push ax
    push ax
    push ax
    pop edx
    pop ebx
    pop cx

    ; Rewrite jpouter and jpinner
    xor al, jpouter-256-96     ; Get a printable-range 8-bit value
    xor ax, 0x235B
    xor ax, 0x225B             ; Add 256
    push ax
    pop si                     ; Use this as the address
    push bx
    pop ax
    xor al, 0x20
    sub al, 0x55               ; AL = 0xCB
    xor [si+0x60], al          ; Opcode = 0xEB (JMP rel)
    xor [si+0x62], al
    xor al,0x34                ; AL = 0xFF
    xor [si+0x61], al          ; Set jump point to known value
    xor [si+0x63], al

    inc ax                     ; AX = 0x0100
    push ax
    pop bp                     ; BP = 0x0100

    ; Set destination point (CS:1000 - 32)
    xor ax,0x3E30
    xor ax,0x3030
    sub al,32
    push ax
    pop di

    ; Set source point (+256+8)
    push bx
    pop ax
    xor ax,0x3120
    xor ax,0x307E
    push ax
    pop si

    ; Perform the decoding
lpouter:
    ; In blocks of 4
    push bx
    push bx
    pop eax
    and [di+0x20],eax
    inc cx
    inc cx
    inc cx
    inc cx

    ; Read 6 bits
lpinner:
    push bx
    pop ax

    db 0x32, 0x44, 0x41           ; xor al, [si+(source-$)]
    sub al,32
    cmp al,94
    je end

    ; Push along by 6 bits, shift in the new bits
    imul edx,[di+0x20],byte 64

    and [di+0x20],ebx
    xor [di+0x20],edx          ; [Dest] = Shifted val
    xor [di+0x20],al           ; [Dest] += next
    inc si
    dec cx
    jnz jpinner

    ; Once the block of 4 is done, DI += 3
    inc di
    inc di
    inc di
    jnz jpouter                ; Jump indirect

    ; Jump padding, since the above code is 25 bytes long
    db '@@@@@@@'

end:
    push bp
    pop ax                     ; AX = 0x0100
    
    ; Rewrite jump to CS:1000
    xor al,jpouter-256-64
    push ax
    pop si
    push bx
    pop ax
    and [si+0x60],ax
    and [si+0x62],ax           ; Clear opcode
    sub al,0x37
    xor al,0x20                ; AL = 0xE9
    xor [si+0x60],al           ; JMP (rel16)
    push bx
    pop ax
    xor ax,0x3E30
    xor ax,0x3072              ; AX = 0x1000 - $
    xor [si+0x61],ax
    dec di                     ; Clear Z flag
    popa                       ; Retrieve all registers
    jnz jpouter+32             ; Jump indirect to CS:1000

    ; Rewritable jump points
jpouter:
    db 0x20
    db ($-lpouter)
jpinner:
    db 0x20
    db ($-lpinner)

    ; The encoded program
source:
    db 'S1.P &@0N0>@Y0% OR5 X?< Y,>)JO- _F#DZG7(___#'
    db 126                     ; EOF marker