The core · features 010, 013 and 035

On a ULX3S

The SoC, core and devices together, as a bitstream for a ULX3S board with Lattice's ECP5-85F, built with the open toolchain alone: Yosys, nextpnr-ecp5, ecppack and openFPGALoader. Its program sits in block RAM from the bitstream, its 32 MB SDRAM is the rest of its RAM, its console is the board's USB serial port, and sysk boots and runs its shell on it in a simulation of the board from power-up.

What is in it

Almost all of the board is sysr. boards/ulx3s/ulx3s.sysr is the board's top: the SoC (soc, from cores/rv64im/soc.sysr) with its parameters, the SDRAM's pins under the board's names (the 16 data lines an inout), the serial input through two flip-flops (it is asynchronous to the clock), the LEDs, and the ESP32's GPIO0 held high so the radio module does not reset the board. Its parameters are the board's facts: the block RAM's size and the SDRAM's after it, the loads' and stores' latencies, the SDRAM's power-up, the UART's rate and mtime's rate, and clock-hz says what the clock is; tools/platform.rkt reads all of them to generate platform/ulx3s (Devices and the platform).

LEDs: 0 is lit once the test device has halted, 1 shows the line out, 2 the line in, 3 to 5 the halt code's low bits, 6 is a heartbeat, bit 24 of a free-running counter (about 0.45 Hz), and 7 is the light only machine mode drives (mattn's bit 0; sysr 046). FIRE2 is the attention button: through two flip-flops, then sampled once every 16,384 cycles (about 1.1 ms), so that a switch's bounce makes one press (The core). FIRE1 resets.

• top.v is the only hand-written logic: the pins under the names of the board's constraint file, the ECP5's PLL (an EHXPLLL, settings from ecppll: 25 MHz / 5 × 3 × 40 = 600 MHz inside, / 40 = 15 MHz), the SDRAM chip's clock, the SoC's inverted through an output DDR register (ODDRX1F), and a reset while FIRE1 is pressed or the PLL has not locked, through two flip-flops. sysr's generated registers start at their reset values from configuration, so nothing else needs a reset pulse.
• ulx3s.lpf places those pins: the 25 MHz oscillator, the FTDI chip's two serial lines, eight LEDs, seven buttons, GPIO0, the SDRAM's 39 (sites from the board's ulx3s_v20.lpf), and the microSD slot's four in SPI mode: CLK, CMD (MOSI), DAT0 (MISO) and DAT3 (the chip select), named by what they carry, for the SD card's host (sysr 042).
• hello.s is the default program: a banner, then every line typed comes back upper-cased. Both pass through the SDRAM: the banner is copied there and printed from there, and every byte typed is stored there and loaded back, so first light on a board shows the SDRAM working.
• tb_board.v simulates the board top as it starts on the board, never reset, at 15 MHz, playing the far end of the serial line at 130 cycles per bit, with devices/sdram.sysr's model of the chip, as large as the board's, on its SDRAM pins (make ulx3s-sim); its microSD slot is empty (MISO is the pull-up's 1). A card in simulation is virt's (The SD card).

From sources to a bitstream

make ulx3s runs the chain below; make ulx3s-prog loads the result into the FPGA over USB (until power-off). Each step is a Makefile rule, quoted from the Makefile as it is. Step through it.

PlayBackStep

With sysk in the RAM, nextpnr-ecp5 reports, per the board's README: 19,146 LUT4 (23% of the 85F), 2,014 flip-flops, 192 of 208 DP16KD block RAMs, 36 multipliers and one PLL, and 16.7 MHz after routing against the 15 MHz needed. The core does one instruction a cycle with nothing pipelined, so its longest path runs from the block RAM's output (the instruction) through decode and execute into the next access, about 60 ns, two thirds of it routing across the 192 RAM blocks. A faster clock needs a pipelined core, not a better placement. Place and route takes about 20 minutes.

The program in block RAM: eight byte lanes

The core's RAM is eight memories, one per byte of a doubleword, so a store can write any of its bytes (b0 to b7 in rv64im.sysr's ram). Yosys maps each to block RAM, and the bitstream can hold their initial contents. tools/lanes.rkt turns a program, an assembly file or an ELF executable, into eight hex files, one per lane, each with ram-words lines; byte k of the image goes to lane k mod 8, line k div 8. sysr build --mem-init b0=.../b0.hex … puts them into the generated Verilog as $readmemh. A program larger than the RAM is an error. Type some bytes to see where they land; the default is hello.s's banner, read from its .word line.

Bytes from 0x80000000

Select a byte.

Two choices made 384 KB fit and meet timing. Each lane is one copy: the fetch port now sees bytes stored in the same cycle through a bypass outside the RAM, which Yosys recognizes, where reading the old data would have taken two copies of each lane. And 384 KB is 49,152 doublewords, which Yosys builds from 16384 × 1-bit blocks, so each bit of a doubleword is one of three blocks behind a small multiplexer; at 400 KB only 2048 × 9 blocks fit, each bit is one of 25, and routing did not finish. The RAM may be any number of doublewords: since feature 013 an access past its end is an access fault in the core, and the RAM itself reads word 0 there and writes nothing, so a kernel that runs out of RAM traps instead of overwriting its start.

The SDRAM

The RAM is 32 MB at 0x80000000 (sysr 031 to 035). Its first 384 KB are the block RAM, which the bitstream loads and the only memory instructions are fetched from; the rest is the board's SDRAM (xram-words, 4,145,152 doublewords; the chip's own first 384 KB are unused), its contents unknown at power-up. devices/sdram.sysr drives it closed-page, so every load takes 11 cycles and every store 10, whichever of the two memories it reaches: no address shows in time, and the core's Zkt proof stands unchanged (Devices). The chip needs 200 µs before its first command: the controller waits 3000 cycles, precharges, refreshes 8 times and sets its mode, and the core fetches its first instruction at cycle 3019 (boot-cycles). The chip's clock is the SoC's inverted, so it takes each command half a cycle after the SoC puts it on the pins, and its data is taken at the end of the cycle it drives it in, half a cycle later.

Clock and timing

The board's oscillator is 25 MHz, but place and route found the single-cycle core's longest path too slow for it, so top.v's PLL makes 15 MHz. The UART's div is 130: 15,000,000 / 130 is 115,385 bits per second, 0.2% above 115,200, well within what a serial port accepts.

sysk's scheduler counts time in mtime ticks, and QEMU's virt machine counts mtime at 10 MHz. So that a 1 ms quantum is 1 ms on the board too, the core counts mtime on tick-num of every tick-den cycles, evenly: a register tick adds tick-num each cycle, and when the sum reaches tick-den it subtracts tick-den and mtime counts. The board sets 2 of 3, and the core proves tick-below: tick stays below tick-den.

tick-num tick-den PlayStepReset

sysk on the board, in simulation

Feature 013 put sysk in the board's block RAM. Two things stood in the way, and both are now in the core: its RAM had to be a power of two that wrapped around (now any size, with an end that faults), and its timer counted the 15 MHz clock (now 2 of every 3 cycles). sysk (its version 028, with the shell) fits in 384 KB once its stacks are 4 KB instead of 16 KB; the deepest any of its stacks went on the ISS was 1.5 KB.

make ulx3s ULX3S_PROG=../sysk/build/sysk.elf
make ulx3s-sim ULX3S_PROG=../sysk/build/sysk.elf ULX3S_INPUT='ls\nhello\n\004' ULX3S_CYCLES=2000000
make sysk-ulx3s ULX3S_PROG=../sysk/build/sysk.elf    # the core against the ISS, to the exit

As recorded in the board's README and feature 013: in the board's configuration (384 KB, mtime at 2 of every 3 cycles, 130 cycles per bit), sysk boots, answers ls with sys/ and hello with unknown command hello, and exits with the pass code after 53 timer interrupts, at cycle 862,075, 57 ms at 15 MHz. The core agrees with the ISS there on everything: registers, CSRs, the devices, all 384 KB of memory, every I/O store, every byte on the serial line. The board top, simulated in Icarus from power-up with its input on the serial pin from cycle 25,000, prints the same and halts at cycle 889,227.

sysk has grown since. On 2026-10-19 (sysk 070) it is 356 KB of code, and with 4 KB stacks 504 KB with its .bss, more than the 384 KB here or the 416 KB of block RAM the 85F has; it also waits at boot for its virtio devices, which the board does not have. A board build of sysk needs its own device list (platform/ulx3s/devices.sysk: the UART alone) and to leave out what the board cannot use. Until then, sysk's timing on the board is simulated with more RAM than it has (--ram-words 131072), which changes nothing about the timer or the UART: A paste at the prompt.

With the SDRAM (sysr 035) the RAM is 32 MB. The image, code and initialized data, must still fit the first 384 KB, but stacks, heap and .bss can be anywhere after it, at their QEMU sizes. sysk 098 builds that image (make board in sysk, 282 KB), and core-test runs its whole session in the board's whole configuration, core against ISS (sysr 039, 040): the boot lines, ls /prg typed once they are out, then Ctrl-D, until every task has finished. The session ends at cycle 2,932,892, 0.2 s at 15 MHz, after 190 timer interrupts, and the core agrees with the ISS there on everything. Before sysp 082 it did not end: sysk's C build copied its 384-byte kernel state at every call that took it by value, a doubleword at a time from the SDRAM, and the shell never came back from the prompt in 8 million cycles. Passing it by pointer made that copy go away, and the SDRAM's fixed latency stays as it is.

Who can replace the machine

The core keeps tasks from each other and from the kernel; the FPGA beneath it is a different question, since a new bitstream replaces the whole machine, kernel and memory included. Read from Lattice's ECP5 and ECP5-5G sysCONFIG Usage Guide (FPGA-TN-02039), the board's manual and the open toolchain's source (sysr 050), and stated in the board's CONTRACT.md (Ways in):

• JTAG cannot be turned off. "As a dedicated port on the ECP5 and ECP5-5G devices, the JTAG port is always available." On the ULX3S three things drive it: the JTAG header, the ESP32 (esp32ecp5), and the FTDI chip behind the USB port, which carries the serial console and is JTAG for fujprog too. Whoever holds the console's cable can load another bitstream.
• The security bit blocks reading, not replacing. With CONFIG_SECURE, "the read-back of the SRAM memory is blocked"; reprogramming the device clears it. prjtrellis's ecppack has no option for it and leaves its place in the bitstream empty, so this build does not set it.
• The one lock against another bitstream is encryption. With the key programmed into the device and its Decrypt Only bit set, "only encrypted data is accepted". The key and the bit are programmed with Lattice's own tools (Diamond Programmer; the Advanced Security Encryption Key Programming Guide), the fuses are one-time, and the open toolchain cannot make an encrypted bitstream: adopting it would put a closed tool in the build of every image, the owner's choice.
• The configuration flash (an IS25LP128F, or a Winbond W25Q128JV on some boards) has block protection, and its status register can be locked until power-off, or for good, through its write-protect pin, which the board wires to the FPGA. A bitstream could hold it; this one does not, and a replacement bitstream would release it anyway.

So on this board, physical access to the USB port, the JTAG header or the ESP32's radio is access to the machine, and the contract says so rather than leaving it to be assumed.

What has not been done

• Running it on a board. Everything above is a build or a simulation.
• An entropy source. The board ties the SoC's noise input to 0, so Zkr's seed answers DEAD (sysr 028; Entropy). A ring-oscillator source in the fabric needs measurements on the board.
• Place and route with the SDRAM. Yosys's count with it: 14,634 LUT4, 2,833 flip-flops, 192 DP16KD, 36 multipliers, one PLL, one ODDRX1F; nextpnr has not run on it yet.
• Code past 384 KB. A loader from the configuration flash and instruction fetch from the SDRAM, when sysk's image outgrows the block RAM.
• A faster clock, which needs a pipelined core and, with it, a new timing model in the ISS.