Announcing a new Smart Keymap Library
Tags: keyboards, firmware.fak, firmware.kirei, programming.ch32x, programming.ch58x, programming.embedded, programming.rust
For the last few months, I’ve been working on a new “smart keymap” library.
The code is published over at: https://github.com/rgoulter/smart-keymap.
This post is to introduce this, go over what it is, and why I’ve found it exciting to work on.
“Smart Keymap”
By “smart keymap”, I mean the keymap behaviour for “smart keyboards”.
And by “smart keyboards”, I mean keyboards like those which run QMK or ZMK firmware.
(I first saw the term used on the Hands Down
Layout webpage).
Smart keyboards often provide functionality like “Layers” or “Tap-Hold” keys, which allow the keys to have sophisticated behaviour.
For example, these features allow bringing the full functionality of the keyboard to within easy reach of the hands resting on home row. (e.g. here’s a video demonstrating how the miryoku layout does this).
What the Project is
The smart keymap library allows declaring a “smart keymap” which can be used as part of a smart keyboard’s firmware.
The keymap is declared using the Nickel configuration
language, a nifty “JSON + functions + types”
language.
(If you’re familiar with the FAK keyboard
firmware, you can understand the smart
keymap libary as like FAK, but targetting a broader set of MCUs than just the
CH55x).
Example Keymap
To illustrate, here’s an example keymap declared in Nickel for a 5x15 ortholinear keyboard:
let K = import "keys.ncl" in
let DVORAK_ = 0 in
let GAMING_ = 1 in
let RAISE_ = 2 in
let LOWER_ = RAISE_ + 1 in
let ADJUST_ = RAISE_ + 2 in
{
chords =
let K = import "keys.ncl" in
[
{ indices = [48, 49], key = K.LeftGUI & K.PageUp, },
{ indices = [55, 56], key = K.LeftGUI & K.PageDown, },
],
config.tap_hold.interrupt_response = "HoldOnKeyTap",
custom_keys = fun K =>
let HoldLayerMod = fun layer_index => K.hold (K.layer_mod.hold layer_index) in
{
DVOR = K.layer_mod.set_default DVORAK_,
GAME = K.layer_mod.set_default GAMING_,
ENT_R = K.Return & HoldLayerMod RAISE_,
ESC_L = K.Escape & HoldLayerMod LOWER_,
ADJ = K.layer_mod.hold ADJUST_,
# Tap-Hold Home Row Modifiers
A_A = K.A & K.hold K.LeftAlt,
G_O = K.O & K.hold K.LeftGUI,
C_E = K.E & K.hold K.LeftCtrl,
S_U = K.U & K.hold K.LeftShift,
S_H = K.H & K.hold K.RightShift,
C_T = K.T & K.hold K.LeftCtrl,
G_N = K.N & K.hold K.RightGUI,
A_S = K.S & K.hold K.LeftAlt,
# Sticky modifier keys
SK_S = K.sticky K.LeftShift,
SK_C = K.sticky K.LeftCtrl,
SK_G = K.sticky K.LeftGUI,
SK_A = K.sticky K.LeftAlt,
},
layers = [
# Base: Dvorak
m%"
` 1 2 3 4 5 XXXX XXXX XXXX 6 7 8 9 0 DEL
TAB ' , . P Y 7 8 9 F G C R L BSPC
ESC A_A G_O C_E S_U I 4 5 6 D S_H C_T G_N A_S /
LSFT ; Q J K X 1 2 3 B M W V Z RSFT
LCTL LGUI LALT TAB ESC_L SPC SPC 0 BSPC BSPC ENT_R DEL RALT RGUI RCTL
"%,
# Base: Gaming
m%"
` 1 2 3 4 5 XXXX XXXX XXXX 6 7 8 9 0 DEL
TAB Q W E R T XXXX XXXX XXXX Y U I O P BSPC
ESC A S D F G XXXX XXXX XXXX H J K L ; '
LSFT Z X C V B XXXX XXXX XXXX N M , . / RSFT
LCTL LGUI LALT TAB ESC_L SPC XXXX XXXX XXXX BSPC ENT_R DEL RALT RGUI RCTL
"%,
# Raise
m%"
TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT
` 1 2 3 4 5 TTTT TTTT TTTT 6 7 8 9 0 \
DEL F1 F2 F3 F4 F5 TTTT TTTT TTTT F6 - = [ ] /
TTTT F7 F8 F9 F10 F11 TTTT TTTT TTTT F12 TTTT TTTT TTTT TTTT TTTT
TTTT TTTT TTTT TTTT ADJ TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT
"%,
# Lower
m%"
TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT
~ ! @ # $ % TTTT TTTT TTTT ^ & * ( ) |
INS F1 F2 F3 F4 F5 TTTT TTTT TTTT F6 _ + { } ?
TTTT F7 F8 F9 F10 F11 TTTT TTTT TTTT F12 TTTT HOME PGDN PGUP END
TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT ADJ LEFT DOWN UP RGHT
"%,
# Adjust
m%"
DVOR GAME TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT
TTTT BOOT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT PSCR SCRL PAUS TTTT TTTT
CAPS SK_A SK_G SK_C SK_S TTTT TTTT TTTT TTTT TTTT SK_S SK_C SK_G SK_A CWTG
TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT
TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT
"%,
],
}
Using an expressive language like Nickel has some compelling advantages for text-based or declarative keymap configuration, compared to what’s required to write keymaps for other keyboard firmware.
e.g. QMK’s keymaps are written in C, or a keymap.json
as downloaded from the online QMK
Configurator.
e.g. ZMK uses kconfig and devicetree files for its configuration.
Using C to write keymaps allows for a breadth and depth of custom implementation
of behaviour.
C is relatively simple. C & CPP are somewhat limited and cumbersome when it comes
to making abstractions, such as re-using keymap functionality across keymaps.
Using a configuration language to define a keymap limits what the keymap can configure to a smaller interface. But, Nickel’s highly expressive features allow it to construct abstractions and reuse/customization more elegantly than CPP. – A Nickel keymap configuration is going to be easier to maintain or re-use than code which makes heavy use of CPP.
(Using Rust to write keymaps, as Keyberon does: it’s practically nicer to use Rust than to make use of CPP; but, Rust’s strictness make it practically much more difficult for Rust code to match how bespoke a keymap’s implementation can be. Certainly in the case of QMK keymaps vs keyberon keymaps. – My own keymaps written in Rust can be found here).
Context and Motivation: The Keymap Abstraction
From my experience writing firmware for different keyboards / MCUs with keyberon, in order to best re-use code across those different firmware binaries, a fairly natural pair of abstractions emerged: the first was abstracting over the logic which produces input key press/release events from matrix scanning; the second, abstracting over the mechanism which produces HID keyboard reports from those key press/release events. – I’d describe the first as the “keyboard” or “keyboard frontend”, the second as the “keymap” or “keyboard backend”.
Essentially, the smart keymap’s interface is along the lines of
Keymap { handle_event(InputEvent); tick() -> Output; }
.
Later, I made a prototype keyboard which used the CH58x MCU. The CH58x EVT provides example firmware code, including an example HID keyboard, and a BLE keyboard. – The keymap logic in these examples is essentially “in a loop, tap a b c d …”. – That is, to make a practical keyboard firmware from the EVT examples, you’d need to implement matrix scanning, and implement the keymap behaviour. So, the same abstraction which was useful for code-reuse in writing keyberon firmware would also be useful for writing keyboard firmware for the CH58x.
Keymap, not Keyboard
The smart-keymap library’s scope is limited to “keymap functionality”, rather than the broader scope of “keyboard functionality”.
This allows for the project to have a narrower focus on what it supports.
‘Keymap features’ are the behaviours from manipulating inputs; ‘keyboard features’
are related to physical parts of the keyboard.
e.g. “tap hold key” functionality is definitely a keymap feature.
Whereas e.g. “split keyboard” functionality is definitely a keyboard feature.
(The keymap behaviour would be the same whether the keyboard is split or not).
For things like “change the RGB effect”, the keymap doesn’t need to know how the
RGB system is implemented. – All the keymap needs to provide is a way of invoking things
like “change the RGB effect”.
Library, not Framework
Providing smart keymap functionality as a library also narrows the project’s scope compared to writing an entire (keyboard) firmware framework.
Practically, I think it’s always going to be easier to bring up keyboard firmware for some unsupported MCU by integrating a library into the an HID keyboard example from an EVT, compared to the effort it’d take to adapt the EVT into the platform-supporting code of a framework.
Why This Has Been Exciting to Write
Nickel is a Powerful Config Language
I first came across Nickel in relation to the Nix package manager. – Nix is pretty neat; so Nix-related things are likely to be neat.
Nickel, at a glance, is “JSON + functions (+ types)”.
The smart-keymap library obviously draws a lot of inspiration from semickolon’s fak and kirei projects; both of which are keyboard firmware which use Nickel for configuring the keymap.
My initial experience working with fak can be found in my fak-config.
After having written much more Nickel code for smart-keymap, a couple of things which stand out are its contracts, and its modular configuration.
Nickel Contracts
The Nickel manual discusses how the language helps write ‘correct’ configurations.
It took me some time to build an intuition for use of contracts.
Roughly, in Nickel, ‘types’ are only ever the concrete shape of the values. e.g. Number, String, Array of type, Record of type. Types are checked statically. Nickel doesn’t have user-defined types / aliases.
Whereas, contracts are a more expressive mechanism for describing more abstract aspects of & assertions about a value. Contracts are evaluated at runtime. Contracts can be user defined; and can even be constructed from predicate or validator functions on values.
Nickel Modular Configuration
The Nickel manual discusses how its record merging enables modular configuration.
In the Nix ecosystem, several tools make use of modules. e.g. NixOS configuration is done with modules, DevEnv configuration modules, flake-parts provides an interface for writing flakes as modules.
Modules provide a way of writing code as composable elements.
Nickel’s records are evaluated lazily, & can be combined, which allows Nickel records to be used as modules.
In the smart-keymap codebase, the codegen for the CH32X
firmware
is written in a modular style.
e.g. keyboard_split.ncl
is a module which describes what its inputs are, what
contracts it expects those inputs to satisfy, and its outputs.
How might that apply to keymaps?
For example: a keymap might be defined using colemak by default, { alphas | default = colemak_dh, .. }
.
Nickel syntax would then allow re-using this keymap and changing alphas
with other_keymap & { alphas = qwerty }
.
Inspirations: Kirei, and “Key as the primary keymap abstraction”
Aside from using Nickel as the configuration language, and targeting the CH58x MCU, smart-keymap also takes further inspiration from Kirei with its “key as the primary keymap abstraction” idea.
Notice, it’s easy to consider generalisations of various smart keymap features.
e.g. QMK’s tri-layer feature can be
generalised as conditional
layers, QMK’s
grave-esc can be generalised as
mod-morph. QMK’s
leader-sequences are kinda like its one-shot layers.
Or, generally: smart keymap functionality is about using different techniques to modify what happens when a key is pressed & released.
You can even consider that “layers” fit into this pattern.
I thought this made for an interesting design spec: a keymap implementation where “key behaviour” was the primary abstraction. (& e.g. not “layers”).
Similarly, a neat observation kirei’s author made on the fak discord that a smart keymap’s output (what keycodes get sent by a keymap, from a sequence of key presses & releases) resembles parsing a grammar. (What AST is constructed from tokens for a grammar, from a sequence of input tokens).
Cheap, Powerful MCUs: CH32X, CH58x
One of the great things about the fak firmware is its support for low-cost MCUs.
WCH’s CH552T has an MCU in TSSOP-20 for about $0.5, with 16KB flash, and about 1KB XRAM.
(For more GPIO / a larger package, the CH559 has LQFP-48 for about $1.5, with 60KB flash and 6KB XRAM).
Since smart-keymap is written in Rust, it can run on targets that Rust supports.
In my case, I care about CH32X and CH58x:
MCUs: CH32X
WCH’s CH32X MCUs are about as cheap as the CH552, but have a more powerful processor.
CH32X in the TSSOP-20 package costs around $0.45, its LQFP-48 package costs
0.75.
The MCU has 62KB flash, 20KB SRAM.
The CH32X is just as simple to use in PCBs as the CH55x is.
The CH32X runs risc-v, and can run Rust firmware.
MCUs: CH58X
WCH’s CH58x is notable for being relatively a BLE-capable MCU that’s relatively cheap.
e.g. WeAct Studio’s CH58x core board costs about $2.5.
It also can run Rust firmware.
What’s Been Done So Far
Quite a bit has been done! I’ve been using keyboards running smart-keymap-based firmware for quite some time.
Smart Key functionality
A broad set of features have been implemented.
At the time of writing, the following have been implemented:
- basic keyboard keys,
- sticky modifier keys,
- caps word,
- tap hold keys,
- tap dance keys,
- layered keys,
- chorded keys.
The implementation for tap-hold has some support for configuration, such as configuring the tap-hold resolution behaviour for key interrupts, or configuring a required idle time before a tap-hold key can resolve as its hold key.
(Tap-hold was surprisingly difficult to implement!)
Example Firmware
I’ve written some basic firmware for some of the different keyboards I’ve made:
- Using Rust’s RTIC, firmware for the RP2040 and STM32F4.
- Using Rust’s embassy, firmware for STM32.
- For the WCH MCUs, firmware which makes small changes to the HID keyboard examples from the C EVT code, for CH32X and for CH58x.
The Rust implementations cover examples of col-to-row and direct pins matrices, as well as split keyboards (both full-duplex usart, and 1-wire half-duplex usart).
CH32X PCBs
Previously, I had designed a couple of keyboards which used the CH552 and fak firmware.
The CH32X’s TSSOP-20 pinout isn’t compatible with CH552T, but porting a PCB from using CH552T to CH32X is nevertheless straightforward.
I was able to port over CH32X-36, and CH32X-48, as well as making use of the LQFP-48 CH32X to make the CH32X-75.
If you’re interested in designing a keyboard with the CH32X, the kicad sources for these PCBs are available.
Getting Started
If that sounds interesting enough to give a try, I’d recommend digging through the firmware code in the smart-keymap repository to figure out how you might adapt that to your own keyboard.
If you happen to not have a suitable fancy custom keyboard, the Pico42 would be relatively easy to find the parts for & solder. Or, the cost of getting the PCB fabricated with PCBA for the CH32X-48 is about $5 per PCB, before S&H. (You’d only need to solder the switches, which is easy to do).
If you’ve already written keyboard firmware that uses keyberon, I wrote some notes on how migrating from keyberon-based code.