Announcing a new Smart Keymap Library

For the last few months, I’ve been working on a new “smart keymap” library.

The code is published over at: https://github.com/rgoulter/smart-keymap.

This post is to introduce this, go over what it is, and why I’ve found it exciting to work on.

“Smart Keymap”

By “smart keymap”, I mean the keymap behaviour for “smart keyboards”.

And by “smart keyboards”, I mean keyboards like those which run QMK or ZMK firmware.
(I first saw the term used on the Hands Down Layout webpage).

Smart keyboards often provide functionality like “Layers” or “Tap-Hold” keys, which allow the keys to have sophisticated behaviour.

For example, these features allow bringing the full functionality of the keyboard to within easy reach of the hands resting on home row. (e.g. here’s a video demonstrating how the miryoku layout does this).

What the Project is

The smart keymap library allows declaring a “smart keymap” which can be used as part of a smart keyboard’s firmware.

The keymap is declared using the Nickel configuration language, a nifty “JSON + functions + types” language.
(If you’re familiar with the FAK keyboard firmware, you can understand the smart keymap libary as like FAK, but targetting a broader set of MCUs than just the CH55x).

Example Keymap

To illustrate, here’s an example keymap declared in Nickel for a 5x15 ortholinear keyboard:

let K = import "keys.ncl" in

let DVORAK_  = 0 in
let GAMING_  = 1 in
let RAISE_   = 2 in
let LOWER_   = RAISE_ + 1 in
let ADJUST_  = RAISE_ + 2 in

{
  chords =
    let K = import "keys.ncl" in
    [
        { indices = [48, 49], key = K.LeftGUI & K.PageUp, },
        { indices = [55, 56], key = K.LeftGUI & K.PageDown, },
    ],

  config.tap_hold.interrupt_response = "HoldOnKeyTap",

  custom_keys = fun K =>
    let HoldLayerMod = fun layer_index => K.hold (K.layer_mod.hold layer_index) in
    {
      DVOR = K.layer_mod.set_default DVORAK_,
      GAME = K.layer_mod.set_default GAMING_,

      ENT_R = K.Return & HoldLayerMod RAISE_,
      ESC_L = K.Escape & HoldLayerMod LOWER_,
      ADJ   = K.layer_mod.hold ADJUST_,

      # Tap-Hold Home Row Modifiers
      A_A = K.A & K.hold K.LeftAlt,
      G_O = K.O & K.hold K.LeftGUI,
      C_E = K.E & K.hold K.LeftCtrl,
      S_U = K.U & K.hold K.LeftShift,

      S_H = K.H & K.hold K.RightShift,
      C_T = K.T & K.hold K.LeftCtrl,
      G_N = K.N & K.hold K.RightGUI,
      A_S = K.S & K.hold K.LeftAlt,

      # Sticky modifier keys
      SK_S = K.sticky K.LeftShift,
      SK_C = K.sticky K.LeftCtrl,
      SK_G = K.sticky K.LeftGUI,
      SK_A = K.sticky K.LeftAlt,
    },

  layers = [
    # Base: Dvorak
    m%"
      `    1    2    3   4     5   XXXX XXXX XXXX 6    7     8    9    0    DEL
      TAB  '    ,    .   P     Y   7    8    9    F    G     C    R    L    BSPC
      ESC  A_A  G_O  C_E S_U   I   4    5    6    D    S_H   C_T  G_N  A_S  /
      LSFT ;    Q    J   K     X   1    2    3    B    M     W    V    Z    RSFT
      LCTL LGUI LALT TAB ESC_L SPC SPC  0    BSPC BSPC ENT_R DEL  RALT RGUI RCTL
    "%,
    # Base: Gaming
    m%"
      `    1    2    3   4     5   XXXX XXXX XXXX 6    7     8    9    0    DEL
      TAB  Q    W    E   R     T   XXXX XXXX XXXX Y    U     I    O    P    BSPC
      ESC  A    S    D   F     G   XXXX XXXX XXXX H    J     K    L    ;    '
      LSFT Z    X    C   V     B   XXXX XXXX XXXX N    M     ,    .    /    RSFT
      LCTL LGUI LALT TAB ESC_L SPC XXXX XXXX XXXX BSPC ENT_R DEL  RALT RGUI RCTL
    "%,
    # Raise
    m%"
      TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT
      `    1    2    3    4    5    TTTT TTTT TTTT 6    7    8    9    0    \
      DEL  F1   F2   F3   F4   F5   TTTT TTTT TTTT F6   -    =    [    ]    /
      TTTT F7   F8   F9   F10  F11  TTTT TTTT TTTT F12  TTTT TTTT TTTT TTTT TTTT
      TTTT TTTT TTTT TTTT ADJ  TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT
    "%,
    # Lower
    m%"
      TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT
      ~    !    @    #    $    %    TTTT TTTT TTTT ^    &    *    (    )    |
      INS  F1   F2   F3   F4   F5   TTTT TTTT TTTT F6   _    +    {    }    ?
      TTTT F7   F8   F9   F10  F11  TTTT TTTT TTTT F12  TTTT HOME PGDN PGUP END
      TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT ADJ  LEFT DOWN UP   RGHT
    "%,
    # Adjust
    m%"
      DVOR GAME TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT
      TTTT BOOT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT PSCR SCRL PAUS TTTT TTTT
      CAPS SK_A SK_G SK_C SK_S TTTT TTTT TTTT TTTT TTTT SK_S SK_C SK_G SK_A CWTG
      TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT
      TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT TTTT
    "%,
  ],
}

Using an expressive language like Nickel has some compelling advantages for text-based or declarative keymap configuration, compared to what’s required to write keymaps for other keyboard firmware.

e.g. QMK’s keymaps are written in C, or a keymap.json as downloaded from the online QMK Configurator.

e.g. ZMK uses kconfig and devicetree files for its configuration.

Using C to write keymaps allows for a breadth and depth of custom implementation of behaviour.
C is relatively simple. C & CPP are somewhat limited and cumbersome when it comes to making abstractions, such as re-using keymap functionality across keymaps.

Using a configuration language to define a keymap limits what the keymap can configure to a smaller interface. But, Nickel’s highly expressive features allow it to construct abstractions and reuse/customization more elegantly than CPP. – A Nickel keymap configuration is going to be easier to maintain or re-use than code which makes heavy use of CPP.

(Using Rust to write keymaps, as Keyberon does: it’s practically nicer to use Rust than to make use of CPP; but, Rust’s strictness make it practically much more difficult for Rust code to match how bespoke a keymap’s implementation can be. Certainly in the case of QMK keymaps vs keyberon keymaps. – My own keymaps written in Rust can be found here).

Context and Motivation: The Keymap Abstraction

From my experience writing firmware for different keyboards / MCUs with keyberon, in order to best re-use code across those different firmware binaries, a fairly natural pair of abstractions emerged: the first was abstracting over the logic which produces input key press/release events from matrix scanning; the second, abstracting over the mechanism which produces HID keyboard reports from those key press/release events. – I’d describe the first as the “keyboard” or “keyboard frontend”, the second as the “keymap” or “keyboard backend”.

Essentially, the smart keymap’s interface is along the lines of Keymap { handle_event(InputEvent); tick() -> Output; }.

Later, I made a prototype keyboard which used the CH58x MCU. The CH58x EVT provides example firmware code, including an example HID keyboard, and a BLE keyboard. – The keymap logic in these examples is essentially “in a loop, tap a b c d …”. – That is, to make a practical keyboard firmware from the EVT examples, you’d need to implement matrix scanning, and implement the keymap behaviour. So, the same abstraction which was useful for code-reuse in writing keyberon firmware would also be useful for writing keyboard firmware for the CH58x.

Keymap, not Keyboard

The smart-keymap library’s scope is limited to “keymap functionality”, rather than the broader scope of “keyboard functionality”.

This allows for the project to have a narrower focus on what it supports.

‘Keymap features’ are the behaviours from manipulating inputs; ‘keyboard features’ are related to physical parts of the keyboard.
e.g. “tap hold key” functionality is definitely a keymap feature.
Whereas e.g. “split keyboard” functionality is definitely a keyboard feature. (The keymap behaviour would be the same whether the keyboard is split or not).
For things like “change the RGB effect”, the keymap doesn’t need to know how the RGB system is implemented. – All the keymap needs to provide is a way of invoking things like “change the RGB effect”.

Library, not Framework

Providing smart keymap functionality as a library also narrows the project’s scope compared to writing an entire (keyboard) firmware framework.

Practically, I think it’s always going to be easier to bring up keyboard firmware for some unsupported MCU by integrating a library into the an HID keyboard example from an EVT, compared to the effort it’d take to adapt the EVT into the platform-supporting code of a framework.

Why This Has Been Exciting to Write

Nickel is a Powerful Config Language

I first came across Nickel in relation to the Nix package manager. – Nix is pretty neat; so Nix-related things are likely to be neat.

Nickel, at a glance, is “JSON + functions (+ types)”.

The smart-keymap library obviously draws a lot of inspiration from semickolon’s fak and kirei projects; both of which are keyboard firmware which use Nickel for configuring the keymap.

My initial experience working with fak can be found in my fak-config.

After having written much more Nickel code for smart-keymap, a couple of things which stand out are its contracts, and its modular configuration.

Nickel Contracts

The Nickel manual discusses how the language helps write ‘correct’ configurations.

It took me some time to build an intuition for use of contracts.

Roughly, in Nickel, ‘types’ are only ever the concrete shape of the values. e.g. Number, String, Array of type, Record of type. Types are checked statically. Nickel doesn’t have user-defined types / aliases.

Whereas, contracts are a more expressive mechanism for describing more abstract aspects of & assertions about a value. Contracts are evaluated at runtime. Contracts can be user defined; and can even be constructed from predicate or validator functions on values.

Nickel Modular Configuration

The Nickel manual discusses how its record merging enables modular configuration.

In the Nix ecosystem, several tools make use of modules. e.g. NixOS configuration is done with modules, DevEnv configuration modules, flake-parts provides an interface for writing flakes as modules.

Modules provide a way of writing code as composable elements.

Nickel’s records are evaluated lazily, & can be combined, which allows Nickel records to be used as modules.

In the smart-keymap codebase, the codegen for the CH32X firmware is written in a modular style.
e.g. keyboard_split.ncl is a module which describes what its inputs are, what contracts it expects those inputs to satisfy, and its outputs.

How might that apply to keymaps?
For example: a keymap might be defined using colemak by default, { alphas | default = colemak_dh, .. }. Nickel syntax would then allow re-using this keymap and changing alphas with other_keymap & { alphas = qwerty }.

Inspirations: Kirei, and “Key as the primary keymap abstraction”

Aside from using Nickel as the configuration language, and targeting the CH58x MCU, smart-keymap also takes further inspiration from Kirei with its “key as the primary keymap abstraction” idea.

Notice, it’s easy to consider generalisations of various smart keymap features.
e.g. QMK’s tri-layer feature can be generalised as conditional layers, QMK’s grave-esc can be generalised as mod-morph. QMK’s leader-sequences are kinda like its one-shot layers.

Or, generally: smart keymap functionality is about using different techniques to modify what happens when a key is pressed & released.

You can even consider that “layers” fit into this pattern.

I thought this made for an interesting design spec: a keymap implementation where “key behaviour” was the primary abstraction. (& e.g. not “layers”).

Similarly, a neat observation kirei’s author made on the fak discord that a smart keymap’s output (what keycodes get sent by a keymap, from a sequence of key presses & releases) resembles parsing a grammar. (What AST is constructed from tokens for a grammar, from a sequence of input tokens).

Cheap, Powerful MCUs: CH32X, CH58x

One of the great things about the fak firmware is its support for low-cost MCUs.

WCH’s CH552T has an MCU in TSSOP-20 for about $0.5, with 16KB flash, and about 1KB XRAM.

(For more GPIO / a larger package, the CH559 has LQFP-48 for about $1.5, with 60KB flash and 6KB XRAM).

Since smart-keymap is written in Rust, it can run on targets that Rust supports.

In my case, I care about CH32X and CH58x:

MCUs: CH32X

WCH’s CH32X MCUs are about as cheap as the CH552, but have a more powerful processor.

CH32X in the TSSOP-20 package costs around $0.45, its LQFP-48 package costs 0.75.
The MCU has 62KB flash, 20KB SRAM.

The CH32X is just as simple to use in PCBs as the CH55x is.

The CH32X runs risc-v, and can run Rust firmware.

MCUs: CH58X

WCH’s CH58x is notable for being relatively a BLE-capable MCU that’s relatively cheap.

e.g. WeAct Studio’s CH58x core board costs about $2.5.

It also can run Rust firmware.

What’s Been Done So Far

Quite a bit has been done! I’ve been using keyboards running smart-keymap-based firmware for quite some time.

Smart Key functionality

A broad set of features have been implemented.

At the time of writing, the following have been implemented:

The implementation for tap-hold has some support for configuration, such as configuring the tap-hold resolution behaviour for key interrupts, or configuring a required idle time before a tap-hold key can resolve as its hold key.

(Tap-hold was surprisingly difficult to implement!)

Example Firmware

I’ve written some basic firmware for some of the different keyboards I’ve made:

The Rust implementations cover examples of col-to-row and direct pins matrices, as well as split keyboards (both full-duplex usart, and 1-wire half-duplex usart).

CH32X PCBs

Previously, I had designed a couple of keyboards which used the CH552 and fak firmware.

The CH32X’s TSSOP-20 pinout isn’t compatible with CH552T, but porting a PCB from using CH552T to CH32X is nevertheless straightforward.

I was able to port over CH32X-36, and CH32X-48, as well as making use of the LQFP-48 CH32X to make the CH32X-75.

If you’re interested in designing a keyboard with the CH32X, the kicad sources for these PCBs are available.

Getting Started

If that sounds interesting enough to give a try, I’d recommend digging through the firmware code in the smart-keymap repository to figure out how you might adapt that to your own keyboard.

If you happen to not have a suitable fancy custom keyboard, the Pico42 would be relatively easy to find the parts for & solder. Or, the cost of getting the PCB fabricated with PCBA for the CH32X-48 is about $5 per PCB, before S&H. (You’d only need to solder the switches, which is easy to do).

If you’ve already written keyboard firmware that uses keyberon, I wrote some notes on how migrating from keyberon-based code.


Older post