<sub>2025-04-11 @19:28</sub>
#digital-garden #lua #programming #luajit #luajit/ffi #lua/learning-lua
# Problem
I’ve been working on a Neovim plugin for a side project at home. As part of this plugin, I need to define a C struct, allocate memory for it, dump it, manipulate it, and create a binary file from it.
Neovim uses LuaJIT. As far as I know, that means I need to use LuaJIT instead of standard Lua like Lua 5.4. So what now? Hmm... oh hey, there’s a C FFI (Foreign Function Interface) I can use!
# FFI
The FFI library is built right into LuaJIT. It looks super clean. The documentation makes some bold claims. It says FFI is a zero copy solution. It is also high performance and ergonomic. The documentation gives a few examples, but honestly, it falls a bit short.
## Zero Copy
Reading further, it implies that I can access C memory directly from Lua without copying it into Lua-managed memory. In standard Lua without FFI, if I want to interact with binary data, I would typically:
1. Write a C binding that copies the data into a Lua table or string.
2. Work with that copy inside Lua.
With FFI, there’s no copy. I am reading directly from the allocated C array. This is crucial for performance and memory efficiency. This is especially important in embedded or real-time systems. That is another reason why LuaJIT’s FFI library is a good solution.
```lua
local arr = ffi.new("uint8_t[4]", {1, 2, 3, 4})
print(arr[2]) -- directly reads from C memory
```
## High Performance
The documentation also notes that FFI data access and function calls are JIT optimized. They can approach native C performance. LuaJIT can inline and optimize FFI calls. That means accessing struct members or calling external functions is incredibly fast.
```lua
ffi.cdef[[ void sleep(int); ]]
ffi.C.sleep(1) -- direct system call
```
## Ergonomic
The API is clean. It feels like native Lua. It’s easy to define C types, create instances, and call C functions. FFI gives me:
- One liner `ffi.cdef` declarations
- Array and struct initialization using table syntax
- Direct indexing like `foo.x` or `array[i]`
As we will see later, FFI is a far simpler solution than writing a C module and exposing it manually using the Lua C API.
# A Very Simple Struct
According to the documentation, the two core functions I should use are `ffi.cdef` and `ffi.typeof`.
- `ffi.cdef` lets me define a C struct. LuaJIT parses and registers the type internally.
- `ffi.typeof()` creates a `cdata` type constructor I can use to allocate memory.
I call these two functions. Then I can use the constructor to allocate memory. It also looks like I can initialize the struct directly.
```lua
local cdef = [[
typedef struct
{
uint32_t x;
uint32_t y;
uint8_t c;
} Model_t;
]]
ffi.cdef(cdef)
local Point = ffi.typeof('Model_t')
local a = Point({x = 1})
```
# A Very Simple Union
The fact that LuaJIT stores this in its FFI-managed memory subsystem is really cool. Let’s try a simple union. This will help me simplify some I/O methods later. I also want to pack my structure so it’s easier to serialize and deserialize.
```lua
local cdef = [[
typedef union
{
struct
{
uint32_t x;
uint32_t y;
uint8_t c;
};
uint8_t raw[2];
} __attribute__((__packed__)) Model_u;
]]
```
It looks like I can’t use `#pragma` here. That is because preprocessors are not supported. But `__attribute__((__packed__))` works just fine. Nice. Moving on.
# One Step Further
Now I want to impose some behavior on my `cdata` type constructor. Since I understand the setup, I can use `ffi.metatype` to attach a metatable.
What behavior makes sense here? I know I’ll want to write this to a file. But `io.write` expects a string. I’ll also want to know the size of the struct.
From the docs:
- `ffi.string` will work for writing to a file. I don’t think overriding `__tostring` is the best fit for that. So I’ll create a `to_bytes()` method inside `__index`.
- `ffi.sizeof` gives me the size. I’ll expose that as a method too.
Here’s how I wired this up:
```lua
M = {}
function M.new(cdef, type)
ffi.cdef(cdef)
return ffi.metatype(ffi.typeof(type), {
__index = {
to_bytes = function(self)
return ffi.string(self.raw, self:size())
end,
size = function(self)
return ffi.sizeof(self)
end,
},
__tostring = function(self)
local out = {}
for i = 0, self:size() - 1 do
local fmt = string.format("%02X", self.raw[i])
table.insert(out, fmt)
end
return table.concat(out, " ")
end,
})
end
```
Nice. I might add `__metatable = false` to prevent accidental modification. I could also add a `dump` method for debugging. I might make `__tostring` a cleaner pretty printer. I like this direction. File I/O is now intuitive.
```lua
local Model = M.new(cdef, 'Model_u')
local m1 = Model({c = 1})
local m2 = Model({c = 3})
print(m1)
print(m2)
print(m1:size())
print(m2:size())
local file = assert(io.open("m1.bin", "wb"))
file:write(m1:to_bytes())
file:close()
```
Here’s the console output from running that:
```text
00 00 00 00 00 00 00 00 01 00 00 00
00 00 00 00 00 00 00 00 03 00 00 00
12
12
```
Lastly, here’s a hexdump of the file:
```
> hexdump -C m1.bin
00000000 00 00 00 00 00 00 00 00 01 00 00 00
0000000c
```