https://filipnikolovski.com/posts/ebpf/
I was doing some research at work for tracing and observability for microservices, when I came across Pixielabs. This tool advertises that you can instantly troubleshoot applications without any instrumentation or special code inside the apps, which sounded ✨magical✨ to me. So naturally I wanted to know a little more about what enables this technology to work, and after scrolling through the site, under the “No Instrumentation” section was this acronym eBPF.
After some further digging on the internet, reading the design papers and watching several videos, it was safe to say this technology caught my attention, so I wanted to write some notes on it and I hope this post will spark some further interest in you as well.
The TL;DR version is that this technology enables sandboxed user space applications to run in the Linux kernel itself. In a way it makes the kernel programmable which unlocks tremendous possibilities.
You can pick any function from the kernel and execute the program every time that function runs. Running a program that “attaches” to network sockets, tracepoints and perf events can be extremely useful. Developers can debug the kernel without having to re-compile it. There are plenty of use cases such as:
That’s what enables tools like Pixie to do blackbox analysis, without requiring any tinkering inside the applications that we’re trying to debug.
Originally BPF was designed for capturing and filtering network packets that matched specific rules, discarding any unwanted packets. That’s why it was called the “Berkeley Packet Filter”. If you’ve ever used tcpdump you’ve probably already used it. Nowadays eBPF can be used for many different things and not just packet filtering but the name stuck for some reason.
This architecture of network monitoring was vastly superior to the current existing packet capture technologies in terms of performance, mainly because the filtering was done in the kernel instead of copying all of the packets in user space and doing the calculations there. But with the advancement of processors it didn’t hold up so well with it’s initial design. That’s why the extended BPF (eBPF) design was introduced, to take advantage of the modern processing power.
Initially eBPF was to be also used as a network packet filtering solution, but the ability to run user-space programs inside the kernel proved to be very powerful. The new design extended what the BPF virtual machine could do, which was to allow it to run on all kinds of events, not just packets, and also do certain actions instead of just the ability to filter things.
An eBPF program is just a sequence of 64-bit instructions. The virtual machine’s instruction set is fairly limited and it has two goals in mind: