How to use atomic operations with user-space variable in BPF?

Question

I am trying to do a CAS with __sync_val_compare_and_swap in my eBPF code. As this operation needs to be atomic I cannot use bpf_probe_read_user. With a regular kernel variable __sync_val_compare_and_swap works with no issue.

However when using a pointer allocated in the user space, the verifier raises misaligned access off (0x0; 0xffffffffffffffff)+0+0 size 8.

Here is the code I am trying to run:

#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include <bpf/bpf_core_read.h>

__u64 *state_ptr;

char _license[4] SEC("license") = "GPL";

SEC("tp_btf/sched_switch")
int BPF_PROG(sched_switch_btf, bool preempt, struct task_struct *prev, struct task_struct *next)
{
    if (!preempt || state_ptr == NULL)
        return 0;

    bpf_printk("State: %ld", state_ptr);
    __sync_val_compare_and_swap(state_ptr, 0, 1);

    return 0;
}

Here is the full verifier log:

libbpf: prog 'sched_switch_btf': BPF program load failed: Permission denied
libbpf: prog 'sched_switch_btf': -- BEGIN PROG LOAD LOG --
reg type unsupported for arg#0 function sched_switch_btf#29
0: R1=ctx(off=0,imm=0) R10=fp0
; int BPF_PROG(sched_switch_btf, bool preempt, struct task_struct *prev, struct task_struct *next)
0: (79) r1 = *(u64 *)(r1 +0)          ; R1_w=scalar()
; if (!preempt || state_ptr == NULL)
1: (15) if r1 == 0x0 goto pc+14       ; R1_w=scalar()
2: (18) r1 = 0xffffb4aa86b6e000       ; R1_w=map_value(off=0,ks=4,vs=8,imm=0)
4: (79) r3 = *(u64 *)(r1 +0)          ; R1_w=map_value(off=0,ks=4,vs=8,imm=0) R3_w=scalar()
5: (15) if r3 == 0x0 goto pc+10       ; R3_w=scalar()
; bpf_printk("State: %ld", state_ptr);
6: (18) r1 = 0xffff9b028dc40f10       ; R1_w=map_value(off=0,ks=4,vs=11,imm=0)
8: (b7) r2 = 11                       ; R2_w=11
9: (85) call bpf_trace_printk#6       ; R0=scalar()
; __sync_val_compare_and_swap(state_ptr, 0, 1);
10: (18) r1 = 0xffffb4aa86b6e000      ; R1_w=map_value(off=0,ks=4,vs=8,imm=0)
12: (79) r1 = *(u64 *)(r1 +0)         ; R1_w=scalar()
13: (b7) r2 = 1                       ; R2_w=1
; __sync_val_compare_and_swap(state_ptr, 0, 1);
14: (b7) r0 = 0                       ; R0_w=0
15: (db) r0 = atomic64_cmpxchg((u64 *)(r1 +0), r0, r2)
misaligned access off (0x0; 0xffffffffffffffff)+0+0 size 8
processed 13 insns (limit 1000000) max_states_per_insn 0 total_states 1 peak_states 1 mark_read 0
-- END PROG LOAD LOG --

state_ptr is assigned when the bpf code is loaded. I have checked that the address stored in state_ptr is equal to the address of my variable.

I am using clang-16 and Linux kernel 6.1.0.

I tried the same code with a kernel variable. This worked I tried ensuring state_ptr was not null to make the verifier happy. Did not work

AlexSchell · Accepted Answer · 2023-06-22 19:53:49Z

0

What architecture do you use?

Atomic operations will only work on aligned data on ARM/ARM64. Align your data in the user mode application (or add fill bytes in a struct). The kernel data will be aligned by the compiler settings in the kernel Makefile.

edited Jun 22, 2023 at 19:53

AlexSchell

1,7051 gold badge4 silver badges24 bronze badges

answered Jun 17, 2023 at 20:15

Bernd Benner

571 bronze badge

Sign up to request clarification or add additional context in comments.

2 Comments

Victor Over a year ago

I am on x86_64. I will try aligning the user space data and report back. Thank you!

Victor Over a year ago

I just noticed you said on ARM/ARM64 and I am on x86_64 so this wouldn't have worked. Anyway, my data was already padded and aligned.

Collectives™ on Stack Overflow

How to use atomic operations with user-space variable in BPF?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related