It always worked for me except in some cases the 'hardware' compositor (ie the wayland side) is a bit buggy for clipboards and inputs in general. I had issues with lxc network in past but that's long ago.
I still don't understand what borked your system. Waydroid downloads the images, mounts and runs them inside lxc just like normal android. It doesn't touch your /usr or anything else. Works well in immutable os too.
zig build