There's been some work getting CLIP to run in pure C++ with quantization in GGML, and there's a curious FasterViT model I've seen months ago, so hopefully this can be made faster to inference and easier to host as one binary soon enough.
this post was submitted on 23 Jun 2023
3 points (100.0% liked)
Selfhosting
7 readers
1 users here now
All things selfhosting and homelab related Resources: - https://github.com/awesome-selfhosted/awesome-selfhosted - https://github.com/awesome-foss/awesome-sysadmin
founded 1 year ago