I save and backup all the photos on a Synology NAS instead of using one of the online providers. However Synology Photos doesn't have good search capabilities. So I built a project to search through the images using natural language captions, and found that it works really well.
I have published the project publicly with a GPL-3.0 license - synology-photos-nlp-search. Anyone is welcome to use and contribute to the project.
It was really cool to see that I can try two search terms like food and eating, and the embeddings model would understand the difference and provide relevant images for both.
The project runs the model and stores any model-related files locally, so besides downloading the model and necessary python packages, there are no API calls being made to any outside services. I have containerized the application to make it easier to deploy and use. That said, some programming experience might be needed as it's not an open-and-use application.
This is my first major project that I am publishing, and would welcome any feedback for improvements from the community.
There's been some work getting CLIP to run in pure C++ with quantization in GGML, and there's a curious FasterViT model I've seen months ago, so hopefully this can be made faster to inference and easier to host as one binary soon enough.