This may not interest you, but Ente checks most of these boxes for me. It has face recognition and AI-based object search out of the box, and you can self-host their open-source server without any restrictions. The models they used might be useful for your project.
Ente is a tremendous proposal. I don't know why I hadn't heard of it before, but I don't think it meets what I'm looking for. But the fact that the software is completely open is impressive.
I currently use Ente.io as a secondary photo syncing service in addition to Google Photos.
While I really like it — snappy and encrypted — I was surprised by how much the missing Ultra HDR implementation affects me. Photos are currently uploaded with brightness information but not displayed with it. Therefore, my photos look great in Google Photos but far less vivid in Ente.
Their pricing page doesn't say anything as far as I can find but do you still pay pay Ente if you self host the server as well as the photos ("S3-compatible object storage")?
The Ente self-hosting proposition seems strange. Why would I want to e2e encrypt my photos that I self-host? Sounds like it will only make life more difficult.
> Some people rent VPSes. This helps keep their data safe.
This is exactly how I self-host Ente and it has been great.
Machine leaning for image detection has worked really well for me, especially facial recognition for family members (easy to find that photo to share).
I have the client on my Android mobile, Fire tablet (via F-Droid), and my Windows laptop.
My initial motivation was to replace "cloud" storage for getting photos copied off the phone as soon as possible.
Because you want to access your photos remotely, or give access to more people to certain albums. If the point is to just store them locally and no remote access is needed, a hard drive would probably be enough.
The server admin can still access their own photos via the client. They wouldn't be able to access the photos of other users.
edit: To explain further why it's almost always desirable:
You guarantee that you and your users' information is safe if the server is compromised, if an admin goes rogue, or if local bodies of power request their information from you.
The information can't be sent to third-parties by design.
Any operations / transformations that need to be applied to the information will have to either be done via homomorphic encryption or on the client-side (which is much more likely to be open source / easy-to-deobfuscate compared to blackbox server code).
I understand what e2ee is, thank you. I just don't think it’s justified for self-hosted photo servers.
E. g., “Any operations / transformations” includes facial recognition, CLIP embeddings, &c; you want to run this on the server, overnight, and to be able to re-run at a later date when new models become available. Under e2ee, that’s a round-trip through a client device at every model update. So that’s a significant downside, for no important upsides in the case when you and your family are the only users.
I was explaining why e2ee has important upsides, not how e2ee works. With Ente (and I think Immich as well), facial recognition and generating new CLIP embeddings are done on-device[0], usually right when the photo is taken / before they're uploaded to the server.
What happens if there’s a new, better model? You’d need to re-download, decrypt, and run inference on all your past media, which is in terabytes for many.
I understand the benefit of e2ee in a situation where there is no trust between user and admin. In personal self-hosting, that’s the same person (or family), and the upsides are not as relevant. The downsides (possibility of data loss for, e. g., kids who are not very good with passwords/keys; difficulties with updating models / thumbs; …) remain important, and outweigh the benefits, even assuming the e2ee is implemented well.
You do you, but the trust is beyond just admin and users. And family photos are treated as treasures. Data loss is a fair point, but if you're self-hosting a photos app I imagine server/db backups are part of your routine. Account recovery is all that's needed to recover lost photos from there. Well, unless your VPS is compromised in a manner of data loss for longer than you wished before your backups ran, in which case it's still better that such sensitive info was e2ee'd.
edit: also feel like I'm echoing the classic dropbox comment, but self-hosting in a sane and secure manner is harder than it's made out to be. It needs to be taken seriously.
People have found decent solutions for that. Proton's is essentially a backup password/phrase or a file you keep safe. Not as simple as a magic link, and could still lose your backup phrase/file, but alas. Security is always a compromise on convenience.
That’s a very good point. For a long time I was advocating for self-hosting for increasing one’s privacy, but I always was hitting the “I’ve got nothing to hide” wall. Now, the concern is losing access to your data. What do you do if you’re kicked out of your email account?
e2ee makes it easier to sell their hosted version, and there's probably not enough incentive to justify the additional overhead of having an unencrypted option.
Also, my house is less secure than commercial data centers, so e2ee gives me greater peace of mind about data safety.
