Lemmy Server Performance

1 readers

1 users here now

lemmy_server uses the Diesel ORM that automatically generates SQL statements. There are serious performance problems in June and July 2023 preventing Lemmy from scaling. Topics include caching, PostgreSQL extensions for troubleshooting, Client/Server Code/SQL Data/server operator apps/sever operator API (performance and storage monitoring), etc.

founded 1 year ago

MODERATORS

[email protected]

De-facto memory leak (lemmy.world)

submitted 1 year ago by [email protected] to c/[email protected]

8 comments fedilink hide all child comments

It looks like the lack of persistent storage for the federated activity queue is leading to instances running out of memory in a matter of hours. See my comment for more details.

Furthermore, this leads to data loss, since there is no other consistency mechanism. I think it might be a high priority issue, taking into account the current momentum behind growth of Lemmy...

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 3 points 1 year ago (4 children)

Using Couchbase as your eventual consistency database is perfect for this scenario. It's designed for this type of thing. Even if systems are offline for a few days they will queue up and replicate when they come back online. Cruise ships use it for this very reason.

[–] [email protected] 2 points 1 year ago (3 children)

Can you provide a link? Only thing I see on Couchbase is for NoSQL databases.

[–] [email protected] 1 points 1 year ago

Actually XDCR is not available in the community edition so that option is gone. Sorry about that.

load more comments (2 replies)