By not all ending up on the same server.
Asklemmy
A loosely moderated place to ask open-ended questions
Search asklemmy π
If your post meets the following criteria, it's welcome here!
- Open-ended question
- Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
- Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
- Not ad nauseam inducing: please make sure it is a question that would be new to most members
- An actual topic of discussion
Looking for support?
Looking for a community?
- Lemmyverse: community search
- sub.rehab: maps old subreddits to fediverse options, marks official as such
- [email protected]: a community for finding communities
~Icon~ ~by~ ~@Double_[email protected]~
A small cloud server + a domain name costs less than a Netflix subscription. The developers have taken care to package lemmy in ways that are relatively straight forward to deploy, so a dedicated person with a small amount of experience can have an instance up and running in an evening. As long as a few percentage of users are willing to pay a netflix subscription to keep a server running, the financial burden would be spread.
I think this underestimates how users will naturally gravitate towards more centralized instances, or they'll give up because the bigger instances are closed. Someone's gotta pay for it, and it's going to cost more than a Netflix subscription. Servers aren't cheap.
This also ignores that the system isn't horizontally scalable at all, so scaling up gets even more expensive
I think this underestimates how users will naturally gravitate towards more centralized instances, or theyβll give up because the bigger instances are closed.
(This is purely my personal opinion, of course!) In the scenario in which a few large instances dominate, the idea of the fediverse failed. One may estimate the likelyhood of success or failure given how they expect humans to behave, but in the end experiment beats theory. I think that for the fediverse to work a significant cultural shift has to occur, but I don't think that it is an impossible shift. I would like the fediverse to succeed, and so I choose to take part in the experiment.
This also ignores that the system isnβt horizontally scalable at all, so scaling up gets even more expensive
Yes, that might cause some serious issues. The project is still in an early-development phase, and I don't understand the technical aspects well enough yet to be able to identify whether there is obviously a fundamentally invincible barrier when it comes to scalability. My optimistic hope is that the developers are able to optimize horizontal scalability fast enough to meet the demand for scale. If it turns out to be impossible to scale, then only rich enough parties would be able to have viable instances, and that could be a reason for failure.
It's certainly a challenge. I run three instances on the #fediverse. Two small ones and a larger one with 450 users. I have a donation page; initially people were enthusiastic and I covered costs. Now, I have regular monthly donators but not enough to cover costs so I am subsidising it. I took the decision when I launched that it could happen and it's my problem.
I think there will be many instances will fall away in the coming months due to costs. Especially if you are thousands of users and associated costs.
We need to come up with a new funding model, where people appreciate you get nothing for nothing. All the large corporates sell your data as advertising for revenue. The greater public do not appreciate they are selling their soul.
Is there an approximate specs per number of users guide to size a lemmy instance?
As everyone else has already said that's a very good question, one that doesn't necessarily have an answer, but Im not too concerned.
I'd point out (rather excitedly) that this really isn't unlike how the Internet used to be up until the late 00s or very early 2010s and the rise of insta, FB, birdsite, digg and reddit. EVERYone had to shoulder hosting costs (unless you were on Geocities,Myspace then it was ads)
Yes, we've had bulletin boards and discussion forums since perl and CGI were a thing; each was self hosted at the hoster's expense. Newsgroup and IRC servers too - THOSE all acted like "federated" instances - common newsgroups and chat channels would be synchronized and replicated from server to server EXACTLY how federated Lemmy/Kbin/etc. instances do it now.
And the infrastructure costs were a struggle then and they will be now. Back then to have a capable CGI forum host, or to colocate your server in someone's data center it cost a lot - like decent hosting/co-loc plans started at $50/month and went up from there. Most hosting plans had steep bandwidth caps, think like 5GB included and +$5 per GB - if you hosted a popular site 40-50GB of traffic wasn't abnormal. If you ran a newsgroup server you frequently had to futz with how long newsgroup msgs were retained to save disk space; like 48 hrs or less (then the data would be purged).
What you can get for $50/month THESE days is quite a lot more capable, and you can run a low retention instance for a lot less. Bandwidth and disk space are ludicrously cheap (at least compared to 10-15+ yrs ago). If your instance is low user, low community, and reasonable data retention/cloning, you could run Lemmy or a Mastodon or Calkey server on an old computer you have kicking around and host it from your home internet connection with a dynamic DNS mapping.
Obviously the big instances with gobs of users will struggle with how they pay for the server infrastructure - some will use crowdfunding, patrons, donations etc. Others will run ads, or subscriptions.
My home instance lemmy.ca is at 1400 users (as of right now) and is on a $25-30/month hosting plan and so far the site is doing just fine (or seems to be). I'd guess that a massive instance like lemmy.ml might be north of $1-200. But, if you think about it, all you need are 20 ppl to donate $10/month. I donate yearly to Wikipedia. As they discuss in this thread here https://lemmy.ca/post/599590 Mastodon gets $28k Euros a month in donations and pays for two? full time developers, so its not like there aren't people donating to open source projects... and so far Fediverse servers are doing fine.
Do you happen to know what service that's with? Trying to see what resources that takes. ($25-$30 can mean very different specs depending on where it is hosted). Ty.
That's why I'm running my own server. Mastodon is much bigger than Lemmy and it does fine with community run servers.
Yeah technical users and people who are friends with technical users can just use a micro instance.
Yep, I'm using lemmy.world mostly as a trial period, but I plan on getting on my own instance to help reduce the load on the community any way I can. Will probably get a few friends on it as well.
I think the biggest cost will be image/video storage. The text takes very little space in today's standards. The good thing is that symmetric fibre internet connections are becoming more common so it may be possible for members of the instance to contribute unused disk space to help with its image/video storage. This plus limiting the image/video sizes (and maybe forbidding video uploads altogether) will allow the instances to scale with user count.
In all honesty, there are a ton of us tech enthusiasts who have no problem paying 10-20$ per month to run an instance out of our own pockets. We get the ability to subscribe to content we used to use Reddit for, and we can have a few folks hop on with us. Multiply that by a bunch, and add in community funded instances, and we'll be fine.
Gotta consider server costs were only a fraction of Reddit's costs. Salaries are quite pricey, and we have lots of folks volunteering time which will make it all work.
Don't forget to make a donation to your instance if you love it. For most of us it's a bit early but I give my 10 EUR per year to my Mastodon admin. Also, if you can choose instance ran by a non profit rather than a person as it ease the whole donation mechanism and give you the right to check where your membership due go.
As paradoxical as it is, I think that these open source non-profit projects are a lot more efficient than profit-driven, debt-fueled corporations.
First of all, the main contributors to a FOSS project do it for passion and do not take a salary.
Secondly, they don't have the infinite growth mindset that pushes enterpreneurs to to spend as much as possible for maximum growth, all financed by a growing amount of investors (and debt, which costs interest fees).
If a FOSS project reaches maximum capacity, they will close subscriptions, they will throttle traffic, i.e. they will slow down growth, but they will not go into debt. Slowing down growth is something that a for-profit company would never do (at least until the interest rates were low and the investors were plenty, today idk). Eventually someone else in the community will decide to do a generous donation or open their own instance.
You bring up a very good point. Currently lemmy.ml has thousands of users. Lemmy.world has thousands of users. The hardware they have selected to run their instances is adequate for now, but, what is the plan for scaling out if the user base grows? Is there one? They have a donation page on each lemmy instance (click or tap the heart icon,) but that canβt be enough to pay for the cost of running something used by millions of people, even if only 100s of thousands are ever only online at any given time.
In terms of UI/UX, @[email protected] has mentioned in a post they are currently working on major performance improvements and enhancements.
It is an old programming trope that premature optimization is a waste of time. As Lemmy scales, several bottlenecks will be hit. Some might be predictable, but many will only become evident after crossing a specific threshold. There are a lot of guides for scaling Mastodon servers after hitting certain bottlenecks, but this is all uncharted territory for Lemmy and we're going to find out the fun way.
Real devs do it in prod!
Iβm seriously tempted to write some performance tests in jmeter, locust, or k6, and fire up some live traffic simulations / simulated load against my lemmy instance to see what happens. But at the same time that would feel too much like work and I donβt want to work over the weekend.
Ideally, I think no one instance should have a million users to begin with.
User caps might be a good way to decentralize and ensure that we don't end up with just a few mega-instances. If there were a page showing available instances with percent of max users then people could use that when selecting.
Ideally, yes. If that can be the reality, and I suppose that is how it should would with federation, then server costs should never get out of hand.
For that to happen, I believe that interacting with people from other instances and moving your community and account from one instance to another have to become possible / easier.
At present, people flock to the instances with most users as those often have more local content (local content is generally easier to find than federated content) and they often have a smaller risk of shutting down. If I create a community on a smaller instance, the chance of it being found and interacted with are also much smaller than if it had been created on a bigger instance (because of, as I said, local content being user to find).
Sure, I can create an account on myfirstlemmyinstance.com (example URL, not an actual instance) with 10 users, but if my instance decides to shut down, my community of, say, 500 users will now have to move somewhere else and all old content will be deleted.
A βtransfer my communityβ feature that allowed an entire community to be moved between instances would certainly help. Thatβs a great idea.
From what Iβve seen so far looking through the Postgres db, every instance has data from most other instances. I see users in my local Postgres db from other instances. So, theoretically moving a community from one instance to another could be as simple as changing a few values in the database. Of course in practice itβs never that simple. π
I signed up for the lemmy.ml Patreon and am happy to support an open, federated site like this. I'd never pay for Reddit Gold, Twitter Blue, Discord Nitro, or any of those other nasty pay-to-win commericalized things but I'll pay to keep an open platform from implementing stupid "premium" bullshit.
this works on the same principals as fidonet, UseNet, email, etc. These protocols are more like fundamental services. The idea behind these was that instead of running a bunch if proprietary garbage you would run things that support A LOT of standard protocols. Why? Because NO ONE should be allowed to own our communications but ourselves.
The corps did not build these networks, we did. Software will improve over time, OSS shows the way.
The cost will be spread out, and people can monetize how they see fit. I'm wondering if there will be additional benefits you can add to your instance for a charge that people might be willing to pay.
I'm considering offering an Element server and maybe email on mine with a shared username for each service. That's going to take time to setup, though.
We must prove that it's valued and let the monetization come later. I'm working on this in my spare time. Once I can grow, maybe I can put more effort into it. I think it'll be a lot of people like me for a while financing it out of pocket.
The distributed nature of Lemmy should make things more manageable. Personally, I'm running an instance on a dedicated machine I already pay for, so it's not costing me anything unless storage skyrockets. Many other instance hosts are also hobbyists that don't mind covering the costs, and may take some form of donations locally on their sidebars.
There probably should be a built-in feature for instance admins to enable a local donation button to contribute to their costs, though. While Lemmy is fairly resource-efficient, larger instances are eventually going to require pretty beefy VMs to keep up with the traffic, image uploads, etc. I could see some instances randomly vanishing when their owners can't/don't keep up with their bills (which would force users over to other instances), but ideally if any instance owners can't afford to cover it, they hand control over to another community member to pick it up.
In general, Fedi admins simply close registrations when they can't keep up with an influx of new users, and point people to other, smaller instances
I just signed up for the Lemmy.ml patreon. I wish they had a $5 per month option, but I can just not skip ordering doordash one extra time and help pay for this instance. I use the hell out of it so it's the least I can do.
Each instance funds itself. Some might try ads, but as of now, most are just funded by donations.
Donating $20 a month over here, hope it helps
Lemmy is never going to handle an entire migration of reddit's whole user base. Most redditors use the official app, and the mainstream audience for the platform now represents the largest user group. They're not going to wholesale make the jump to Lemmy. That wouldn't even be possible without widespread coordination of resources. Each instance can only handle so many users, so new instances will have to continually be created to accommodate influx. Theres no profit incentive either, meaning whoever is running the instance server is purely doing it out of their passion for the platform. That doesn't scale linearly, there's only so many people out there with the resources to run a large instance.
Measuring the cost isn't possible. It depends on electricity and equipment costs which vary a lot. And the question doesn't make sense either.
Basically, volunteer code commits, volunteer admins, and donations for hosting costs.
Fosstodon is a pretty great example. It's a fairly large mastodon instance which makes enough in donation revenue to pass some on to other open source projects. It's not heaps ($600 in 2021), but I think it demonstrates that donations are a viable funding model. If things got tight I expect the community would meet the challenge.
It's not like you need to build a custom data centre - it's just renting a server, maybe even a VPS.
That said, of course the admins and mods are volunteers. I'd like to imagine that one day a few lemmy instances could charge a subscription fee for a premium, well managed experience.
@psylancer If you got 100 million users so that it's costing 400 million dollars a year, then ideally you need one million servers with 100 users on each. They need to all pass around a hat between their 100 users to raise the 50 dollars in server costs a month.
Reddit has over 2,000 employees most of whom are doing bullshit nobody using the site actually needs or wants, it's possible to run a lot leaner than that. Like Reddit itself used to, before they started burning hundreds of millions trying to compete with every other social media site at once instead of being Reddit
bullshit nobody using the site actually needs or wants
Like being CEO.
Unlike Reddit there will be many lemmy servers that exist, and I assume most will be supported by donations. A lot of servers have an Open Collective or Patreon option if you want to support them.
donations to my favorite instances, like wikipedia i hope :)
I'm running a barebones server for myself and a few communities (not many subs yet) which will run for less than a Starbucks coffee a month... (Assuming I don't need more storage space... Lemmy seems pretty light. The main servers are gonna carry the load unfortunately... Beehaw.org had a transparency post about financials as of about a week ago they said something that their instance was costing like 50-75ish a month of I recall.
I think the idea is that many people can run lemmy servers so the load is split between everyone hosting them.
I think Lemmy, like Mastodon, will crumble if people don't wrap their heads around federation. Mastodon stuggled because everyone just joined mastodon.social, not understanding that the server you join only affects your local timeline.
We need to teach people that you can join a small instance and still get 99% of the stuff you want from every other instance