Showerthoughts

29577 readers

1929 users here now

A "Showerthought" is a simple term used to describe the thoughts that pop into your head while you're doing everyday things like taking a shower, driving, or just daydreaming. The best ones are thoughts that many people can relate to and they find something funny or interesting in regular stuff.

Rules

All posts must be showerthoughts
The entire showerthought must be in the title
Posts must be original/unique
Be good to others - no bigotry - including racism, sexism, ableism, homophobia, transphobia, or xenophobia
Adhere to Lemmy's Code of Conduct

founded 1 year ago

MODERATORS

[email protected]

-8

Sleep is when your brain's LLM retrains on the day's conversation (lemm.ee)

submitted 10 months ago by [email protected] to c/[email protected]

1 comments fedilink hide all child comments

I’ve noticed ChatGPT gets less able to do precise reasoning or respond to instructions, the longer the conversation gets.

It felt exactly like working with a student who was getting tired and needed to rest.

Then I had above shower thought. Pretty cool right?

Every few months a new ChatGPT v4 is deployed. It’s got new training data, up through X date. They train up a new model on the new content in the world, including ChatGPT conversations from users who’ve opted into that (or didn’t opt out, can’t remember how it’s presented).

It’s like GPT is “sleeping”, to consolidate “the day’s” knowledge into long term memory. All the data in the current conversation is its short term memory. After handling a certain amount of complexity in one conversation, the coherence of responses breaks down, becomes more habitual and less responsive to nuance. It gets tired and can’t go much further.

top 1 comments

sorted by: hot top controversial new old

[–] [email protected] 2 points 8 months ago

I wonder if it has more to do with the fact that the more inputs are available, the more complicated and prone to error something is. This is true of all things, and computer/programming stuff is no different. The more complex it is or the larger the dataset, the greater potential for noise vs signal.