Marcos Pereira

Posts  •  About me  •  Notes  •  Concepts

Skinner boxes, reinforcement schedules, and fun in games

May 2, 2024

Gabe Newell’s conception of fun

In this clip from Half-Life’s 25th Anniversary Documentary Gabe Newell shares a working definition of fun (working as in work in progress) at the time of Half Life’s development:

the degrees to which the game recognizes and responded to the player’s choices and actions

Gabe also highlights that

at that point in time that was a useful way of making design decisions

so I do wonder how this definition has evolved for them over time.

Realism is not fun?

In the clip Gabe also mentions that realism isn’t fun and gives the example of managing a grocery list.

Not all aspects of reality are inherently fun, but I wouldn’t go so far as to say any realism is necessarily not fun.

In fact, I think one anchor statement here is the fact that what makes a game fun is the same as what makes anything else fun in life.

Games are in essence a fake reality that our lower brain systems interpret as the real one.

This leads to an interesting concept - experiences and fantasies that one would enjoy experiencing in real life also make for a good game idea.

Skinner boxes

In the video Gabe also mentions reinforcement schedules.

The idea of Skinner boxes and reinforcement schedules is all around us, especially in social media apps with infinite feeds. They stalk us from all sides like predators.

Quick aside - the solution is mainly to increase the friction for unintended behaviors, and creating competing behaviors + minimizing friction to doing those.

For example, I have been having Huel for most meals (nutritionally complete shakes & savory crunchy food prepared with just water) which incredibly reduces the friction to eating healthy, and makes fast food have less of an advantage on my dopaminergic system.

Anyway, back to business.

I think what we take for granted in classical and operant conditioning is the fact that there is a reward. This area of study is all about how rewards (and negative stimulus) interact with other neutral stimuli and actions taken by the subject.

The thing is that to determine what fun is, we need to determine what makes something a reward. This is not what the “field” of reinforcement studies.

What we can take away from this field of study is mainly that a variable ratio reinforcement schedule is the strongest at provoking action, and the most resistant to extinction. This is what gambling at a slot machine is like.

This all comes down to reward unpredictability - of its nature, timing, dimension, and so on - as well as requiring action from the subject (otherwise it would be a variable interval reward schedule).

Unpredictability

So I think unpredictability is another key idea.

The brain loves making mistakes - it is how it improves its prediction model of the world, so it naturally feels great. As long as it doesn’t result in a negative effect that is, in which case it will feel particularly painful.

This is related to a concept called “dopamine reward prediction error coding”.

Dopaminergic activity spikes as a result of unexpected reward, and as a result of expectation of imminent reward.

This means it can’t represent a feeling of pleasure nor a feeling of motivation.I’m not even sure it correlates to any subjective experience directly.

What is clear is that dopamine encodes a reward prediction signal. This may be interpreted by higher level systems which are in charge of updating the brain’s prediction model of the world and surfacing subjective feelings.

Hard problem of fun

In the end understanding the meaning of fun is equivalent to reverse engineering the brain’s dopaminergic system.

Each person has a slightly different one, and it is incredibly complex.

Imagination and experimentation still seem to be the best tools for uncovering fun.

Takeaways

So Gabe’s conception of fun is only part of the story, but indeed an important one - as what makes games different from other kinds of media is the interaction, and also what makes them the most immersive.

If I were to compile a list of what makes fun fun, it would be something like:

Now the onus is all on the real world to figure out what people fantasize about.

(note that real world fantasy does not mean it has to be possible in the real world - just that if it were, it would be fun)