Rewards is a mechanism for telling users that they have done well – that their actions have been judged favorably.
There are two fundamental types of reward schedules which fundamentally change how rewards are experienced: fixed- and variable reward schedules.
Fixed rewards
Fixed rewards are given at a set time, amount, and type and are opposed to variable rewards, which feel more like random rewards.
In computer games, fixed rewards are given when you complete a level or achieve some other kind of clearly defined goal. Variable rewards are usually given when killing monsters.
In web applications fixed rewards are the most commonly used type of reward as they provide clear goals for users to strive for. At Hacker News, features such as voting on comments, or changing template colors are unlocked as you collect Karma points for your activities. At Stackoverflow.com, you receive a badge as you engage more and more in the community. Both provide clear set goals that users can strive for in order to climb up the ladder of status in the community.
The right reward at the right time and amount
Fixed rewards on the web
Hacker News – the priviledges of voting on comments and changing template colors are unlocked as you collect Karma points for your actions.
Stackoverflow.com – receive badges and priviledges the more you engage in the community
Everyone likes to be told they are doing a good job, but it is essential for rewards to work that they are given at the right time, in the right amount, and that it is the right rewards that is being given. Ask these questions for each opportune moment to determine what is right1:
Getting a reward you don't understand is like getting no reward at all.
- What rewards is the system giving at the moment? Can it give out others as well?
- Are users excited when they get rewards or bored by them? Why is this?
- Do users understand the rewards they are given? Getting a reward you don’t understand is like getting no reward at all.
- Are rewards given on a too regular schedule? Can they be given in a more variable way?
- How are rewards related to one another? Is there a way that they can be connected?
- How are the rewards of the system building? Too fast? Too slow? Or just right?
There is only one way to find out the right balance of time, amount, and kind of reward: through trial and error.
There is only one way to find out the right balance of time, amount, and kind of reward: through trial and error. Balancing rewards is often a question of “good enough”1.
In behaviorism, the rate or probability of a behavior (“response”) is tried increased through stimulus (e.g. candy). The quality, or response strength, is assessed by measuring frequency, duration, latency, and accuracy2.
Positive and negative rewards (and punishments)
There are two ways to strengthen behavior through rewards: bring pleasure or excuse from pain – positive or negative rewards (also called reinforcements). Opposite of rewards are punishments3.
Positive rewards and punishments introduce stimuli to modify behavior where negative rewards and punishments take away stimuli to modify behavior. Rewards increase possibility of behavior while punishments decrease possibility of behavior.
Primary and secondary rewards
A secondary reward are the stimuli we have come to associate with the primary reward. When we see the message number notification appear on facebook, we associate that number with the feeling of receiving a message from our friends, which is the primary reward.
Variable rewards
Use variable rewards rather than fixed rewards when there is a chance that users will get acclimated to rewards the more they receive them1.
The activity level of users is a function of how soon they expect a reward to be given. The more certain they are that something good or interesting will happen soon, the more activity they will produce. When users know a reward is a long way off, the motivation is low and so is user activity4. This reward schedule is called a fixed one, as users are rewarded again and again with a fixed ratio or interval. Variable ratios and intervals on the other hand randomize rewards around an average. The latter produce the highest activity in users.
Read on to find out what variable reward ratios are and how they are different to fixed ratios and intervals.
Why variable rewards work
As humans, and animals, we react differently to certain patterns of rewards. Behaviorism has studies these patterns4 and have come to the conclusion that variable reward schedules and contingencies motivate us more than fixed schedules and contingencies.
Reward schedules
Fixed schedule: Users are rewarded again and again with a fixed ratio or interval.
Variable schedule: Variable ratios and intervals randomize rewards around an average.
Variable reward schedules produce higher activity in users.
Contingencies
Contingencies are sets of rules defining when rewards are given.
Ratio schedules: Provide a reward after a certain amount of actions have been carried out
Interval schedules: Provide a reward after a certain amount of time has passed.
Contingencies are rules or sets of rules defining when rewards are given. There are two fundamental sort of contingencies: ratios and intervals. Ratios schedules provide a reward after a certain amount of actions have been carried out – the more you do, the more you get. Interval schedules provide a reward after a certain amount of time has passed.
Fixed vs variable ratios
Rewards with a fixed ratio are given again and again after completing the same action the same amount of times. It could be that you will receive 10 karma points every 5th time you reply to a comment or that you would increase your level every 10th time you uploaded a video.
The problem with fixed ratios is that users distinctly pause completing actions when they receive a reward, as they know receiving a new reward will take a while. This creates an opportune moment for the user to walk away. However, the break in rewards caused by fixed reward ratios might also give the user an opportunity to explore different aspects the system.
Variable ratios are rewarded after a specific number of actions have been carried out, but that number changes every time. A user might know to upload approximately 10 videos to rise in levels, but the precise number is randomly generated every time – everything has a chance of reward. Such variable ratios have proven to stimulate more activity than fixed ratios – even when on average the same amount of rewards are given.
Variable ratios are free from the pause in activity generated from fixed ratios. It’s important to note that users do not know how many actions are required this time, just the average number from previous experience4.
Rewarding with fixed ratios produces a pause in activity after a reward has been given and a burst of activity just before being rewarded. While users typically respond at a higher rate in the fixed ratios bursts, variable reward ratios provide a more consistent rate free from pauses of the fixed ratios.
Fixed vs variable intervals
Instead of providing a reward after a certain number of actions has been completed, interval schedules provide rewards after a certain amount of time has passed. Users being rewarded in fixed intervals will pause activity once an award has been given and wander around for a while. They will return frequently to check if their reward has been “refilled” or has reappeared. Gradually, checks will become more frequent as the proper time nears.
Vimeo.com utilizes fixed reward intervals for its regular users, who are allowed to upload only 500 mb of video every week. After the fixed interval of one week, the user’s upload quota is refilled. As users reach their upload quota on vimeo.com, their activity will pause until its refilled next week. Vimeo hopes users will use the pause to consider buying a pro account with no upload quota.
With variable reward intervals, the period of time changes after each reward as with the variable ratios. As with variable ratios, variable intervals also produce a steady and continuous flow of activity – there is always a reason to be active.
Example: Lomography.com
Lomography.com sells retro analogue cameras on their website and in physical stores. To spark the enthusiasm of their lomographic society (fans taking pictures with lomography cameras), Lomography has created an online community for sharing pictures.
Community activity is backed by a “piggie bank” system where users can earn piggie points by having their photos selected as “photo of the day”, by submitting reviews of cameras and accessories, by winning rumbles and competitions, by translating content, and much more. Piggie points, which have an expiration date, can be used in the online shop to by cameras and thus translate into cold cash. The piggie bank system utilizes a mix of fixed and variables ratios and intervals.
At the retro camera company, Lomography, you can earn “piggie points” for your online activity, which translates to cold cash in the lomography online store. Piggy point rewards are given both at fixed ratios and at variable intervals.
Vimeo.com utilizes fixed reward intervals for its regular users, who are allowed to upload only 500 mb of video every week.
What else?
Variable and fixed ratios and intervals can successfully be combined. Say that you need to upload 20 videos to become an elite member, where after you will have the chance of picking up a free pro-membership appearing at variable intervals.
Extinction
Extinction is what happens when you stop providing a reward and will feel as a punishment that cause anger and frustration in the user. Behavior learned on the variable schedules are much harder to extinguish, as users will never know if what they are experiencing is just an unusually long run of misses and that next reward will justify their patience.
Avoidance
The principle of avoidance uses the negative reward of decay, pain, or other forms of punishment to keep users active. An example could be to force users to revisit your site and log in once a week to avoid their “elite” status to be taken away.
Sources
1 Jesse Schell, The Art of Game Design: A Book of Lenses
2 Reinforcement at Wikipedia.org
3 Psychology is fun at gamasutra.com
4 Behavioral Game Design at Gamasutra.com
5 Only a game by Chris Bateman
Notes
What we call rewards in this article is also called reinforcement. The opposite of a reinforcement is punishment.
2 comments
katy english on Sep 29, 2011
I’m studying teaching, and what you’re describing is known as Pavlov’s conductism.
There’s a famous American whose name I can’t recall in this moment, who employed the same method very effectively.
Anders Toxboe on Sep 19, 2023
You’re correct in noting that the principles described in the article are related to behaviorist theories, which have been influential in fields ranging from psychology to education. Ivan Pavlov is indeed a well-known figure in this area, famous for his work on classical conditioning with dogs.
The American researcher you might be thinking of is probably B.F. Skinner, who is known for his work on operant conditioning. Skinner’s experiments expanded upon Pavlov’s findings and delved deeper into the concept of rewards and punishments as mechanisms for behavior modification. Operant conditioning has been particularly influential in the field of education, where it has informed teaching methods and classroom management strategies.
Thank you for connecting the dots between the article and educational theory. Understanding the psychological underpinnings of reward mechanisms can indeed offer valuable insights for educators like yourself.
Comments have been closed