Operant conditioning

A few reminders regarding learning processes

What is a learning process?

A learning process is a durable change in an individual’s behaviour pattern resulting from past experience, expressed by an animal when exposed to environmental stimuli*. The animal will reproduce the new behaviour pattern when faced with the same stimulus again. Generally speaking, animals will alter their behaviour in a way which is most favourable to them.

*stimulus = Any change in the environment which leads to a behavioural response on the animal’s part (sight of another horse/ a new object, arrival of the groom/ veterinarian, feed distribution, approaching the lunge whip, action of the rider’s aids, tarpaulin sheet flapping, smell of a care product…).

Operant conditioning : an associative learning processe

conditionnement opérant — A horse moving forward when the rider applies leg pressure and/or uses the stick is an example of operant conditioning© A. Laurioux

Also called Skinner conditioning or instrumental conditioning, operant conditioning is an « associative » learning process. This means quite simply that the horse learns to consciously associate a specific action on his part (for example changing to a faster gait) to the consequences of the action (in this case, the pressure from the rider’s leg stops).

The basic principles of operant conditioning

The aim of operant conditioning is to get the horse to voluntarily carry out the action expected of him.

Motivating the horse to voluntarily carry out the expected action

To get the horse to carry out the action voluntarily, he must be motivated to do so. This is where reinforcement comes in. But what exacty does that mean ?

Reinforcement, or how to obtain the desired behaviour

When a reinforcement is used it increases the liklihood of a behavioural response being repeated. In other words, reinforcement serves to promote the expression of a desired behaviour. It allows the horseman to tell the horse that the behaviour he adopted was the expected. Behaviour. Two different approaches are possible :

Either the rider/tainer motivates the horse by giving him something which he naturally values and which is pleasant (food treat, scratching the withers…) once the desired action has been performed. The horse gradually learns that he will get the reward by performing the said action. This is known as a learning process using positive reinforcement(R+). A food treat remains by far the most effective.
Or the rider/trainer motivates the horse by placing him in an uncomfortable/unpleasant situation (pressure from the legs/hands, stimulating him with slight touches of the whip/ dressage stick…) until the desired action is performed. The stimulation then ceases as soon as the horse gives even the slightest beginning of a correct response, however small. In equestrian langage this is called « giving » (releasing the aids). It is the withdrawal of pressure which reinforces the response. This is known as alearning process using negative reinforcement (R-). Classical equestrianism is mainly based on the use of this type of reinforcement.

In both cases, the horse associates his behaviour to favourable consequences for him, due to this behaviour : provision of something pleasant (R+), or the ceasing of an uncomfortable or unpleasant stimulation (R-).

There is no judgemental noton behind the terms « positive reinforcement » and « negative reinforcement ». The idea is simply mathematical : « positive » merely means that you are adding something and « negative » that you are removing something.

The horse should understand 1) what is expected of him….

Motivating the horse is not enough, he must also understand the action expected of him. To do this, he should be placed in a favourable context, which will allow him to accidentally produce the desired action. For example, when teaching leg yield to a young horse, it is advisable to begin by doing the exercise on a small diagonal going towards the track. As it is easier to move towards the track than away from it.

…. and 2) when his action is expected

The next thing is to add an order or signal which enables the horse to understand when he should perform the said action. In the example above, the rider places his aids (right leg moved back and left leg at the girth to get the horse to move laterally to the left). The pressure from the right leg ceases immediately, as soon as the horse has moved one step sideways. The horse then understands that all he has to do to stop the pressure is to move sideways.

To begin with, the horse will need time to decompose and understand the association [order – action – R-] or [order – action – R+]. With experience, the codes will become established and the horse will learn to react to the lightest demand from his rider/trainer. The learning process is carried out gently, through shaping. This type of learning process is done by « trial and error ».

When does operant conditioning apply?

All the time, and in all disciplines ! Absolutely everything in horsemanship is based on the principles of operant conditioning. This is why it is paramount to have a good knowledge of the principles, and to know how to use them appropriately.

How should one use operant conditioning appropriately?

Here is the method step by step, divided up into 6 major steps.

Step 1 : identify the action the horse is expected to perform

The first thing to do is to define precisely what the horse is meant to perform. At the beginning, be careful to remain gradual in the complexity of your demands, especially with young horses in training : request simple actions such as going forward, turning, halting, changing gait… One should remember to be patient, and be satisfied with little. On gaining experience, the horse can be asked to perform more and more complex actions.

Step 2 : choose the order or signal

The next step is choosing an order/signal or a combination of orders/signals (touch, visual signals, sounds…) which are clearly identifiable by the horse. In the same way, the lower the horse’s experience, the simpler the signal should be. With time and experience, the signals will become more subtle, until they even become imperceptible to an outside observer. It is always surprising to observe how an experienced horse is capable of differentiating between complex aid combinations.

Step 3 : choosing the right reinforcement, and the dosage

Depending on the exercise requested, on the context, on the horse’s sensitivity and temperament, the choice of the nature and intensity of the reinforcement will differ. Therefore, being able to use both types of reinforcement (R+ or R-) is a great asset. It will allow you to juggle and to choose the type best adapted to the context, to the desired result and to the horse, and even to combine the two types. To be effective, the reinforcement used should be very motivating for the horse, and appropriately dosed. In other words, the horse should clearly want to get R+, or to avoid the discomfort of R-, and the intensity of the reinforcement should be measured (handful of pellets vs a bucket, pressure from the lower leg vs kicking with the heels).

Step 4 : think about how you can get the horse to accidentally produce the desired response

Horses have absolutely no clue as to what is expected of them. The rider/trainer therefore has to place the horse in a favorable context, to bring the horses to perform the desired movement accidentally. This is simply logical. For example, there is little chance of teaching a young inexperienced horse to break into canter by kicking him with your heels from a halt or from the walk. On the contrary, it is probably the best way for the horse to spook or resist due to incomprehension ! The learning process should be gradual, by first teaching the horse to mobilise his haunches and lighten his forehand on a circle for example. Once the horse is balanced and in front of the rider’s leg, breaking into canter will come naturally.

Step 5 : plan a sufficient number of sessions

A certain number of tries (number of associations [order – action – R-] or [order – action – R+]), sufficient for the horse to begin to understand, but reasonable enough to kep him motivated throughout the session, and from one session to the next, without boring him. Short sessions (of a few minutes) with several tries (10-20 tries) should be favored instead of over long sessions. It is of more benefit to know when to stop, even if the goal is not entirely reached, and to resume a few hours or days later. The horse will be more available. « Request often, be satisfied with little, reward a lot ». Be careful however to at least get the beginning of a response, even very slight, as a try which does not acheive anything does not teach the horse anything.

There is no general rule ; it all depends on the horse and his capacities, on what we want him to learn, and how we go about teaching him. Some horses will require a great number of short sessions. Others will understand quicker and will be able to cope with slightly longer sessions.

Step 6 : go for it and adjust

Start in a quiet and familiar environment for the horse so that his attention is focused on the exercise, and not on the outside environment. Your horse should be listening to you. If the request is not understood, then adjust it. To begin with, do not hesitate to reward a lot or to yield quickly, at the slightest sign of a correct response on the horse’s part.

Very often the horse will try out several actions, by trial and error, before understanding what is expected of him. The horse’s work is much more the consequence of mental training, rather than just mechanical training. This is why operant conditioning is a learning process which requires correct timing and precise dosage in the use of reinforcements. The latter should be perfectly in sync with the the appearance of the desired behavioural response, otherwise they may be misunderstood and therefore ineffective if they are delayed with regard to the horse’s response. A good teacher is one who makes himself understood quickly, with vey simple things.

What about punishment?

Punishment is another learning strategy used in operant conditioning. Unlike reinforcement, which serves to promote a desired behaviour, the aim of punishmentis to eradicate unwanted behaviour. There again there are two types :

Either the rider/trainer motivates the horse to stop unwanted behaviour by introducing an unpleasant/uncomfortable stimulus (tap with the crop, tapping the nose…) straight after the behaviour. This is known as a learning process with positive punishment (P+). In this case it can be the same stimulation as negative reinforcement, but the moment when it is used is different. Instead of using the stimulus before the desired response, the stimulus is used after the behaviour which we want to eradicate. In this way the horse learns that if he does not wish to receive the uncomfortable stimulus, he must not produce the behaviour. Although it can have a negative effect on the horse’s well-being and should be avoided, it is the punishment which is classically used in equestrianism.
Or the rider/trainer motivates the horse to cease an unwanted behaviour by taking away something of value to the horse. This is known as a learning process with negative punishment (P-). It could be for example, moving away with the feed bucket when the horse paws at his stable door. This type of punishment is rarely used in equestrianism.

The horse then associates his behaviour and the unpleasant consequences of this behaviour : something unpleasant happens (P+) or something of value is taken away (P-)

Punishwith parcimony

In the same way as with reinforcements, the intensity of the punishment should be moderate:

Too weak and it will have no value for the horse and will be useles.
Too strong, it will place the horse in an anxious frame of mind detrimental to his well-being and to the learning processes.

It is important to get the dosage right according to the horse. For some, just raising your voice suddenly will be enough, whereas for others, it may be necessary to go as far as touching or surprising him.

Always apply the same rules

Once you have punished an action, it should not be tolerated again, and inversly. A horse needs clear precise markers ; the horseman should remain coherent. A horse who has been allowed to nibble for a long period, will not associate his behaviour and the punishment if he is suddenly punished for this particular behaviour.

An unjustified punishment, will cause incomprehension on the horse’s part.
In the same way, punishing a frightened horse, will only increase his fear.

What if the learning process I am trying to establish does not work?

Is the order used clear and precise enough? Rule 1 « 1 order = 1 action"
Have you repeated the association enough times [order – response - reinforcement]?
Did you apply the correct dosage of reinforcement?
Did you respect a good time contiguity?

The time lapse between the response and the reinforcement should be as short as possible, so that the horse associated the two. This is called time contiguity.

• With what frequency have you been using reinforcements?

At the start of the learning process, the reinforcement should be used every time the horse gives a correct response. This is known as continuous reinforcement. Each correct response is associated to R- being ceased or to R+ being given. Once the horse has grasped the causal link between his response and the reinforcement, the reinforcement does not then need to be given everytime (2 times out of three, then every other time, then more seldom, e.g. when the horse performs the exercise exceptionally well). This is known as partial reinforcement.

• Have you respected the tidea of contingency?

Contingency is a complex idea, but it is determining. It is the likelihood of giving a reward after an action by the horse, as opposed to to the likelihood of rewarding at another time. It has been demonstrated that deteriorating contingency, i.e. free rewards, and not only to reward a specific action, will interfere with the learning process. The horse soon learns that he does not need to do the exercise requested, as he obtains the reward even when he does nothing.

To remember

The basic idea of operant conditioning learning process is to bring the horse to associate a voluntary action on his part to a consequence which is favorable for him : either receiving positive reinforcement (R+), or stopping a negative reinforcement (R-).

Having a good knowledge of the elements of this type of conditioningis a toolbox which the rider should master so as to properly educate and train his horse. The choice of tools is adapted to the exercise, the horse, the context… These tools can also be used to complement one another to teach something ! In any case, a good trainer will always try to reduce excitement or fear, and favour light codes.

The operant conditioning learning process is also sometimes referred to as « learning by trial and error ». Accept mistakes as they are part of the learning process ! Dressage of a horse is shaping him over a long period of time, over several years. It is little by little, through a combination of learning processes, that you will bring your horse to perform complex actions such as dressage figures.