260247.qxd

John Monterosso · George Ainslie
Beyond discounting:possible experimental models of impulse control Received: 17 March 1999 / Final version: 6 June 1999 Abstract Animal studies of impulsivity have typically
core of most if not all conceptions, though, is the notion used one of three models: a delay of reward procedure, a of irrationality. In the animal literature, this irrationality differential reinforcement for low rate responding (DRL) manifests as failure to maximize overall reward. In hu- procedure, or an autoshaping procedure. In each of these man research, it is the individual’s failure to follow her paradigms, we argue, measurement of impulsivity is im- own recognized best interest. The word “recognized” is plicitly or explicitly equated with the effect delay has on important; choices that a person consistently regards as the value of reward. The steepness by which delay di- desirable but which observers regard as maladaptive be- minishes value (the temporal discount function) is treat- long to a distinct category, for which we will not be us- ed as an index of impulsivity. In order to provide a better analog of human impulsivity, this model needs to be ex- On this view, the study of impulsivity is a particularly panded to include the converse of impulsivity – self-con- challenging frontier for psychopharmacology. Under- trol. Through mechanisms such as committing to long standing counterproductive behavior is straightforward range interests before the onset of temptation, or through when the culprit is misinformation. However, the thorny bundling individual choices into classes of choices that fact is that we repeatedly and wittingly subvert our own are made at once, human decision-making can often look recognized long-term best interests. This utterly central far less myopic than single trial experiments predict. For aspect of the human condition defies conventional utility people, impulsive behavior may be more often the result theory that attributes an ultimate rationality to all behav- of the breakdown of self-control mechanisms than of ior. As typically defined, the construct of “impulsivity” steep discount functions. Existing animal models of self- is an acknowledgment of at least a subset of systematic control are discussed, and future directions are suggested Given the diversity of behaviors most often talked about in the context of impulsivity (including violence, Key words Hyperbolic discounting · Impulsivity ·
gambling, credit card binges, sexual behavior, substance Self-control · Reward bundling · Psychopharmacology abuse, and self-injurious behaviors), the phenomenoncannot be understood as a product of a single appetite ora particular consumed substance. Instead, impulsive be- havior must be related to the pattern of rewards availablefor certain behaviors. Even at the level of physiology, “Impulsivity” has been an imprecise construct in the non-substance related addictive behaviors like gambling clinical literature. Dozens of scales, sub-scales, and be- have been shown to look remarkably like substance-di- havioral measures have been created to measure impulsi- rected addictive behaviors (Wray 1981; Griffiths 1993; vity, and the modest intercorrelations of many of these Comings et al. 1996; 1997). There is reason to believe measures suggests different underlying conceptions of that impulsiveness arises from the properties that re- the construct (Corulla 1987; Parker et al. 1993). At the 1 Dickman’s typology of impulsivity distinguishes between “dys- functional impulsivity” and “functional impulsivity” (which he de- Department of Psychiatry, Coatesville VA Medical Center, fines as acting with little forethought when doing so is beneficial).
While this explicitly allows for impulsivity to be rational, Dick- e-mail: ainslie.george@coatesville.va.gov man points out that 1) most work in the field focuses on dysfunc-tional impulsivity, and 2) that the two types of impulsivity are not highly correlated (Dickman 1990). As such, the inconsistency be- Department of Psychiatry, University of Pennsylvania, tween “dysfunctional impulsivity” and the conception of impulsiv- 3900 Chestnut Street, Philadelphia, PA 19104, USA ity we are following lies mainly at the level of labeling equal increments of delay produce progressively less ad-ditional discounting. Among other consequences, hyper-bolic discounting predicts the motivational inconsistencyover time that characterizes impulsive behavior. The rel-ative value of two alternatives available at fixed timescan switch, based solely on the addition or subtraction ofan equal delay to both alternatives. For example, withthe passage of time, the delay to fixed alternatives de-creases by an equal amount. Given the form of the dis-count function, an alternative that was inferior from adistance may become preferred when its availability be-comes immediate (Fig. 1b). Thus hyperbolic discountingleads to regular and systematic inconsistencies in prefer-ences.
Particularly in the animal literature on the psycho- pharmacology of impulsivity, the concept of hyperbolicdiscounting has been widely used to model impulsivechoice and track its pharmacological manipulation. In Fig. 1 a Exponential discount curves from two rewards of differ-
this article, we have three goals. First we will compare ent sizes available at different times. There is no delay at which paradigms in the animal psychopharmacological litera- preference switches. b Hyperbolic discount curves from two re-
ture on impulsive choice which do and do not use a tem- wards of different sizes available at different times. The smaller poral discounting framework. In doing so, we will argue reward is more valued just in the period when its availability isrelatively immediate that those that do not can and should be readily incorpo-rated into such a framework. Second, we attempt tobroaden the model by showing how, in addition to pro- wards behavior generally (Mazur et al. 1987; Shizgal ducing impulsivity, hyperbolic discounting can give rise to mechanisms of self-control. And third, with each It has long been observed that the basic machinery of mechanism we discuss animal models of self-control, motivation favors more immediate reward. An organism, and we suggest related directions for psychopharmaco- be it person or pigeon, will engage more readily in a be- havior which immediately brings a positive outcomethan one in which the same outcome occurs after a peri-od of delay. The function relating the length of delay to the diminution in the motivating force of the reward isthe temporal discount function. Some possible temporal In the animal laboratory, studies of the impact of psycho- discount functions are inconsistent with impulsivity. Ex- pharmacological interventions on impulsivity have vari- ponential temporal discount functions (discounting by a ously used either a delay of reward paradigm (Charrier fixed proportion per period of time) do not predict im- and Thiebot 1996; Evenden and Ryan 1996; Poulos et al.
pulsive choice, regardless of how steep they are. An ex- 1998; Tomie et al. 1998a), a DRL paradigm (Evenden ponential discounter’s temporal distance to a fixed set of 1998b; McMillen et al. 1998), or an autoshaping para- alternatives will not affect the order of preference among digm (Poling and Appel 1979; Picker et al. 1986; Tomie those alternatives. The situation is like that of a set of savings bonds with different maturity dates and differentdenominations. Though both maturity dates and denomi-nation will affect the current value of all the bonds, the relative value of the bonds will not change. The bondthat has the greatest value at time T is certain to continue The delay of reward procedure is explicitly grounded in to have the greatest value at time T+X. For the same rea- the temporal discounting conception of impulsivity. In son, if delay attenuates rewards by a fixed exponent, the this procedure, the organism chooses between two re- reward that is most valued at delay T will also be the wards – one larger but more delayed, the other smaller most valued at delay T+X (Fig. 1a). Temporal discount- but more immediate. Impulsivity in the procedure is ing at a fixed exponential rate, therefore, predicts stable equated with the animal’s tendency to choose the more immediate of the alternatives and in so doing, decrease However, empirical research using a variety of organ- the reward he receives. In an especially useful version of isms and a range of procedures has consistently found this, called the delay adjusting procedure (Mazur 1987), that temporal discount functions are not exponential, but the delay associated with the larger reward is changed rather hyperbolic (Ainslie 1975; Green et al. 1994; Har- within session to determine the exact amount of delay vey 1994; Kirby and Herrnstein 1995; Richards 1997).
necessary to make the larger alternative equally pre- Discounting is steeper at short delays, and additional ferred to the more immediate alternative which remains fixed. If the animal repeatedly chooses the larger reward, only go unrewarded, but reset the expired time to zero, the delay to the larger reward is increased; if the animal often after a salient brief time-out interval. Impulsivity repeatedly chooses the smaller more immediate reward, here is equated to the extent to which an organism be- the delay to the larger reward is decreased. When a crite- haves prematurely, undermining its own attainment of rion of stability is reached such that neither choice is reward. In recent times this procedure has also gone by a clearly preferred, the parameters of the fixed and adjust- more descriptive name – “interresponse time greater than ing alternatives reveal an indifference point that indi- t” or “IRT>t” for short. One important variant on the cates an equality between an additional delay and an ad- procedure called the fixed consecutive number procedure ditional quantity of reward. From an array of indiffer- (FCN) minimizes the confound between impulsive per- ence points, the temporal discount function of the organ- formance and general level of motor activity. In the FCN procedure, the animal has to make a required string of al- The resulting function represents a radical departure ternative operants (for instance pecking a green key) from the exponential curve of conventional utility theory.
rather than wait out an interval before making the rele- The following simple hyperbolic function has generally vant operant (for instance pecking a red key). As in the provided a reasonably good fit to the data over a range of DRL procedure, performing the operant associated with reward before satisfying the FCN criterion resets thecounter and is not rewarded (Mechner and Latranyi where V equals the value of the delayed reward, V equals the value of the reward if immediate, D equals the delay, and K equals a scaling constant which can be usedas an index of discounting, and by extension, as an index A third type of procedure sometimes used in the animal of impulsivity. With data from human choices, this for- literature on impulsivity is autoshaping. Autoshaped be- mula holds for time scales ranging from seconds to years haviors are those that organisms spontaneously engage (Solnick et al. 1980; Harvey 1994; Kirby and Herrnstein in without obvious reinforcement. For example, if a key- 1995). Importantly, while the form is consistently hyper- light signals the impending arrival of food, a pigeon will bolic, the steepness of the discounting (the parameter k) come to peck at the key-light, even though doing so does varies enormously in human studies of choice. Most no- not affect the arrival of food. The robustness of this tably, the parameter k can be smaller by orders of magni- seemingly unrewarded behavior is evident from a variant tude in human choice experiments with money as the re- on the procedure called negative automaintenance. In a ward (Ainslie and Haendel 1983; Green et al. 1994) as negative automaintenance procedure, the autoshaped be- compared to when a primary reward is used such as the havior is punished by canceling the arrival of reward elimination of an aversive sound (Solnick et al. 1980; (Williams and Williams 1969). While autoshaped behav- Navarick 1982). As we will argue below, the sophisticat- iors are diminished in the negative automaintenance pro- ed strategies of self-control employed by humans can cedure, they are usually not eliminated entirely dramatically alter their observed discount functions.
(Schwartz and Williams 1972). The extent of autoshaped Presumably because of practical limitations many responding has been taken as a measure of impulsivity pharmacological studies have not used Mazur’s delay ad- (Tomie et al. 1998a, 1998b). Given the common view of justing procedure which derives indifference points. In- autoshaped behaviors as classically conditioned (elicit- stead, choice alternatives are presented and the propor- ed) responses rather than as operant (emitted) behaviors, tion of times the smaller more immediate alternative is this may seem a strange procedure to use for modeling chosen (over varying delays) is taken as an index of dis- impulsivity. However, autoshaped behaviors have also counting (Evenden and Ryan 1996; Poulos et al. 1998; been argued to be completely operants (Dougan et al.
Tomie et al. 1998a). Although there are difficulties with 1983), or combinations of operant responses and Pavlov- the interpretation of these data, the underlying logic is ian responses(Schwartz et al. 1974).
unchanged. In these choice procedures, impulsivity isoperationalized as the extent to which an organism willtrade reward magnitude for reward immediacy.
On the surface, delay of reward procedures seem to re- flect one concept of impulsivity, and DRL and autoshap-ing procedures seem to reflect another concept of impu- The second class of procedure used to measure impulsi- lsivity. As Ho et al. succintly put it, “Research in the ex- vity in the animal psychopharmacology literature is perimental analysis of behaviour suggests that two im- called DRL, “differential reinforcement of low rate re- portant characteristics of “impulsiveness” are i) deficient sponding”. In this procedure an operant response is re- tolerance to delay of gratification [a steep temporal dis- warded only if it occurs after a fixed interval of time has count rate] and ii) inability to inhibit or delay voluntary expired since the last response. Premature responses not behavior [poor inhibitory control];” (Ho et al. 1998, p.
68). Delay of reward procedures seem well suited to tap However, we almost certainly make a mistake if we the temporal discount rate version of impulsivity, and assume that all or even most of what rewards an organ- DRL and autoshaping procedures (particularly the nega- ism’s choice is observable. A person singing while in a tive automaintenance variant) seem better suited to tap state of joy may have no possibility of any externally ob- the inhibitory control version of impulsivity. However, servable reward. And yet the operant (reward driven) na- on closer scrutiny, both DRL and autoshaping proce- ture of that behavior is evident from the fact that its val- dures lend themselves to a tolerance to delay interpreta- ue is in competition with the value of observable re- tion in which poor performance amounts to the choice of wards. The offer of money to refrain from singing, for a smaller but more immediate alternative. As such, the instance, might be persuasive. Of course, if the sum of first of the two above characteristics alone (temporal dis- money was sufficiently modest, or the joy sufficiently in- count rate) is sufficient to characterize impulsivity as it tense, the observable reward could lose to the unobserv- has been measured in the animal psychopharmacology able reward of singing. In the same way, pecking may be intrinsically rewarding for a pigeon that is anticipating In order to view poor performance on a DRL schedule food. The extension of this argument is an argument as reflective of a steep underlying discount function, it is against two-factor theory generally. For the complete necessary to view premature responses as the choice of a form, see Ainslie (1992 , pp. 102–122), Ainslie (1999) smaller more immediate reward over a later larger one.
and John Donahoe’s work suggesting a “Unified Rein- This requires only the reasonable assumption that imper- forcement Principle” (Donahoe et al. 1993; Donahoe and fect time sense makes the organism’s estimate of reward Palmer 1994). This operant interpretation of autoshaping vary as a function of delay. At the eighth second of a is consistent with the fact that the frequency and duration DRL10 trial, the organism may have what amounts to, of the autoshaped response in negative maintenance pro- say, a 50% expectancy of reward given an operant at that cedures is greatly diminished, sometimes to the point of moment. Making the simplifying assumption of norma- extinction (Woodruff and Williams 1976). Automainte- tive expected value, the reward availability for engaging nance behaviors can be understood as the choice of an in the operant after 8 s would in effect be 0.5 times the immediate but possibly relatively small reward (the val- value of the reward when its availability is certain. The ue of pecking when expecting food), over a more distant organism in effect has a choice after 8 s between an im- but larger reward (actual access to food). As such, mediate 0.5*V, and a continuous range of later larger al- changes in the frequency of these behaviors may reflect ternatives, such as, perhaps, 0.8*V after 2 more seconds, changes in the temporal discount function of the organ- and 1.0*V a few seconds after that. A premature re- sponse amounts to choosing a smaller-sooner expected It is less clear to us whether autoshaping outside the value over a later-larger expected value. As such DRL negative automaintenance procedure can be reasonably procedures do not necessitate the positing of any alterna- taken as impulsivity as operationalized here. Since there tive or addition to the temporal discounting conception is no apparent cost to the behavior, it is not clear that the of impulsivity. This explanation does not account well animal engaging in autoshaped behaviors is trading off for the burst responses often seen at the onset of DRL potential reward for immediacy. If “irrationality” is at trials. However, at such time, when sunk cost is low, re- the core of the concept being modeled, as we think it sponses may be more like autoshaped responses (see be- should be, then it is probably preferable to use proce- low). On this view, prevalence of onset burst responding dures in which the presumed impulsive behavior has a in particular ought to be inversely related to the inter-tri- more obvious cost. It is nevertheless interesting that indi- al interval (ITI), since a long ITI will add to the animal’s vidual differences in rats’ sensitivity to delay on a delay of reward procedure were shown to be correlated with The case of autoshaping requires further explanation, individual differences in the quantity of autoshaped be- but we believe it also can be incorporated into the dis- haviors performed (Tomie et al. 1998a).
counting framework, at least in the negative automainte- Temporal discount rate seems to be the key underly- nance variant. Again, the temporal discount rate concep- ing property in procedures used to model impulsivity.
tion of impulsivity requires viewing impulsivity as the We believe that the delay of reward procedure is the best choice of a smaller more immediate reward over a later alternative as an animal model, since it alone allows the larger alternative. Given a narrow definition of reward, discount rate parameter k to be directly estimated. The autoshaped behavior has no measurable reward (and in parameter k can serve as an index of impulsivity, thereby the case of negative automaintenance, it has a measurable allowing impulsivity to be quantified in absolute terms reward cost). As such, automaintenance behaviors have rather than having to be satisfied with relative compari- typically been treated as principally Pavlovian (elicited by the pairing with an unconditioned stimulus) ratherthan instrumental (emitted based on contingency of rein-forcement). On a Pavlovian account, autoshaped behav- iors and negative automaintenance behaviors could there-fore not be viewed as the choice of a smaller more imme- Most of the interest in pharmacological manipulations of diate reward since they are not choice behaviors at all.
rats’ impulsive behavior is based on the hope that the an- imal can serve as a useful model of human impulsivity.
The act of employing a mechanism to precommit However, humans are capable of feats of self-control that against a future temptation may seem a sophisticated and look qualitatively different from the spontaneous behav- exclusively human behavior. However, even pigeons ior of rats. For instance, people often make financial de- sometimes precommit against a predictable preference cisions that are consistent with very low, exponential dis- reversal. One can easily establish parameters in a choice count rates (Logue et al. 1986, 1993).
experiment in which pigeons will regularly peck a key Since people and less cognitively sophisticated ani- that brings a smaller-sooner reward rather than wait for a mals do not differ in the hyperbolic form of their dis- later-larger reward. However, if given the option, some count curves, but differ greatly in their capacity for self- pigeons will learn to peck a key ahead of time which re- control, failures of this capacity are apt to come from a sults in the removal of the smaller-sooner option from phenomenon not being measured in present psychophar- the choice set that will follow (Rachlin and Green 1972; macology experiments. If we hope to manage impulsi- Ainslie 1974). Thus a pigeon will make a binding pre- vity pharmacologically in people, then it makes sense to commitment to secure its current preference. It may be consider not just how hyperbolic discounting can pro- worth exploring whether some drugs increase or de- duce impulsive choice, but also how normal adults man- crease the animal’s learning of this response, quite apart age to behave in farsighted ways. A person who is more from whether they change the impulsiveness that moti- impulsive than other people may not have a steeper un- derlying discount function, but instead may be impairedin whatever makes human choice farsighted. While sur-veys have often shown a general correlation between self-reported discount rates and impulsive behaviors, themost striking clinical observation is that individuals are Another method by which current preferences can guard markedly impulsive for a limited range of activities and against future changes of preference is by the control of for limited periods in their lives. Pharmacological inter- attention. Someone struggling to maintain fidelity to a ventions may affect mechanisms of self-control in addi- spouse may not allow themselves to notice the flirtations tion to underlying discount functions.
of an attractive third party. Attending to such informa- A number of mechanisms of self-control are predicted tion may foreseeably lead to the likelihood of creating by hyperbolic discounting. Since a person’s preference preferences in opposition to current preferences. Atten- among a fixed set of alternatives can vary predictably as tional control can occur as either deliberate avoidance of a function of the passage of time, it follows that one of information or an avoidance that is itself not acknowl- the obstacles they may face in trying to attain her current edged. The latter case is the repression that Freud at one preferences is the expected preferences of their future time held to be the cornerstone of all defensive processes selves. The dieter who has just finished bingeing has (Freud 1956). The repressive individual avoids unwanted both a current clear preference for moderate consump- thoughts, feelings or behaviors by not attending to the tion in the future, and an equally clear expectation that psychically loaded information. Aside from the distor- their own future self may pose a threat to this current tions that Freud noted, its disadvantage is a loss of infor- preference. In order to secure what they currently want, mation that may be needed for other decisions.
it may be necessary for the person to find a way to pre- A simplified animal model of impulse-control commit their future choices. They may, for example, pre- through manipulation of attention may be suggested by a commit their future self to abstaining from savory snacks recent experiment by Siegel and Rachlin (1995). In the by ridding the kitchen of all such foods. Several distin- experiment, pigeons were first exposed to an FR1 sched- guishable methods of precommitment have been identi- ule with two reward alternatives in which the smaller more immediate reward was reliably chosen. Next, thereinforcement schedule was altered to an FR31 schedulewhich first required the pigeon to peck either of the two keys 30 times, after which he could then peck either keyand attain the associated reward. In this schedule, the The most direct method of precommitment is to arrange key chosen on the first 30 responses had no effect on re- for some external control or influence – taking Antabuse, inforcement possibilities; the 31st peck alone determined checking in to a “fat-farm”, setting up a binding contract, which reward the pigeon received. In accordance with or cultivating friends with particular expectations all can hyperbolic discounting, the larger more delayed reward serve this role. The alcoholic who currently prefers not was more valued from a distance. As such, it is not sur- to get drunk later, but who doubts they will still feel that prising that pigeons more often pecked the key associat- way when “later” arrives may stack the deck in favor of ed with the larger reward during the first 30 responses.
their current preference by taking Antabuse and adding Importantly, more often than not the pigeons went on to the cost of nausea to a future self’s decision to drink.
peck the key associated with the larger more delayed re- The problem with extrapsychic devices is that they also ward even on the 31st trial which presented the exact reduce desirable flexibility, and may create a wasteful same options as in the FR-1 condition. Though some- motive to try to evade or undo them.
what diminished, this shift away from impulsivity was still present when a one-second timeout signaled the on-set of the 31st trial. This phenomenon, which the authorslabeled “soft commitment”(Siegel and Rachlin 1995),may work by fixing the pigeon’s attention on the alterna-tive that is preferred at the start of a trial. On this ac-count, enhanced performance is the result of a failure toshift attention to the alternative option. The interactionbetween “attentional set-shifting” (Owen et al. 1991) andimpulsivity is of particular interest given the growing Fig. 2 Summed hyperbolic discount functions for six pairs of re-
wards. As the number of future rewards being summed increases,
body of research reporting pharmacological manipula- the region of impulsive preference reversal is diminished tions that diminish attentional set-shifting performance(Coull et al. 1995) and manipulations that enhance atten-tional set-shifting performance (Roberts et al., 1994).
made far in advance. If a current choice is “bundled” Pharmacotherapies that diminish attentional set-shifting with series of like future choices, and made at once, may increase the effectiveness of attentional manipula- preference can be made less impulsive. Consider a tion as a mechanism of self-control since attention is smoker interested in quitting. Suppose that, for whatever reason, the smoker believed that his choice of whether tolight up after breakfast would be a binding determinantof all future smoking choices; if he lit up he would con- tinue smoking at the current rate for the rest of his life,but if he did not he would never smoke again. Presum- Emotions such as fear, jealousy, and arousal can, to a ably the aspiring former smoker in this situation would point be vicious circles. After the emotion has gotten un- be more likely to refrain from smoking that one ciga- derway, there is a lower threshold for further emotional activity of the same kind, until some satiation point is Such a decrease in impulsiveness when future choices reached (Skinner 1953, pp. 235–236, 239–240). If a per- are grouped with current choices is predicted by hyper- son expects an emotion to make currently unpreferred re- bolic discounting. A single immediate reward that is ward dominant, he may commit himself not to choose more compelling than a larger but delayed alternative the reward through early inhibition of that emotion.
can be made relatively less compelling if both alterna- There have been some experimental demonstrations of tives are summed over a series of identical future choic- emotional control. For instance, Walter Mischel and col- es. The cigarette right now may be more valued than the leagues found that while children below around 6 years expected beneficial consequences of abstaining from that were poor at self-control, many older children were often cigarette, but each delayed cigarette may be less valued able to resist the temptation of an immediately available than the expected consequences of abstaining. If some- marshmallow in favor of a more preferred reward. Those one could bundle a series of these future cigarettes with that succeeded in avoiding the impulsive preference re- the current cigarette she could tip the balance away from versal often used emotional control in the form of think- ing about the immediately available marshmallow in a Hyperbolic discounting further predicts just such a “cool” way, or by imagining it to be undesirable (Mis- possibility: A person’s successive motivational states chel and Moore 1980; Mischel and Mischel 1983). Emo- will be in a relationship of limited warfare with each oth- tion-forestalling devices tend to distort rather than nor- er (Schelling 1960), sharing some goals, but at times in malize motivation, and may make people emotionally conflict over ones imminent when she makes each unresponsive, as in alexithymia (Nemiah 1977). At the choice. This kind of conflict is stabilized by the percep- moment, there seems to be no way to analyze them using tion that it is a variety of repeated prisoner’s dilemma where the impulsive choice represents defection. The on-ly way the present self can expect most future ones tocooperate is to make each choice a test case, so that cur- rent cooperation motivates future agents to preserve thechain of cooperation, and current defection destroys this The above mechanisms of self-control do not seem to future incentive. By seeing current choices as precedents match what people mean by “will-power”; they seem too for similar choices in the future, a person does arrange to mechanical. As William James (1890, p. 534) said, will- choose between a whole series at once (Ainslie 1975, power seems to occur “with both alternatives steadily held in view.” Hyperbolic discounting provides a mecha- Although choice bundling has the advantage of not nism for just such a method of self-control. Because the directly limiting freedom or information, it is far from a diminution of value is great with the addition of initial flawless solution to the problem of impulsivity. For one delay, but small with the addition of the same delay to an thing, it is precarious because it creates a motive for ra- already far delayed alternative, it follows that the choice tionalization. The person faced with the possibility of between alternatives will tend not to be impulsive if immediate reward has a constantly present motivation to view the choice as singular. The prospect of (the imme- with rigidities like “compulsive saving”, possibly be- diate impulsive choice+an expectation of abstaining in cause the latter has lacked a distinct recognized cause.
the future), if credible, is better than either universal im- However, the two are rather more like opposites. As pulsive choice or universal abstinence. One consequence Howard Rachlin put it, “The workaholic is less like the of this is the creation of a market for finding a reason alcoholic than he is like the teetotaler. The workaholic’s that makes the current choice exceptional. As William so-called addiction to work is really the compulsive avoidance of a narrow choice between work and leisure,a choice that by definition will always be resolved (if it How many excuses does the drunkard find when each is framed narrowly enough) in favor of leisure” (Rachlin new temptation comes! It is a new brand of liquor which the interests of intellectual culture in such matters oblige The need for choice bundling suggests a second route him to test; moreover it is poured out and it is a sin to to impulsivity, in addition to the steepness of the dis- waste it; or others are drinking and it would be churlish- count function. Impulsive behavior can occur because of ness to refuse. or it is Christmas day; or it is a means of the absence or breakdown of this rather brittle strategy.
stimulating him to make a more powerful resolution in The underlying discount function might also interact favor of abstinence than he has hitherto made; or it is with mechanisms of self-control. For instance, given a just this once, and once does not count, etc., etc., ad lib- constant level of precommitment or choice-bundling itum – it is, in fact, anything you like except being a across a set of individuals, it might still be the case that the individuals with the steeper underlying discount Even when choices are bundled together successfully, function will be more likely to exhibit breakdowns of the cure can be worse than the disease. Grouping choices self-control, and exhibit impulsivity. However, at the increases the incentive against impulsivity, but in so do- same time, the individual with the steeper discount func- ing, it also increases the costs of a single failure to resist tion might also have a built-in motivation to precommit temptation. Consistent with this, it has been demonstrat- to non-impulsive behavior sooner, and also to employ ed that among individuals who construct self-control more rigid rules for self-control. Unusually disciplined rules which strictly group all members of a category (i.e.
behavior might in some cases be the product of compen- “I must never eat any sweets again”), lapses are less fre- sation for steep underlying discounting.
quent, but more likely to turn into binge lapses of impul- Of the two elements of choice-bundling (summation sive choice when they do occur. Interestingly, AA and of serial rewards and perception of current choices as other 12 step programs may have found a way to deal test cases) only the former has been shown in animals.
with some of the tension between the benefits and costs Mazur showed that the value to pigeons of at least a of choice grouping. By promoting the conception that small series of rewards was equal to the sum of the value the addict is helpless to have just one drink, the grouping of each reward individually (Mazur 1986). Thus it implied is that one choice to drink will necessitate a should follow that impulsivity is diminished in a delay of whole binge of drinks. However, AA attempts to mini- reward procedure in which single choices determines a mize the consequences of failure by advising resolutions series of rewards rather than a single presentation of the for only “one day at a time” (Marlatt and Gordon 1980; chosen reward. If experimental paradigms are created which can robustly demonstrate reductions in impulsi- Another danger of choice bundling is that by valuing vity based on reward summation, we may have an ani- choices as precedents more than as events in themselves, mal model of one element in a more sophisticated self- choice-bundling can put the individual at risk for becom- control mechanism. Psychopharmacological research can ing rigid and legalistic. Obsessive-compulsive personali- then look at how reward-summation responds to various ty disorder (OCPD) and related everyday phenomena manipulations. Those that facilitate summation in animal like workaholism can be viewed as pitfalls of extreme models should reduce impulsivity and those that under- choice bundling. For example, a patient with OCPD may mine summation should increase impulsivity. We know not use too large a postage stamp even once for fear that of no work that has looked at pharmacological impact on she will become a spendthrift. Furthermore, fear of laps- the summation of serial rewards, an issue that is highly es creates a motive to adopt highly explicit rules for one- relevant to the study of self-control.
self in preference to subtler ones that may be more richly Thus far, the animal models of impulsivity used in pharmacological research have relied directly on phe- The inadequacy of bundling as a route to true ratio- nomena that depend on hyperbolic discount functions.
nality creates a second kind of maladaptive choice, While we think this is the critical foundational element which has only partially been differentiated from impul- of impulsivity, animal models should be elaborated to in- siveness in the literature of motivational science. While clude the converse of impulsivity – self-control. Rela- psychotherapists from Freud onward have recognized tively impulsive behavior in humans may be more a problems from “punitive superegos”, “cognitive maps”, function of the extent to which mechanisms of self-con- “overgeneralization” etc. as distinct from spontaneous trol are employed than the steepness of the individual’s over-valuation of the immediate, our language still underlying discount function. Mechanisms of self-con- lumps together impulses like “compulsive spending” trol grow directly out of hyperbolic discounting, and in many cases can be modeled in animals. We think choice- Evenden JL (1998b) The pharmacology of impulsive behaviour in bundling is a particularly critical mechanism of self con- rats III: the effects of amphetamine, haloperidol, imipramine,chlordiazepoxide and ethanol on a paced fixed consecutive trol, and we are encouraged that the animal literature number schedule. Psychopharmacology 138:295–304 supports one of its properties – the summation of serial Evenden JL, Ryan CN (1996) The pharmacology of impulsive be- rewards. Clinical observation suggests that choice bun- haviour in rats: the effects of drugs on response choice with dling may both precipitate impulsiveness by its failures, varying delays of reinforcement. Psychopharmacology 128: and impose another kind of maladaptive trait, compul- Freud S (ed) (1956) On the history of the psychoanalytic move- siveness, by its overgrowth. Pharmacologic research on ment. The standard edition of the complete works of Sigmund impulsiveness should be done with these complex possi- Green L, Fristoe N, Myerson J (1994) Temporal discounting and preference reversals in choice between delayed outcomes. Psy-chon Bull Rev 1:383–389 Griffiths M (1993) Tolerance in gambling: an objective measure using the psychophysiological analysis of male fruit machine Harvey C (1994) The reasonableness of non-constant discounting.
Ainslie G (1974) Impulse control in pigeons. J Exp Anal Behav Ho MY, Al-Zahrani SS, Al-Ruwaitea AS, Bradshaw CM, Szabadi Ainslie G (1975) Specious reward: a behavioral theory of impul- E (1998) 5-Hydroxytryptamine and impulse control: prospects siveness and impulse control. Psychol Bull 82:463–496 for a behavioural analysis. J Psychopharmacol 12:68–78 Ainslie G (1992) Picoeconomics: the strategic interaction of suc- James W (1890) Principles of psychology. Holt, New York cessive motivational states within the person. Cambridge Uni- Kirby KN, Herrnstein RJ (1995) Preference reversals due to myo- pic discounting of delayed reward. Psychol Sci 6:83–89 Ainslie G (1999) The intuitive explanation of passionate mistakes Logue AW, Pena-Correal TE, Rodriguez ML, Kabela E (1986) and why it’s not adequate. In: Elster J (ed) Addiction: entranc- Self-control in adult humans: variation in positive reinforcer es and exits. Russell Sage, New York (in press) amount and delay. J Exp Anal Behav 46:159–173 Ainslie G, Haendel V (1993) The motives of the will. In: Gottheil Logue AW, Forzano LB, Tobin H (1993) Independence of rein- E, Druley K, Skodola T, Waxman H (eds) Etiology aspects of forcer amount and delay: the generalized matching law and alcohol and drug abuse. Thomas, Springfield, Illinois self-control in humans. Learn Motiv 23:326–342 Bizot JC, Thiebot MH, le Bihan C (1988) Effects of imipramine- Marlatt G, Gordon J (1980) Determinants of relapse: implications like drugs and serotonin uptake blockers on delay of reward in for the maintenance of behavior change. In: Davidson P, Da- rats: possible implication in the behavioral mechanism of vidson S (eds) Behavioral medicine: changing health life- action of antidepressants. J Pharmacol Exp Ther 246:1144– styles. Pergamon Press, Elmsford, N.Y., pp 410–452 Mazur J (1986) Choice between single and multiple delayed rein- Charrier D, Thiebot MH (1996) Effects of psychotropic drugs on rat responding in an operant paradigm involving choice be- Mazur J (1987) An adjusting procedure for studying delayed rein- tween delayed reinforcers. Pharmacol Biochem Behav 54: forcement. In: Commons M, Mazur J, Nevin J, Rachlin H (eds) The effect of delay and of intervening events on reinforcement Comings DE, Gade R, Wu S, Chiu C, Dietz G, Muhleman D, value. Lawrence Erlbaum, Hillsdale, N.J., pp 55–73 Saucier G, Ferry L, Rosenthal RJ, Lesieur HR, Rugle LJ, Mac Mazur J, Stellar J, Waraczynski M (1987) Self-control choice with Murray P (1997) Studies of the potential role of the dopamine electrical stimulation of the brain as a reinforcer. Behav Proc D1 receptor gene in addictive behaviors. Mol Psychiatry Mazur JE (1984) Tests of an equivalence rule for fixed and Comings DE, Rosenthal RJ, Lesieur HR, Rugle LJ, Muhleman D, variable reinforcer delays. J Exp Psychol Anim Behav Proc Chiu C, Dietz G, Gade R (1996) A study of the dopamine D receptor gene in pathological gambling. Pharmacogenetics McMillen BA, Means LW, Matthews JN (1998) Comparison of the alcohol-preferring P rat to the Wistar rat in behavioral tests Corulla WJ (1987) A psychometric investigation of the Sensation of impulsivity and anxiety. Physiol Behav 63:371–375 Seeking Scale Form V and its relation to the 1.7 Impulsiveness Mechner F, Latranyi M (1963) Behavioral effects of caffeine, questionnaire. Person Indiv Diff 8:651–658 methamphetamine, and methylphenidate in the rat. J Exp Anal Coull JT, Middleton HC, Robbins TW, Sahakian BJ (1995) Cloni- dine and diazepam have differential effects on tests of atten- Mischel H, Mischel W (1983) The development of children’s tion and learning. Psychopharmacology 120:322–332 knowledge of self-control strategies. Child Dev 54:603–619 Curry S, Marlatt GA, Gordon JR (1987) Abstinence violation ef- Mischel W, Moore B (1980) The role of ideation in voluntary de- fect: validation of an attributional construct with smoking ces- lay for symbolically-presented rewards. Cognit Ther Res sation. J Consult Clin Psychol 55:145–149 Dickman S (1990) Functional and dysfunctional impulsivity: per- Navarick D (1982) Negative reinforcement and choice in humans.
sonality and cognitive correlates. J Person Soc Psychol 58: Nemiah J (1977) Alexithhymia: theoretical considerations.
Donahoe JW, Burgos JE, Palmer DC (1993) Selectionist approach to reinforcement. J Exp Anal Behav 58:17–40 Owen AM, Roberts AC, Polkey CE, Sahakian BJ, Robbins TW Donahoe JW, Palmer DC (1994) Learning and complex behavior.
(1991) Extra-dimensional versus intra-dimensional set shifting performance following frontal lobe excisions, temporal lobe Dougan J, McSweeney F, O’Reilly P, Eacker J (1983) Negative excisions or amygdalo-hippocampectomy in man. Neuropsy- automaintenance: Pavlovian conditioning or differential rein- Parker JDA, Bagby RM, Webster CD (1993) Domains of the Evenden JL (1998a) The pharmacology of impulsive behaviour in impulsivity construct: a factor analytic investigation. Person rats II: the effects of amphetamine, haloperidol, imipramine, chlordiazepoxide and other drugs on fixed consecutive number Picker M, Blakely E, Poling A (1986) Effects of anticonvulsant schedules (FCN 8 and FCN 32). Psychopharmacology 138: drugs under automaintenance and negative automaintenance procedures. Pharmacol Biochem Behav 24:555–560 Poling A, Appel J (1979) Drug effects on the performance of pi- Shizgal P, Conover K (1996) On the neural computation of utility.
geons under a negative automaintenance schedule. Psycho- Siegel E, Rachlin H (1995) Soft commitment: self-control Poulos CX, Parker JL, Le DA (1998) Increased impulsivity after achieved by response persistence. J Exp Anal Behav 64:117– injected alcohol predicts later alcohol consumption in rats: ev- idence for “loss-of-control drinking” and marked individual Skinner BF (1953) Science and human behavior. Free Press, New differences. Behav Neurosci 112:1247–1257 Rachlin H (1995) Behavioral economics without anomalies. J Exp Solnick JV, Kannenberg CH, Eckerman DA, Waller MB (1980) An experimental analysis of impulsivity and impulse control Rachlin H, Green L (1972) Commitment, choice and self-control.
Tomie A, Aguado AS, Pohorecky LA, Benjamin D (1998a) Etha- Richards J, Mitchell SH, De Wit H, Seiden LS (1997) Determina- nol induces impulsive-like responding in a delay-of-reward tion of discount functions in rats with an adjusting-amount operant choice procedure: impulsivity predicts autoshaping.
Roberts AC, De Salvia MA, Wilkinson LS, Collins P, Muir JL, Tomie A, Cunha C, Mosakowski EM, Quartarolo NM, Pohorecky Everitt BJ, Robbins TW (1994) 6-Hydroxydopamine lesions LA, Benjamin D (1998b) Effects of ethanol on Pavlovian au- of the prefrontal cortex in monkeys enhance performance on toshaping in rats. Psychopharmacology 139:154–159 an analog of the Wisconsin Card Sort Test: possible interac- Williams D, Williams H (1969) Automaintenance in the pigeon: tions with subcortical dopamine. J Neurosci 14:2531–2544 sustained pecking despite contingent nonreinforcement. J Exp Schelling, TC (1960) The strategy of conflict. Harvard University Woodruff G, Williams D (1976) The associative relation underly- Schwartz B, Williams D (1972) The role of the response-reinforc- ing autoshaping in the pigeon. J Exp Anal Behav 26:1–13 er contingency in negative automaintenance. J Exp Anal Be- Wray I, Dickerson M (1981) Cessation of high frequency gam- bling and “withdrawal” symptoms. Br J Addict 76:401–405 Schwartz B, Reisberg D, Vollmecke T (1974) Effects of treadle training on autoshaped keypecking: learned laziness andlearned industriousness or response competition? Bull Psy-chon Soc 3:369–372

Source: ftp://psyftp.mcmaster.ca/dalywilson/Papers/futurediscounting/discount1.pdf