The purpose of this review is to refine the neuroanatomical framework of the dopamine reward circuitry, in light of recent observations on drug reward trigger zones using intracranial self-administration procedures in rats. This review consists of two major parts following this introduction (section 1). The first part concerns anatomy and consists of two sections. The ascending dopamine projection from the ventral midbrain to the nucleus accumbens and the olfactory tubercle is discussed based on previous studies and present experiments (section 2). In addition, the data on the efferents from the nucleus accumbens and olfactory tubercle are reviewed (section 3). The second part of the manuscript reviews behavioral studies and proposes a hypothesis to address how the dopamine projection systems interact with each other in drug reward and motivated behaviors (section 4).

The shell portion of the accumbens appears to be more important than the core for drug reward. Rats learn to self-administer psychomotor stimulants such as amphetamine or cocaine or dopamine receptor agonists into the accumbens shell, but not the core (Carlezon et al., 1995; Ikemoto et al., 1997a; Rodd-Henricks et al., 2002; Ikemoto, 2003; Ikemoto et al., 2005). In addition, microinjections of dopaminergic antagonists into the shell, but not the core, disrupt conditioned place preference induced by systemic nicotine or morphine (Fenu et al., 2006; Spina et al., 2006). Conditioned place preference induced by intravenous administration of cocaine or amphetamine is attenuated by lesions on dopamine terminals in the shell rather than the core (Sellings and Clarke, 2003; Sellings et al., 2006a; 2006b). These results confirm functional differences between the two accumbens compartments inferred from anatomical observations that the afferents to and efferents from the accumbens differ significantly between the shell and core (Zahm and Brog, 1992).

The VTA at least partly mediates the rewarding effects of nicotine, opiates, cannabinoids and ethanol. Intravenous self-administration of nicotine in rats is attenuated by selective lesions to VTA dopamine neurons projecting to the ventral striatum (Corrigall et al., 1992) as well as by the blockade of nicotinic receptors administered into the VTA (Corrigall et al., 1994). Mice learn to run on a Y-maze to self-administer nicotine into the VTA, an effect that can be diminished by systemic administration of dopamine antagonists (Maskos et al., 2005; David et al., 2006). Administration of the nicotinic receptor agonists cytisine or nicotine into the vicinity of the VTA induces conditioned place preference (Museo and Wise, 1994; Laviolette and van der Kooy, 2003). Similarly, rats and mice learn to self-administer opiates (Bozarth and Wise, 1981; Welzl et al., 1989; David and Cazala, 1994a; 1994b; Devine and Wise, 1994), cannabinoids (Zangen et al., 2006), cocaine (David et al., 2004; Rodd et al., 2005) or ethanol (Gatto et al., 1994; Rodd-Henricks et al., 2000) into the VTA. These behavioral effects are mediated by dopaminergic systems as shown by pharmacological manipulations (no data is available on cannabinoids in this context).

Taken together, these data suggest that dopaminergic neurons localized in the VTA projecting to the nucleus accumbens, particularly the shell, are activated by administration of drugs of abuse, and the activation of this system is rewarding.

Figure 1 Self-administration of cocaine into the olfactory tubercle. (A) Injection sites were placed in the medial portion of the olfactory tubercle and are plotted (white dots) on TH stained sections. Abbreviations: ac, anterior commissure; CO, core; OTm, medial (more ...)

The mechanisms of drug reward in the vicinity of the VTA are more complex than previously thought. Rats learn to lever-press for administration of drugs such as nicotine (Ikemoto et al., 2006), carbachol (Ikemoto and Wise, 2002), opiates (Zangen et al., 2002), cannabinoids (Zangen et al., 2006), cocaine (Rodd et al., 2005), ethanol (Rodd-Henricks et al., 2000) and its metabolite acetaldehyde (Rodd et al., 2004) into the posterior VTA more readily than into the anterior VTA. Data obtained with viral technology also indicate that the anterior and posterior VTA play different roles in conditioned place preference induced by systemic cocaine or opiate administration (Carlezon et al., 2000; Bolanos et al., 2003; Olson et al., 2005). In addition to the posterior VTA, the central (or caudal) linear nucleus raphe, localized posterodorsal to the VTA, is involved in drug reward. Rats learn to self-administer nicotine and other drugs into the central linear nucleus (Ikemoto et al., 1998; 2006). Interestingly, this midline structure contains both dopaminergic (Phillipson, 1979b; Swanson, 1982) and serotonergic (Steinbusch, 1981; Halliday and Tork, 1989) neurons.

The pattern of drug-reward trigger zones described above appears to be partly explained by cellular connectivity between the ventral midbrain and the ventral striatum. Previous data clearly suggest that the neurons in the VTA project into the nucleus accumbens with mediolateral topography (Fallon and Moore, 1978; Nauta et al., 1978; Beckstead et al., 1979; Phillipson and Griffiths, 1985; Brog et al., 1993; Hasue and Shammah-Lagnado, 2002). Similar mediolateral topography appears to exist for the projection from the ventral midbrain to the olfactory tubercle (Newman and Winans, 1980). The set of data described below refines this mediolateral characterization and provides additional details. The cell-bodies of dopaminergic neurons projecting to the nucleus accumbens and olfactory tubercle distribute from the posteromedial to anterolateral dimensions at an approximate 45° angle to the anteroposterior axis. The medial olfactory tubercle and medial accumbens shell (drug-reward trigger zones in the striatum) receive strong dopaminergic innervation from the posteromedial VTA and central linear nucleus (drug-reward trigger zones in the ventral midbrain), but little innervation from anteromedial or lateral VTA. This arrangement of dopaminergic neurons can provide an explanation for the behavioral data, suggesting that the anterior and posterior VTA are functionally heterogeneous. Before describing in detail how cell-bodies of dopaminergic neurons in the ventral midbrain distribute and how they project to the ventral striatum, the historical background of relevant concepts will be reviewed. This includes a discussion of the VTA, A10 nucleus, ventral striatum and meso-limbic dopamine system. These anatomical concepts are not as definitive as generally thought.

2.1.1. VTA and A10 dopamine nucleus There has been reciprocal influence between the concepts of the VTA and the A10 dopamine nucleus. The understanding of the VTA as a discrete brain area influenced the definition of the A10 nucleus, which in turn re-shaped the concept of the VTA. Since the VTA in human and other primate brains is difficult to distinguish from the substantia nigra, the VTA was initially identified in animals with less developed substantia nigra. Tsai (1925) was the first to acknowledge the “nucleus tegmenti ventralis” (or “ventral tegmental nucleus”) in the opossum brain, and Papez (1932) provided the first description of the region (in the armadillo), although he referred it as the “nucleus of the mammillary peduncle”. Tsai’s nomenclature was adopted by subsequent studies that examined the region in other mammalian species (e.g., Rioch, 1929; Brown, 1943; Taber, 1961) and became the basis for the current term “VTA”. In his influential paper, Nauta (1958) used the term “ventral tegmental area of Tsai” to refer to the region roughly corresponding to the ventral tegmental nucleus, and he appears to have been the first to use this term. The word “area” seems to be more appropriate than “nucleus” considering the heterogeneous cytoarchitectonic features of the region and the lack of clear borders distinguishing it from adjacent regions. Nauta’s findings (Nauta, 1956; 1958; 1960) that limbic-related structures, especially the lateral hypothalamic area, project to the medial midbrain including the VTA, but not the substantia nigra, affirmed the distinction of the cytoarchitectonically obscure VTA from the substantia nigra.

Nauta and his colleagues’ work influenced the way dopaminergic nuclei are labeled in the ventral midbrain. Dahlstrom and Fuxe (1964) identified the locations of cell groups containing catecholamines and serotonin throughout the rat brain. Because neurochemically-identified cell groups were not necessarily confined in the structures that were largely defined by cytoarchitectonic features, Dahlstrom and Fuxe introduced a series of terms, A1-12 and B1-9 nuclei, to label neurochemically-defined cell groups. In the ventral midbrain, areas containing dopaminergic neurons were classified into three nuclei, A8, A9 and A10, although these nuclei were linked continuously. The A10 nucleus was differentiated from others on the basis of selective limbic-related afferents to the VTA (Dahlstrom and Fuxe, 1964).

The VTA has been defined in several different ways. There are two popular versions, differing in their inclusion or exclusion of the midline nuclei. Earlier influential work performed in the rat brain (Dahlstrom and Fuxe, 1964; Fallon and Moore, 1978; Moore and Bloom, 1978) treated the VTA as synonymous with the A10 nucleus. The emphases on dopamine content rather than cytoarchitectonic features of the area led to a definition of the VTA that includes the midline nuclei – the interfascicular nucleus, rostral linear nucleus and central linear nucleus (Oades and Halliday, 1987). These midline nuclei, which had long been documented in other species (Brown, 1943; Taber, 1961), were not fully acknowledged in the rat brain (e.g., Pellegrino and Cushman, 1967; Sherwood and Timiras, 1970; Pellegrino et al., 1979) until later. Phillipson (1979a; 1979b; 1979c) described the interfascicular nucleus and caudal (or central) linear nucleus in the rat and found that these nuclei contain dopamine cell-bodies. Swanson (1982) described the rostral linear nucleus as well as interfascicular and central linear nuclei in the rat and also reported that they contain dopamine cell-bodies. Consideration of distinct cytoarchitectonic features of the midline nuclei led to the other popular definition of the VTA that excludes midline nuclei (Swanson, 1982).

Hokfelt et al. (1984) extended the concept of the A10 nucleus. Using more sensitive procedures, they described dopaminergic neurons localized in the supramammillary nucleus, medial habenula, dorsal raphe nucleus and periaqueductal gray, and included these regions into the A10 nucleus (Hokfelt et al., 1984). Accordingly, the revised A10 area has lost its intimate relationship to the VTA. Nevertheless, it is the original concept of the A10 that influenced how early and current investigators thought of the VTA. The extended notion of the A10 area has not yet been accepted.

2.1.2. Ventral striatum The concept of the ventral striatum emerged in 1973 when Heimer and Wilson presented in the Golgi Centennial Symposium. They suggested that the well-established isocortico-striato-pallidal system has a parallel system consisting of limbic (piriform, hippocampal, and amygdaloid) cortices projecting to the olfactory tubercle and nucleus accumbens, which in turn, project to the ventral pallidum (sections 2.4 and 3.1) (Heimer and Wilson, 1975; Heimer, 1978). The olfactory tubercle and nucleus accumbens are conceptualized as part of the striatal system. This understanding is consistent with massive dopaminergic innervations to these structures from the ventral midbrain, which also supplies dopamine to the dorsal striatum. Swanson and Cowan (1975) also suggested that the nucleus accumbens is a medial extension of the caudate-putamen complex, an idea based on cytoarchitecture, cellular connection and developmental links (although they did not consider the olfactory tubercle). The concept of the ventral striatum is useful because it figures the functions of the tubercle and accumbens as parts of a large functional system, namely the basal ganglia (Zahm and Brog, 1992; Heimer et al., 1995).

It should be noted that the entire portion of the olfactory tubercle does not belong to the striatal system. The olfactory tubercle consists of several heterogeneous elements (Fig. 1A) – the islands of Calleja, the zone consisting of the ventral pallidum and the medial forebrain bundle, and the striatal zone containing medium spiny neurons (Heimer, 1978).

2.1.3. The meso-limbic dopamine system As discussed above, drug reward is partly mediated by the meso-limbic dopamine system. However, this anatomical concept has been used slightly differently by different researchers. Ungerstedt (1971) originally reported the observation that the A10 dopamine nucleus projects primarily to the nucleus accumbens and olfactory tubercle. This projection was named the meso-limbic dopamine system to contrast with the nigro-striatal dopamine system, projecting from the A9 nucleus to the “striatum”, which is now referred to as the “dorsal striatum” because of the introduction of the concept of the ventral striatum. Therefore, one definition of the meso-limbic dopamine system is the dopaminergic projection from the VTA to the ventral striatum, parallel to the nigro-striatal dopamine projection.

Subsequent studies employing more sensitive methods showed that VTA dopamine neurons project not only to the accumbens and tubercle but also to other limbic-related regions including the septum, hippocampus, amygdala and prefrontal cortex (Lindvall and Bjorklund, 1974; Lindvall et al., 1974; Fallon and Moore, 1978; Swanson, 1982), although these regions receive much less dopaminergic innervation from the VTA than does the ventral striatum. Thus, another definition of the meso-limbic dopamine system not only includes the limbic striatum but also other limbic forebrain regions that receive dopaminergic inputs from the ventral midbrain. It should be noted that some limbic regions such as the amygdala and piriform cortex receive significant inputs from the substantia nigra; thus, in this definition, “meso” may include both the VTA and substantia nigra.

A third definition is based on the fact that the concept of A10 has been extended to include areas such as the supramammillary nucleus and dorsal raphe nucleus (Hokfelt et al., 1984), which also project into limbic forebrain areas (Hasue and Shammah-Lagnado, 2002). Thus, the meso-limbic dopamine system could be construed as the extended A10 area and substantia nigra projecting to extensive limbic forebrain regions.

In summary, the meso-limbic dopamine system can be defined in different ways. There are many dopamine-containing neurons localized inside and outside the ventral midbrain projecting to the ventral striatum and non-striatal forebrain regions. The challenge is how to group them up with respect to function.

2.2.1. Cytoarchitectonic features of the VTA and surrounding areas Cytoarchitectonic data have long suggested distinct regions within the VTA. However, those regions within the VTA are not necessarily consistent across different studies, especially when different species are involved. Olszewski and Baxter (1954) identified the paranigral nucleus (PN) and parabrachial pigmented area (PBP) in the human brain, in the approximate area where Tsai (1925) identified the ventral tegmental nucleus. Taber (1961) identified three nuclei in the cat VTA - the PN, PBP and ventral tegmental nucleus of Tsai. In rats, the two nuclei – the PN and PBP – have commonly been identified in the VTA (Phillipson, 1979b; Halliday and Tork, 1986). The divisions made by Phillipson (1979b) and those of Halliday and Tork (1986) are similar, and these divisions appear to have been adopted in many rat studies including the present study. However, other divisions have been made in the rat brain, especially when neurochemical profiles of the VTA are taken into consideration (e.g., McRitchie et al. 1996).

The present paper divides the VTA into four major zones (Fig. 2; see section 6.1 for methods): PN, PBP, parafasciculus retroflexus area, and ventral tegmental tail (VTT). The PN and PBP are dopaminergic cell-body rich zones, whereas the parafasciculus retroflexus area and VTT are dopaminergic cell-body poor zones. The parafasciculus retroflexus area consists of a low density of tyrosine hydroxylase (TH)-positive cell-bodies, which are small to medium in size and light to moderate in staining (Fig. 3, 4, and 5). The TH-positive cell-bodies in the parafasciculus retroflexus area are continuous with those in the posterior hypothalamic area including the supramammillary nucleus, which contains TH-positive cell-bodies centered in its medial part (Fig. 3), as previously reported (Swanson, 1982; Hokfelt et al.,1984; Shepard et al., 1988). These TH-positive cell-bodies are mostly mid-sized, round-shaped, and moderately stained.

Figure 2 Divisions of the ventral midbrain shown on TH-stained horizontal sections. Sections, separated by 160 μm, are arranged from dorsal (A) to ventral (F). For the purpose of projection analyses, each section is divided into anterior-posterior and (more ...)

Figure 3 Enlarged panel E of Figure 2, showing cytoarchitectonic features of TH-stained cells at the level of the PN. See the legend of Figure 2 for abbreviations.

Figure 4 Enlarged panel B of Figure 2, showing TH-stained cells at the level of the central linear nucleus.

Figure 5 Coronal sections showing TH-positive cells at the level of the parafasciculus retroflexus area. The areas within the green rectangular frames are enlarged in the bottom panels. The 0.1-mm scale applies to the bottom five images.

The middle two-thirds of the VTA consist of dense TH-positive cell-bodies. This dopamine rich sub-area is divided into two zones consistent with previous studies (Phillipson, 1979b; Halliday and Tork, 1986). The zone adjacent to the anterolateral portion of the interpeduncular nucleus is referred to as the PN. Its TH-positive cells are relatively homogeneous, mostly medium in size, medium to dark in staining (Fig. 3), and oriented mediolaterally, tilting up toward the interfascicular nucleus, a feature that is best captured in the coronal section (Fig. 6).

Figure 6 Coronal sections showing TH-stained cells at the level of the PN. The areas within the green rectangular frames are enlarged in the bottom panels. The 0.1-mm scale applies to the bottom four images.

The rest of the dopamine cell-body rich zone is referred to as the PBP. The borders of the PBP have not been consistently defined in the rat, probably because the cytoarchitectonic features of the PBP are heterogeneous; the borders of the present PBP appear to roughly correspond to those of Halliday and Tork (1986). This zone consists of large- and medium-sized cell-bodies, which do not have unified orientations. The anterior PBP contains large TH-positive cells that are continuous with those in the anterior compacta part of the substantia nigra (SNC), and is packed with large, intensely stained TH-positive cells (Fig. 3, 4 and 5). In the posterior VTA, the PBP occupies the area just dorsal and lateral to the PN. The cell-bodies and fibers of the posterodorsal portion of the PBP form a net-like structure (Fig. 6 and 7), which may correspond to the description of the PBP in the cat (Taber, 1961). Although the heterogeneous cytoarchitectonic features of the PBP suggest that it can be divided into multiple units, the present paper treats the PBP as an entity and defines it as the dopamine cell-body rich zone that is not the PN.

Figure 7 Sagital sections showing TH-positive cells at the level of PN. The areas within the green rectangular frames are enlarged in the bottom panels. The 0.1-mm scale applies to the bottom four images.

The VTT contains a low density of TH-positive cell-bodies, localized in the zone just posterior to the PN and just lateral to the posterior interpeduncular nucleus. Its TH-positive cell-bodies are small and moderately stained (Fig. 3, 7 and 8). Although this zone is clearly distinct from the PN and PBP in TH-stained sections, the cytoarchitectonic features of the VTT in Nissl-stained sections (Fig. 8) are similar to and continuous with those of more anterior VTA zones, a key factor that places this zone into the VTA.

Figure 8 Coronal sections showing TH- and Nissl-stained cells at the level of the central linear nucleus. The areas within the green rectangular frames of panel A are enlarged in panels B and C. The area in the green frame of Panel D is enlarged in panel E showing (more ...)

Groups of TH-positive cell-bodies are found along the midline. The interfascicular nucleus, localized just dorsal to the interpeduncular nucleus, contains densely packed dopaminergic neurons (Fig. 6 and 9). Its density may be characterized as the greatest in the ventral midbrain. Although three linear nuclei, the rostral, central and caudal linear nuclei, are identified in dog and cat brains (Brown, 1943), only two linear nuclei have been recognized in the rat brain (Phillipson, 1979b; Swanson, 1982). The rostral linear nucleus is found anterodorsal to the interfascicular nucleus and its dopaminergic neurons are sparse (Fig. 5 and 9). The central linear nucleus, found along the midline between the interfascicular nucleus and the dorsal raphe nucleus, is filled with TH-positive neurons, which are relatively homogeneous, mostly medium in size, and medium to dark in staining (Fig. 8 and 9). The location of this nucleus appears to correspond to “the median fiber flow of the tegmental catecholamine radiations” described by Lindvall and Bjorklund (1974).

Figure 9 Sagital sections showing TH-positive cell-bodies at the midline. The areas within the green rectangular frames are enlarged in the bottom panels. BV, blood vessel (other abbreviations in the legend for Figure 2).

2.2.2. Locations of cell-bodies projecting to the ventral striatum Figure 10 depicts the representative distribution of Fluoro-Gold (FG, a retrograde tracer) labeled cell-bodies in the ventral midbrain for respective deposit sites, and Table 1 shows the mean counts of labeled cells with respect to midbrain zones (see section 6.2 for detailed results, discussion and methods). The majority of neurons projecting to the ventral striatum from the ventral midbrain appear to be dopaminergic, whereas the majority of neurons projecting to the nucleus of the diagonal band are non-dopaminergic (see section 6.2.2 for estimates and methodological issues). Overall, the ventromedial striatum, consisting of the medial olfactory tubercle and medial accumbens shell, receives dopaminergic innervations from the posteromedial VTA, interfascicular nucleus and central linear nucleus. However, the ventrolateral striatum, consisting of the lateral tubercle, lateral shell and accumbens core, receives dopaminergic innervations from the lateral half of the VTA (Fig. 11).

Figure 10 A chart of the distribution of retrogradely-labeled cell-bodies in the ventral midbrain after FG injections into ventral striatal sites and adjacent regions. Photomicrographs (top) show FG deposit sites in representative coronal sections. The numbers (more ...)

Table 1 Percent contributions (SEM) of 10 selected zones in the ventral midbrain to respective terminal regions deposited with FG.

Figure 11 The meso-striatal dopamine systems. The data shown in Figure 10 are re-drawn on single sections to contrast ventral midbrain projections of striatal zones. Horizontal sections are arranged from (a) most dorsal to (f) most ventral. Although dopaminergic (more ...)

These ventral midbrain projections to the ventral striatum differ among sub-areas within the ventral midbrain (Fig. 12). The medial part of the PBP provides a selective projection to the ventromedial striatum. The lateral part of the PBP provides heavy projections to the ventrolateral striatum. Thus, the PBP appears to send its efferents to the ventral striatum with mediolateral topography. Dopamine neurons in the PN appear to selectively project to the ventromedial striatum. Similarly, dopamine neurons in the central linear nucleus selectively project to the ventromedial striatum, connectivity that is consistent with recent data obtained with the anterograde tracer PHA-L (Del-Fava et al., 2007). Interestingly, the interfascicular nucleus may provide selective projection to the dorsomedial part of the shell, but not other striatal regions.

Figure 12 Midbrain distributions of cells projecting to the ventral striatum. Projection data of sections (a) through (f) of Figure 11 are combined onto a single plane to show how ventral midbrain projections to the striatum are organized with respect to sub-areas (more ...)

The rostral linear nucleus appears to project into the diagonal band, as well as into the pallidal zone of the olfactory tubercle. Although it is not demonstrated by the data presented here, recent anterograde-tracer data suggest that neurons in this structure projects to the pallidal zone of the tubercle (Del-Fava et al., 2007). The parafasciculus retroflexus area and the VTT of the VTA project little, if at all, into the ventral striatum and thus are not part of the meso-striatal dopamine systems. The VTT appears to contain an aggregate of GABAergic neurons (Olson and Nestler, 2007). It is unclear, at this time, whether those neurons have any functional link to dopaminergic neurons projecting to the ventral striatum. Consistent with previous data (Fallon and Moore, 1978), the parafasciculus retroflexus area projects heavily into the nucleus of the diagonal band, which also receives innervations from the posterior hypothalamic area including the supramammillary nucleus (Vertes, 1992). As described above, the morphological features of TH-positive neurons in the parafasciculus retroflexus area are more similar to those of the supramammillary nucleus than to those in the PN or PBP, and are continuous with the posterior hypothalamic area. These data suggest that the parafasciculus retroflexus area is more closely related to the posterior hypothalamus than the PN or PBP of the VTA.

In summary, these data indicate that not all VTA and midline zones project to the ventral striatum. As shown in Figure 12, the cells of dopaminergic neurons projecting to the ventral striatum distribute with posteromedio-anterolateral topography at an approximate 45° angle to the midline. This organization of dopaminergic projections appears to provide an explanation for the pattern of self-administration data described in section 1.2 and discussed below.

Rewarding effects of such disparate drugs as cocaine, nicotine, heroin, alcohol and cannabis have been suggested to be mediated by a common circuitry (Wise and Bozarth, 1987; Wise, 1996). The meso-ventromedial striatal dopamine system appears to be a common circuitry for triggering drug reward. Psychomotor stimulants such as cocaine and amphetamine act at protein targets on the axon terminals of the medial accumbens shell and medial olfactory tubercle (Carlezon et al., 1995; Ikemoto et al., 1997a; Rodd-Henricks et al., 2002; Ikemoto, 2003; Sellings and Clarke, 2003; Ikemoto et al., 2005; Sellings et al., 2006b). Other drugs of abuse such as nicotine and opiates appear to act in the posteromedial VTA and central linear nucleus.

Functional heterogeneity of the VTA has been described in terms of differences between the anterior and posterior VTA (Ikemoto et al., 1997b; 1998). In light of the pattern of midbrain projections to the ventral striatum (Fig. 12), the heterogeneity of the VTA may be better described by differences between the posteromedial VTA, centering around the PN, versus the rest of the VTA. The study that initially suggested possible VTA heterogeneity in drug reward reported that rats self-administered the GABA_A antagonist picrotoxin into the anterior VTA, but not the posterior VTA (Ikemoto et al., 1997b). The notion that drugs trigger rewarding effects from the anterior VTA, however, is no longer viable. It was found that rats learn to self-administer the excitatory amino acid AMPA or nicotine into the supramammillary nucleus (Ikemoto et al., 2004; 2006), localized just anteromedial to the anterior VTA and dorsal to the mammillary body (Fig. 5 and 9). Importantly, they do not learn to self-administer these drugs into the anterior VTA. Moreover, picrotoxin administration into the supramammillary nucleus appears to be more rewarding than administration into the anterior VTA (Ikemoto, 2005). Therefore, the rewarding effects of picrotoxin into the anterior VTA are readily explained by the notion that picrotoxin administered into the anterior VTA diffuses to the supramammillary nucleus for its rewarding effects. Indeed, no other drug has been reported to trigger rewarding effects from the anterior VTA. These observations make sense in light of the pattern of midbrain dopaminergic projections to the ventral striatum. Drug injections into the anterior VTA are likely not rewarding because either anteromedial VTA (parafasciculus retroflexus area) or anterolateral VTA (anterior PBP) do not significantly project to the ventromedial striatum, the drug-reward trigger zone in the striatum.

The data from many studies are consistent with the notion that the posterior VTA, but not anterior VTA, centering around the PN mediates the rewarding effects of drugs including nicotine (Ikemoto et al., 2006), carbachol (Ikemoto and Wise, 2002), opiates (Zangen et al., 2002), cannabinoids (Zangen et al., 2006), cocaine (Rodd et al., 2005), ethanol (Rodd-Henricks et al., 2000) and its metabolite acetaldehyde (Rodd et al., 2004). It is most likely that protein targets localized in the posteromedial VTA mediate these drugs’ actions. It should be emphasized that their targets are not necessarily on dopaminergic neurons. The administration of these drugs into the VTA triggers multiple actions, and the net effect of these actions appears to increase firing of dopamine neurons. To illustrate this point, findings on the actions of drugs in the VTA are reviewed.

2.3.1. Nicotine Nicotine can directly excite dopaminergic neurons via the nicotinic receptors on the somatodendritic region of dopaminergic neurons (Calabresi et al., 1989; Pidoplichko et al., 1997). However, these receptors appear to be desensitized within a few seconds after exposure to high, reward-mediating concentrations of nicotine (Pidoplichko et al., 1997; 2004; Wooltorton et al., 2003). The brief excitatory capacity that nicotine has on dopaminergic neurons does not readily explain the finding that rats self-administer nicotine continuously over minutes or hours and that a single systemic injection of nicotine increases and maintains extracellular dopamine levels in the nucleus accumbens for over an hour (Imperato et al., 1986; Pidoplichko et al., 2004). Therefore, nicotinic receptors on dopaminergic neurons may be responsive to the first puff of a cigarette or the first infusion of intravenous nicotine following abstinence, but may not be responsive to subsequent administration of nicotine.

Other mechanisms are needed to explain the maintenance of nicotine self-administration and prolonged dopamine release. Nicotinic receptors are also found on presynaptic glutamatergic terminals in the VTA. Glutamatergic transmission can be facilitated by the presynaptic action of nicotine (McGehee et al., 1995), which then may activate dopaminergic neurons. The action of nicotine on glutamatergic transmission appears to persist and does not readily desensitize (Mansvelder and McGehee, 2000; Mansvelder et al., 2002; Pidoplichko et al., 2004). Therefore, this mechanism may play a major role in nicotine’s persistent reward-related effects. Nicotine administration into the VTA also exerts inhibitory effects on dopaminergic neurons. Nicotine can directly excite local GABAergic neurons, which may inhibit adjacent dopaminergic neurons, and enhances GABAergic inhibitory input via presynaptic receptors. However, nicotinic receptors on GABAergic neurons appear to be quickly desensitized and to stay desensitized for prolonged periods (Mansvelder et al., 2002; Pidoplichko et al., 2004). The net effect of nicotine administration into the VTA appears to increase the frequency of dopaminergic neuron firing.

2.3.2. Opiates, cannabinoids and ethanol The reinforcing effects of opiates administered into the VTA are thought to be mediated via local GABAergic neurons. Administration of opiates, systemically or locally, appears to disinhibit dopaminergic neurons in the VTA (Gysling and Wang, 1983; Matthews and German, 1984; Johnson and North, 1992). Opiate administration appears to inhibit ventral tegmental GABAergic neurons, which tonically inhibit dopaminergic neurons, thus removing tonic inhibition over dopamine neurons and, in turn, activating dopamine neurons (Gysling and Wang, 1983; Johnson and North, 1992). The actions of cannabinoids in the VTA may be more complex than those of opiates, but appear to involve disinhibitory mechanisms to activate dopamine neurons (Lupica et al., 2004; Gardner, 2005). The actions of ethanol have not yet been clearly elucidated.

2.3.3. Cell groups consisting of the meso-ventromedial striatal dopamine system In addition to the posteromedial VTA, the central linear nucleus and interfascicular nucleus should be included in the dopamine reward circuitry in the meso-ventromedial striatal dopamine system (Fig. 11). The central linear nucleus, into which rats quickly learn to self-administer nicotine or the GABA_A receptor agonist muscimol (Ikemoto et al., 1998; Ikemoto et al., 2006), provides heavy dopaminergic inputs to the ventromedial striatum. Thus, the function and projection of the central linear nucleus are similar to that of the posteromedial VTA. The interfascicular nucleus projects to the dorsomedial shell (Phillipson and Griffiths, 1985; Brog et al., 1993; section 2.2.2), in which psychomotor stimulants trigger reward. Thus, the axon-terminals of the neurons originating from the interfascicular nucleus are most likely targets of psychomotor stimulants, which then increase extracellular dopamine in the medial shell.

Functional roles of these dopamine projection systems are further discussed below, following a discussion of the non-dopaminergic afferents to the ventral striatum and the efferent systems of the striatum.

Ventral striatal dopamine modulates glutamatergic input onto median spiny projection neurons (Nicola et al., 2000; O’Donnell, 2003). The glutamatergic innervations to the ventral striatum come from various limbic regions, including the prefrontal cortex, hippocampus, amygdala, and thalamus, and vary with mediolateral topography (for the nucleus accumbens, see Groenewegen et al., 1999). Both the medial olfactory tubercle and the medial accumbens shell receive afferents from common zones, including the infralimbic cortex (Berendse et al., 1992), the posterior basolateral amygdaloid nucleus (Krettek and Price, 1978; Russchen and Price, 1984), the ventral subiculum of the hippocampal formation (Kelley and Domesick, 1982; Groenewegen et al., 1987), and the paraventricular thalamic nucleus (Newman and Winans, 1980; Berendse and Groenewegen, 1990; Moga et al., 1995). The lateral tubercle, lateral accumbens shell, and accumbens core receive afferents from the parataenial thalamic nucleus (Newman and Winans, 1980; Berendse and Groenewegen, 1990; Moga et al., 1995). However, there are differences. The lateral tubercle and lateral accumbens shell receive innervation from the anterior basolateral amygdala (Krettek and Price, 1978; Russchen and Price, 1984) and the ventral agranular insular cortex (Berendse et al., 1992), whereas the core receives input from the central medial thalamic nuclei (Berendse and Groenewegen, 1990), prelimbic cortex and dorsal agranular insular cortex (Berendse et al., 1992).

In addition, the ventral striatum receives input from the ventral pallidum with mediolateral topography. The medial ventral pallidum projects to the medial shell and medial tubercle, whereas the lateral ventral pallidum projects to the core, lateral shell and lateral tubercle (Groenewegen et al., 1993).

The olfactory tubercle receives input from primary olfactory regions, which receive monosynaptic input from the olfactory bulb. Mediolateral functional differences are also apparent in afferents to the tubercle from olfactory structures. The medial tubercle receives extensive input from the posterior piriform cortex, lateral entorhinal cortex, dorsal peduncular cortex, and ventral tenia tecta, whereas the lateral tubercle receives input from the olfactory bulb, anterior olfactory nucleus, and anterior piriform cortex (Luskin and Price, 1983). It should be noted that these olfactory inputs appear to go to the striatal zone, which consists of medium spiny neurons that are morphologically similar to the medium spiny neurons in the accumbens and dorsal striatum (Millhouse and Heimer, 1984). Thus, as the name suggests, the olfactory tubercle is closely connected to the olfactory system (Shipley et al., 2004). It should also be noted that the lateral shell appears to receive inputs from primary olfactory regions (Luskin and Price, 1983), although these inputs are not as extensive as those to the tubercle.

In summary, medium spiny neurons in the ventral striatum receive varying afferents between the structure’s medial and lateral areas. Afferents including dopaminergic afferents from the midbrain to the ventral striatum most likely change gradually rather than divide abruptly. Therefore, functional differences between them may be thought of as a continuum. It may be stated that the ventral striatum is functionally more different on the mediolateral dimension than the dorsoventral dimension. In other words, differences between the medial shell and the medial tubercle appear to be smaller than differences between the medial shell and the lateral shell or the medial tubercle and the lateral tubercle. The afferents to the core are somewhat unique. It is hard to say how much the core differs from the lateral shell/lateral tubercle. Based on afferents and efferents, which will be discussed next, the core appears to be more similar to the lateral shell/lateral tubercle than to the medial shell/medial tubercle. Although the olfactory tubercle receives olfactory inputs, the striatal district of the tubercle shares strikingly similar inputs with the nucleus accumbens, especially the accumbens shell. Common zones, except the olfactory regions, provide inputs to both medial shell and medial tubercle and both lateral shell and lateral tubercle. These similarities between the shell and tubercle suggest that the striatal district of the tubercle is a ventral extension of the shell. This concept should be useful for the investigation of functions of the tubercle.

Parallel to similarities and differences in afferents are morphological features of medium spiny neurons that receive these afferents. Features such as spine density and terminal and branch segments appear to differ between the medial shell and the lateral shell, and between the shell and the core (Meredith et al., 1992; 1995). In general, the features of spiny neurons in the lateral shell are intermediate between those of the medial shell and the core. Although morphological features of the medium spiny neurons of the olfactory tubercle have been examined (Millhouse and Heimer, 1984), comparisons between its medial and lateral parts have yet to be performed. Based on the notion that the tubercle is an extension of the shell, we can predict that medium spiny neurons in the medial tubercle resemble those of the medial shell more than those of the lateral tubercle, which resemble those in the lateral shell.

This section reviews data on efferent projections from the nucleus accumbens-olfactory tubercle complex. The reviewed data suggest strong mediolateral topography for the outputs from the ventral striatum. Figure 13 summarizes, in a schematic manner, how the outputs of the meso-striatal dopamine systems are organized with parallel circuits (Alexander et al., 1986) and split circuits (Joel and Weiner, 1994) leading to a hierarchy. This is a working model designed for understanding dopamine reward functions. Different lines coming into and out of each of the major structures such as the striatum, pallidum and thalamus do not necessarily indicate distinct connections, but rather graded connections. In addition, because the focus of the present paper is the nucleus accumbens-olfactory tubercle complex, the meso-dorsal striatal dopamine system, which has been divided into multiple components (Alexander et al., 1986), is simplified in Figure 13.

Figure 13 The meso-ventromedial and ventrolateral striatal dopamine systems and output connections. Abbreviations: Cg, cingulate cortex; CL, central linear nucleus; CO, core; DS, dorsal striatum; GPe, external globus pallidus; GPi, internal globus pallidus; IF, (more ...)

The ventral pallidum participates in lateral information flow via pallido-mesencephalic projections. The medial ventral pallidum, which receives its afferents from the medial shell and medial tubercle, provides its projections to both the posteromedial and lateral VTA, whereas the lateral ventral pallidum, which receives its afferents from the core, lateral shell and lateral tubercle, provides its projections to both the lateral VTA and the medial substantia nigra pars compacta/reticulata (Zahm, 1989; Groenewegen et al., 1993). Thus, signals elaborated in the ventromedial and ventrolateral striatum also reach the meso-dorsal striatal dopamine system, via the pallido-mesencephalic systems.

The dorsolateral ventral pallidum, which receives its afferent from the core, sends few efferent to the mediodorsal thalamus (Zahm et al., 1996). However, the ventrolateral ventral pallidum, which receives its afferents from the lateral shell and lateral tubercle, sends its efferent to the central and lateral segments of the dorsomedial thalamic nucleus (Groenewegen, 1988; Ray and Price, 1992; Zahm et al., 1996; O’Donnell et al., 1997). The central segment projects to the ventral agranular insular cortex (Groenewegen, 1988; Ray and Price, 1992), which in turn projects back to the lateral shell and lateral tubercle. The lateral segment, on the other hand, projects to the cingulate cortex, which in turn projects to the dorsal striatum, resulting in splitting signals.

Via these striato-mesencephalic, striato-pallido-mesencephalic and striato-pallido-thalamo-cortical split circuits, signals conducted in the basal ganglia including the meso-ventromedial and ventrolateral striatal dopamine systems eventually reach the primary motor and premotor cortices, which control rational, intelligent and cognition-guided voluntary movements; this motor system is called “the rational motor system”. This circuitry is consistent with the notion of the limbic/motor interaction held by Nauta and his colleagues (Nauta and Domesick, 1978; Nauta et al., 1978) and the motivation-to-action concept of Mogenson and his colleagues (Mogenson et al., 1980), in that emotional information elaborated by the ventral striatum influences cognition-guided movements.

The purpose of this section is to provide a conceptual framework for understanding how the meso-ventromedial and ventrolateral striatal dopamine systems play roles in motivated behavior, especially drug reward* (asterisks indicate technical terms that are described in Table 2). The present theoretical framework is a product of synthesis of previous incentive motivation* hypotheses (Bindra, 1968; Trowill et al., 1969; Bolles, 1972, Panksepp, 1982; Stewart et al., 1984; Fibiger and Phillips, 1986) and related perspectives (Hebb, 1955; Schneirla, 1959; Glickman and Schiff, 1967). Specifically, the present paper adopts the perspective that the meso-striatal dopamine systems are part of the set of coordinated neuronal mechanisms that allows organisms to find biologically important stimuli from the environment, to promote and sustain life (SEEKING or expectancy/foraging system) (Panksepp, 1982; 1998). This perspective has been applied in a review of the functional roles of nucleus accumbens dopamine (Ikemoto and Panksepp, 1999).

Table 2 GLOSSARY

Scientific explanations of goal-directed learning began with seminal studies by Thorndike (1898; 1911). He introduced the notion of “the law of effect” to explain how animals learn to acquire adaptive responses when they are confronted with a new environmental situation. For example, a hungry rat placed in a novel, unthreatening environment would explore the environment; if the rat finds food, it would consume food. When this procedure is repeated, the rat would display seeking behavior ever more efficiently to obtain food in that environment. Thorndike (1911) proposed that “Of several responses made to the same situation, those which are accompanied or closely followed by satisfaction to the animal will, other things being equal, be more firmly connected with the situation, so that, when it recurs, they will be more likely to recur” (p. 244). In other words, when animals are challenged by a new situation, they display an “impulse to act” in a trial-and-error fashion, and some actions are followed closely in time by adaptive consequences which are then strengthened or stamped in, thus occurring more often in the future. It is not difficult to recognize a remarkable conceptual parallel between learning by Thorndike’s law of effect and evolution by Darwin’s variation-selection. However, this parallel largely escaped the attention of dominant learning theorists (e.g., Hull, 1943) for decades to come. This state of affairs undermined early investigation of how, in the beginning of learning sessions, a variety of responses occur at high frequencies, so that some responses would be “strengthened” by consequences. Instead, goal-directed learning had been explained with the concept of reinforcement* as the major mechanism supplemented by motivational concepts such as drive. Skinner (1938), who played a key role in establishing instrumental (operant) conditioning as a scientific paradigm, proposed that “If the occurrence of an operant [instrumental response] is followed by presentation of a reinforcing stimulus, the strength is increased” and “If the occurrence of an operant already strengthened through conditioning is not followed by the reinforcing stimulus, the strength is decreased” (p. 21).

Decades later, Skinner (1981) suggested reinforcement as Darwinian selection operating at the level of behavior. He wrote, “Selection by consequences is a causal mode found only in living things, or in machines made by living things. It was first recognized in natural selection, but it also accounts for the shaping and maintenance of the behavior of the individual and the evolution of cultures” (p. 501). But Skinner left out variation, which must come before selection. Staddon and Simmelhag (1971) suggested, 10 years earlier than Skinner, a concept that unifies motivation and learning, and stated that “both evolution and learning can be regarded as the outcome of two independent processes: a process of variation that generates either phenotypes, in the case of evolution, or behavior, in the case of learning; and a process of selection that acts within the limits set by the first process” (p. 19). Staddon and Simmelhag’s variation* and selection* hypothesis of goal-directed learning provide more seamless explanations than the concept of reinforcement for learning and other anomalous learning phenomena such as “superstition” and “autoshaping”.

In his influential study, Skinner (1948) argued that response-independent deliveries of food to hungry animals are sufficient to induce instrumental conditioning, a phenomenon he called “superstition”. In his study, pigeons were maintained at 75% of their original body weights and, thus, were presumably highly motivated for food at the time of testing. They received a piece of food every 15 seconds, and food deliveries were independent of their behavior. Overtime, they developed certain specific responses that were displayed just before food delivery. Skinner accounted for this with the idea of reinforcement, arguing that the animals adventitiously made the association between their responses and food delivery and, hence, increased responses. His explanation may partly account for this phenomenon, but is at best incomplete because it does not explain how this procedure so reliably induced adjunctive behavior. After all, the animals did not have to do anything to get food. Staddon and Simmelhag’s variation-selection hypothesis explains that intermittent scheduled delivery of food elicited responses via the variation process. Some of those responses were then selected or reinforced via the selection process.

Another behavioral phenomenon as intriguing as superstition is “autoshaping” or “sign-tracking” (Brown and Jenkins, 1968, Hearst and Jenkins, 1974), initially discovered in pigeons. When a key is illuminated just before food is delivered to food-restricted, experimentally naïve pigeons, they quickly learn to peck at the illuminated key, even though food delivery is response-independent. This phenomenon cannot be readily explained by reinforcement, because animals in an autoshaping procedure keep responding at the illuminated key even when responding at the key actually prevents food delivery (Williams and Williams, 1969). According to Staddon and Simmelhag’s variation-selection hypothesis, intermittent delivery of food in hungry animals triggers various responses via the variation process. In addition, salient cue illumination that predicted food delivery guided (or selected) state-appropriate responses, in this case, pecking via the selection process. Thus, the variation-selection hypothesis offers an explanation for increased responses in which the delivery of reward is not dependent on response-contingency*.

Behavioral variation concerns diversity in behavior. It is not appropriate, however, to surmise that environmental demands generate any variation in physical movements, because behavior of organisms is constrained by phylogeny (e.g., Breland and Breland, 1961), just like evolutionary variation. It would be more appropriate to think of variation as increased episodes of unconditioned responding that are elicited by the perception of novelty or uncertainty in the environment. As a result of variation, some behavioral episodes that lead to significant consequences will be strengthened, and others that do not lead to adaptive consequences will be inhibited by selection mechanisms, leaving adaptive ones to recur.

4.1.1. Variation-selection hypothesis of striatal functional organization The meso-dorsal striatal (i.e., nigro-striatal) dopamine system has long been thought to control behavioral selection (Robbins, 1976; Cools, 1980; Wickens, 1993; Mink, 1996; Yin and Knowlton, 2006). Based on the hierarchical organization summarized in Figure 13, the variation-selection hypothesis of goal-directed learning (Staddon and Simmelhag, 1971), and behavioral data reviewed below, it is argued that the striatal complex plays a key role in both behavioral variation and selection. In particular, the meso-ventromedial striatal dopamine system appears to participate in the variation process, generating unconditioned responding. The meso-ventrolateral striatal dopamine system, in addition to the meso-dorsal striatal dopamine system, appears to participate in behavioral selection by modulating associative learning. The present proposal of the variation-selection hypothesis is a further elaboration of the concept of flexible and habit response systems generated to explain functional roles of nucleus accumbens and dorsal striatal dopamine (Ikemoto and Panksepp, 1999).

Discussions of the roles played by the dopamine systems in variation and selection are preceded by discussion of the methodological issues involved in investigating behavioral functions and functional organization of the nucleus accumbens-olfactory tubercle complex.

The compensation issue is minimal in temporal neuronal modulations induced by microinjection of drugs, because behavioral tests typically follow immediately after microinjections. Microinjection procedures can modulate selective neuronal communication when ligands of selective receptors are used. Disadvantages of this technique (for extended discussion, see Ikemoto and Wise, 2004) include the difficulty of identifying the exact regions of drug action, because drugs diffuse as soon as they are microinjected into a site. Thus, it is essential to address the issue of anatomical specificity. The most effective approach in freely moving animals may be to perform control injections into neighboring regions when functions of selective regions are being investigated. In addition, effects induced by microinjections of drugs are transient, typically lasting for only a few minutes. Whether or not this property of the method is helpful or not depends on tasks and research questions.

Because dopamine signals fluctuate phasically and tonically (see Table 1 in Ikemoto and Panksepp, 1999), it is reasonable to hypothesize that phasic dopamine signals have different functional roles than tonic signals. No consensus has been reached as to how exactly phasic and tonic dopamine signals should be defined; in this paper, a phasic dopamine signal is defined as a brief increase (lasting up to 2 sec) in dopamine concentration (or “transient”) in terminal regions, which is thought to result from an episode of burst firing of a neuron (temporal summation) (Gonon, 1988; Wightman and Zimmerman, 1990; Suaud-Chagny et al., 1992) or simultaneous firing of multiple cells projecting to the same target neurons (spatial summation). A tonic signal is defined as a slow change in concentration, lasting from tens of seconds to hours to days or longer (for another definition, see Grace, 1995; Goto and Grace, 2005). Phasic changes, which are temporally and spatially selective, may play a critical role in learning, because in most cases associative learning depends on temporal contiguity*. Tonic changes may play a critical role in motivation and affect*, because motivation and affect concern tonic states. Indeed, tonic signals in the ventral striatum as measured by microdialysis procedures appear to correlate with hyper-activity effects as indicated by locomotion (Sharp et al., 1987; Steinpreis and Salamone, 1993). The current literature on function does not offer sufficient information to address phasic and tonic issues; the last section of this paper will offer some suggestions for future investigation.

Table 3 summarizes popular research tools and their effectiveness for the investigation of functional roles of phasic and tonic dopamine signals. Although not neurotransmitter-selective, brief electrical stimulation administered at a frequency of 10 Hz or higher could mimic an episode of burst firing of dopamine neurons, which results in transients (Gonon, 1988; Wightman and Zimmerman, 1990; Suaud-Chagny et al., 1992; Marinelli et al., 2006). Injections of dopaminergic agents such as amphetamine and cocaine may mimic phasic increase in dopamine levels. Microdialysis techniques are useful for detecting tonic, but not phasic, changes, whereas the fast-scan cyclic voltammetry or chronoamperometry are useful for detecting phasic changes in dopamine levels (Wightman and Robinson, 2002).

Table 3 Research tools and their capacities to study phasic or tonic changes

Microinjections of cocaine into the medial shell elicit more robust forward locomotion and rearing than cocaine injections into the lateral part of the accumbens shell (Ikemoto, 2002). As mentioned above (section 1.2), rats readily learn to self-administer cocaine or amphetamine into the medial shell, but not the lateral shell (Ikemoto, 2003; Ikemoto et al., 2005). As for the olfactory tubercle, microinjections of cocaine into the medial olfactory tubercle elicit locomotion and rearing more effectively than those into the lateral tubercle (Ikemoto, 2002), and rats readily learn to self-administer cocaine or amphetamine into the medial tubercle, but not the lateral tubercle (Ikemoto, 2003; Ikemoto et al., 2005). These behavioral data suggest functional differences between the medial shell and lateral shell and between the medial tubercle and lateral tubercle, and functional similarity between the medial shell and medial tubercle. The data are therefore consistent with the two dopamine systems proposed here (Fig. 11). The following review focuses on functional studies of the nucleus accumbens, but not the olfactory tubercle, because the behavioral functions of the olfactory tubercle have not received much research attention.

Fibiger and Phillips (1986) suggested a close relationship between conditioned reinforcement and incentive motivation. They stated “Incentive motivational stimuli therefore have both activational and directional or cue features” and “It is possible that the mesolimbic DA system is concerned with mediating the activational or energizing properties of such stimuli” (p. 667).

In addition to such energizing effects, particularly with the presence of incentive stimuli*, dopamine in the accumbens plays an important role in incentive learning (Ikemoto and Panksepp, 1999). For example, McFarland and Ettenberg (1995) provided particularly strong evidence for dopamine’s role in incentive learning using the dopamine antagonist haloperidol (although this study does not indicate brain regions where the dopamine antagonist acts). Rats were trained to run down a straight runway to a goal box in which they received intravenous heroin reward. Olfactory cues in the runway predicted whether responding would be rewarded with the drug. Rats learned to run fast when a cue paired with the reward was present and run slowly when another cue paired with no reward was present. Pretreatment with a moderate dose of haloperidol in these rats did not significantly change running speeds, i.e., the motivation for the drug. However, when these rats were tested again 24 hours later without haloperidol treatment, the rats that had received the reward cue and the drug reward under the influence of haloperidol ran significantly slowly even with the presence of the reward cue. On the other hand, rats that had received the no-reward cue and no drug reward under the influence of haloperidol ran fast with the reward cue. These data strongly suggest that dopaminergic activity at the time of reward consumption is involved in incentive learning (for more evidence see Beninger, 1983); a predictive stimulus will no longer energize conditioned responding if dopaminergic activity was disrupted during previous consumption of the reward. In other words, dopamine is involved in memory consolidation of incentive representation. The experience of approaching and consuming the reward appears to make incentive representation of environmental knowledge labile. Activity levels of dopamine transmission at the time of or just after such experience appear to determine how incentive representation will be consolidated.

Dopamine in the nucleus accumbens is thought to play such a role, because blockade of dopamine receptors in the nucleus accumbens results in a similar deficit (Fig. 2 of Ikemoto and Panksepp, 1999). Also, dopamine receptor blockade in the accumbens disrupts a random foraging task: haloperidol-treated rats increase re-entries (errors) in an eight arm maze as they collect bait from four randomly baited arms. On the other hand, if rats can use information that they learned prior to receiving intra-accumbens haloperidol, no impairment is found in collecting bait from the eight arm maze (Floresco et al., 1996).

Since our review (Ikemoto and Panksepp, 1999), many studies have been conducted providing rich data indicating differential functions between the medial portion of the nucleus accumbens shell and the core in appetitive tasks (see Di Chiara, 2002; Kelley, 2004; Everitt and Robbins, 2005). The present review will focus on new studies that have addressed functional differences within the ventral striatum, and will try to address how the meso-ventromedial and ventrolateral striatal dopamine systems participate in invigoration of approach, incentive learning and drug reward.

The functional role of the meso-ventromedial striatal dopamine system is also characterized in part by the notion of “general drive state” that Hebb (1955) conceived. By drive, Hebb meant “an energizer, but not a guide; an engine, but not steering gear” (p. 249). In the present paper, such affective and drive states of mind/body interaction modulated by the meso-ventromedial striatal dopamine system is referred to as “action-arousal*”. The meso-ventromedial striatal dopamine system conducts signals from the limbic system, which detects significant changes in the environment with respect to self-preservation and procreation (MacLean, 1990). This dopamine system is sensitized by regulatory imbalances (e.g., hunger) and activated when animals detect incentive stimuli and especially when procurement of reward is uncertain.

4.5.1. Action-arousal in unconditioned contexts: behavior Behavioral variation, a process that generates unconditioned responding, is normally triggered by the perception of novelty, opportunity, danger or uncertainty. The induction of this process appears to be mimicked by the activation of the meso-ventromedial striatal dopamine system. Administration of drugs into the posteromedial VTA or the central linear nucleus, which activates dopaminergic projections to the ventromedial striatum, elicits heightened locomotion and rearing, behaviors that enable organisms to interact with the environment. Locomotor activity is elicited by rewarding treatments such as microinjections of the cholinergic receptor agonists carbachol (Ikemoto et al., 2003) or cytisine (Museo and Wise, 1990), NMDA receptor agonists (Ikemoto, 2004), μ-opiate receptor agonists (Joyce et al., 1981; Zangen et al., 2002) or the cannabinoid receptor agonist delta-9 THC (Zangen et al., 2006) into the posteromedial VTA, or the GABA_A receptor agonist muscimol into the central linear nucleus (Klitenick and Wirshafter, 1988). Thus, these observations are consistent with the role of the meso-ventromedial striatal dopamine system in action-arousal and behavioral variation. The significance of these findings is that the activation of the meso-ventromedial striatal dopamine system elicits unconditioned responses.

Administration of dopaminergic drugs such as cocaine and amphetamine into the medial accumbens shell or the medial olfactory tubercle elicits forward locomotion and rearing in rats (Ikemoto, 2002, Ikemoto and Witkin, 2003). Ventrolateral striatal dopamine may also participate in locomotion, although the data regarding this are not so consistent. Microinjections of dopamine or amphetamine into the accumbens core can induce heightened locomotion comparable to that induced by microinjections of the same substances into the medial shell (Johnson et al., 1996; Swanson et al., 1997; Ikemoto, 2002, Ikemoto and Witkin, 2003). Interestingly, 6-OHDA lesions of dopaminergic terminals in the core, but not the medial shell, disrupt locomotor activity induced by intravenous psychomotor stimulants (Boye et al., 2001; Sellings and Clarke, 2003; Sellings et al., 2006a; 2006b). On the other hand, excitotoxic lesions of the core enhance locomotor activity induced by systemic amphetamine or cocaine (Parkinson et al., 1999; Ito et al., 2004), suggesting an inhibitory role of the core in psychomotor stimulant-induced locomotion. Several studies showed that mixtures of D₁- and D₂-like dopamine receptor agonists microinjected into the medial shell or medial tubercle, but not the core, increased locomotor activity (Swanson et al., 1997; Choi et al., 2000; Ikemoto, 2002). These data raise a question as to how exactly core dopamine is involved in locomotion. Nevertheless, heightened activity of dopamine in ventromedial striatum appears to lead to forward locomotion and rearing in rats, observations that are thought to reflect an action-arousal state or general drive state. The next section reviews the evidence that ventromedial striatal dopamine also energizes conditioned responses*.

4.5.2. Action-arousal in conditioned contexts: behavior Activation of the meso-ventromedial striatal dopamine system appears to energize conditioned responses in the environment where extensive conditioning has taken place. These effects have been shown using cocaine reinstatement models. In this procedure, rats are first trained to lever-press for intravenous cocaine administration. When the animals learn to lever-press for cocaine over a few weeks, lever-pressing is extinguished by saline infusions instead of cocaine over several days (i.e., extinction). It should be noted that the extinction of conditioned responses is not erasure of what was previously learned, but is new learning. In other words, in this procedure animals first learn excitatory conditioning over selective response and then inhibitory conditioning over the same response, leading to no expression of conditioned responding. The balance between excitatory and inhibitory conditioning can be tipped by certain manipulations. For instance, in animals that have extinguished conditioned responding (lever-pressing), systemic administration of cocaine reinstates the conditioned responding, and this cocaine-induced reinstatement is blocked by the administration of dopamine receptor antagonists into the medial shell, but not the core (McFarland and Kalivas, 2001; Anderson et al., 2003; Bachtell et al., 2005; Anderson et al., 2006). Furthermore, microinjections of cocaine or dopamine receptor agonists directly into the medial shell, but not the core, reinstate lever-pressing (Schmidt et al., 2006). Reinstatement of responding is selective, because these animals only reinstate responding on the active lever, but not the inactive lever. Thus, these results suggest that the medial accumbens shell, but not the core, exerts excitatory control over conditioned responding. Findings from a conditioned reinforcement study are also consistent with microinjection data. The vigor of conditioned reinforcement induced by intra-accumbens administration of amphetamine is diminished following selective excitotoxic lesions of the medial shell, but not the core (Parkinson et al., 1999).

However, these findings from instrumental tasks allow an alternative interpretation: increased activity of ventromedial striatal dopamine merely energizes learned motor-habits, but not motivation or action-arousal, because conditioned stimuli are learned through instrumental tasks. This motor habit hypothesis is no longer viable because of the data obtained by Wyvell and Berridge (2000). They trained rats in two distinct learning procedures and first gave an instrumental conditioning* task in which they had to lever-press to receive sucrose pellets, and then trained rats in a Pavlovian conditioning* task in which a light cue was conditioned with the delivery of sucrose pellets. Thus, conditioned stimulus (light cue) was learned in a Pavlovian task, but not an instrumental task. The test session was carried out in an extinction* procedure in which lever-presses did not deliver the reward. Amphetamine microinjections into the medial shell resulted in heightened conditioned responding (active lever-pressing but not inactive lever-pressing) upon the presentation of a Pavlovian conditioned stimulus, but not upon presentation of a control stimulus or the absence of a stimulus. This and related studies (Wyvell and Berridge, 2000; 2001; Pecina et al., 2006) demonstrate that increased activity of intra-shell dopamine does not merely enhance motor-habits, but also enhances motivation for rewards, a finding consistent with the notion of action-arousal or drive.

Considering the role of ventromedial striatal dopamine in unconditioned contexts, acute heightened activity of ventromedial striatal dopamine appears to energize responding without directing it (Hebb’s general drive state). The goal-directed nature of conditioned responding appears to be exerted by other selection mechanisms such as the accumbens core and dorsal striatum; this point is further elaborated in sections 4.6.3 and 4.9.5. This hypothesis is consistent with the suggestion by Everitt and his colleagues that “The NAc shell can thus serve to amplify the expression in behavior of information flowing through the NAc core” (p. 395, Ito et al., 2004).

4.5.3. Action-arousal in unconditioned contexts: vocalization Heightened dopaminergic activity in the ventromedial striatum not only energizes somatic motor expressions, but also appears to induce an emotional state. Microinjections of amphetamine into the medial shell, more readily than the core, elicit high frequency (around 50-Hz) ultrasonic vocalizations (Burgdorf et al., 2001; Thompson et al., 2006), which have been implicated in positive motivational states in rats (Knutson et al., 2002).

4.5.4. Action-arousal in unconditioned contexts: physiological measures The activation of the meso-limbic dopamine system also leads to physiological responses consistent with the notion of action-arousal. Overall, such physiological responses resemble those triggered by mild stress. Electrical brain stimulation at the medial forebrain bundle/VTA is rewarding and triggers dopamine release in the ventral striatum (Fiorino et al., 1993; Garris et al., 1999; Cheer et al., 2005; Hernandez et al., 2006) and investigatory responses of sniffing (Clarke and Trowill, 1971; Ikemoto and Panksepp, 1994). Such electrical brain stimulation also increases blood pressure, an effect that is blocked by pretreatment with dopamine antagonists or 6-OHDA (Spring and Winkelmuller, 1975; Tan et al., 1983; Burgess et al., 1993; Cornish and van den Buuse, 1994), suggesting dopamine mediation. This electrical brain stimulation not only elicits reward, investigatory responses and heightened blood pressure, but also heightened norepinephrine, epinephrine and glucocorticoid levels in the plasma of rats (Burgess et al., 1993), characterized as a set of sympathetic arousal responses. Increased blood pressure is also observed after microinjections of the substance P analog DiMe-C7 into the VTA and is abolished by pretreatment with systemic dopamine antagonists (Cornish and van den Buuse, 1995). These physiological responses appear to be more readily triggered by the activation of the meso-ventromedial striatal dopamine system rather than other dopamine systems, because microinjections of cocaine or dopamine receptor agonists into the ventromedial striatum increase plasma glucocorticoid levels more effectively than microinjections of the same drugs into the medial prefrontal cortex or dorsal striatum (Ikemoto and Goeders, 1998).

Consideration of the natural environment in which wild mammals live may be helpful for understanding the relationships between two apparently distinct effects of sympathetic arousal and reward triggered by the activation of the meso-ventromedial striatal dopamine system. For example, think of hungry lions that have just spotted a zebra cub in distance. Heightened blood pressure and circulatory levels of “stress” hormones enable lions to maintain energy homeostasis in the event of vigorous physical activity and, thereby, ensure the completion of predatory pursuit leading to the procurement of their prey. Glickman and Schiff (1967) suggested that the engagement of approach such as predatory behavior or the activation of neuronal mechanisms underlying such approach is rewarding. From this perspective, it is not surprising that common neuronal mechanisms underlie both stress responses, such as those of lions getting ready for predatory action, and reward.

Stress-related physiological responses, stimulated by the activation of the meso-ventromedial striatal dopamine system, appear to regulate the dopamine systems (see Marinelli, 2007). Certain stressful stimuli appear to selectively activate the meso-ventromedial striatal dopamine system. Exposure to foot shock increases extracellular dopamine levels in the medial shell, but not the core (Kalivas and Duffy, 1995). Interestingly, extracellular basal dopamine levels in the medial shell, but not the core, decrease after adrenalectomy, which diminishes plasma glucocorticoids (Barrot et al., 2000). Moreover, diminishment of glucocorticoids induced by adrenalectomy blunts increased dopamine concentration in the medial shell, but not the core, induced by morphine or cocaine administration (Barrot et al., 2000). Indeed, previous data suggest that stress hormones play a critical role in the reinstatement of drug seeking (Shalev et al., 2002) and the rewarding effects of psychomotor stimulants (Sarnyai et al., 2001; Goeders, 2002; Marinelli and Piazza, 2002). These data suggest that dopamine in the ventromedial striatum is more responsive to stressful stimuli than dopamine in the ventrolateral striatum. Overall, the meso-ventromedial striatal dopamine system and some stress responses appear to interact reciprocally. Such relationships are consistent with the notion of states of mind/body interaction; both heightened ventromedial striatal dopamine and stress-related physiological responses are coordinated to help organisms to actively interact with the environment for survival.

4.5.5. Action-arousal: possible mechanisms These coordinated effects at behavioral, psychological and physiological levels induced by the activation of the meso-ventromedial striatal dopamine system appear to be mediated, in part, by the medial ventral pallidum, the major recipient of the outputs from the ventromedial striatum (section 3.1). Heightened locomotion induced by intra-accumbens dopamine is facilitated by microinjections of the GABA_A receptor antagonist picrotoxin and attenuated by GABA injections into the vicinity of medial ventral pallidum (Jones and Mogenson, 1980). In addition, hyper-sensitive locomotion to systemic administration of the dopamine receptor agonist apomorphine following 6-OHDA lesions of the nucleus accumbens is attenuated by injections of the GABA_A receptor agonist muscimol into the vicinity of the medial ventral pallidum (Swerdlow and Koob, 1984) or excitotoxic or electrolytic lesions of the region (Swerdlow et al., 1984a; 1984b). Moreover, excitotoxic lesions of the vicinity of the medial ventral pallidum appear to attenuate the rewarding effects of intravenous administration of cocaine or heroin (Hubner and Koob, 1990). Therefore, these data are consistent with the idea that the medial ventral pallidum plays a major role in mediating the outputs from the ventromedial striatum.

The medial ventral pallidum sends its efferents to the medial mediodorsal thalamic nucleus, the lateral hypothalamic area (Zahm, 1989; Groenewegen et al., 1993; O’Donnell et al., 1997) and the midbrain extrapyramidal area, dorsomedial to the pedunculopontine tegmental nucleus (Swanson et al., 1984; Rye et al., 1987). Although data from some studies suggest that one region is more important in locomotion than others, all of these regions appear to be involved in locomotion and possibly action-arousal. Lesions of the medial mediodorsal thalamic nucleus, but not the midbrain extrapyramidal area, attenuate hyper-sensitive locomotion to apomorphine following 6-OHDA accumbens lesions (Swerdlow and Koob, 1987). In addition, microinjections of GABA receptor agonists muscimol or baclofen into the mediodorsal thalamic nucleus elicit locomotion in rats (Churchill et al., 1996). However, microinjections of the local anesthesia procaine into the midbrain extrapyramidal area, but not the medial mediodorsal thalamic nucleus, attenuate heightened locomotion induced by picrotoxin injections into the ventral pallidum (Mogenson and Wu, 1988). Moreover, heightened locomotion induced by amphetamine injections into the nucleus accumbens is also attenuated by procaine injections or excitotoxic lesions of the midbrain extrapyramidal area (Brudzynski and Mogenson, 1985).

Additional data confirm that both mediodorsal thalamic nucleus and midbrain extrapyramidal area are involved in locomotion and suggest that distinct mechanisms exist within the ventral pallidum projecting between these regions, to mediate locomotion. Microinjections of the μ-opiate-receptor agonist DAMGO or the glutamate receptor agonist AMPA into the ventral pallidum elicit locomotion. Locomotion induced by intra-pallidal DAMGO is blocked by procaine microinjections into the mediodorsal thalamic nucleus, but not the midbrain extrapyramidal area, whereas locomotion induced by intra-pallidal AMPA is blocked by procaine microinjections into the midbrain extrapyramidal area, but not the mediodorsal thalamic nucleus (Churchill and Kalivas, 1999). The medial mediodorsal thalamic nucleus may send arousal signals to the prelimbic and infralimbic prefrontal cortices, which then relay them back to the ventromedial striatum and to the lateral hypothalamic area. The midbrain extrapyramidal area, which may conduct arousal information, sends its efferents to the spinal cord (Swanson et al., 1984) (apparently to modulate motor processes), the VTA (Klitenick and Kalivas, 1994) (completing a circuit) and the lateral hypothalamic area.

The action-arousal state may be regulated by a network of nuclei or cell assemblies localized from the spinal cord to the telencephalon. A key structure of the network may be the lateral hypothalamic area, which receives innervation from the medial ventral pallidum and ventromedial striatum and from the infralimbic prefrontal cortex and midbrain extrapyramidal area. Recent data suggest that the lateral hypothalamic area contains hyporcretin/orexin (de Lecea et al., 1998, Sakurai et al., 1998) neurons (Peyron et al., 1998), which appear to play a critical role in arousal (Siegel, 2004; Saper et al., 2005) and probably reward (Harris et al., 2005; Harris and Aston-Jones, 2006).

The lateral hypothalamic area belongs to the phylogenically old brain structure referred to as “isodendritic core” (or “reticular formation”), as do the ventral pallidum and VTA (Leontovich and Zhukova, 1963; Ramon-Moliner and Nauta, 1966; Geisler and Zahm, 2005). The isodendritic core is a type of neuronal tissue that consists of “isodendritic” neurons, characterized as long, straight, thick, poorly ramified dendrites and long axons with many collaterals and poorly ramified terminals. The dendritic field of an isodendritic cell is extensive and overlaps with those of other isodendritic cells, forming a continuum localized in the core of the central nervous system from the spinal cord to the telencephalon (Leontovich and Zhukova, 1963; Ramon-Moliner and Nauta, 1966). The isodendritic neurons’ morphological features enable them to receive extensive afferents and, thus, appear to be optimal for integrating a variety of inputs, including strong inputs from protopathic somato-visceral sensibility (Leontovich and Zhukova, 1963). In other words, the lateral hypothalamic area is reciprocally connected with the brain regions that conduct information on visceral and somatic outputs characterized as emotional (Bandler et al., 1991; Holstege, 1991; Nieuwenhuys, 1996; Saper, 2002). Electrical brain stimulation along the pathways of these structures elicits sniffing in anesthetized rats (Ikemoto and Panksepp, 1994) and a set of physiological responses characterized as a heightened sympathetic state (Hilton, 1982; Yardley and Hilton, 1986), effects that resemble those elicited by the stimulation of the VTA, a recipient of extensive afferents from other isodendritic core regions (Geisler and Zahm, 2005). Thus, the lateral hypothalamic area may control the autonomic nervous system via the connections to the nucleus of the solitary tract and parabrachial nucleus (Saper, 2002; 2004), endocrine secretion via the connection to the periventricular hypothalamus (Swanson, 1987), and emotional movements via the connection to the periaqueductal gray and the midbrain extrapyramidal area (Bandler et al., 1991; Holstege, 1991; Nieuwenhuys, 1996).

Therefore, an extensive network of brainstem regions may mediate action-arousal modulated by the meso-ventromedial striatal dopamine system. Such a network organization would be consistent with observations that the rewarding effects of electrical brain stimulation at the medial forebrain bundle/lateral hypothalamic area are not readily abolished by lesions of the tissues just anterior or posterior to the stimulating electrodes (e.g., Janas and Stellar, 1987). Although precise circuitry regulating the action-arousal state is not clear at this time, the lateral hypothalamic area and its associated brainstem isodendritic core regions are tentatively referred to as the “action-arousal system”. The activation of this global system leads to not only motor arousal but also emotional and cognitive arousal or vigilance (via the activation of the thalamus and cortices), which enables the organism to learn about the environment in relation to itself (Hebb, 1955).

4.6.1. Action-outcome and stimulus-response association Instrumental learning tasks have been found to depend on two basic processes (Yin and Knowlton, 2006), in addition to those typically involved in Pavlovian learning. Action-outcome* learning consists of encoding in memory the relationship between animals’ actions and the value of their outcomes (Dickinson and Balleine, 1994). As a result of this learning, rats can initiate an action while anticipating the outcomes of the action from previous experience. This learning process is thought to be partly mediated by the dorsomedial striatum (or the caudate) (Yin et al., 2005a; 2005b). The other process is stimulus-response* or habit learning, which involves encoding the relationship between environmental stimuli and responses. Over repeated trials of learning, stimuli alone automatically elicit fixed, adaptive patterns of responses, i.e., goal-directed habit. This learning process appears to be mediated, in part, by the dorsolateral striatum (or the putamen) (Knowlton et al., 1996; White, 1997; Graybiel, 1998; Hikosaka, 1998; Yin and Knowlton, 2006). After weeks or months of the experience of self-administration of drugs, characters of drug self-administration become more and more persistent and extinction-resistant in rats (Deroche-Gamonet et al., 2004; Vanderschuren and Everitt, 2004). These observations are thought to be, in part, a consequence of abnormal stimulus-response learning (Everitt and Robbins, 2005). Such functional divisions of the dorsal striatum are consistent with recent analyses of the connectivity of the striatum (Haber, 2003; Voorn et al., 2004).

4.6.2. Stimulus-outcome association and its performance Pavlovian conditioning involves repeated parings between predictive stimuli and outcomes. As a result, predictive stimuli acquire the affective properties of outcomes (Rescorla, 1988). Hence, Pavlovian conditioning depends in part on stimulus-outcome association*, which enables animals to retrieve the representation of an outcome upon the presentation of a predictive stimulus. The formation of stimulus-outcome association is evident when the presentation of stimuli that have previously elicited no response now elicits responses (i.e., conditioned responses). Therefore, Pavlovian conditioning also involves the execution of actions upon the presentation of conditioned stimuli. The evidence indicating different neural mechanisms for these two processes of Pavlovian conditioning will be discussed later.

To better understand dopamine’s role in Pavlovian conditioning and drug reward, two points concerning the nature of unconditioned stimuli and conditioning are elaborated here. Food nutrition is an affective outcome (or unconditioned stimuli), which triggers association with stimuli. Sight, smell and other senses of food may not be affective outcomes until they acquire affective properties of nutrients contained in food over the repeated experience of consuming it (the notion of “cathexis” by Tolman, 1949). For example, the sight of a banana may be used as an unconditioned stimulus, but will not trigger affective responses unless the animal has consumed it before. Unlike the consumption of nutrients via food, drug administration could bypass the five senses when administered to animals via the intravenous or intracranial routes (for discussion, see Wise, 2002). Thus, such drug administration lacks intrinsic sensory properties. However, environmental stimuli associated with drug administration will acquire the affective properties of drugs.

In addition, Pavlovian conditioning appears to be a collective name for a set of heterogeneous phenomena. For example, conditioned autonomic responses such as heart rate change appear to be learned via different mechanisms than those for conditioned somatic responses such as eye-blinks (Kao and Powell, 1988). Unlike most types of Pavlovian learning, in which temporal contiguity plays an important role, conditioned taste aversion (the association of illness with food-intake) can be established even though a predictive stimulus associated with food intake occurs hours before the aversive effects of an unconditioned stimulus (Rozin and Kalat, 1971). In addition, the same unconditioned stimulus can elicit conditioned responses via different mechanisms under some circumstances. Cocaine administration appears to induce conditioned responses via different mechanisms, depending on how the drug is delivered (intravenous vs. intraperitoneal). These points will be elaborated later on in section 4.9.2. The focus of the present paper is Pavlovian conditioning mediated via dopaminergic mechanisms.

The conditioned place preference (or conditioned cue preference) procedure, a variant of Pavlovian procedures, is commonly used to measure the rewarding properties of drugs in rats and other animals. Conditioned place preference induced by drugs or other stimuli involves two phases. The first, or acquisition, phase involves pairing of a compartment (place cues) with rewards like drug administration and another compartment with no reward. Following such pairings, animals are thought to associate cues present in the reward-paired compartment with some affective properties of rewards (Carr et al., 1989). The testing or performance phase involves choice behavior between the compartments. The reward-paired compartment is thought to attract animals more than the non-drug compartment, because of its affective properties acquired through association. Thus, conditioned place preference performance involves retrieval of affective information for action and has been found to involve stimulus-outcome association (Perks and Clifton, 1997; Yin and Knowlton, 2002). Because the performance phase is conducted during extinction, conditioned place preference performance does not involve other types of association such as action-outcome and stimulus-response associations.

Because its conditioning and performance phases take place separately, conditioned place preference tests allow experimenters to distinguish neuronal mechanisms involved in stimulus-outcome learning from those involved in performance guided by stimulus-outcome association. Indeed, it has been shown that the acquisition and performance phases of conditioned place preference procedures are affected by different neuronal manipulations (e.g., Hiroi and White, 1991a; 1991b). On the other hand, most Pavlovian procedural tasks typically involve both conditioning and performance phases occurring simultaneously and, thereby, do not allow experimenters to examine them separately or pinpoint which basic process is affected by particular treatments. For example, when presentation of a light cue precedes the delivery of food by several seconds in a Pavlovian procedure, hungry rats learn to exhibit conditioned responding to the lighted cue. Because the conditioning phase in this typical Pavlovian task is not separated from the performance phase, it is difficult to investigate mechanisms involved in the stimulus-outcome component distinguished from the performance component. Moreover, one cannot be sure whether the stimulus-outcome association really controls conditioned responses in this procedure. Even if outcomes are not dependent on any responses, organisms could develop “anomalous beliefs” that their action produces outcomes, incidental action-outcome or stimulus-response associations, which could maintain the performance (Hearst and Jenkins, 1974).

4.6.3. Hypothesis on associative roles of striatal regions in selection Figure 14 summarizes the roles of striatal regions in associative processes. Although evidence is scant at this time, to facilitate research, explicit, easily falsifiable functions are suggested. The ventromedial striatum and its dopamine are suggested to be involved in stimulus-outcome learning, while the ventrolateral striatum, particularly that of the nucleus accumbens core, is involved in selection of stimulus-appropriate responding, a process that is referred to as stimulus-action* association. Here, a distinction is made between stimulus-action and stimulus-response associations. The term action signifies that performance depends on a stimulus-outcome association, while the term response signifies that performance does not depend on such association. Core dopamine may be important for the acquisition of stimulus-action association, i.e., conditioned response learning dependent on stimulus-outcome association.

Figure 14 The variation-selection hypothesis of striatal functional organization. Striatal regions, shown on top, are divided into four regions: the ventromedial (VM), ventrolateral (VL), dorsomedial (DM), and dorsolateral (DL) striatum. Stimulus-outcome (S→O) (more ...)

These associative processes participate in behavioral selection. Stimulus-outcome association is proposed to be the foundation for selection via stimulus-action, action-outcome and stimulus-response associative mechanisms through the hierarchical organization of the striatal complex described in Figure 13. The variation-selection processes are envisioned to generate adaptive responding, from unconditioned to stimulus-dependent to outcome-dependent to habit responding, as organisms adapt to a new environment. It should be emphasized, however, that these associative processes critically depend on the interactions of striatal regions with various other structures such as the limbic (amygdala, hippocampus, prefrontal) cortices (Everitt et al., 1999; Everitt and Robbins, 2005; Goto and Grace, 2005), associative and sensory cortices (Haber, 2003; Yin and Knowlton, 2006).

4.7.1. Nucleus accumbens medial shell Limited evidence suggests that the meso-ventromedial striatal dopamine system plays a critical role in incentive learning, particularly that of stimulus-outcome association. Ventromedial striatal dopamine modulates the formation of incentive representation of the environment such that stimuli associated with the increase of ventromedial striatal dopamine gains an incentive motivation property. Microinjections of dopamine receptor antagonists into the medial accumbens shell, but not the core, during the acquisition phase disrupt conditioned place preference induced by systemic administration of nicotine or opiates (Fenu et al., 2006; Spina et al., 2006). Consistent with this finding, researchers have found that microinjections of amphetamine into the medial accumbens shell, but not the core, enhanced the acquisition of Pavlovian conditioned responding induced by conditioned stimuli signaling food deliveries (Phillips et al., 2003), and that 6-OHDA lesions of the medial shell or medial tubercle, but not the core, disrupted conditioned place preference involving psychomotor stimulants (Sellings and Clarke, 2003; Sellings et al., 2006a and 2006b). In addition, microinjections of the D₂ dopamine receptor agonist quinpirole into the posteromedial VTA just before the acquisition phase diminish conditioned place preference induced by food. This treatment inhibits dopamine activity and dopamine release in the ventromedial striatum. Microinjections of the same drug into the anterolateral VTA or substantia nigra, which respectively project to the ventrolateral striatum and dorsal striatum, are much less effective (Liu and Ikemoto, 2006). Importantly, the doses of quinpirole that diminish conditioned place preference for food do not influence food consumption and cannot alone induce conditioned place avoidance. Moreover, the same manipulation disrupts the acquisition of conditioned place avoidance induced by systemic administration of naloxone (Liu and Ikemoto, 2006). Therefore, these findings are consistent with the idea that the meso-ventromedial striatal dopamine system plays an important role in incentive learning, particularly learning linked to stimulus-outcome association.

Acute, moderate disruption of ventromedial striatal dopamine transmission, which impairs incentive learning, does not appear to interfere with performance based on a stimulus-outcome association. The blockade of dopamine receptors in the medial shell just before preference testing does not change conditioned place preference induced by nicotine or morphine administration, even though the same treatments during the acquisition phase disrupt the acquisition of conditioned place preference (Fenu et al., 2006; Spina et al., 2006). Therefore, once learning takes place, moderate blockade of ventromedial striatal dopamine receptors does not appear to influence performance based on stimulus-outcome learning or the retrieval of memories of stimulus-outcome association. This is not to say that ventromedial striatal dopamine is only involved in acquisition of stimulus-outcome association. Although adequate evidence is lacking at this time, it is reasonable to postulate that ventromedial striatal dopamine is involved in the re-organization of incentive representation or reconsolidation of incentive memory in stimulus-outcome association (section 4.9.4).

These findings are not always consistent with others, however. Selective excitotoxic lesions of the medial shell did not disrupt Pavlovian procedural learning in which approach responses were elicited by the presentation of conditioned stimuli paired with food (Parkinson et al., 2000; Corbit et al., 2001; Hall et al., 2001). Such data may be explained by compensatory mechanisms after lesioning. The loss of medial shell function may be compensated during the “recovery” period (typically a week or more) from surgery by remaining mechanisms including the medial olfactory tubercle, which serves similar functions to the medial shell.

4.7.2. Nucleus accumbens core The accumbens core appears to be involved in generating conditioned responding (or selection of adaptive responding) based on stimulus-outcome association. Dopamine in the nucleus accumbens core may be involved in learning such conditioned responding. The acquisition of instrumental tasks to obtain food is severely retarded when rats receive microinjections of D₁ receptor antagonists into the core immediately before learning sessions (Smith-Roe and Kelley, 2000). The same is true when rats receive the protein synthesis inhibitor anisomycin into the core, but not the shell or dorsolateral striatum, immediately after each learning session (Hernandez et al., 2002). The latter finding in particular is consistent with the idea that the accumbens core is involved in consolidation of memory concerning conditioned responding. These instrumental studies, however, do not pinpoint which basic processes are disrupted by the core manipulations.

Findings from excitotoxic lesion studies shed some light on this issue, and more specifically suggest that the core plays an important role in conditioned responding that is guided and reinforced by conditioned stimuli. Some inconsistency in finding should be mentioned first. Excitotoxic lesion studies (Corbit et al., 2001; Ito et al., 2004) found that core lesions have no detectable effect on the acquisition of a simple instrumental task; this lack of effects of core lesions appears to contradict the findings mentioned above (Smith-Roe and Kelley, 2000; Hernandez et al., 2002). This discrepancy may be explained by the difference in recovery periods given following permanent lesions, some of which may have resulted in some compensation. In any case, the core lesions do disrupt instrumental performance that is maintained by the presentation of cocaine-paired conditioned stimuli (Ito et al., 2004). Similarly, core-lesioned rats do not learn to perform instrumental tasks for food as effectively as sham-lesioned controls, when they have to extend responding guided by conditioned stimulus (Corbit et al., 2001). Moreover, microinjections of dopamine receptor antagonists into the core disrupt cocaine or food seeking maintained by conditioned stimuli (Bari and Pierce, 2005) and inactivation of the core by microinjections of GABA receptor agonists disrupts the reinstatement of cocaine seeking triggered by the presentation of a conditioned stimulus (Fuchs et al., 2004).

In addition, Corbit et al. (2001) showed that core lesions disrupt the so-called devaluation effect. When rats are trained to perform a task for a food and another task for a different food, and when they are pre-fed with one of the foods, intact rats will seek the unfed outcome more than fed outcome. When core lesioned rats, which had learned to perform these tasks, were pre-fed with one of the foods, they showed decreased responding for both tasks; or, they showed no selective responding. In contrast, after rats received extinction (no food) sessions instead of devaluation, core-lesioned rats showed selective responding with decreased responses for the extinguished task. These results suggest that the deficit induced by core lesions in the devaluation test is not due to impaired action-outcome association, but rather the inability to select adaptive responding based on conditioned stimuli informing values of the outcome.

Studies using Pavlovian procedural tasks suggest that the core is involved in consolidation and retrieval of information concerning selection of action. Dalley et al., (2005) injected the D₁ receptor antagonist SCH 23390 into the lateral nucleus accumbens (the core and lateral shell) immediately after a Pavlovian approach task with food in hungry rats, and found that this disrupted the acquisition of conditioned responding. SCH 23390 administration into the accumbens core does not disrupt stimulus-outcome association in conditioned place preference procedures (Fenu et al., 2006; Spina et al., 2006). Taken together, these data suggest that dopamine in the lateral nucleus accumbens is involved in memory consolidation concerning conditioned responses or stimulus-action association.

Using a conditioned place preference, Miller and Marshall (2005) showed that performance of conditioned place preference induced by intravenous cocaine administration results in activation of specific molecular signaling (ERK, CREB, Elk-1 and Fos) in the accumbens core, but not the shell. Furthermore, pharmacological blockade of this molecular signaling cascade in the core disrupted conditioned place preference performance immediately after the injection and up to 14 days later. Although it is not clear how this core manipulation may have affected stimulus-outcome association, these data are consistent with the hypothesis that the core plays a role in generating responses based on stimulus-outcome association. Although additional research is needed to substantiate these findings, the overall pattern of recent data suggests that the core is important for the acquisition and the selection of conditioned responding based on stimulus-outcome association.

The response-independent cocaine administration into the medial olfactory tubercle significantly increased lever-pressing (Fig. 15A and B). Comparison of the present data with those from the self-administration study (Fig. 15B vs. C), which involved virtually identical procedures with the same equipment, suggests that the levels of lever-pressing obtained with the response-independent schedule are strikingly similar to those obtained with the response-dependent schedule. These data suggest that intermittent injections of cocaine into the medial olfactory tubercle have marked arousal effects in rats and lead to heightened lever-pressing. As discussed in sections 4.5.1 and 4.5.2, cocaine administration into the medial tubercle appears to increase a general drive state or action-arousal, as it elicits locomotor activity and rearing (Ikemoto, 2002). In addition, high extracellular levels of ventromedial striatal dopamine appear to energize instrumental responding controlled by incentive stimuli. Taken together, these data are consistent with the explanation generated by the variation-selection hypothesis that administration of cocaine into the medial tubercle energized responses (variation) reinforced by the presentation of light signals upon lever-pressing (selection by action-outcome association). Although this explanation must be substantiated by additional experiments, the present experiment demonstrates that response-independent delivery of cocaine into the medial olfactory tubercle leads to heightened lever-pressing, an effect that the reinforcement concept alone does not readily explain. Cocaine’s capacity to enhance the incentive effects of other rewards such as brain stimulation reward or conditioned stimuli has been documented (Phillips and Fibiger, 1990). Recently, Caggiula and his colleagues (Chaudhri et al., 2006) have shown that non-contingent nicotine administration increases lever-pressing for the presentation of a light signal. Light stimulus presentation appears to be rewarding to rats especially when they are food-restricted (Stewart and Hurwitz, 1958). Their rats were food-restricted, and moderately responded on the lever that delivered light signals. Response-dependent, but not response-independent, administration of intravenous nicotine also supported moderate lever-pressing. Combining the light signal and nicotine administration synergistically increased lever-pressing even when nicotine was delivered in a response-independent procedure (Donny et al., 2003). I should remind readers that cocaine administration into this region has a dual role: it does not only energize responding elicited by incentive stimuli, but also elicits a positive affective effect, because it induces conditioned place preference without the presence of incentive stimuli (Ikemoto, 2003; Ikemoto and Donahue, 2005).

Figure 15 Effects of response-independent and dependent cocaine administration into the olfactory tubercle on lever-pressing. Panel A shows cumulative lever-press records from a representative rat that received cocaine infusions with response-independent schedules. (more ...)

The present data also have implications for schedule-induced adjunctive behaviors such as autoshaping induced by food in hungry animals (section 4.1.1). These behaviors may be partly mediated by the striatal complex and basal ganglia. The meso-ventromedial striatal dopamine system may play a major role in energizing such behaviors. Indeed, intermittent delivery of small pieces of food in food-restricted rats more effectively increases extracellular dopamine levels in the nucleus accumbens than single deliveries of large amounts of food (McCullough and Salamone, 1992). Lesions of the meso-limbic dopamine system by 6-OHDA disrupt adjunctive behaviors induced by intermittent delivery of food to hungry rats, including heightened water consumption (schedule induced-polydipsia), wheel-running and plasma glucocorticoid (Robbins and Koob, 1980; Wallace et al., 1983). The variation-selection hypothesis suggests that inactivation of the meso-ventromedial striatal dopamine system disrupts a variety of schedule-induced adjunctive behaviors because of attenuated action-arousal, leading to low response levels, whereas inactivation of the meso-ventrolateral or dorsal striatal dopamine systems may not disrupt arousing effects but instead disrupts conditioned responding (selection of a certain response over others). In addition, food-restriction procedures used in scheduled-induced adjunctive behaviors may potentiate the meso-ventromedial striatal dopamine system or its upstream or downstream systems. Indeed, Carr (2002) suggested that food-restriction “sensitizes” the meso-limbic dopamine system.

4.9.1. The anhedonia hypothesis and subjective effect issues Research in human subjects has documented that administration of drugs of abuse such as cocaine and amphetamine elicits subjective effects characterized as “euphoria” and “high”. Such an effect is likely mediated, in part, via dopamine. Volkow and her colleagues (1999 colleagues (2002) have shown that the greater the velocity in ventral striatal dopamine receptor binding induced by the administration of psychomotor stimulants, the greater the subjective effect of “high”. Based on animal research, Wise (1982) offered a hypothesis that the blockade of dopamine receptors takes away the pleasurable experience of rewards. In his words, “all of life’s pleasures – the pleasures of primary reinforcement and the pleasures of their associated stimuli – lose the ability to arouse the animal” (Wise, 1982), after the treatment with dopamine receptor antagonists.

To better understand subjective effect issues, two points must be recognized. First, subjective experience of pleasure does not necessarily dictate animals’ behavior. It is not clear how subjective experience interacts with ongoing subconscious brain processes to modify behavioral outputs, and to what extent it controls animals’ actions. This point is further elaborated in Figure 16.

Figure 16 Conscious experience in relation to stimulus-response processes of the central nervous system (CNS). Arrows within rectangular boxes indicate how the perception of stimuli is processed in relation to conscious awareness process before it reaches the motor (more ...)

In addition, pleasure is not a unitary phenomenon and, thus, dopamine may only be involved in a certain kind of pleasure. At least two different types of pleasure have been suggested. One is associated with the consumption of rewards and is referred to as sensory pleasure, which does not appear to depend on dopamine. For example, feeding is a pleasurable experience. A series of recent studies using dopamine deficient mice (Zhou and Palmiter, 1995) make compelling points on related issues. Without dopamine, hungry mice are hypoactive and starve to death, even though food and water are readily available, literally in front of their noses. However, these mice still have the capacity to consume food or water if these are directly delivered to their mouths (Szczypka et al., 1999; 2001). In addition, dopamine deficient mice prefer sweet solutions over plain water (Cannon and Palmiter, 2003). These data suggest that dopamine does not appear to be essential for feeding or experiencing some pleasure out of it, while it is critical for seeking out and delivering food into the mouth. Other lines of research are consistent with this notion (for a recent review, see Baldo and Kelley, 2007). Ikemoto and Panksepp (1996) showed a dissociation between reward-seeking and reward-consumption in that blockade of dopamine receptors in the nucleus accumbens disrupts seeking for a sucrose reward, but not sucrose consumption, in rats. Berridge and Robinson (1998) showed that almost complete depletion of striatal dopamine does not disrupt “facial pleasure expression” induced by sucrose delivered into rats’ mouths.

The other type of pleasure is the anticipation of rewards, referred to as emotional pleasure. In his influential book, The Expression of the Emotions in Man and Animals, Charles Darwin (1872) described this type of pleasure as being accompanied by physical movements.

As suggested by Panksepp (1982; 1998), this type of pleasure is likely mediated, in part, by dopamine. The activation of the meso-ventromedial striatal dopamine system is hypothesized to be particularly important. As discussed above, extracellular tonic increases of dopamine in the ventromedial striatum elicit unconditioned physical movements and vocalization in rats, consistent with Darwin’s description. Therefore, ventromedial striatal dopamine may play an important role in emotional pleasure of reward expectancy, but not sensory pleasure. However, it is unclear to what extent the subjective experience of such pleasure controls behavior.

4.9.2. The psychomotor stimulant theory and reward-arousal homology issues The theoretical framework and data discussed above indicate a close relationship between the rewarding effects and behavioral arousal effects of drugs. Indeed, a theory linking the reinforcing effects and the motor stimulant effects of drugs has been proposed by Wise and Bozarth (1987). They argue, “all drugs that are positive reinforcers should elicit forward locomotion… they do so by activating the dopaminergic circuitry of the medial forebrain bundle”. It should be emphasized that there are multiple tasks or measures that qualify stimuli as rewarding. By positive reinforcers*, Wise and Bozarth (1987) meant stimuli for which animals learn instrumental tasks. However, stimuli can be said to be rewarding if they induce conditioned place preference or if they are orally consumed. Wise and Bozarth (1987) did not suggest that rewarding drugs defined by non-instrumental tasks elicit forward locomotion. As discussed above, mice without dopamine can consume food and water (Szczypka et al., 1999; 2001) and prefer sweet solutions over plain water (Cannon and Palmiter, 2003). Thus, these results and many other data demonstrate a dissociation between consumption and seeking/procurement (see Ikemoto and Panksepp, 1999; Baldo and Kelley, 2007) suggesting that dopamine appears to be essential for seeking, but not necessary for generating preference for the oral consumption of particular stimuli.

In addition, conditioned place preference procedures appear to detect the rewarding effects of stimuli that are dopamine-independent. Whereas intravenous administration of cocaine induces conditioned place preference mediated via a dopamine dependent mechanism (Spyraki et al., 1987), particularly that of the accumbens shell and olfactory tubercle (Sellings et al., 2006b), intraperitoneal administration of cocaine induces conditioned place preference, which is not blocked by pretreatments with systemic dopamine antagonists or 6-OHDA lesions of the nucleus accumbens (Spyraki et al., 1982a; Sellings et al., 2006b). Spyraki et al., (1982a) suggested that intraperitoneal cocaine induced conditioned place preference results from the local anesthetic actions of cocaine, because intraperitoneal administration of procaine, which has a similar chemical structure to cocaine and similar local anesthetic properties, but little capacity to block dopamine uptake, induces conditioned place preference. Interestingly, intraperitoneal administration of procaine, which induces conditioned place preference, does not increase but instead decreases forward locomotion (Wiechman et al., 1981; Reith et al., 1985).

Similarly, opiates appear to have both dopamine-dependent and -independent actions. Dopamine deficient mice can learn to associate place cues and the affective action of morphine, and if they are treated with caffeine or L-DOPA, which temporarily restores their dopamine, just before the performance phase, they display a preference for the compartment paired with morphine (Hnasko et al., 2005). These data suggest that dopamine is not necessary for stimulus-outcome learning between environmental cues and some affective effects of morphine, but is necessary for expression of that learning.

Dopamine deficient mice display heightened locomotion when treated with psychomotor stimulants if dopamine is selectively restored in the ventral striatum, but not the dorsal striatum (Heusner et al., 2003), findings that are consistent with other studies (section 4.5). Therefore, as Wise and Bozarth (1987) suggested, drug administration that is rewarding as assessed by instrumental tasks should elicit heightened locomotion.

Wise and Bozarth (1987) also stated, “the locomotor effects and the positive reinforcing effects of these drugs are homologous.” These effects are triggered via a common mechanism, namely the meso-limbic dopamine system, especially the meso-ventromedial striatal dopamine system. However, rewarding and arousal effects appear to be differentially dependent on the downstream mechanisms of this dopamine system; that is, these effects should be dissociated on anatomical grounds. The meso-ventromedial striatal dopamine system interacts with various other systems: the ventrolateral and dorsal striatal dopamine systems, action-arousal system and limbic, associative and sensory cortices (Fig. 17A). The rewarding effects of drugs, which are assessed by behavioral tasks such as conditioned place preference or self-administration, appear to rely on basic associative processes (stimulus-outcome, stimulus-action, action-outcome and stimulus-response associations); the nature of the task and extent of experience with the task make a difference in the way these associative mechanisms interact and control animals’ behavior to display the rewarding effects of drugs (Fig. 17B). On the other hand, drugs’ unconditioned motor stimulant effects, initiated via the meso-ventromedial striatal dopamine system, may not require elaboration of information with other meso-striatal systems or other higher systems (Fig. 17C). This point is consistent with the above-mentioned finding that the restoration of dorsal striatal dopamine is not necessary for psychomotor stimulant-induced locomotion, if ventral striatal dopamine is restored in dopamine deficient mice (Heusner et al., 2003).

Figure 17 Interaction of dopamine systems with cortical and action-arousal systems (A). Conditioned responses that indicate drug reinforcement depend on dopamine-cortical interactions (B), whereas unconditioned responses do not as much (C).

In addition, rewarding and arousing effects may be dissociated on temporal grounds. This point is discussed in section 5.1.

4.9.3. Anergia hypothesis Salamone and his colleagues (Salamone and Correa, 2002; Salamone et al., 2003) have suggested that nucleus accumbens dopamine is involved in energizing animals performing instrumental tasks. In their words, “interference with accumbens DA impairs the exertion of sustained effort over time” (p. 7, Salamone et al., 2003). This suggestion is in general agreement with the role of ventromedial striatal dopamine in action-arousal. Dopaminergic activity in the ventromedial striatum may determine animals’ capacity to pursue seeking tasks of varying degrees of “difficulty”. The level of difficulty for which ventromedial striatal dopamine is important has not yet been determined. Because ventromedial striatal dopamine is involved in action-arousal or general drive states, the deficiency of ventromedial striatal dopamine may affect performance based on both physical and mental challenges.

4.9.4. Appetitive motivation/”wanting” vs. consummatory motivation/”liking” and incentive formation hypothesis Goal-directed behavior is thought to be divided into two general phases, appetitive and consummatory phases (Craig, 1918; Konorski, 1967), served by distinct neuronal mechanisms. Appetitive motivation, or “wanting”, is hypothesized to be mediated partly by brain dopamine (Blackburn et al., 1992; Ikemoto and Panksepp, 1996; Berridge and Robinson, 1998). However, the argument that dopamine is not important for the consummatory or “liking” phase needs careful consideration. As discussed above, dopaminergic neurons fire and dopamine is released during the consummatory, or liking, phase. Moreover, as exemplified by the McFarland and Ettenberg (1995) study, conditioned responding is attenuated after the disruption of dopaminergic activity by systemic or intra-accumbens administration of dopamine antagonists during the consummatory, or liking, phase.

To account for the role of dopamine during consummatory behavior, Ikemoto and Panksepp (1999) hypothesized that dopamine in the nucleus accumbens plays an essential role in incentive learning; increased dopaminergic activity in the accumbens dopamine during the consummatory phase increases the incentive value of stimuli, while dopaminergic inactivity during the consummatory phase decreases the incentive value of stimuli. Fast-scan cyclic voltammetry data indicate that the magnitude of dopaminergic signals in the accumbens occurring just after lever-pressing (the consummatory phase) becomes smaller and smaller during the extinction phase, while response intervals increase more and more (Stuber et al., 2005b), as if the magnitude indicates incentive motivation for the next lever-presses. The present variation-selection model considers incentive learning in two forms: stimulus-outcome and stimulus-action learning (Fig. 14). Treatment with dopamine receptor antagonists by itself does not appear to change stimulus-outcome or stimulus-action associations that have already been formed. Therefore, treatment with dopamine receptor antagonist does not affect immediate appetitive performance, if the task is not physically or mentally demanding or the blockade of dopamine receptors in the ventromedial striatum is not extensive (severe inhibition leads to the lack of drive, see section 4.9.6). When rats execute an action and receive a reward, ventral striatal dopamine participates in the reconsolidation of incentive memory. The disruption of normal dopaminergic activity in the ventral striatum during consummatory behavior followed by appetitive behavior will lead to re-organization of a stimulus-outcome and stimulus-action association, such that the presentation of conditioned stimuli in the next trial will be ineffective in eliciting conditioned responding.

4.9.5. The incentive-salience hypothesis and incentive-sensitization theory of addiction Berridge and Robinson (Robinson and Berridge, 1993; 2003; Berridge and Robinson, 1998) proposed an incentive salience hypothesis of dopamine function. Accordingly, striatal dopamine “transforms the brain’s neural representations of conditioned stimuli, converting an event or stimulus from a neutral ‘cold’ representation (mere information) into an attractive and ‘wanted’ incentive that can ‘grab attention’” (p. 5, Robinson and Berridge, 1993). The present variation-selection hypothesis offers a mechanism for understanding how ventral striatal dopamine is involved in incentive salience. Increased ventromedial striatal dopamine energizes responding, while activities in the ventrolateral and dorsal striatum interacting with cortical regions guide responding toward novel, salient and conditioned stimuli. Ventromedial striatal dopamine during consummatory behavior is involved in the acquisition and, perhaps, the re-organization of stimulus-outcome association, and ventrolateral striatal dopamine during consummatory behavior is involved in the acquisition and re-organization of conditioned responding (stimulus-action association).

Although the present variation-selection model is not designed to address drug addiction (or dependence), it can offer some insights. Robinson and Berridge (1993; 2003) proposed a theory to explain how prolonged drug use makes drugs addictive. They stated:

As mentioned above, Wyvell and Berridge (2000) found that amphetamine injections into the accumbens shell energize rats to respond on the active lever only when a Pavlovian-conditioned stimulus is present, a finding that is consistent with an incentive salience attribution effect of dopamine. Wyvell and Berridge (2001) further found that previous experience with repeated administration of amphetamine has essentially the same effect on incentive salience attribution as intra-shell administration of amphetamine. The presentation of a Pavlovian-conditioned stimulus energizes rats with a history of repeated administration of amphetamine to respond on the active lever, even if the rats are free of the drug at the time of testing. These results raise the possibility that repeated drug experience “sensitizes” the meso-ventromedial striatal dopamine system, or its upstream or downstream systems, in response to conditioned stimuli.

Interestingly, intra-shell administration of amphetamine in those sensitized rats did not further increase conditioned responding upon the presentation of a conditioned stimulus, but rather increased general investigatory behaviors such as sniffing and rearing not targeted at the active lever (Wyvell and Berridge, 2001). This observation may be explained by the hypothesis that the combination of previous repeated amphetamine and intra-shell amphetamine activated the meso-ventromedial striatal dopamine system so much that it overrode inhibitory control of selection mechanisms or conditioning effects, leading to an increase in unconditioned responding.

4.9.6. Hedonic homeostatic dysregulation In contrary to Robinson and Berridge’s addiction theory based on hyper-function of reward systems, Koob and Le Moal (1997; 2001; 2005) proposed an addiction theory based on hypo-function of reward systems. They adopted an opponent-process model to conceptualize changing mood states as a function of drug taking. In this model, drug taking results partly in a chronic deviation of reward mechanisms from a set point, leading to dysregulation of reward systems and increased drug taking.

The identification of the two dopamine systems and our recent unpublished data may offer some insights on Koob and Le Moal’s thesis. The meso-ventromedial striatal dopamine system may participate in not only the positive mood state (a-process), but also the negative mood state (b-process). As mentioned above, microinjections of the dopamine D₂ receptor agonist quinpirole into the posteromedial VTA, which inhibit dopaminergic neurons projecting to the ventromedial striatum, disrupt conditioned place preference induced by food and conditioned place avoidance induced by naloxone (Liu and Ikemoto, 2006), suggesting that inactivation of these dopamine neurons disrupts stimulus-outcome learning. Higher doses of quinpirole into the posteromedial VTA that lowered basal dopamine levels in the ventromedial striatum induced conditioned place avoidance by themselves, and in the presence of food, reduced food intake (Z. H. Liu and S. Ikemoto, unpublished observation). High doses of quinpirole injections into the anterolateral VTA or substantia nigra did not induce these effects. Our data suggest that hypo-activity of the meso-ventromedial dopamine system leads to a negative affective state. Some clinical data suggest that chronic psychomotor-stimulant abuse decrease activity of brain dopamine in patients (Lago and Kosten, 1994, Koob and Le Moal, 1997). These findings suggest that chronic drug taking leading to hypo-activity of the meso-ventromedial striatal dopamine system may result in a negative mood state which in turn accelerates drug-taking (“self-medication”) in an attempt to improve the mood state (Markou et al., 1998).

4.9.7. Prediction-error hypothesis and phasic-tonic functional issues In a series of studies, Schultz (1998; 2002) and his colleagues characterized how dopaminergic neurons in the ventral midbrain are excited by incentive stimuli in learning tasks. They found that those neurons respond to incentive stimuli as if their signals are conveying discrepancies between the expectation of a reward (with respect to timing and magnitude) and the actual result. The way those signals change in a learning task is found to fit well with the Rescorla-Wagner equation (Rescorla and Wagner, 1972) or, more recently, temporal-difference models (Sutton and Barto, 1981), which describe how organisms learn the relationship between a neutral stimulus and an unconditioned stimulus in Pavlovian learning tasks. Therefore, it is suggested that midbrain dopaminergic neurons encode a prediction error between the actual and predicted rewards.

Two points are needed to clarify these observations. First, these electrophysiological signals are correlates of incentive stimuli during reward-seeking tasks, and the functional significance of these signals is not clearly understood. The observation that dopaminergic neurons increase firing activity similarly across the VTA and SNC in relation to reward-seeking tasks does not necessarily mean that dopamine released in various striatal target sites has the same functional consequence. Indeed, behavioral data indicate otherwise (Di Chiara, 2002; Kelley, 2004; Everitt and Robbins, 2005, Yin and Knowlton, 2006). Therefore, these phasic dopaminergic signals, when they reach the ventromedial, ventrolateral, and dorsal striatum, appear to be utilized differently by terminal regions and their downstream circuits.

Secondly, those phasic dopamine signals should be distinguished from tonic signals and may be critically involved in associative learning between environmental stimuli and internal states, between stimuli and actions and between actions and consequences (Fig. 14). Latencies of firing of dopamine neurons in relation to reward-related stimuli may be too short to signal rewards (Redgrave et al., 1999). Indeed, dopaminergic neurons appear to fire not only in response to reward-related stimuli, but also those of potential importance such as loud noise (Kiyatkin, 1988; Pan et al., 2005). Redgrave et al. (1999) suggested, “the initial burst of dopaminergic-neurone firing could represent an essential component in the process of switching attentional and behavioural selections to unexpected, behaviourally important stimuli. This switching response could be a crucial prerequisite for associative learning and might be part of a general short-latency response that is mediated by catecholamines and prepares the organism for an appropriate reaction to biologically significant events” (p. 146, Redgrave et al., 1999). Phasic dopamine signals, therefore, may be involved in associative learning by enabling salient stimuli to be associated with internal states and actions, and actions to be associated with consequences. A similar conclusion is suggested based largely on observations concerning the effects of dopaminergic manipulations on latent inhibition (Joseph et al., 2003), an effect in which a previously exposed (i.e., familiar) stimulus is more difficult to condition with an unconditioned stimulus than a novel stimulus.

Table 4 Possible functions of phasic and tonic changes in dopamine

Phasic dopamine signals in the ventromedial striatum and the accumbens core may be important in associative learning, stimulus-outcome and stimulus-action learning, respectively. Tonically high levels of extracellular dopamine may be responsible for heightened drive states. As mentioned above, tonic levels of ventral striatal dopamine correlate well with levels of locomotor activity. It is not easy to investigate functional roles of phasic and tonic dopamine signaling because tonic signals influence phasic signals and vice versa; systemic administration of cocaine, which increases tonic dopamine signals by blocking dopamine uptake, also increases frequencies of phasic signals and transients as detected by the fast-scan cyclic voltammetry (Heien et al., 2005; Stuber et al., 2005a; 2005b).

The reward function may be regulated differently from drive and stimulus-outcome associative effects by the magnitude and speed of change in ventromedial striatal dopamine levels. As mentioned above, the greater the positive velocity in dopamine receptor binding in the ventral striatum induced by the administration of psychomotor stimulants, the greater the subjective effect of “high” (Volkow et al., 1999; 2002). Studies in rats found that intravenous administration, which delivers drugs into the brain in seconds (Mateo et al., 2004; Heien et al., 2005; Stuber et al., 2005), establishes robust conditioned place preference at much lower doses than slower intraperitoneal or subcutaneous administration (Spyraki et al., 1982b; Nomikos and Spyraki, 1988; Sellings and Clarke, 2003), which delivers drugs into the brain over minutes (Riffee et al., 1978). Also, lateral hypothalamic stimulation is rewarding when its duration is short (0.4 sec), whereas it elicits escape responses when its duration is several seconds or longer, suggesting that the onset of stimulation is involved in reward and the continuation of the stimulation is not (Bower and Miller, 1958).

Another important step is to clarify how dopamine is involved in behavior controlled by aversive stimuli. The meso-limbic dopamine system has been implicated in motivated behavior triggered by aversive stimuli (Salamone, 1994; Ikemoto and Panksepp, 1999; Pezze and Feldon, 2004). As mentioned above, our preliminary studies found that moderate inhibition of the meso-ventromedial striatal dopamine system disrupts conditioned place avoidance induced by aversive stimuli (Liu and Ikemoto, 2006). In addition, more severe inhibition of this dopamine system alone induced conditioned place avoidance and reduced food intake and locomotor activity (Z. H. Liu and S. Ikemoto, unpublished observation). These observations may be explained by hypotheses that phasic dopamine signals are important for stimulus-outcome association involving aversive stimuli, that tonic changes of dopamine from a basal level to a low level trigger a negative affective state, and that tonically low basal dopamine levels result in depression-like symptoms (Willner, 1983).

Dopaminergic functions in the ventrolateral striatum have yet to be substantiated. In particular, the functions of the lateral shell and lateral olfactory tubercle have not been investigated. Based on cellular connection data, the functions of the two regions are expected to be somewhat similar, but not identical. The functions of accumbens core are likely somewhat different from those of the lateral shell and lateral tubercle.

Although drug reward was the focus of the present review, the two dopaminergic systems discussed in the paper are thought to be integral parts of the general SEEKING system (Panksepp, 1998) that has evolved from mechanisms underlying basic activities such as hunting and gathering energy sources and avoiding predators. The meso-ventromedial striatal dopamine system is hypothesized to be a critical component of the basic processes regulating mood states and allowing organisms to engage in effortful activities. Dysregulated interactions of this dopamine system with other closely-related systems may lead to mood disorders and various types of addiction.

Cytoarchitectonic features of the ventral midbrain were examined in horizontal, coronal and sagital sections that were immuno-reacted for TH, and sub-divisions of the ventral midbrain were drawn in horizontal sections. TH is a specific maker for dopaminergic neurons in the ventral midbrain, because dopamine-β-hydroxylase, the enzyme that converts dopamine to norepinephrine, is not found in this area (Swanson and Hartman, 1975). Sections stained for Nissl substance were also used to aid the determination of structure borders.

6.1.1. Methods Eleven male Wistar rats (275–350 g; Harlan Industries, Dublin, VA) were used. The following procedures were approved by the Animal Care and Use Committee of the NIDA Intramural Research Program and were in accordance with NIH guidelines. Serial 40-μm thick sections (one-in-four) of the brains of 11 naïve rats were cut with a cryostat for horizontal (n = 5), coronal (n = 3), or sagital (n = 3) views. One complete series of sections was stained with cresyl violet to detect Nissl substance. Another was processed to detect TH by using the following immunohistochemical procedure. To remove peroxidase activity, sections were placed in 0.3% hydrogen peroxide for 20 min and washed with PBS three times each for 10 min. The sections were then incubated in a 4% bovine serum albumin solution supplemented with 0.3% Triton X-100 in PBS for 1 hr, followed by incubation with a mouse anti-TH monoclonal antibody (1:500–1,000, Chemicon) in PBS containing 4% bovine serum albumin and 0.3% Triton X-100 for 48 hrs. After being washed with PBS three times each for 10 min, the sections were incubated in a 1:200 dilution of biotinylated secondary antibody for 1 hr, then rinsed with PBS, and incubated with avidin–biotinylated horseradish peroxidase for 1 hr (ABC kit, Vector Labs., Burlingame, CA). The samples were rinsed and the peroxidase reaction was developed with 0.05% 3,3-diaminobenzidine-4 HCl and 0.003% hydrogen peroxide. The sections were then mounted on gelatin coated slides and cover-slipped using embedding medium (Cryo-Gel, Instrumedics, Hackensack, NJ) for light microscopic examination. Photographs were taken with a digital camera installed on a microscope (Nikon Eclipse E 800) using 10 X and 20 X objectives. All photographs were adjusted for contrast and brightness with Adobe Photoshop (version 8). Drawings of horizontal sections were produced by tracing photo images using Adobe Illustrator (version 11).

6.2.1. Results The retrograde tracer FG was selectively deposited into medial and lateral zones within the nucleus accumbens and olfactory tubercle, which have been studied for behavioral functions (section 1.2). Figure 18 depicts the locations and extents of FG deposited into the medial tubercle (n = 5), lateral tubercle (n = 4), medial accumbens shell (n = 4), lateral shell (n = 4) and accumbens core (n = 5). FG was also deposited into the dorsal striatum (n = 4) and the nucleus of the diagonal band (n = 3) as site controls for comparison. FG deposits in fourteen rats were found in in-between regions including core-dorsal striatum (n = 1), core-medial shell (n = 3), medial shell-medial tubercle (n = 5), medial shell-diagonal band (n = 3), medial tubercle-diagonal band (n = 1), and ventral shell-lateral tubercle (n = 1); the sections from these rats were studied, but not included for analysis.

Figure 18 Extent of iontophoretic FG deposits for individual cases shown by lines. Each case is assigned with a unique number and color to help distinguish from others within the zone. The drawings are coronal sections adapted from Swanson (1998).

Figure 10 depicts representative distributions of retrogradely labeled cell-bodies in the ventral midbrain for respective deposit sites, and Table 1 shows the mean counts of labeled cells for respective terminal zones. FG deposits confined in the medial olfactory tubercle labeled many cells in the posteromedial portion of the PBP, PN and central linear nucleus, but few in other ventral midbrain zones such as anterolateral VTA or SNC (Fig. 10). An FG deposit slightly more lateral than that of case 241 resulted in labeling cells more laterally in the ventral midbrain than 241, suggesting a mediolateral topography. Indeed, FG deposited into the lateral tubercle mostly labeled cells in the lateral portion of the PBP.

FG deposited into the dorsomedial portion of the accumbens shell (cases 234, 279, and 280) retrogradely labeled cells mostly in the PN, central linear nucleus and interfascicular nucleus (Fig. 10). Significant retrogradely labeled cells were not observed in the interfascicular nucleus after FG was deposited into any other striatal sites, suggesting that the interfascicular nucleus selectively projects to the dorsomedial shell within the ventral striatum. FG deposited into the slightly more ventral area of the medial shell (case 219) labeled cells that were mostly confined to the medial portion of the PBP and central linear nucleus. Although previous studies reported that injections of horseradish peroxidase or unconjugated WGA into the medial nucleus accumbens labeled the supramammillary nucleus (Carter and Fibiger, 1977; Phillipson and Griffiths, 1985), the deposits of FG confined to the medial shell in the present experiment did not result in labeling in the supramammillary nucleus. FG deposited into the lateral shell, like FG deposited into the lateral tubercle, largely labeled cells in the lateral portion of the PBP (Fig. 10). Like deposits into the lateral shell and lateral tubercle, FG deposited into the accumbens core extensively labeled cells in the lateral portion of the PBP (Fig. 10); unlike those of the lateral shell and lateral tubercle, core FG also labeled notable levels of cells in the SNC.

FG deposited into the dorsal striatum (cases 244–247) labeled many cells in the SNC and the anterolateral PBP and labeled some in the retrorubral area and posterolateral PBP (Fig. 10). Unlike deposits into the striatal regions, FG deposited into the nucleus of the diagonal band retrogradely labeled many cells in the parafasciculus retroflexus area and the posterior hypothalamic area including the supramammillary nucleus, and a few in the rostral linear nucleus and medial PBP (Fig. 10). Slight overlaps in the locations of the labeled neurons between the diagonal band and medial tubercle and between the diagonal band and medial shell may be explained by either minor diffusions of the tracers into each other or the existence of transition zones between them. In addition, FG labeling found in the rostral linear nucleus and parafasciculus retroflexus area after FG deposits in the olfactory tubercle most likely resulted from the projection to the pallidal, but not striatal, zone of the olfactory tubercle; recent PHA-L data suggest that neurons in the rostral linear nucleus and parafasciculus retroflexus area project to the pallidal zone (Del-Fava et al., 2007). Heterogeneous elements of the olfactory tubercle raise the issue that FG deposited in the olfactory tubercle must have been taken up by neurons of all of these zones. However, the striatal zone is by far the largest in the tubercle. In addition, the ventral pallidum/medial forebrain bundle zone stains very lightly with the TH immunohistochemical procedures (Fig. 1A) and appears to receive inputs from zones different from those projecting to the striatal district. Therefore, the striatal zone of the tubercle is likely the dominant source of the FG labeling found in the midbrain.

6.2.2. Co-localization of FG and TH containing neurons To determine whether retrogradely labeled cells were dopaminergic, TH was detected with an immunohistochemical procedure and labeled with a fluorescent marker. No immuno-reaction was performed for FG, a fluorescent tracer. Table 5 summarizes the extent of co-localization between FG and TH. Over 96% of retrogradely labeled cell-bodies that were identified in the ventral midbrain after FG was deposited into the olfactory tubercle, nucleus accumbens or dorsal striatum were also labeled with TH, whereas only 10% of cell-bodies labeled after FG application into the nucleus of the diagonal band were co-localized with TH. These values of co-localization are slightly higher than the 85% value for the VTA TH neurons projecting to the nucleus provided by Swanson (1982), although within the same range with 95% or higher values for the dopaminergic cells projecting to the dorsal striatum (van der Kooy et al., 1981). Slightly higher estimates than those of Swanson (1982) may be explained by one or all of the following factors. First, the present experiment utilized small injections confined in small areas within the ventral striatum, while Swanson used large injections affecting the entire nucleus accumbens, which may have affected sites outside of the nucleus such as the diagonal band. Second, the detection of FG labeling relied on intrinsic fluorescence of FG, but not immunoreactions to FG, for the present co-localization. This procedure resulted in reduction in the detection of retrogradely labeled cells by approximately 50% from those detected by the immunohistochemical detection procedure. If, for unknown reasons, non-dopaminergic projection neurons reflect less fluorescence than dopaminergic neurons, this would result in overestimation. Third, it was not always easy to determine co-expression between TH and FG, because of interference by overlapping TH immunofluorescent cells in the VTA (Fig. 19). In any case, the majority of FG-positive cells projecting from the ventral midbrain to the ventral striatum appear to be dopaminergic, whereas the vast majority of FG-positive cells in the parafasciculus retroflexus area projecting to the diagonal band appear to be non-dopaminergic.

Table 5 Percent of midbrain FG-positive cell-bodies that are also positive for TH.

Figure 19 Co-localization of FG- and TH-labeling. (A) FG-labeling is shown. (B) TH-labeling is shown. (C) The red and green channels of panel A were removed and then the image was superimposed over panel B. Arrows indicate FG-labeled cells that are not labeled (more ...)

6.2.3. Topography In order to systematically address differences in the locations of cell-bodies between the medial and lateral, anterior and posterior, and dorsal and ventral units in relation to projection sites in the ventral striatum, each of the midbrain horizontal sections was divided into four compartments (Fig. 2) and counts for retrogradely labeled cell-bodies within those compartments were compared between the dorsal three and ventral three horizontal sections. In general, the cells in medial compartments of the ventral midbrain projected to medial regions of the olfactory tubercle and accumbens shell, whereas those in lateral compartments projected to the lateral regions of the olfactory tubercle and accumbens shell and the accumbens core (Fig. 20). In addition, the present analyses revealed the roles of the anteroposterior and dorsoventral dimensions in projection patterns. Very little FG labeling was found in the anteromedial zone corresponding the parafasciculus retroflexus area after FG injections into the ventral striatum, suggesting that the parafasciculus retroflexus area, which contains only few dopaminergic neurons, provides limited projection to the ventral striatum. The ventral anterolateral compartments, containing dense dopaminergic cells, had extensive labeling after FG injections into the ventrolateral striatum including the lateral tubercle, lateral accumbens shell and accumbens core. These observations are consistent with the finding that leucine and proline application confined within the anterolateral VTA anterogradely labeled axon terminals predominantly in the accumbens core, lateral accumbens shell and lateral olfactory tubercle (Beckstead et al., 1979). The dorsal posteromedial compartments had extensive FG labeling after FG injections into both the medial tubercle and the medial accumbens shell, reflecting heavy projections from the CL to these regions. The ventral posteromedial compartments, on the other hand, had heavy labeling after FG injections into the medial accumbens, but not the medial tubercle. This difference is explained by selective projection of the interfascicular nucleus to the dorsomedial shell, but not the medial tubercle. Overall, these data are in general agreement with previous data from rats, suggesting that the cells in the VTA project to the accumbens (Fallon and Moore, 1978; Nauta et al., 1978; Beckstead et al., 1979; Phillipson and Griffiths, 1985; Brog et al., 1993; Hasue and Shammah-Lagnado, 2002) or to the tubercle (Newman and Winans, 1980) with mediolateral topography. The present data highlight the importance of an anteroposterior dimension as shown in Figure 12.

Figure 20 Retrogradely labeled cell counts by compartments, divided by the anterior-posterior (ANT-PST), medial-lateral (M-L) and ventral-dorsal (V-D) dimensions. Error bars indicate SEM. See Figure 2 for divisions of ANT-PST, M-L, and D-V (divided between sections (more ...)

6.2.4. Limitations The present study focused on limited anatomical dimensions and did not examine other important anatomical dimensions. The VTA projects to other brain regions such as dorsomedial striatum just lateral to the lateral ventricle, septum, hippocampus and prefrontal cortex. At the present, it is unclear how VTA zones and the midline nuclei differentially project to other regions of the brain. The present study did not examine antero-posterior differences of the ventral striatum in dopaminergic projection. In fact, anatomical and functional differences between the anterior and posterior ventral striatum have been reported by many researchers for the nucleus accumbens in particular (e.g., Fallon and Moore, 1978; Voorn and Docter, 1992; Zahm and Brog, 1992; Reynolds and Berridge, 2002; Zangen et al., 2006). The anterior and posterior olfactory tubercle may also differ with respect to responses to psychomotor stimulants (Ikemoto, 2002; 2003).

6.2.5. Methods Forty-three male Wistar rats (275–350 g; Harlan Industries, Dublin, VA) were used. The following procedures were approved by the Animal Care and Use Committee of the NIDA Intramural Research Program and were in accordance with NIH guidelines. Each rat was anesthetized with sodium pentobarbital (31 mg/kg, i.p.) and chloral hydrate (142 mg/kg, i.p.) and placed in a stereotaxic instrument. A glass micropipette with a 10-μm tip diameter was filled with 1% FG (Fluorochrome LLC, Denver, CO) in physiological saline and was lowered into one of the target sites. The micropipettes for medial shell, medial tubercle, and diagonal band sites were inserted at a lateral 20° angle from the other hemisphere through the midline to minimize diffusion of the tracer to the core or shell, whereas the micropipettes for the core, lateral shell, lateral tubercle, and dorsal striatum were inserted vertically. The incisor bar was set at 3.3 mm below the interaural line. The stereotaxic coordinates were 2.0 mm anterior to bregma (A), 1.9 mm lateral to the midline (L), and 9.2 mm ventral to the skull surface (V) (measured along the trajectory of the angled pipette) for medial tubercle placements; A2.0, L1.6, V8.2 for medial shell placements; A1.5, L1.9, V8.8 for diagonal band placements; A2.0, L2.6, V9.4 for lateral tubercle placements; A2.0, L2.2, V8.7 for lateral shell placements; A2.0, L1.8, V7.5 for core placements; A1.6, L2.5, V6.0 for dorsal striatum placements. The tracer was iontophoretically delivered via the micropipette using positive 2.0 μA current pulsed 5 sec on and 5 sec off over 15 min for the dorsal striatum and using positive 1.0 μA current pulsed 5 sec on and 5 sec off over 15–25 min for other sites.

Survival time of rats for tracer experiments ranged from 10 to 14 days after FG application. As described in section 6.1.1, the brains were processed for histological procedures. One complete series of sections was stained with cresyl violet to detect Nissl substance, another was processed to detect FG, and a third to detect TH with a fluorescent marker. FG was visualized by the immunohistochemical procedure (section 6.1.1) with a rabbit anti-FG polyclonal antibody (1:1000, Chemicon, Temecula, CA). In order to visualize TH, tissues were processed for the anti-TH antibody as described in section 6.1.1 and then incubated in the 4% bovine serum albumin/0.3% Triton X-100 PBS containing Cy² donkey anti-mouse secondary antibody (1:100 dilution; Jackson ImmunoResearch Lab, West Grove, PA) for 24 hours, and then washed with PBS. The sections were then mounted on gelatin coated slides and cover-slipped using embedding medium for fluorescent microscopic examination.

The locations of retrogradely labeled cells found on photo images taken with 10X and 20X objectives were plotted onto the drawings (Figure 2) using Adobe Illustrator. Only labeled cells in ipsilateral sides to FG deposit sites were examined, because previous studies indicated that midbrain projections to the ventral striatum are primarily unilateral (Fallon and Moore, 1978; Nauta et al., 1978; Beckstead et al., 1979; Phillipson and Griffiths, 1985; Brog et al., 1993). Indeed, no significant FG-labeling was observed on the contralateral side, except in the contralateral sides of the midline structures, including the central and rostral linear nuclei and interfascicular nucleus, in which significant labeled cells were often found and were analyzed.

Co-localizations of FG- and TH-positive neurons were determined in photomicrographs taken under fluorescent light using a 20X objective using Adobe Photoshop. Each area was photographed twice, one to detect FG-labeling and the other to detect TH-labeling under appropriate filters. Photomicrographs were edited by Adobe Photoshop. Because RGB image files contain three different color channels: red, green and blue, the color of RGB images can be altered by removing one or two of the channels. FG-labeling is white in original images (Fig. 19A). Because superimposing white labeling over green labeling of TH (Fig. 19B) makes it hard to detect co-localization, the Red and green channels were removed from the original images for FG-labeling, a process that transformed white FG-labeling to blue. Superimposing blue (FG) over green (TH) made it easier to detect co-localization (Fig. 19C). Co-localizations of FG- and TH-positive cells were determined by comparing the three images: FG-labeling only, TH-labeling only and FG-labeling superimposed over TH-labeling. While these images are displayed on a 24” monitor, the numbers of all FG-labeled cells and those co-expressing FG- and TH-labeling were counted.

6.3.1. Results As shown in Fig. 15A, this rat did not respond in session 1 when it received vehicle infusions; the median responses in session 1 among the rats were 27. When the rat received 60 mM cocaine in session 3, it lever-pressed at a steady rate during the first 30 min and slowed down thereafter. When the rat received vehicle infusions in session 5, it started but stopped lever-pressing quickly. When it received 200 mM cocaine in session 7, it kept lever-pressing for the entire session; rates of lever-presses were faster in the beginning of the session and slowed gradually. These observations are confirmed by ANOVAs on lever-press rates (Fig. 15B): lever-press rates obtained with 60-mM cocaine (sessions 3 and 4) and 200-mM cocaine (sessions 6 and 7) were significantly higher than those obtained with vehicle in sessions 1 and 5 (F_1,4 = 9.20, P < .05) and in sessions 5 and 8 (F_1,4 = 16.88, P < .05), respectively.

6.3.2. Methods Five male Wistar rats (Harlan, Dublin, Virginia; 280–330 g at the time of surgery) were used. Food and water were freely available except during testing. The procedures were approved by the Animal Care and Use Committee of the NIDA Intramural Research Program and were in accordance with NIH guidelines. Each animal was implanted with a unilateral guide cannula (24 gauge) that ended 1.0 mm above one of seven target sites (Ikemoto, 2002; Ikemoto and Wise, 2002). The cannulae were inserted at a 20° angle from the other hemisphere through the midline to minimize diffusion of drug solution to the shell. The stereotaxic coordinates were A2.0, L2.0, V8.5 for medial tubercle placements. (−)-Cocaine HCl (Sigma) was dissolved in an artificial cerebrospinal fluid consisting of 148 mM NaCl, 2.7 mM KCl, 1.2 mM CaCl₂, and 0.85 mM MgCl₂ (pH adjusted to 6.5–7.8).

Each animal was placed in a 30 × 22 × 24 cm chamber equipped with a lever (45mm wide × 2mm thick, protruding 20 mm from the wall) and a cue light just above the lever. Each rat’s 31-gauge injection cannula was connected by polyethylene tubing to a micropump (Ikemoto and Sharpe, 2001) hanging a few millimeters above the rat’s head. A lever-press illuminated the cue light for 1 sec and had no other programmed consequences. These experimental procedures were identical to those of the Ikemoto study (2003), except for response contingences. Five rats with no prior instrumental training received 60 mM cocaine in sessions 2, 3 and 4, 200 mM in sessions 6 and 7, and vehicle in sessions 1, 5 and 8. Infusions (75 nl per infusion) in each session were delivered at equal intervals; the rates of infusions were the median rates of self-administration in each session from the Ikemoto study and were 0.16, 0.29, 0.38, 0.29, 0.21, 0.59, 0.67 and 0.20 per min for sessions 1–8, respectively. The maximum number of infusions available per session was limited to 60. When each rat completed the experimental procedure, the brain was removed and processed as described previously (Ikemoto, 2003) for microscopic examination to confirm placements of cannulae.

Effects of 60 mM cocaine on infusion rates were analyzed with 2 × 2 within-subjects ANOVAs with session and treatment (vehicle sessions 1 and 5 vs. cocaine sessions 3 and 4). Effects of 200 mM cocaine were also analyzed with 2 × 2 within-subjects ANOVAs with session and treatment (vehicle sessions 5 and 8 vs. cocaine sessions 6 and 7). The data highly varied and, thus, were square root transformed to maintain homogeneity of variance.

The present work was supported by the Intramural Research Program of NIDA, NIH. The author would like to thank Stefanie Geisler and Yavin Shaham for comments on an earlier draft of the paper, Roy Wise for comments and discussions on related conceptual issues, Robert Ator for technical assistance, Marisela Morales for advice on immunohistochemistry, and Emily Wentzell and the NIH Fellows Editorial Board for editorial assistance.