Figure 1. Statuary from ancient Crete, depicting rituals in the Minoan civilization, about 2000 BC.

The Neural Architecture Underlying Habit Learning: An Evolving View Ann Martin Graybiel Massachusetts Institute of Technology I hope to tell this story of our work as an adventure story, for that is what it is and has been. I never would have predicted at the start that we would now be working on how we form habits and rituals — and how these turn out to be related to disorders in neurology and psychiatry. But I have become fascinated with habits and rituals — and with trying to understand the neurobiology that underlies these behaviors of ours. Our habits are so familiar to us, so common in our lives, that for many of the little habits and mannerisms that we have, we almost are unaware that we are doing them — from morning routines to evening routines. These, of course, are individual habits; but we all share in rituals and habits that are social and even societal. These rituals are like threads running through the history of mankind (Fig. 1); once shared as cultural habits, they can have great power. As I began this work, and decided to do particular experiments, I was delighted by unexpected findings; and we — for it became a small band of adventurers — had to find new methods in order to follow the implications of our findings. At each step, there was a sort of leap of faith that if we tried in this new way, or that, we might truly learn something deeply important about the brain. Our early work on the striatum started with discovering the chemical architecture of the striatum in the human brain (striosomes and surrounding matrix) and then the more global compartmental architecture (matrisomes as well as striosomes) that we found in laboratory animal experiments. Our findings made me think that the striatum might have a learning architecture. And so we needed to — and did — begin to record striatal activity day by day as animals learned tasks to the point of their becoming habitual. We discovered learning-related patterns of striatal activity in freely moving rodents that seemed to bracket behaviors that were becoming habitual, and then in primates we found similar patterns: it was as though with the transition from deliberative behavior to habitual behavior, the entire circuit-level activity of the striatum changed so as to bracket the habitual behavior. This all has all led to our current work, in which we are trying to manipulate these habit-related patterns to probe the underlying circuits and also to develop therapeutic strategies. I wanted at the same time to push toward understanding the molecular bases of these behaviors; and, from using early-gene assays, to discovering striatum-enriched genes, to analyzing the effects of their deletion, the idea is emerging that some behavioral problems related to repetitive, over-focused behaviors could result in part from imbalances between striosome and matrix processing. And through this all, despite the still limited methods for seeing into the depth of the functioning brain at even 1 mm levels of resolution, we have tested as directly

Ann Martin Graybiel

2

as possible for how the chemical architecture might be important in neurologic and neuropsychiatric disorders. What follows is, as a consequence, full of jumps across different methodologies and strategies, but the big jumps were necessary. The hope is that the findings are coalescing to point toward the beauty and intricacy of deep-brain functions. These functions seem somehow to lie very close to the core of ourselves as humans. The Neocortex When I began to study the brain, as a student in the late 1960's, there was enormous excitement about work on the neocortex. Surely this was the organ of thought and creativity, the organ underlying our ability to see and hear and feel, our ability to act deliberatively, to do mathematics. And, building on the great discoveries about the anatomy of the neocortex, with its finely organized layers of neurons, many with apical dendrites oriented at right angles to these, it seemed reasonable that this elaborate biological structure underpinned such complex functions. The excitement came with the new findings from microelectrode recordings made by Mountcastle and Hubel and Wiesel and others — recordings that showed that the functional organization of the neocortex was related to its anatomical architecture, and that the representations of the world were somehow built up step by step by groupings of neurons, in cortical columns and micro-circuits (Mountcastle, 1957; Hubel and Wiesel, 1962). This all had a big influence on me. I was being trained to use anatomical tract-tracing methods in the laboratory of W.J. H. Nauta, and I was lucky that he let me work not on his main theme, but on the outlying regions of the neocortex then known as the association cortex, which were considered also to have sensory functions but presumably 'higher' ones — association regions innervated by the pulvinar.

There seemed to be a logic in the connectivity patterns I found. This is still a deep value of working on what is now called the wiring diagram of the brain. As I studied, I added more and more work related to the motor system. There were terrible limitations in the methods then available. The anatomical tracing methods were based on silver stains to track degenerating fibers induced to degenerate by an earlier experimental lesion. This meant that there were confounds from damage to fibers just passing through the region of the lesions. And there was no good way to look at a connection 'backwards’, to identify neurons projecting to a given site in the brain. When two new methods were introduced to get around these problems, I immediately turned to the long-neglected brainstem — where it had been almost impossible to work before because of the thicket of passing fibers lacing through the region. I reveled in being able to study the brain's wiring diagram for regions that earlier workers had been unable to chart, and roamed across many regions documenting ‘new’ pathways related to vision and oculomotor control. (The most amusing was the finding that one nucleus known to be related to control of the tongue and so named for that function — the perihypoglossal nucleus — actually was a pre-oculomotor nucleus with direct projections to motor neurons controlling eye movements!) Chemical Architecture of the Striatum: Striosomes I was increasingly questioning whether to continue in anatomy, because I wanted to return to functional issues that had attracted me to work on the brain in the first place. As a first step, I wanted to find a way to link the experimental studies to work on the human brain, to which I had been brilliantly exposed by Hans-Lukas Teuber and others at MIT. This was before fMRI was

Ann Martin Graybiel

3

Figure 2. Photomicrograph of section from human brain, stained for acetylcholinesterase, showing pockets of low enzyme activity. These are the histochemically identified striosomes. From Graybiel and Ragsdale, 1978.

developed, when even PET images were new. Having become interested in histochemistry, and being originally trained in chemistry, I decided to try to find histochemical stains that would work both on human brain (post-mortem) and on the brains of the laboratory animals in which I had traced anatomical connections.

I decided to use enzyme stains for the cholinergic enzyme, acetylcholinesterase, and after testing several methods began to stain the experimental brains. I was astonished. I could identify many patterns that I had seen in the connectivity studies. But how to look at the human brain? With some trepidation, I found my way to the Massachusetts General Hospital, where I met its wonderful diener. I explained that I wanted to take a brain to MIT to stain it. This began the period in which I would receive these contributions and take them in a carefully covered bucket back to MIT, only one subway stop away! I learned how to fix and handle the brains, and how to stain carefully cut sections, and our small lab began to work in parallel on the human and animal material.

The striatum, part of the basal ganglia, was well known to be highly enriched in molecules related to cholinergic function, and the striatum was strongly stained for acetylcholinesterase activity, that was for sure. But what we saw attracted my interest. The striatum was known as a primitive part of the forebrain — a vast ball of neurons jammed underneath the elegant neocortex. So it was surprising that when Cliff Ragsdale, an MIT undergraduate, and Henry Hall and I did the staining, the stains did not look homogeneous; there were small zones scattered through the tissue in which the staining was weak (Graybiel and Ragsdale, 1978) . We were just learning how to incubate the sections in the staining solutions, so we stayed up all night testing, and it was about 4 AM when we saw these. Doubtful, we checked again after some sleep, and they were still there (Fig. 2).

What was so tantalizing was that these little zones had about the same dimensions as cortical columns. We had no hint that connections might also form column-like structures in the striatum; after all, the striatum was known as an anatomically primitive part of the forebrain. But we looked, and soon we found that indeed, the cholinesterase-poor zones corresponded to striatal zones with special input and output connections that we could mark in the experimental animals. I reconstructed these in serial sections (the old way, on large plastic plates onto which I projected images of the sections — done nights, at home, with the understanding of my kind and patient husband).

This meant that there were three-dimensional labyrinths running through the striatum — for all the world as though there was a subcortical column or layer system in the striatum. We called these zones striosomes, for striatal bodies, a name I came up with after a delightful time pouring over Latin and Greek dictionaries.

Ann Martin Graybiel

4

Figure 3. Evidence that neurons and striosomes share birth-dates. Matched adjoining transverse sections through the striatum of an adult cat illustrating clusters of striatal neurons born on embryonic day 26 (A) and acetylcholinesterase-poor striosomes (B). Scale bar: 1 mm. From Graybiel and Hickey, 1982.

Figure 4. A cross section through the brain of a human fetus at 22 weeks of gestation, illustrating the distribution of muscarinic cholinergic binding sites, regions that correspond to developing striosomes. Scale bar: 1 mm. From Nastuk and Graybiel, 1985.

To test the layer and column idea, I managed to borrow some brains that had been experimentally labeled with 3H-thymidine for neuronal birth-dating studies from Terry Hickey, a friend who worked on the visual system. Striosomal neurons were born in a relatively narrow time-window, just like a cortical layer — in fact, like layer 6 of the neocortex (Fig. 3) (Graybiel and Hickey, 1982). This finding strengthened our thought that the striatum, far from being primitive, had a hidden neurochemical architecture that in some ways was comparable to that of the neocortex, even though there were not strictly aligned layers and columns. Striosomes Correspond to Regions in the Embryonic Brain Receiving Early-Arriving Dopamine Innervation and Having Early Cholinergic Receptor Expression I had learned that Dahlstrom and Fuxe had discovered in very young brains so-called 'dopamine islands', zones of intense fluorescence marking the developing nigrostriatal innervation. We made direct comparisons of these to the 'young' striosomes, and I was excited that the 3H-thymidine birth-dating would give a way to find out whether the striosomes that we found at adulthood corresponded to the dopamine islands of the developing brain. To make the comparisons at adulthood, we needed the marker to bridge across time. It turned out that there was a precise correspondence (Graybiel et al., 1981; Graybiel, 1984) (Fig. 3). Moreover, I was able to obtain fetal brain material from the Children's Hospital, again with trips back and forth from hospital to lab. We soon found the striosomal compartments were visible early in the human striatum, and remarkably, that through the course of embryonic development and early postnatal months, the zones switched from being cholinesterase-rich to being cholinesterase-poor (Graybiel and Ragsdale, 1980). We later found that many neurotransmitter-related substances — including dopamine — make these switches. Fu-Chin Liu and I spent many hours at the microscope together looking at developing brains, prepared with great skill by Diane Major. But the critical point was that now we knew that there was a close connection of the striosomal system to the two key neurotransmitters famously important for

Ann Martin Graybiel

5

Figure 5. Diagram illustrating mosaic of striosomes and matrisomes that in the aggregate form a global compartmentation in the striatum. IN-striosome; OUT-extrastriosomal matrix. From Malach and Graybiel, 1986.

disorders like Parkinson's disease through the concept of a cholinergic-dopaminergic balance in the striatum. This development was vividly shown in ligand binding for the M1 acetylcholine receptor, work that Mary Nastuk and I did for human and laboratory animal brains (Fig. 4) (Nastuk and Graybiel, 1985, 1988). This cholinergic-dopaminergic linkage is again a focal point for our work now. Striosome-Matrix Compartmental Organization Holds for Most Neurotransmitter-Related Molecules in the Striatum During these years we and many other people stained and labeled for tens and tens of neurotransmitter-related substances, and, with a series of wonderfully gifted students, postdocs, and occasional visitors, we all found that the substances tended to be enriched either in striosomes or in the extrastriosomal matrix (which we called matrix for short). In other words, there was an elaborate, developmentally regulated molecular compartmentation of the striatum that had been totally unseen in the standard classic stains (Graybiel, 1990).

We were already finding that the striosomes received inputs from parts of the neocortex related to the limbic system, thought to regulate emotion, and that striosomes projected either to the dopamine-containing substantia nigra pars compacta, or close to them. This meant that the striosomes might be like outlying limbic parts of the striatum, embedded in the large matrix region, which we and others found to be more related to sensorimotor and associative cortex. But this was all vague, functionally, and besides, why would there be only one set of compartments not a whole series, if the compartments were like layers or columns? Global Compartmentation of the Striatum: Striosomes and Matrisomes We clearly needed physiology. Fortunately, Carl Olson came to the lab, and though we did not work together on the striatum, I learned from him about mapping the somatosensory representations of the brain. I learned about sleep deprivation, too: some of these experiments lasted for up to three days straight! Rafi Malach and I then decided to try to use the cortical maps as a way to learn more about the functional organization of the striatum. We carefully mapped the primary sensory cortex and adjoining area 3a in acute recording experiments, and then traced corticostriatal connections from small, identified parts of the somatotopic representation, using the improved anatomical tracing methods. What we found was, to us, remarkable (Malach and Graybiel, 1986): if we injected tracer into the cortex representing one small body part, say the hindpaw, we found multiple, relatively confined zones of labeling — about the size of striosomes — but these were all in the matrix compartment. These dispersed zones formed a map globally, but they had interesting local order as well, for example, interdigitation patterns suggesting the possibility of local computation across compartments related to similar functions (Fig. 5).

Ann Martin Graybiel

6

Figure 6. Correspondence of ‘input matrisomes’ labeled by an anterograde tracer injection in the foot region of the motor cortex (A) and ‘output matrisomes’ labeled by a retrograde tracer injection in the pallidum (B) in matched adjoining sections. Scale bar: 1 mm. From Flaherty and Graybiel, 1994.

Alice Flaherty and I then worked on the somatosensory maps in the squirrel monkey, applying tracers in different combinations, with kind advice from Mriganka Sur, who had done the early mapping of this system and to our good fortune had a lab next door to ours. Alice and I confirmed the Malach results but then studied the mapping in detail, eventually also combining different tracers to see how the corticostriatal mappings were related to the output organization of the striatum mapped by using retrograde labeling of the output cells (Flaherty and Graybiel, 1991). The output neurons, we found with Alice and Juan Jimenez-Castellanos and Jose Giminez-Amaya, were also in dispersed clusters about the size of striosomes! We made injections of retrograde tracers at many different sites in different experiments (Jiménez-Castellanos and Graybiel, 1989; Giménez-Amaya and Graybiel, 1990; Flaherty and Graybiel, 1993), and we found labeling of either neurons in striosomes or neurons in the matrix, depending on the pallidal or nigral site that we injected. We named the zones in the matrix 'matrisomes' (for matrix bodies).

In later experiments, Hemai Parthasarathy and I found, with the help of Jeff Schall, that the oculomotor regions of the cortex also sent distributed clustered projections to the striatum, and moreover, that these two corticostriatal input systems converged in the input matrisomes, just as the somatotopically matched sensory and motor maps had in the Flaherty experiments (Parthasarathy et al., 1992). We gradually reached the conclusion that the entire matrix compartment was tiled (three-dimensionally) with zones that were not visible with current staining methods but that nonetheless were highly organized and visible by looking at the connectivity of the striatum. This surely meant that compartmentation was as crucial to striatal function as the columns and layers of the neocortex. Potential Learning Architecture of Striatal Networks In a subset of our animals, Alice and I had found that the input matrisomes labeled from the sensorimotor cortex were precisely aligned with output matrisomes labeled in the same monkeys by pallidal injections (Fig. 6) (Flaherty and Graybiel, 1994). I had seen the computational mixture-of-experts learning architecture being developed by Michael Jordan, and the divergent-reconvergent patterns that Alice and I were seeing were highly reminiscent of this architecture. We suggested that the striatum, by redistributing incoming sensory information as though giving the information to little expert sub-networks, and then sending it on within a re-convergent mapping framework to the output structures, could function to promote neuroplasticity of the cortico-basal ganglia system. If so, the striatum could be the learning machine of the basal ganglia (Fig. 7).

Ann Martin Graybiel

7

Figure 8. Diagram illustrating concept of cortico-basal ganglia loops as helping in the selection of actions under the guidance of reinforcement signals from the midbrain dopamine system (and related neural modulatory influences).

Figure 7. Diagram illustrating the analogy of the input-output architecture of the striatum to a mixture of experts computational architecture introduced by Jordan and colleagues. Adapted from Graybiel, 1998.

These results were emerging in our lab just as Schultz and Romo were first reporting their stunning finding that the dopamine-containing neurons of the substantia nigra pars compacta responded in relation to primary rewards and then acquired responses to stimuli that predicted those rewards (Romo and Schultz, 1990). The two sets of findings seemed fully complementary (Fig. 8). The relationship between these was bolstered when Minoru Kimura and I began to collaborate and, with his coworker, Toshihiko Aosaki, found that the so-called tonically active neurons (then thought and now known to be the famous cholinergic interneurons of the striatum) also had responses to primary rewards and acquired responses to conditioning stimuli predicting reward, and that these acquired responses depended on dopamine in the striatum (Aosaki et al., 1994). We mapped them in acute recording and marking experiments and found that these neurons were disproportionately adjoining striosomes (Graybiel et al., 1994; Aosaki et al., 1995). These experiments were very encouraging to me, and I finally decided to try to set up chronic physiological recording in the lab to follow up on our mapping studies. Reorganization of Ensemble Spike Activity in the Sensorimotor Striatum during Learning Setting up chronic recording suitable for long-term recording during learning was quite a challenge, given my background, and it seemed smart to start in rats, for which ensemble recording was being done in the hippocampus already. Mandar Jog, Chris Connolly and Yasuo Kubota, and I, with initial technical advice from Matt Wilson's lab, then initiated a series of experiments in which we inserted multiple tetrodes into the striatum. Experiments of this type still are ongoing on in our lab.

We began with a simple conditional maze task, influenced by the success psychologists had had in doing behavioral experiments. We first made peri-event histograms of firing at different parts of the maze runs, and found that there were many neurons in the sensorimotor striatum that were active at different times during the runs. We noted this array of responses, but what piqued our interest was that as the rats acquired the task behaviorally, we began to see changes in the global patterns of ensemble firing of the striatal neurons. We developed analyses to look at the entire time that the rats were performing the maze runs — not just small peri-event times. This strategy of looking at the entire behavioral sequences turned out to be critical. What we found was that, instead of the ensemble activity being strong throughout the maze runs, the activity gradually

Ann Martin Graybiel

8

Figure 9. Dynamic reorganization of neural activity in the striatum during habit learning. Schematic activity diagrams illustrating the changes in proportions of task-responsive neurons (A, top), spikes proportions (A, bottom), and ensemble activity (B) that occur in the sensorimotor striatum as rats learn to perform a conditional T-maze task. The pseudocolor scale indicates the population spike activity (red: high activity). The ensemble activity patterns gradually shift toward emphasizing the beginning and end of the maze-runs. A: from Jog et al., 1999. B: from Barnes et al., 2005.

Figure 10. Diagram illustrating the idea of chunking of action repertoires, whereby entire sequences of behavior, accepted by cortico-basal ganglia circuits as valuable, are marked by activity bracketing the entire sequence.

became stronger toward the beginning and the end of the runs (Fig. 9). Fewer neurons, on average, had high activity during the middle of the runs (Jog et al., 1999).

The Idea of Chunking of Action Repertoires as a Function of Corticostriatal Circuits This learning-related pattern suggested that as the animals acquired the maze runs as habits, the activity in the sensorimotor striatum was reconfigured so as to bracket the entire learned behavior. This was an idea that we began to explore in further experiments. With Terra Barnes, we found that this pattern could be seen in firing rate plots as well as cell count plots, and strikingly, that if we took away the rewards, in extinction trials, the pattern faded, but if we put back the rewards, the pattern reappeared almost immediately (Barnes et al., 2005). This meant that as the animals learned, patterns of spike firing were laid down in the striatum in such a way that they could be covert or overt, depending on the situation. I likened this to the notion of chunking that had been introduced by George Miller as a way to make memories manageable: maybe when we learn behaviors, the sensorimotor striatum brackets the ones that have the most positive and least negative value (Graybiel, 1998). And at the same time, 'expert neurons' developed, freeing many other striatal neurons from full-time duty in driving the habitual behavior. Thus through the stream of behavior, episodes could be marked as scripts ready to be called up (Fig. 10). Action Circuits and Permissive Circuits: An Idea about Corticostriatal Circuit Dynamics Katy Thorn and I decided to ask whether similar patterns would form in the 'associative' part of the striatum, the part receiving inputs form the association areas of the neocortex. The answer was no. In fact, in the medial part of the stratum, the activity patterns were almost the reverse of those in the sensorimotor striatum (Thorn et al., 2010). The activity was strongest mid-run! We developed the idea that the activity in the associative part of the stratum had to die down before the

Ann Martin Graybiel

9

beginning-and-end pattern in the sensorimotor stratum could actually take over control of the behavior (Thorn et al., 2010). This was a proposal for a new dynamic: one in which one set of cortico-basal ganglia loops could control another set during the course of learning.

We had not yet proved that the maze runs in the over-trained rats actually were habits. Only much more recently, when Kyle Smith joined the lab, could we do this. Kyle introduced to our lab the devaluation protocol that psychologists use to test for the presence of habitual behavior. With this protocol, our over-trained rats continued to do the same behaviors over and over again even when the former reward had lost its positive value (Smith et al., 2012). The maze runs in the highly trained animals passed the test for being habits!

The idea of an executive network overseeing habitual behavior is an idea that psychologists had formulated from lesion studies of the cortex. In new experiments, Kyle and I decided to record simultaneously in the medial prefrontal cortex and the sensory motor striatum — the two regions singled out as 'habit-related' in prior lesion work — as our rats learned maze tasks. We found that the familiar beginning-and-end pattern formed not only in the sensorimotor striatum, but also in the medial prefrontal cortex. But, unlike the striatal pattern, the prefrontal pattern only emerged as the habitual behavior became deeply ingrained during the over-training period. Moreover, when the animals were then given devaluation exposure to degrade the reward value of one of the maze rewards, the sensorimotor striatal task-bracketing pattern was rock-steady, but the prefrontal cortical bracketing pattern became obscured. It was as though once a habit is engrained, its bracketing representation in the sensorimotor striatum survives changes in reward value, but the neocortical bracketing pattern remains sensitive to reward value and quickly weakens. These findings are fresh and only now being submitted for publication.

This result has led us to test for the effects of inhibiting this region of the medial prefrontal cortex by using on-line temporally controlled optogenetic inhibition with virally delivered opsin contributed by Karl Deisseroth. We not only could block the habitual behavior, as predicted from earlier lesion work, but also turn the habit back on later with a reapplication of the optogenetic silencing to the same cortical site! We now can, in effect, toggle the habits on and off. This work suggests that Pavlov was quite right when he suggested that we never forget a habit, we just inhibit it. We are hard at work on further experiments now. Cost-Benefit Coding in the Primate Striatum and Neocortex Once we knew that we could record in rodents, I, with some trepidation, decided to set up recordings in behaving primates. Luckily, Jun Kojima, Naotaka Fujii and Pablo Blazquez joined the lab. The beginnings were almost amusing. We had one totally empty room. Jun came first, from Japan, where he had learned monkey neurophysiology, and he was still learning English. I myself knew English, all right, but I had never set up a monkey lab. But eventually, it all worked. Pablo and I found, by using airpuffs for negative reinforcement, that many of the tonically active neurons responded to the unrewarding air puffs as well as to rewarding juice treats (Blazquez et al., 2002). That is, both cost and punishment could modulate their firing rates. Moreover, the development of these responses beautifully matched the development of EMG activity in the orbicularis oculi muscle, which controls eye blinks. Having had exposure early on to the invertebrate field, I was thrilled that we could actually, in non-human primates, record from a single identified neuronal cell type and record activity in a single muscle and find a highly systematic relation between the two

Ann Martin Graybiel

10

Figure 11. Responses of tonically active neurons are tightly related with the conditioned eye blink behavior. (A) Proportions of conditioned responses (red) and neuronal responses (blue) during conditioning (shaded) and extinction (non-shaded) training. (B) Significant correlations between behavioral and neuronal responses. (C) Neuronal responses predict the occurrence of behavioral responses. From Blazquez et al., 2002.

Figure 12. Accentuated activity of neurons in the prefrontal cortex at the start and end of sequences of saccadic eye movements. (A) Task cartoon. (B) Raster plot (above) and peri-stimulus time histogram (below) illustrating the spike activity of a single prefrontal neuron. The monkey made four saccades. Note the fifth, ‘extra’, peak of activity. From Fujii and Graybiel, 2003.

(Fig. 11). In fact, with the activity of about 150 of the tonically active neurons we could predict the occurrence of a blink with a 90% probability of being accurate!

This contrasting reward and punishment sensitivity has become important in current studies of the cortico-basal ganglia system, and became important for experiments that Ken-ichi Amemori and I have done using approach-avoidance conflict paradigms to study neural responses in the cortex and striatum. Action Boundary Coding by Primate Corticostriatal Ensembles As we were working on the early maze experiments in the rats, Naotaka Fujii and I found responses in the prefrontal cortex of monkeys that, as in the rodent experiments, formed a task-bracketing pattern around a well-learned behavior — saccadic eye movement sequences that the monkeys made in response to visual cues (Fujii and Graybiel, 2003). It was as though action-boundaries formed in the prefrontal cortical ensembles, and also in neuronal ensembles in the oculomotor part of the stratum. We found that at the end of a given sequence, many neurons produce a single burst of activity — an 'extra peak' we called it — as though literally marking the end (Fig. 12). This activity didn't appear to be related to reward delivery — instead, it seemed to be related to ending the sequence. Theresa Desrochers now is studying the evolution of such bracketing responses during untutored habit formation

Ann Martin Graybiel

11

Figure 13. Evidence for a neural representation of time in cortico-basal ganglia circuits illustrated in the output of perceptrons driven by prefrontal (A and B) or by striatal (C and D) inputs. Maximum margins for the perceptrons with prefrontal and striatal inputs discriminate single time points from all others at least 50 msec apart. Gray trace indicates the noise level. B and D illustrate weighted sums of inputs for three perceptrons. Red dots indicate the times decoded. From Jin et al., 2009.

in primates. This activity is of much interest to us now as we explore oscillatory patterns of activity occurring in striatal and cortical networks during learning.

Dezhe Jin, a physicist, helped us look at the fine timing of the responses we found in the prefrontal cortex and oculomotor striatum as the monkeys performed the saccade sequences. What we found was that time is embedded in these representations. With perceptron models, we could map task-time with a resolution of at least 50 milliseconds for the prefrontal and striatal ensembles (Fig. 13) (Jin et al., 2009). This was a critical finding; in the maze experiments, we could not separate navigational distance and navigational time. We suggested that these responses might provide the time-stamp representations predicted by spectral timing theory and by reinforcement learning models. Multiple Patterns of Plasticity Emerge Simultaneously in Striatal Circuits as Learning Occurs Altogether, these experiments in both rats and mice and in monkeys suggest that learning-related representations are developed in the striatum as habits form, that such representations form in the neocortex also, and that, among these, task-bracketing and end-task activation patterns occur as a result. From the devaluation work, we are finding that the action-boundary representations in the sensorimotor striatum are stable to changes in value, but that such patterns are not stable in the medial prefrontal cortical region. Activity in this region, as our optogenetic experiments show, is necessary for the performance of the maze behavior as a habit rather than as a goal-directed behavior (Smith et al., 2012). This last finding is curious, as it implies that even nearly automatic behaviors are, despite their apparent automaticity, monitored on a moment-to-moment basis by a neural system that must give permission for the automaticity.

The fact that habitual behavior has some fixed representations and some representations that are malleable and responsive to current reinforcement conditions suggests that network-level neural activity must be coordinated to ensure smooth transitions in behavior. By now, we have found that multiple learning-related patterns of activity develop simultaneously in different parts of the striatum during learning. Task-bracketing patterns form in the sensorimotor striatum, complementary decision-period activity in the medial striatum, and cue-plus-reward-related activity develops in the ventral striatum, activity that is quite similar to that reported for the dopamine-containing neurons of the midbrain in cue-plus-reward conditioning contexts (Atallah et al., in prep.). Others in the field are finding these patterns as well. This diversity means that

Ann Martin Graybiel

12

Figure 14. Illustration demonstrating oscillatory activity recorded in the ventral striatum early during T-maze learning. Power in the gamma range (A and C) decreases with learning, whereas power in the beta range (B and D) increases with learning. From Howe et al., 2011.

different cortico-basal ganglia loops are simultaneously active and developing contrasting patterns as habits are acquired. How do they become integrated? Oscillatory Local Field Potential Activity in the Striatum Changes during Learning and Is Reorganized in Relation to Local Spiking Activity Almost certainly, oscillatory activity indicative of synchrony in networks of neurons is associated with this plasticity in spike patterning. We began to look at the frequency domain in experiments with Richard Courtemanche. We found little zones in which striatal beta activity popped out of synchrony with the beta activity recorded around them when task-related spike activity in the little zones increased (Courtemanche et al., 2003). We think that these might be matrisomes. These experiments alerted us: there was much beta activity in the striatum even in normal animals, and it was modulated according to on-going behavior.

Having Richard in the lab helped as we began to look at oscillatory activity in the rodent maze experiments. With Bill DeCoteau and others in our group, we found that coordination between theta activity in the stratum and theta activity in the hippocampus, peaking during the decision-period of the task, predicted whether individual rats would learn the task successfully (DeCoteau et al., 2007). Mark Howe, Hisham Atallah, Dan Gibson and I have found that in the ventral striatum, the spike firing of the striatal neurons occurring at the end of the maze runs becomes temporally tuned to fleeting bursts of oscillatory activity that are mainly at one frequency early in learning (a gamma frequency) but then shift to a lower (beta) frequency late in learning (Howe et al., 2011). We interpret these experiments as suggesting that the global organization of the ventral striatal networks changes during learning — likely reflecting more widespread reorganization occurring in larger networks. And the detailed patterns suggest that this process has to do with reorganizing the networks to mark the end of the task and to adjust the network plasticity state as a function of learning, as suggested by the 'extra peak' findings in our primate work (Fig. 14). Moreover, with Joey Feingold and Dan Gibson, working with monkeys making series of voluntary joystick movements to visual targets, we are also finding a remarkable end-of-task oscillatory bursting, with fleeting, coordinated beta bursts in the striatum and frontal cortex. The field of brain dynamics is opening up, and we are excited to see these inklings of how powerfully the dynamics in corticostriatal circuits are influenced by learning.

One huge missing piece in this work is any idea of how the striosome-matrix story fits in, let alone the striosome-matrisome patterns in detail. This has been a

Ann Martin Graybiel

13

Figure 15. Illustration of the high correlation between levels of stereotypical behavior elicited by repeated treatments with direct and indirect dopamine receptor agonists (y axis) and the degree to which the drug treatments induced differential labeling of striosomes after the final challenge dose (x axis). From Canales and Graybiel, 2000.

formidable technical challenge. We are working full tilt trying to fill this gap, with Alexander Friedman and Leif Gibb leading the effort in the rodents and with Ken-ichi Amemori, Satoko Amemori, Hideki Shimazu, Patrick Tierney and, most recently, Simon Hong in primates. We have an excited, pioneering feeling about all this — and as I will return to below, there is real hope that we can solve at least part of the puzzle. Another Approach: Circuit-Level Reorganization of Gene Expression in the Striatum During these years of setting up physiology, I continued to wish that we could 'find' striosomes functionally. The excitation of immediate early genes was just being introduced as a way to mark active neurons (I learned this from Steve Hunt, who was working on spinal cord pain pathways in England), and I immediately decided to give rats a dose of a dopamine agonist drug — the habit forming drug amphetamine — to try to activate striosomes. I cut the first brains too excitedly, so rapidly that there were knife marks on the sections, but it was clear what the stain showed: in the rostral part of the striatum, striosomes were differentially labeled with the early-gene protein stains. This was the first 'functional' view that we had had of the striosomes (Graybiel et al., 1990). Rosario Moratalla came to the lab at nearly the same time, and I had met Harry Robinson, who was working with the gene method that he, too, had learned in England, and he joined in those early experiments. I have never gotten over the feeling of amazement that one dose of a drug such as amphetamine could so powerfully and immediately influence the expression of genes in the brain. Epigenetics is now a major field of study, but for anyone thinking about taking a drug, this response should be thought provoking.

Rosario Moratalla and I then found that if we gave the drug repeatedly, the striosome predominance of the patterns became even stronger (Moratalla et al., 1996). When Juan Canales joined the lab, he brought his trained eyes to the experiments and taught us how to score the repetitive behaviors that developed in the animals given the repeated dosing. The animals developed stereotypical behaviors, and we found that the strength of these repetitive behaviors was highly correlated with the levels of striosome- predominant gene expression in the same animals (Fig. 15) (Canales and Graybiel, 2000). Bulent Elibol and then Essen Saka, neurologists from Turkey, joined the lab, and Essen and I teamed up with the New England Primate Research Center's Bertha Madras to do such experiments in monkeys. There, too, chronic dosing led to increased striosome-predominant expression of early response genes, and this expression was highly correlated with the emergence of stereotypical behaviors (Saka et al., 2004). But correlations, enticing as they are, do not provide causal evidence.

I had become determined, as we began the physiology, to try to find striatum-enriched genes so that we could manipulate the system. I learned how to use a PCR machine and to do simple procedures, especially from Moses Chow, and then set up a tiny bit of equipment for doing gene

Ann Martin Graybiel

14

and protein work. I was lucky to enlist the collaboration of Brent Cochran and later David Housman, and lucky that Hiroaki Kawasaki came to the lab. We worked together and eventually discovered a novel class of genes that had at one end second-messenger binding motifs, and at the other end motifs for guanine nucleotide exchange factors that targeted members of the ras superfamily. One pair bound cAMP (Kawasaki et al., 1998a), the other pair calcium and diacylglycerol (Kawasaki et al., 1998b). We called them the cAMP-GEFs and the CalDAG-GEFs. We were lucky. One of the CalDAG-GEFs was highly enriched in striosomes, the other in the matrix. And the matrix-enriched gene, CalDAG-GEFI, was expressed in much lower levels elsewhere. They acted on the MAP kinase/ERK pathways, we found in cell assays. Jill Crittenden came to the lab and made knockouts of the CalDAG-GEFs, and later the other genes, and we have worked together for some years to characterize the behavioral and signaling functions of the genes thanks to her engineering of knockout mice (Crittenden et al., 2009; Crittenden et al., 2010; Crittenden and Graybiel, 2011). Deletion of the matrix-enriched CalDAG-GEFI produces a tendency toward stereotypical, overly focused behavior — just what we might expect if matrix function were decreased relative to striosome function and the striosome-matrix balance idea for controlling the degree of repetitiveness of behavior were correct (Crittenden et al., in prep). And the knockouts have a mild learning deficit. The combination of symptoms in the mice is striking.

With our coworkers, including Carolyn Lacey, we now have the idea that the cholinergic system is critically affected in the mice, which could fit with the heightened expression of CalDAG-GEFI in the matrix compartment and the links between cholinergic control and repetitive behavior suggested by our work and that of others. Putting this work together with other features of striosome-matrix compartmentation is an exciting prospect in view of the potential relevance for these genes in the human brain. Striosomes and Limbic Circuits Thanks to work in many labs, including our own, we quite early on had at least indirect evidence that striosomes were related to the limbic system. With Frank Eblen, we made an explicit attempt to find out what regions of the frontal cortex project to striosomes in non-human primate experiments. Of all the sites in the frontal cortex that we injected with tracer, there were only two from which we could trace differentially strong labeling in striosomes: the far-anterior anterior cingulate cortex (now called the pregenual anterior cingulate cortex, pACC for short), and the far-caudal orbitofrontal cortex (Eblen and Graybiel, 1995). It was just becoming known in then-emerging PET human brain scanning studies that these regions had abnormal activity in states of addiction and in obsessive-compulsive disorder. For us, this was remarkable: drugs that could induce addictive states, particularly on repeated use, preferentially activated genes in the anterior striatum and induced repetitive stereotypical behaviors. Frank and I had found inputs to these anterior striosomes from regions that seemed to correspond to those that, in the human, were abnormal in addictive and repetitive behavioral disorders (Fig. 16).

The anterior cingulate-orbitofrontal link suggested by the experiments with Frank Eblen, along with the other links that we and others found (for example, with the midline thalamus (Ragsdale and Graybiel, 1991)), led us to the notion that striosomes might interact with regions related to emotional control at higher and lower levels on a scale from autonomic-endocrine and

Ann Martin Graybiel

15

Figure 16. Striosomes receive inputs from the pregenual anterior cingulate cortex and the posterior orbitofrontal cortex. Schematic diagrams illustrating sites of anterograde tracer injection made in the anterior cingulate cortex and orbitofrontal cortex of macaque monkeys. Of all the sites, only those illustrated in red produced differential labeling of striosomes. From Eblen and Graybiel, 1995.

visceromotor to deliberative behavior (Graybiel, 2008). This set up the possibility that these dispersed striosomal regions might bring a form of ‘limbic’ control to the sensorimotor-associative processing going on in the much larger, surrounding matrix. From the many experiments that we and others did, there were some remarkable instances of fairly sharp divisions between the two compartments: the dendrites of striatal neurons often tended to shy away from crossing the borders, something that we examined in some detail with Ruth Walker and Gordon Arbuthnott (Walker et al., 1993; Walker and Graybiel, 1993), and with Marie-Francoise Chesselet; but there were also instances of border-crossings, as we called them. Especially interesting was the idea that the cholinergic interneurons, which we had found often to lie next to striosomes, might be integrators across the boundaries, an idea pursued elegantly by Aosaki and Kawaguchi (Aosaki and Kawaguchi, 1996; Miura et al., 2008).

All of this finally can be studied in detail now that genetically assisted identification of neurons, viral transfection methods and optogenetic and other methods for manipulating activity methods are becoming available. We have gone from barely being able to trace axons to their terminals — the condition when I began in the field — to being able to identify neurons and their processes in complete detail. This must be akin to what ardent surfers feel as they catch wave after wave — we in neuroscience have the incredible privilege of using wave after wave of new methods! Striosomes as Sources of Direct Input to Dopamine-Containing Neurons of the Midbrain One other, key suggestion from quite early on, based on indirect evidence, was that striosomes projected to or near to the dopamine-containing neurons in the midbrain. This issue has still not been fully settled, but current single-fiber and targeted tracing done with genetically assisted methods (Fujiyama et al., 2011; Watabe-Uchida et al., 2012) strongly suggests that neurons in striosomes may be the only striatal neurons that project directly to the dopamine-containing neurons. This situation would mean that striosomes are in a position to influence the very dopaminergic system that can signal to other brain sites, including the striatum, about the salience and reinforcement value of stimuli. This possibility suggests a powerful functional position for striosomes not only in influencing reinforcement-guided behavior but also in influencing the cognitive states that accompany visceromotive states. The striosomes, themselves, by receiving limbic/emotion-related inputs, may be part of larger networks controlling positive and negative emotional states and their bodily concommitants.

Some time ago, to begin to convey this idea, I proposed that the outputs of the basal ganglia to the neocortex (via the thalamocortical systems with which they connect) might help to build cognitive patterns much as classic studies have viewed them as control centers for the central pattern generators of the brainstem and spinal cord (Graybiel, 1997). The nervous system probably

Ann Martin Graybiel

16

Figure 17. Microstimulation of the pregenual anterior cingulate cortex in its striosome-projecting region influences cost-benefit decision-making in monkeys. The monkeys made decisions to approach (receive) or avoid combinations of food and airpuff. The diagram shows avoidance responses as squares and approach responses as plus signs. The average pre-stimulation decision boundary is shown by the interrupted red line, and the average of the decisions made during stimulation as shown by the solid line. From Amemori and Graybiel, 2012.

has techniques for using very simple mechanisms for creating behaviors that seem very complex. Some of the signals that we have found — like the extra peaks, the oscillatory bursts and the beginning-and-end signals — seem like good candidates for such computational or algorithmic signals. Striosome-Related Circuits and Emotional Decision-Making Ken-ichi Amemori and I decided, based on all of this accumulating evidence, that it would be important to record from striosome-projecting cortex to gain a hint of what striosomes might do. Ken had developed the idea that striosomes might be important for, and perhaps integrate, both positive and negative reinforcement signals transmitted to the striatum. Until we could record from the striosomes themselves (the technical challenge that we are working on now), studying striosome-projecting cortex could be highly informative. Ken and I adapted for use in monkeys a well known test used to study human emotion, an approach-avoidance conflict task. The monkeys would receive both positive (juice) and negative (airpuff) reinforcement, not one or the other. This meant that the monkeys would have to decide how much reward made how much airpuff worthwhile to experience. Out of this idea, and many long hours of recording in the pACC and related cingulate cortex, we came up with an intriguing result (Amemori and Graybiel, 2012): there were large numbers of anterior cingulate neurons that are active during the decision-making, with about equal numbers of these firing more in relation to positive expected outcome or more for negative expected outcome. But in one site, right in the pACC, and apparently corresponding to the striosome-projecting zone, there were more of the negative type. Microstimulating that region, but not the nearby regions, pushed the monkeys' decisions toward increased pessimism (more avoidance; Fig. 17). And this effect could be cleanly reversed by giving the monkeys a dose of an anxiolytic drug (diazepam).

These experiments brought us right back to thinking about potential striosome-related circuits and emotional control. Surprisingly, there seems to be no effect of the same microstimulation when the monkeys have to decide between two rewards of different size (approach-approach conflict). This set of results could tie in with the discoveries by Okehide Hikosaka and his colleagues (Matsumoto and Hikosaka, 2007) that there is a negative reinforcement pathway leading from the habenula toward the dopamine-containing neurons of the midbrain. Striosomes were found by Rajakumar to project, indirectly, to the same part of the habenula (Rajakumar et al., 1993).

Ken-ichi Amemori, Leif Gibb, Alexander Friedman and I are interested in the idea that striosomes might, together with the nearby cholinergic interneurons, form a dispersed set of decision units that, based on current context could weight relative good and bad and influence basal ganglia outflow (Amemori et al.,

Ann Martin Graybiel

17

2011). Striosomes could provide the responsibility signals in a modular hierarchical learning structure, as introduced by Doya and coworkers. This idea could fit well with the mixture-of-experts template suggested by the earlier anatomy. Perhaps striosomes affect decision-making when relative cost and benefit decisions are made. This idea makes the broad distributions of the striosomes particularly interesting: they might form a distributed biasing mechanism within the striatum — a gridwork for value (Graybiel, 2009). Striatal Compartments in Relation to Neurologic and Neuropsychiatric Disorders So far, the striosome-matrix architecture appears to be beyond the reach of human brain imaging methods; getting adequate resolution at such a great depth has been a roadblock. Frustrating as this has been, there are now hints from anatomical work on post-mortem brains that striosome and matrix compartments are differentially affected in some disorders in humans. With Richard Faull and his team in New Zealand, I had the privilege to join in examining the brains of patients who had suffered from Huntington's disease. This work suggested that degeneration was more likely to occur in striosomes in those individuals who had had early mood dysfunction than in those with early motor signs (Tippett et al., 2007). Other work on human brains has reinforced the idea that striosomes, or an imbalance in relative striosome/matrix function, might contribute to repetitive behavior. Brotchie and colleagues found that in the brains of Parkinson's patients who had suffered L-DOPA induced dyskinesias, striosomes had differentially increased pre-proenkephalin expression, relative to the matrix (Henry et al., 2003). In our own lab, we worked with William Langston to study MPTP-treated monkeys and found differential sparing of dopamine transporter binding in striosomes in parts of the striatum in which the transporter signal was nearly undetectable in the matrix. And in work on the brains of patients who suffered from DYT-3 dystonia, Goto and colleagues have reported differential loss of striosomes, which they relate to the severe mood and motor dysfunction in these patients (Goto et al., 2005). Adding to this are studies in animal models, including our own (Crittenden and Graybiel, 2011).

These and other studies point to the striosome-matrix architecture as one in which compartmentally selective neurodegeneration could be important in disease. Striosomal neurons are born and migrate out into the developing striatum very early, and they are for a prolonged developmental period more mature than are the later-born neurons moving into the matrix. Early genetic or environmental insults, including those occurring during the prenatal period, could differentially affect these compartments and their developing specialized circuits. I hope that imaging methods will soon allow us in the field to see into the depths of the forebrain with adequate resolving power to help in identifying differential patterns of compartmental involvement.

The Future I am writing this essay just as methods are finally becoming powerful enough to let scientists approach the deep brain with methods as powerful as those for some time available for work on the neocortex. But it already seems likely that aspects of our lives that are very basic to us as people depend on these deep brain regions. I have emphasized here our ability to move smoothly from deliberative, conscious decision-making to semi-automatic behavior, our capacity to include emotional states within our behavioral guidance systems as

Ann Martin Graybiel

18

individuals and as communities of people, and our vulnerability to states and disorders that pull apart the balance of systems that seem to control these functions and our mental equilibrium. Given all of this, it is no wonder that the catalogue of basal ganglia disorders has expanded from the original extrapyramidal disorders — hypokinetic and hyperkinetic disorders and dystonias — to include neuropsychiatric disorders ranging from obsessive-compulsive disorder and related OC-spectrum disorders to mania and depression and to autistic and attention deficit disorders. This realization adds great inspiration to work in this field, a source of inspiration that is matched by the beauty of the brain itself. Acknowledgments I have been graced with talented students, post-doctoral fellows, and staff members, to each one of whom I am grateful. I have mentioned the names of many of those who worked with me on the striatum. I especially thank Diane Major and Henry Hall, who have been long-time lab members and friends, along with Pat and Ray Harlan and Hu Dan. We run our lab as nearly a family, and I register my thanks to each person as a tribute to what can come from talented, energized, and dedicated people who work toward a great goal — in our case, understanding the brain and trying to use that information to help humankind. References Amemori K, Gibb LG, Graybiel AM (2011) Shifting responsibly: the importance of striatal

modularity to reinforcement learning in uncertain environments. Front Hum Neurosci 5:47. Amemori KI, Graybiel AM (2012) Localized microstimulation of primate pregenual cingulate cortex

induces negative decision-making. Nat Neurosci. Aosaki T, Kawaguchi Y (1996) Actions of substance P on rat neostriatal neurons in vitro. J Neurosci

16:5141-5153. Aosaki T, Graybiel AM, Kimura M (1994) Effects of the nigrostriatal dopamine system on acquired

neural responses in the striatum of behaving monkeys. Science 265:412-415. Aosaki T, Kimura M, Graybiel AM (1995) Temporal and spatial characteristics of tonically active

neurons of the primate's striatum. J Neurophysiol 73:1234-1252. Barnes T, Kubota Y, Hu D, Jin DZ, Graybiel AM (2005) Activity of striatal neurons reflects dynamic

encoding and recoding of procedural memories. Nature 437:1158-1161. Blazquez P, Fujii N, Kojima J, Graybiel AM (2002) A network representation of response probability

in the striatum. Neuron 33:973-982. Canales JJ, Graybiel AM (2000) A measure of striatal function predicts motor stereotypy. Nat

Neurosci 3:377-383. Courtemanche R, Fujii N, Graybiel A (2003) Synchronous, focally modulated ß-band oscillations

characterize local field potential activity in the striatum of awake behaving monkeys. J Neurosci 23:11741-11752.

Crittenden JR, Graybiel AM (2011) Basal Ganglia disorders associated with imbalances in the striatal striosome and matrix compartments. Front Neuroanat 5:59.

Crittenden JR, Cantuti-Castelvetri I, Saka E, Keller-McGandy CE, Hernandez LF, Kett LR, Young AB, Standaert DG, Graybiel AM (2009) Dysregulation of CalDAG-GEFI and CalDAG-GEFII

Ann Martin Graybiel

19

predicts the severity of motor side-effects induced by anti-parkinsonian therapy. Proc Natl Acad Sci U S A 106:2892-2896.

Crittenden JR, Dunn DE, Merali FI, Woodman B, Yim M, Borkowska AE, Frosch MP, Bates GP, Housman DE, Lo DC, Graybiel AM (2010) CalDAG-GEFI down-regulation in the striatum as a neuroprotective change in Huntington's disease. Hum Mol Genet 19:1756-1765.

DeCoteau WE, Thorn CA, Gibson DJ, Courtemanche R, Mitra P, Kubota Y, Graybiel AM (2007) Learning-related coordination of striatal and hippocampal theta rhythms during acquisition of a procedural maze task. Proc Natl Acad Sci U S A 104:5644-5649.

Eblen F, Graybiel AM (1995) Highly restricted origin of prefrontal cortical inputs to striosomes in the macaque monkey. J Neurosci 15:5999-6013.

Flaherty AW, Graybiel AM (1991) Corticostriatal transformations in the primate somatosensory system. Projections from physiologically mapped body-part representations. J Neurophysiol 66:1249-1263.

Flaherty AW, Graybiel AM (1993) Output architecture of the primate putamen. J Neurosci 13:3222-3237.

Flaherty AW, Graybiel AM (1994) Input-output organization of the sensorimotor striatum in the squirrel monkey. J Neurosci 14:599-610.

Fujii N, Graybiel A (2003) Representation of action sequence boundaries by macaque prefrontal cortical neurons. Science 301:1246-1249.

Fujiyama F, Sohn J, Nakano T, Furuta T, Nakamura KC, Matsuda W, Kaneko T (2011) Exclusive and common targets of neostriatofugal projections of rat striosome neurons: a single neuron-tracing study using a viral vector. Eur J Neurosci 33:668-677.

Giménez-Amaya JM, Graybiel AM (1990) Compartmental origins of the striatopallidal projection in the primate. Neuroscience 34:111-126.

Goto S, Lee LV, Munoz EL, Tooyama I, Tamiya G, Makino S, Ando S, Dantes MB, Yamada K, Matsumoto S, Shimazu H, Kuratsu J, Hirano A, Kaji R (2005) Functional anatomy of the basal ganglia in X-linked recessive dystonia-parkinsonism. Ann Neurol 58:7-17.

Graybiel AM (1984) Correspondence between the dopamine islands and striosomes of the mammalian striatum. Neuroscience 13:1157-1187.

Graybiel AM (1990) Neurotransmitters and neuromodulators in the basal ganglia. Trends Neurosci 13:244-254.

Graybiel AM (1997) The basal ganglia and cognitive pattern generators. Schizophr Bull 23:459-469. Graybiel AM (1998) The basal ganglia and chunking of action repertoires. Neurobiol Learn Mem

70:119-136. Graybiel AM (2008) Habits, rituals and the evaluative brain. Annu Rev Neurosci 31:359-387. Graybiel AM (2009) Dynamic templates for neuroplasticity in the striatum. In: Dopamine Handbook

(Iversen LL, Dunnett SB, Björklund A, eds), pp 333-338. Oxford, UK: Oxford University Press. Graybiel AM, Ragsdale CW, Jr. (1978) Histochemically distinct compartments in the striatum of

human, monkey, and cat demonstrated by acetylthiocholinesterase staining. Proc Natl Acad Sci U S A 75:5723-5726.

Graybiel AM, Ragsdale CW, Jr. (1980) Clumping of acetylcholinesterase activity in the developing striatum of the human fetus and young infant. Proc Natl Acad Sci U S A 77:1214-1218.

Ann Martin Graybiel

20

Graybiel AM, Hickey TL (1982) Chemospecificity of ontogenetic units units in the striatum: Demonstration by combining [3H] thymidine neuronography and histochemical staining. Proc Natl Acad Sci U S A 79:198-202.

Graybiel AM, Moratalla R, Robertson HA (1990) Amphetamine and cocaine induce drug-specific activation of the c-fos gene in striosome-matrix compartments and limbic subdivisions of the striatum. Proc Natl Acad Sci U S A 87:6912-6916.

Graybiel AM, Aosaki T, Flaherty AW, Kimura M (1994) The basal ganglia and adaptive motor control. Science 265:1826-1831.

Graybiel AM, Pickel VM, Joh TH, Reis DJ, Ragsdale CW, Jr. (1981) Direct demonstration of a correspondence between the dopamine islands and acetylcholinesterase patches in the developing striatum. Proc Natl Acad Sci U S A 78:5871-5875.

Henry B, Duty S, Fox SH, Crossman AR, Brotchie JM (2003) Increased striatal pre-proenkephalin B expression is associated with dyskinesia in Parkinson's disease. Exp Neurol 183:458-468.

Howe MW, Atallah HE, McCool A, Gibson DJ, Graybiel AM (2011) Habit learning is associated with major shifts in frequencies of oscillatory activity and synchronized spike firing in striatum. Proc Natl Acad Sci U S A 108:16801-16806.

Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction, and functional architecture in the cat's visual cortex. J Physiol 160:106-154.

Jiménez-Castellanos J, Graybiel AM (1989) Compartmental origins of striatal efferent projections in the cat. Neuroscience 32:297-321.

Jin DZ, Fujii N, Graybiel AM (2009) Neural representation of time in cortico-basal ganglia circuits. Proc Natl Acad Sci U S A 106:19156-19161.

Jog M, Kubota Y, Connolly CI, Hillegaart V, Graybiel AM (1999) Building neural representations of habits. Science 286:1745-1749.

Kawasaki H, Springett GM, Mochizuki N, Toki S, Nakaya M, Matsuda M, Housman DE, Graybiel AM (1998a) A family of cAMP-binding proteins that directly activate Rap1. Science 282:2275-2279.

Kawasaki H, Springett GM, Toki S, Canales JJ, Harlan P, Blumenstiel JP, Chen EJ, Bany IA, Mochizuki N, Ashbacher A, Matsuda M, Housman DE, Graybiel AM (1998b) A Rap guanine nucleotide exchange factor enriched highly in the basal ganglia. Proc Natl Acad Sci U S A 95:13278-13283.

Malach R, Graybiel AM (1986) Mosaic architecture of the somatic sensory-recipient sector of the cat's striatum. J Neurosci 6:3436-3458.

Matsumoto M, Hikosaka O (2007) Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447:1111-1115.

Miura M, Masuda M, Aosaki T (2008) Roles of micro-opioid receptors in GABAergic synaptic transmission in the striosome and matrix compartments of the striatum. Mol Neurobiol 37:104-115.

Moratalla R, Elibol B, Vallejo M, Graybiel AM (1996) Network-level changes in expression of inducible Fos-Jun proteins in the striatum during chronic cocaine treatment and withdrawal. Neuron 17:147-156.

Mountcastle VB (1957) Modality and topographic properties of single neurons of cat's somatic sensory cortex. J Neurophysiol 20:408-434.

Ann Martin Graybiel

21

Nastuk MA, Graybiel AM (1985) Patterns of muscarinic cholinergic binding in the striatum and their relation to dopamine islands and striosomes. J Comp Neurol 237:176-194.

Nastuk MA, Graybiel AM (1988) Autoradiographic localization and biochemical characteristics of M1 and M2 muscarinic binding sites in the striatum of the cat, monkey, and human. J Neurosci 8:1052-1062.

Parthasarathy HB, Schall JD, Graybiel AM (1992) Distributed but convergent ordering of corticostriatal projections: Analysis of the frontal eye field and the supplementary eye field in the macaque monkey. J Neurosci 12:4468-4488.

Ragsdale CW, Jr., Graybiel AM (1991) Compartmental organization of the thalamostriatal connection in the cat. J Comp Neurol 311:134-167.

Rajakumar N, Elisevich K, Flumerfelt BA (1993) Compartmental origin of the striato-entopeduncular projection in the rat. J Comp Neurol 331:286-296.

Romo R, Schultz W (1990) Dopamine neurons of the monkey midbrain: contingencies of response to active touch during self-initiated arm movements. J Neurophysiol 63:592-606.

Saka E, Goodrich C, Harlan P, Madras BK, Graybiel AM (2004) Repetitive behaviors in monkeys are linked to specific striatal activation patterns. J Neurosci 24:7557-7565.

Smith KS, Virkud A, Deissertoth K, Graybiel AM (2012) Reversible on-line control of habitual behavior by optogenetic silencing of medial prefrontal cortex. Proc Natl Acad Sci U S A, under review.

Thorn CA, Atallah H, Howe M, Graybiel A (2010) Differential dynamics of activity changes in dorsolateral and dorsomedial striatal loops during learning. Neuron 66:781-795.

Tippett LJ, Waldvogel HJ, Thomas SJ, Hogg VM, van Roon-Mom W, Synek BJ, Graybiel AM, Faull RLM (2007) Striosomes and mood dysfunction in Huntington's disease. Brain 130:206-221.

Walker RW, Graybiel AM (1993) Dendritic arbors of spiny neurons in the primate striatum are directionally polarized. J Comp Neurol 337:629-639.

Walker RW, Arbuthnott GW, Baughman RW, Graybiel AM (1993) Dendritic domains of medium spiny neurons in the primate striatum: Relationships to striosomal borders. J Comp Neurol 337:614-628.

Watabe-Uchida M, Zhu L, Ogawa SK, Vamanrao A, Uchida N (2012) Whole-brain mapping of direct inputs to midbrain dopamine neurons. Neuron 74:858-873.