Abstract of “Primate frontal eye fields mediate spatial attention in
Transcription
Abstract of “Primate frontal eye fields mediate spatial attention in
Abstract of “Primate frontal eye fields mediate spatial attention in covert visual search” by Ilya E. Monosov Ph.D., Brown University, May 2010. Visual spatial attention serves to select locations of interest in the visual field and enhances the cortical representation of objects at those locations. Previous studies suggest that neural activity in primate frontal eye fields (FEF) is involved in the spatial selection of salient stimuli in complex visual environments for eye movements and spatial attention. Here, we explore the origin of the spatial selection signal in FEF and its relationship to measures of covert spatial attention. We compare the timing of spatial selection for the location of the target in two simultaneously recorded cortical signals: local field potentials (LFPs) and spikes. LFPs are thought to represent synaptic input, while spiking activity is the output, of the area around the electrode tip. We found that spatial selectivity identifying the location of the target in the visual search appeared in the spikes about 30 ms before it appeared in the LFPs. This suggests that the spatial selection signal is computed locally in FEF from spatially non selective inputs. Additionally, we show that the magnitude of spatial selection in FEF is related to behavioral measures of attention during the time period in which the stimulus is being processed by the visual system. This relationship shows that FEF is directly involved in spatial attention. Primate frontal eye fields mediate spatial attention in covert visual search by Ilya E. Monosov B.S., University of California San Diego, 2004 A dissertation submitted in partial fulfillment of the requirements for the Degree of Doctor of Philosophy in the Division of Biology and Medicine Providence, Rhode Island May, 2010 © Copyright 2009 by Ilya E. Monosov This dissertation by Ilya E. Monosov is accepted in its present form by the Division of Biology and Medicine as satisfying the dissertation requirement of the degree of Doctor of Philosophy. Date__________ ___________________________________ Kirk G. Thompson, Ph.D., Advisor Laboratory of Sensorimotor Research National Eye Institute National Institutes of Health Recommended to the Graduate Council Date__________ __________________________________ David L. Sheinberg, Ph.D., Reader Dept. of Neuroscience Brown University Date__________ __________________________________ Robert H. Wurtz, Ph.D., Reader Laboratory of Sensorimotor Research National Eye Institute National Institutes of Health Date__________ __________________________________ Jeffrey D. Schall, Ph.D., Reader Dept. of Psychology Vanderbilt University Approved by the Graduate Council Date__________ __________________________________ Shelia Bonde, Ph.D. Dean of the Graduate School iii Curriculum Vitae Ilya E. Monosov, BS, MS DOB: 01-13-1978 Contact Information Laboratory of Sensorimotor Research 2a50, Bldg 49, National Eye Institute, NIH Bethesda, MD, 20891 Email: ilya.monosov@gmail.com Education 2000-2004 B.S. Biology, UCSD. Graduated with Provost’s Honors 2004-2005 M.S. Architecture, New School Of Architecture (San Diego, CA) Thesis: Cognitive Neuroscience and Architecture: Interdisciplinary Research for K-6 School Design. 2005-current PhD Candidate in Neuroscience, Brown University-NIH partnership Teaching Experience 2004 – Teaching Assistant. Course: Molecular Basis for Disease. Taught by Immo Scheffler at UCSD. 2005 – Teacher. Course: Environmental Biology for Architects at New School of Architecture. San Diego, CA. 2005 – Guest lecturer in UCSD music department (invited by Professor Charles Curtis). Professional Memberships Society for Neuroscience Frogpeak Music Composer’s Collective Peer-Reviewed Publications iv (1) Monosov IE, Traverse JC, Thompson KG. Measurements of simultaneously recorded spiking activity and local field potentials suggest that spatial selection emerges in the frontal eye field. Neuron. 2008 Feb 28;57(4):614-25. (2) Traverse JC, Monosov IE, Zhou Y, Thompson KG. A perceptual representation in the frontal eye field during covert visual search that is more reliable than the behavioral report. Eur J Neurosci. 2008 Dec;28(12):2542-9. Posters (1) Monosov IE, Trageser JC, Thompson KG. A comparison of the time course and spatial tuning of attention in evoked and induced LFP responses, and spikes in the primate frontal eye field. Society for Neuroscience 2006. Professional Talks (1) Monosov IE, Trageser JC, Xu P, Thompson KG. The effects of frontal eye field inactivation in a cued covert visual search task. Society for Neuroscience, 2008. (2) Monosov IE, Trageser JC, Thompson KG. Spatial selection in the frontal eye field predicts accurate object recognition. Society for Neuroscience, 2007. (3) Monosov IE, Bellugi U. William’s Syndrome and Music Creativity. Salk Institute for Biological Studies, 2005. (4) Monosov IE. K-6 Design: Classroom and Brain development. Academy of Neuroscience for Architecture workshop for architects and neuroscientists, 2005. v Preface “Trying to understand perception by studying only neurons is like trying to understand bird flight by studying only feathers: it just cannot be done. In order to understand bird flight, we have to understand aerodynamics; only then do the structure of the feathers and the different shapes of birds’ wings make sense.” (Marr, 1982) Visual spatial attention serves to select locations of interest in the visual field and enhances the cortical representation of objects at those locations. Previous studies suggest that neural activity in primate frontal eye fields (FEF) is involved in the spatial selection of salient stimuli in complex visual environments for eye movements and spatial attention. The main goal of the following thesis is to understand the temporal dynamics of spatial selection and to understand the relationship of this measurable neural signal to behavioral measures of spatial attention. Chapter 1 outlines the hypothesis of this thesis and provides the background that is to guide the experiments outlined in Chapter 2 and 3. I discuss how spatial attention has been measured behaviorally and what is already known about the neurophysiology of spatial selection in the primate frontal eye fields (FEF). My hypothesis is that 1) spatial selection is computed in FEF from non-spatial inputs and 2) that the magnitude of spatial selection in FEF is directly related to behavioral measures of spatial attention in covert visual search. In Chapter 2, I present the results of experiments that demonstrate that FEF is directly involved in the generation of a spatially selective signal that indicates the presence of a target amongst distractors in a complex visual scene. In Chapter 3, I show the results of an experiment that suggests that there is a direct relationship between behavioral measures of spatial attention and the spatial selection signal in FEF. Chapter 5 is a summary and outline of the main results of the thesis. It also includes several proposals for future inquiry. vi Acknowledgements This thesis was completed in the Laboratory of Sensorimotor Research at the National Institutes of Health (NIH) and Brown University between June, 2005 and May, 2009. It was made possible by the Brown University Graduate Partnerships Program (GPP) with the NIH. I would like to thank my advisor Kirk G. Thompson for the enormous amount of support, encouragement, and training he has provided. I would also like to thank my co-advisor David L. Sheinberg for providing a great deal of expertise and support. I hope to repay their kindness with a long and successful collaborative relationship. I would like to thank the members of my thesis committee, Robert H. Wurtz and Jeffrey D. Schall, for helpful discussions and comments. I would like to thank my parents, Edward and Anna Monosov for a lifetime of encouragement and love; and Elizabeth P. Weber for the love and kindness she has given me on a daily basis. vii Table of Contents Curriculum Vitae ........................................................................................................................................iv Preface ........................................................................................................................................................vi Acknowledgements ....................................................................................................................................vii Table of Contents........................................................................................................................................viii List of Figures ............................................................................................................................................x Chapter 1: Introduction...............................................................................................................................1 1.1 The correlates of spatial attention: behavior ....................................................................................1 1.2 Spatial selection in the primate frontal eye fields ............................................................................1 1.3 The correlates of spatial attention: neurophysiology........................................................................3 1.4 Hypothesis and Outline.....................................................................................................................4 Chapter 2: The frontal eye field converts feature-related information into a categorical representation of the target’s location...........................................................................................................................................6 2.1 Introduction ......................................................................................................................................6 2.2 Methods ............................................................................................................................................8 2.3 Results ..................................................................................................................................................20 2.4 Discussion.........................................................................................................................................32 Chapter 3: Spatial selective signals in the frontal eye fields mediate spatial attention and predict accuracy in a covert visual search task that requires object identification.....................................................................38 3.1 Introduction ......................................................................................................................................38 3.2 Methods ............................................................................................................................................39 3.3 Results ..............................................................................................................................................44 3.4 Discussion.........................................................................................................................................62 Chapter 5: Summary and Conclusions .......................................................................................................67 5.1 Overview ..............................................................................................................................................67 5.2 Future directions ...................................................................................................................................69 viii Bibliography................................................................................................................................................71 ix List of Figures Figure 2.1: The behavioral tasks................................................................................................................9 Figure 2.2: Spatial tuning analysis of spiking activity recorded at the same site as the LFP shown in Figure 2.3...........................................................................................................................................................11 Figure 2.3: Spatial tuning analysis of the LFP response recorded at the same site as the spiking activity shown in Figure 2.2.................................................................................................................................13 Figure 2.4: Visual response latency analysis of the spike (left) and LFP (right) responses recorded during the covert visual search task....................................................................................................................14 Figure 2.5: Testing the effects of signal filtering on the determination of selection time.........................19 Figure 2.6: Population results from the covert visual search tasks shown separately for the two monkeys..................................................................................................................................................23 Figure 2.7: Population results from the covert visual search task at each recording site combined across the two monkeys...........................................................................................................................................25 Figure 2.8: The relationship of selection time in LFPs and spikes to LFP visual response latency..........28 Figure 2.9: Comparisons of spatial tuning in spiking activity and LFP responses recorded in the memoryguided saccade and covert visual search tasks........................................................................................31 Figure 3.1: Task and behavior....................................................................................................................44 Figure 3.2: Activity of a single FEF neuron during correct target present trials.......................................48 Figure 3.3: Relationship between cue-related spatial selectivity and behavioral measures of spatial attention for monkey C (open symbols) and for monkey B (solid symbols) ........................................................50 Figure 3.4: Population analysis of activity during correct and error valid cue trials.................................52 Figure 3.5: Population analysis of activity during correct and error invalid cue trials..............................54 Figure 3.6: Population analysis of activity during correct and error neutral cue trials..............................56 Figure 3.7: A comparison of population average activity during (a) correct rejection trials and (b) invalid cue miss trials.............................................................................................................................................58 Figure 3.8: The relationship between the SI and the speed of target identification...................................60 x Chapter 1 Introduction 1.1 The correlates of spatial attention: behavior To function in a complex and constantly changing environment, it is important to select and filter out irrelevant information and to attend to information that is of benefit and interest. Generally, attention is thought to include the following components: working memory, competitive selection, top-down signal sensitivity control, and saliency filters, which select sensory information for the processing of stimuli that are relevant for behavior (Knudsen, 2007). Visual spatial attention serves to select locations of interest in the visual field and enhances the cortical representation of objects at those locations. This type of attention is necessary for recognition of objects in complicated natural environments (Sheinberg and Logothetis, 2001; Rensink, 2002). In psychology experiments, spatial attention is typically measured as improvements in performance accuracy and decreases in reaction time for detection and discrimination of stimuli presented at the attended location (Pashler 1998). This thesis will concentrate on exploring the neural basis of spatial attention in the primate frontal eye fields. 1.2 Spatial selection in the primate frontal eye fields The visual system is thought to be divided into two hierarchical processing streams (Ungerleider and Mishkin, 1982). The dorsal visual stream, or the ‘where’ stream, is concerned with coding space, while the ventral visual stream, or the “what” stream, is thought to process the visual scene and extract meaning from the environment, for example regarding object identity. The frontal eye field (FEF) is an important site of convergence in the visual system. The FEF lies at the end of the dorsal “where” visual processing stream. It was first observed by David Ferrier in the late 1880’s when he discovered that electrical 1 2 stimulation of the macaque prefrontal cortex elicited saccadic eye movements. FEF is located in the prefrontal cortex in the rostral bank of the arcuate sulcus of macaques and receives input from dorsal stream visual areas MT, MST, an LIP; ventral stream visual areas V4, TEO, and TE; and from the supplementary eye field and prefrontal areas 46 and 12 (Schall, 2009). Therefore, FEF is in a unique position to signal the presence of a behaviorally relevant stimulus as that stimulus becomes the target for a saccade. FEF is retinotopically organized and has a map of visual field eccentricity (Schall, 2009). Many functionally different types of neurons have been identified in FEF. Movement neurons begin to fire approximately 50ms before the initiation of a saccadic eye movement (Bruce and Goldberg, 1985; Thompson et al., 1996). These neurons are thought to influence eye movements through projections to the superior colliculus and basal ganglia (Stanton et al., 1988; Schall et al., 1995; Schall, 2009). Visually responsive neurons in FEF respond when a target is placed in their receptive field. Their activity selects the salient target amongst distractors in visual search tasks without eye movements (Thompson et al, 2005). From now on when I discuss FEF activity, I will be referring to visually responsive FEF neurons. Some of these neurons also have eye movement related activity (these are termed visuomotor neurons). We do not discuss the activity of purely movement neurons because movement related activity in FEF is suppressed during the covert visual search tasks used in the thesis (Thompson et al., 2005). Examples of visually responsive FEF neurons are shown in Chapters 2 and 3. Though FEF receives input from the ventral visual pathway, its visually responsive activity is generally not selective for the identity of objects or their specific features; though a few exceptions have been reported. Generally, if a given visual feature is important to the subject, FEF activity will identify its location in space, and not the feature itself (Thompson and Bichot, 2005). For example, if for a given behavior motion becomes more important than the color red, FEF activity will identify the location of the motion amongst red stimuli, conversely, if a color identifies a target that is to guide a behavior, FEF will select the location of the color. Therefore, FEF acts like the final stage of dorsal processing, forming a saliency map of space (Thompson and Bichot, 2005), where higher activity is correlated with the importance of a given location. It is thought that this map of visual salience guides visual spatial attention 3 (Thompson et al., 2005; Schafer and Moore, 2007) and eye movements (Schall, 1995; Thompson et al., 1996). Understanding the time course and origin of the explicit spatial selection signal in FEF is key to understanding the neural computations that underlie it. This thesis explores the origins of the spatial selection signal in FEF and presents evidence that this signal is directly involved in the top-down control of spatial attention. 1.3 The correlates of spatial attention: neurophysiology The neurophysiological literature defines visual spatial attention as an enhanced processing of visual features used to guide a decision process (Desimone and Duncan, 1995; Maunsell and Treue, 2006; Knudsen, 2007). Attentional effects have been demonstrated in the neural responses of V1, V4, MT, and IT in experiments that combined neural recordings and attention-demanding behavior (Reynolds and Chelazzi, 2004). For example in V4, neuronal firing rates are enhanced when the animal pays attention to a behaviorally relevant stimulus, and suppressed when the animal attends away from that same stimulus (Reynolds and Desimone, 2000). It is widely accepted that the spatially selective signals in the frontal eye field (FEF) are associated with the planning and execution of saccadic eye movements (Goldberg and Segraves 1989; Schall and Thompson 1999; Tehovnik et al. 2000). In humans, functional imaging studies show that the FEF is part of what is referred to as the fronto-parietal attention network and is active during the allocation of attention with and without eye movements (Beauchamp et al. 2001; Corbetta and Shulman 2002; Kincade et al. 2005; Bressler et al. 2008; Kelley et al. 2008). Transcranial magnetic stimulation over the FEF modulates perceptual performance in covert attention tasks (Grosbras and Paus 2002; Muggleton et al. 2003; Smith et al. 2005), and also modulates visual activity in extrastriate visual cortex (Silvanto et al. 2006; Taylor et al. 2007). There is a growing body of evidence that FEF plays a causal role in directing covert spatial attention (Awh et al. 2006). In monkeys, electrical microstimulation of FEF enhances perception (Moore and Fallah 2001; Moore and Fallah 2004; Schafer and Moore 2007) and produces enhanced responses in extrastriate visual cortex that resemble the effects of covert spatial attention (Moore and Armstrong 2003; 4 Armstrong and Moore 2007). Inactivation of FEF disrupts target detection during covert visual search (Wardak et al. 2006) Neuron recordings in monkeys have shown that the activity of visually responsive FEF neurons identifies the location of an attended visual stimulus, even in the absence of eye movements (Kodaka et al. 1997; Murthy et al. 2001; Sato and Schall 2003; Thompson et al. 2005b). But thus far, there has not been an experiment relating FEF activity and behavioral measures of spatial attention. This thesis directly assesses the relationship between reaction time and performance accuracy in a cued covert visual search task in order to establish a link between FEF activity and spatial attention. 1.4 Hypothesis and Outline In the following chapters, we will explore the origin of spatial selection in FEF and its relationship to spatial attention. The general hypothesis that guides this thesis is that FEF is one of the sources of spatial selection signals in the brain, and that the magnitude of these signals is correlated with reaction time and performance accuracy in covert visual search. In Chapter 2, we will explore the origin of spatial selection in FEF during covert visual search. This work is an extension of previous work that showed that the activity of visually responsive FEF neurons selects the location of the behaviorally relevant stimulus amongst distractors in covert visual search before the monkeys’ made their response (Thompson et al., 2005). In this study, local field potentials (LFP) and spiking activity were recorded while the monkeys performed a covert visual search and a memory guided saccade tasks. It has been proposed that LFPs represent the sum of dendritic synaptic activity in an area that spans less than 1mm of tissue around the tip of the electrode (Logothetis et al., 2007; Katzner et al., 2009), while spiking activity represents the output of neurons being recorded. Therefore, we will compare the time course of non-selective visually evoked signals in FEF LFPs and spiking activity. We will show that visually evoked activity was detectable first in LFPs and then in spiking activity. This is consistent with the hypothesis that LFPs represent input from local processing and from other brain regions. Next, we will compare the time course of spatial selection in spiking activity and LFPs and show that while the non-selective visually evoked activity was first detectable in LFPs and then spikes, spatial selection was first detectable in spiking activity and 30ms later in LFPs. We suggest that this finding 5 supports the hypothesis that FEF converts spatially non-selective inputs to a categorical representation of target location, and therefore that FEF is a source of spatial selection in the brain. In Chapter 3, we will explore the influence of spatial selection in FEF on visual processing by employing a cued covert visual search task that requires the monkeys to report the presence and identity of a stimulus amongst distractors. We will fill the missing piece in a growing body of evidence that suggests that FEF is directly involved in shifting covert spatial attention and enhancing visual processing of taskrelevant stimuli. Importantly, we will show that the magnitude of spatial selection in FEF predicts accuracy and response time of object identification only during the time when the stimulus is being processed by the visual system. Chapter 2 The frontal eye field converts non-spatial visual information into a categorical representation of the target’s location The frontal eye field (FEF) participates in selecting the location of behaviorally relevant stimuli for guiding attention and eye movements. We simultaneously recorded local field potentials (LFPs) and spiking activity in the FEF of monkeys performing memory-guided saccade and covert visual search tasks. We compared visual latencies and the time course of spatially selective responses in LFPs and spiking activity. Consistent with the view that LFPs represent synaptic input, visual responses appeared first in the LFPs followed by visual responses in the spiking activity. However, spatially selective activity identifying the location of the target in the visual search array appeared in the spikes about 30 ms before it appeared in the LFPs. Because LFPs reflect dendritic input and spikes measure neuronal output in a local brain region, this temporal relationship suggests that spatial selection necessary for attention and eye movements is compute locally in FEF from non-spatially selective inputs. † 2.1 Introduction Visual spatial selection describes the process that guides spatial attention (Serences and Yantis, 2006) and selectively couples perception to action (Allport, 1987). Understanding the time course of this process is key to understanding the neural computations that underlie it. Typically, this question has been addressed by analyzing event-related brain potentials (ERPs) recorded from scalp electrodes in humans † The bulk of this chapter is taken from a previously published manuscript: Monosov IE, Trageser JC, Thompson KG (2008) Measurements of simultaneously recorded spiking activity and local field potentials suggest that spatial selection emerges in the frontal eye field. Neuron 57(4):614-25. 6 7 (Hillyard and Anllo-Vento, 1998; Luck et al., 2000) and neuronal spiking activity in behaving primates (Schall and Thompson, 1999). In visual search studies, in which subjects are required to discriminate a target among distractors, human ERPs (Luck and Hillyard, 1994) and single units recorded in primate frontal eye field (FEF) (Thompson et al., 1996; Sato et al., 2001), lateral intraparietal area (Ipata et al., 2006; Thomas and Pare, 2007) and superior colliculus (McPeek and Keller, 2002) exhibit an initial period of non-selective activation followed by a discrimination process that identifies the location of the target in the search array. Local field potentials (LFPs) are electrical potentials recorded with an electrode positioned in the brain. The LFP signal represents the summed synaptic activity occurring near the tip of the electrode. It is a combined measure of local processing and synaptic inputs from other brain regions regardless of whether or not spikes are generated (Mitzdorf, 1985, 1987; Juergens et al., 1999; Cruikshank et al., 2002; Kaur et al., 2004; Logothetis and Wandell, 2004; Kreiman et al., 2006; Nielsen et al., 2006; Chen et al., 2007). In contrast, spiking activity represents the results of local neural processing and is the output signal, from the neurons near the tip of the electrode, to local circuits or other brain regions. Although both LFPs and spiking activity have been used to measure the time course of spatial attention processes, the relationship between these neurophysiological signals is still unclear. Analysis of concurrently recorded LFP and spiking activity can shed light on how sensory representations in dendritic input are transformed into cognitive signals such as spatial selection (Kreiman et al., 2006; Nielsen et al., 2006). The FEF is a brain area in monkeys and humans that participates in the visual spatial selection process (Schall and Thompson, 1999; Pessoa et al., 2003; Awh et al., 2006; Serences and Yantis, 2007). The spatial selection process localizes behaviorally important objects in a complex visual scene and is necessary for guiding visual attention and goal directed behaviors. It was previously shown that spiking activity in monkey FEF reflects the locus of spatial attention during covert visual search tasks in the absence of eye movements (Thompson et al., 2005a). During the collection of these neuronal spiking data, LFPs were also recorded simultaneously from the same electrodes. The goals of this study were to determine whether LFP responses were spatially selective, and if so, to compare the time course and spatial tuning of the spatially selective signals in neuronal spiking activity and LFP responses. 8 We found that in the covert visual search task both the LFPs and the spiking activity exhibited initial non-selective visual responses that evolved into significant spatial tuning in the time period before the monkeys’ behavioral report. The directional tuning of the spatially selective responses in the visual search task matched the directional tuning of the visually evoked responses to a single visual stimulus in the memory-guided saccade task. Although the initial visual responses appeared first in the LFP signals in both tasks, the spatially selective responses in the visual search task appeared first in the spiking activity. These results suggest that during visual search, spatial selectivity is generated in FEF from non-spatially selective inputs. 2.2 Methods Data collection The data were collected from two male monkeys (Macaca mulatta) weighing 8 kg (monkey S) and 6.5 kg (monkey C). All surgical and experimental protocols were approved by the National Eye Institute Animal Care and Use Committee and complied with the National Institutes of Health Guide for the Care and Use of Laboratory Animals. The surgical procedures, behavioral control, and visual stimulation techniques have been described previously (Thompson et al., 2005b). The single unit spiking activity analyzed in this study is the same as in the previous study (Thompson et al., 2005b). Often two or three units were recorded simultaneously on one electrode and sorted offline. For this study, all the single units recorded at each site were combined to represent the overall spiking activity at each recording site. The local field potentials (LFPs) were recorded simultaneously on the same glass insulated tungsten electrodes as the spikes using a Plexon data acquisition system (Plexon Inc.). The impedance of the head-stage was 40 MΩ at 1 kHz. Electrode impedance ranged from 0.5 to 1.5 MΩ. A stainless steel guide tube resting on the surface of the dura served as the reference. The signals were amplified and filtered between 154 Hz and 8.8 kHz to obtain spike data. LFP signals were digitized and sampled at 1 kHz after filtering the electrode signal between 3 Hz and 88 Hz. Analog eye position and lever position signals were digitized and sampled at 1 kHz. 9 Figure 2.1: The behavioral tasks. (A) The memory-guided saccade task. After the monkey fixated on a central spot, a peripheral stimulus identical to the fixation spot was flashed for 50 ms randomly at one of the six or eight locations matching the stimulus locations in the covert visual search task. After a delay, the fixation spot disappeared, and the monkey was rewarded for making a saccade to the remembered target location. (B) The covert visual search tasks. After the monkey grasped the lever in the vertical position, a small fixation cross appeared. After fixating the central cross, a search array appeared in which one of the stimuli was different. Monkey S was rewarded for turning the lever in the same direction as a different-colored stimulus in relation to the fixation cross. Monkey C was rewarded for turning the lever in the same direction as the gap in the C target stimulus regardless of its location in the search array. The depiction of the lever at the bottom shows the correct behavioral responses for the example trials shown in the search displays. Behavioral tasks At each recording site monkeys performed a memory-guided saccade task (Figure 2.1 A) and one of two visual search tasks (Figure 2.1 B) in separate blocks of trials as described in a previous report (Thompson et al., 2005b). In the memory-guided saccade task, after the monkey fixated on a 0.3° diameter gray spot for 400-800 ms, an identical gray spot was flashed for 50 ms at one of six or eight isoeccentric peripheral target locations spaced equally around the central fixation spot. The eccentricity was adjusted so 10 that at least one of the stimulus locations was inside the receptive field of the neuron being recorded. The eccentricities of the stimuli ranged between 8° and 12° across recording sessions depending on receptive field location. Monkeys were required to maintain fixation on the central fixation spot for a random period ranging from 800 to 1400 ms. After the fixation spot disappeared, the monkeys were rewarded for making a saccade to the remembered target location. In the covert visual search tasks, monkeys initiated a trial by grasping a lever and holding it in a vertical position. Once the lever was within 10° of vertical, a small central yellow fixation cross (0.3°) appeared. After fixating the cross for 400 to 800 ms, a search array appeared that was made up of a target randomly placed at one of the locations used in the memory-guided saccade task and distractors at the remaining locations. Each of the search array stimuli subtended 1.5° of visual angle. The monkeys were rewarded for maintaining fixation on the central cross and making the correct lever turn (> 15° from vertical) within 2 s after search array presentation; in practice, the monkeys nearly always turned the lever to the physical limit of 35° from vertical. If the monkey broke fixation on the central cross, released the lever, or made an incorrect lever turn the trial was aborted immediately. The reward was given after a correct lever turn; however, the search array remained on for an additional 250-500 ms to probe for latent saccade plans. The monkeys did not tend to make saccades to the target of the search array after obtaining the reward (Thompson et al., 2005b). Monkey S was trained to report the location of the color singleton target of the search array (Figure. 2.1 B, upper). The luminance of the stimuli was determined by a Minolta Ca-100 color analyzer. The stimuli were isoluminant green and red discs. The target could be either green or red, but within a block of trials the color of the target and distractors did not change. The singleton target appeared randomly at one of six stimulus locations, three to the left and three to the right of the fixation cross. A correct response was a lever turn to the left or right corresponding to the location of the target stimulus relative to the fixation cross. In order to verify that the direction of the lever turn did not affect FEF activity during the task, Monkey C was trained to report the orientation of a C among O distractors (Figure 2.1 B, lower). The stimuli were gray rings with one of them having a 0.5° gap randomly on the left or right. The C target appeared randomly at one of eight locations positioned around the fixation cross. A correct response was a 11 lever turn to the left or right corresponding to the location of the gap in the C target regardless of its location in the search array. Data analysis The LFP signal is a continuous measure of brain activity. A comparable measure of spiking activity was obtained by convolving each spike with a function that resembles an EPSP (Thompson et al., 1996). With this method, each spike exerts influence only forward in time and represents the post-synaptic consequences of spiking activity. The resulting spike density function reflects the onset of spiking activity at a 1 millisecond time resolution and is comparable to the onset of activity measured in the LFP signal. Examples of the EPSP spike density functions are shown in Figure 2.2 A (see below). Next we describe the analytical methods used to determine the time course of visual activation and spatial selection, and characterize the spatial tuning of spiking activity and LFP responses recorded during the memory-guided saccade and covert visual search tasks. Figure 2.2: Spatial tuning analysis of spiking activity recorded at the same site as the LFP shown in Figure 2.3. (A) Spike density functions, derived from a filter resembling an EPSP, are plotted above tick marks representing times of action potentials for three representative trials. (B) The average target-aligned spiking activity at each target location from the memoryguided saccade task (gray) and the covert visual search task (black). The box-whisker plot in each panel indicates the median, quartiles and range of reaction times in the covert visual search task. The neuron’s preferred target direction (60°) corresponds to the filled 12 circle in the search array at the center. (C) The superimposed average activity for each target position from the memory-guided saccade task (left) and the visual search task (right). The thick line represents the average activity on trials when the target was at the preferred spatial location. (D) The p-value (ANOVA) at each millisecond in the memory-guided saccade task (left) and in the visual search task (right) that estimates the probability that the spiking activity did not vary across target locations. The black triangle at the bottom of the plot marks the selection time (vertical dotted line: memory-guided = 70 ms, visual search = 128 ms) which was defined as the first millisecond that the p-value crossed p = 0.05 (horizontal dotted line), only if it continued past p = 0 001 and p < 0.05 for more than 20 of the next 25 milliseconds. (E) The spatially selective response measured from 50 – 300 ms following the target flash in the memory-guided saccade task (left), and from 100 – 300 ms following the time of search array presentation (right) as a function of target direction. The time ranges for measuring spatial tuning are indicated by black bars in (C). The parameters of the best fit Gaussian curve from the memory guided saccade task (left) are B = 40.26 sp/s, R = 36.54 sp/s, Φ = 64.42°, and Tφ = 38.25°; and from the covert visual search task (right) are B = 45.75 sp/s, R = 55.36 sp/s, Φ = 63.22°, and Tφ = 45.02°. Selection time The time course of spatial selectivity in the LFP and spiking activity was determined with an analysis of variance (ANOVA) at each millisecond following the target flash in the memory-guided saccade task and the presentation of the search array in the visual search tasks (Figures 2.2 and 2.3). The running ANOVA estimated the probability (p) at each millisecond that the response did not vary across target locations. Figures 2.2 and 2.3 illustrate the time course analysis for the spiking activity (Figure 2.2) and the LFP response (Figure 2.3) recorded concurrently at a single site. The selection times of the spiking activity and the LFP response were determined separately and were defined as the first millisecond that the p-value dropped below the 0.05 level only if it continued past the 0.001 level and remained below the 0.05 level for more than 20 of the next 25 milliseconds. To obtain the earliest possible selection times, a threshold of p = 0.05 was used. However, a threshold of p = 0 01 did not alter the temporal relationship between the selection times of the LFP and spiking activity. Again, the important point is that the same statistical analysis and threshold was used to determine selection times in the LFP and spiking activity in the memory-guided saccade task and in the visual search tasks. Our threshold for determination of the time of selection (see above) also insured that our results were reliable. 13 In Figures 2.2D and 2.3D, p-values obtained from the running ANOVA are plotted as a function of time on a log axis from 1 to 10-10 for spikes (Figure 2.2D) and LFPs (Figure 2.3D) recorded concurrently at a single site during the memory-guided saccade task (left) and the visual search task (right). It is important to note that selection time measured in the memory-guided saccade task is qualitatively different from that measured in the visual search task. In the memory-guided saccade task a single target stimulus is presented alone and evokes a different initial response across target locations. Therefore, selection time in the memory-guided saccade task corresponds to the initial visual response latency to a single visual stimulus. In the visual search task, however, selection time measures the first time that the responses to the target of the search array are different from the responses to the distractors. As previously shown for spiking activity (Thompson et al., 1996), and now we demonstrate for LFPs, the initial visually evoked responses in FEF during visual search do not distinguish the target from the distractors. Therefore we used a different method to determine visual response latency in the visual search task. Figure 2.3: Spatial tuning analysis of the LFP response recorded at the same site as the spiking activity shown in Figure 2.2. (A) The LFP responses on three representative visual search trials. (B) The average target-aligned LFP response in the memoryguided saccade task (gray) and in the covert visual search task (black) sorted by target location. (C) The superimposed average LFP 14 response for each target position from the memory guided saccade task (left) and the covert visual search task (right). (D) The p-value (ANOVA) at each millisecond that estimates the probability that the LFP response did not vary across target locations. The selection time of the LFP response at this recording site is 69 ms for the memory-guided saccade task and 142 ms for the visual search task. (E) The spatially selective response measured from 100 – 200 ms following the target flash in the memory-guided saccade task (left), and from 180 – 300 ms following the time of search array presentation (right) as a function of target direction. The time interval used for determining the spatial tuning of the LFP response was the interval that exhibited the most variability in the ANOVA analysis shown in D. The parameters of the best fit Gaussian curve from the memory guided saccade task (left) are B = -4.59, R = -33.14, Φ = 43.05°, and Tφ = 59.24°; and from the covert visual search task (right) are B = 19.59, R = -16.33, Φ = 64.05°, and Tφ = 62.91°. Visual response latency during visual search A paired t-test was performed across all correct trials comparing the average LFP and spiking activity during the 50 ms preceding the appearance of the search array on each trial to the activity at each millisecond following the appearance of the search array. The visual response latency was defined as the first time that the p-value dropped below the 0.01 level only if it continued past the 0.001 level and remained below the 0.01 level for more than 20 of the next 25 milliseconds. The same threshold was used for determining visual response latencies in the LFP and spiking activity recording during the visual search task. Figure 2.4 (see below) illustrates how we measured the initial visual response latencies of the spiking activity and the LFP response recorded simultaneously during the visual search task. Figure 2.4: Visual response latency analysis of the spike (left) and LFP (right) responses recorded during the covert visual search task. 15 The activity is from the same recording session as shown in Figures 2.2 and 2.3. (A) The average spike density function constructed by convolving each spike with a kernel that resembles an EPSP and averaging across all trials. (B) The p-value (paired t-test) at each millisecond that estimates the probability that the spiking activity is equal to the baseline activity (measured from -50 to 0 ms). The visual response latency was defined as the first time that the p-value crossed p = 0.01 (horizontal dotted lines), only if it continued past p = 0 001 and p < 0.01 for more than 20 of the next 25 milliseconds. The spiking visual response latency in the covert visual search task at this recording site is 63 ms (black triangles and vertical dotted lines). (C) The average LFP signal across all trials. (D) The pvalue (paired t-test) at each millisecond that estimates the probability that the LFP signal is equal to the baseline signal (measured from -50 to 0 ms). The LFP visual response latency is 53 ms. Spatial tuning To describe the variation in the spiking and LFP responses with the location of the singleton target, the response averaged over a time interval was fit with a Gaussian function of the form A(φ) = B + R·exp(-½[( φ-Φ)/ Tφ]2), where activation (A) as a function of meridional direction (φ) depends on the baseline response (B), peak response (R), optimum direction (Φ), and tuning width (Tφ). Previous reports have shown that this function effectively characterizes the spatial pattern of FEF spiking activity (Bruce and Goldberg, 1985; Schall et al., 1995a; Schall et al., 2004). The best fit Gaussian curve was obtained for the average activity measured over a time range following visual stimulus presentation. For spiking activity, the time range was from 50 ms to 300 ms for the memory-guided saccade task, and from 100 ms to 300 ms for the visual search task. These time intervals were used because it encompassed the period of spatial selectivity observed across the data (Thompson et al., 2005b). For the LFP response in the memory-guided saccade task, the time range was from 100 ms to 200 ms because this interval encompassed a strong spatially selective negative-going deflection observed across all the LFP recordings (see Figure 2.3C, left panel). For the LFP response in the visual search task, it was necessary to determine the appropriate time interval individually for the different recording sites. This is because a spatially selective response could emerge in a positive or in a negative difference in the LFP signal. Therefore, to determine the spatial tuning of the LFP signal, we made the assumption that the preferred direction was in the visual hemifield contralateral to the brain hemisphere in which the LFP signals were recorded. This was reasonable because the right visual hemifield is represented in the left hemisphere of the brain and the left visual hemifield is represented in the right hemisphere of the brain. In some of the LFP recordings, spatial tuning was evident in positive tuning during 16 one time interval and in negative tuning during another time interval that was separated by a non-selective period during which time the polarity of the spatial tuning switched. The time interval we used for determining the directional tuning was the interval that exhibited the strongest spatial selectivity in the running ANOVA analysis described above because it was most reliable. Selection time was defined as the first time that there was significant spatial tuning (see above). In the covert visual search task, the earliest spatial tuning in the LFP from 18 of the recording sites was shortlived and positively tuned. In each of these 18 recordings there was also a later, longer lasting and negatively tuned response that was more significant statistically. The LFP shown in Figure 2.3 is one example of this pattern. On average in these 18 recordings the polarity of the tuning was opposite in the early and late periods and the responses during the early period were more variable than during the late period, but the preferred target directions were nearly identical. Across the population, there was a strong correlation between the preferred target directions from the early and late time periods (p = 0.005). The preferred target directions of 16 of the 18 LFPs (89%) were separated by less than the distance between adjacent target locations (60° for Monkey S and 45° for Monkey C). Signal Variability and the Reliability of Timing Measurements The accuracy of the timing relationships between LFPs and spikes presented in this study depends on the reliability of the signals and analysis methods. If the amount of variability was different for spikes and LFP signals, then the results of the ANOVA analysis comparing the timing relationships between the spikes and LFPs are questionable. Therefore we compared the variability of the LFP responses and spiking activity at each recording site by calculating a variability index (VI) for each signal, a ratio of two standard deviations that measures signal variability across trials relative to the variability across target positions. The numerator of the VI is the standard deviation across all trials measured at each millisecond and averaged over the first 50 ms following search array presentation, and the denominator is the standard deviation across the mean activity for each target position averaged over the same time period. The VI for spikes (6.8 ± 0.4) was not significantly different from the VI for LFPs (6.6 ± 0.4) (paired t-test; p = 0.6). The results were the same when the initial non-selective visual response was included by calculating VI from the time of search array presentation to the measured selection time. This indicates that the later 17 selection times in the LFPs than in the spiking activity determined from the ANOVA analysis was not a false result caused by more variability in the LFP data than in the spike data. Effects of Signal Filtering We were concerned that the signal filtering during data acquisition distorted the recorded LFP and artificially delayed the selection times measured in the LFPs relative to spikes (Nelson et al., 2008), so we directly tested whether signal distortions during data acquisition could have affected the results of our study. This test is illustrated for two recording sessions in Figure 2.5. First we appended 100 ms of a 40 Hz sine wave to the beginning and end of an actual LFP signal recorded in the covert visual search task. The appended sine waves were used to align temporally the original LFP signal to signals re-recorded in the same Plexon data acquisition system (Plexon Inc.) used to record the original LFP (Figure 2.5A). The original LFP signals with the appended sine waves were converted into “.wav” sound files in Matlab (The MathWorks, Inc). We used a portable battery powered audio compact disc player (Panasonic Model No. SL-CT582V) to feed the .wav file signal into a beaker of 0.9% saline in which we placed a 1 MΩ metal electrode used in the neural recordings. The electrode was connected to the filtered (3.3 – 88 Hz) Plexon LFP input channel. An unfiltered signal was simultaneously recorded on a separate a/d card (National Instruments) in the Plexon system normally used to record eye position. Figure 2.5A shows that the continuous voltage signal recorded on the Plexon analog input channel from the portable audio player effectively reproduced the original LFP signal. (We tried playing the .wav file from various laptop computers, but the sound cards attenuated the lower frequencies.) We replaced the actual LFP signal in the original data file with the re-recorded LFP signals and performed the same time course analysis. Figure 2.5B shows the analysis performed on the .wav signal recorded on the unfiltered analog input channels. One of the original LFP signals shown was the same one as in Figure 2.3. The average stimulus related LFP signals for each target position are nearly identical to the originals, and the ANOVA time course analysis generated the exact same selection times as the originals. This shows that the .wav file and audio signals reliably reproduced the originally recorded LFP signals. Figure 2.5C shows the analysis performed on the .wav signals recorded on the filtered LFP input through a 1 MΩ metal electrode in 0.9% saline. There were 18 some distortions of the average stimulus related signals that are apparent when compared to the original LFP signals, and the significance levels of the ANOVA analysis were lessened. But the important result was that the measurement of selection time was not appreciably affected. We performed this test on the LFP signals from 5 separate recording sessions (3 from monkey S and 2 from monkey C). The selection times determined from the .wav file signals recorded on the LFP input channel ranged from 1 ms before to 2 ms after the selection times determined from the .wav file signal recorded on the analog input channel. In summary, the results were the same from all 5 sessions – the signal filtering during data acquisition somewhat altered the shape of the evoked LFP response, but did not affect the determination of selection time. 19 Figure 2.5: Testing the effects of signal filtering on the determination of selection time. (A) Voltage signals from the original LFP recording (red line) and audio .wav file signals re-recorded through an unfiltered analog 20 input channel (dark blue) and the filtered LFP input channel. (light blue). A 40 Hz sine wave was appended to the original before creating the .wav file to temporally align the re-recorded signals. (B) The superimposed average .wav signals recorded through the analog input channels for two LFP signals from two different monkeys (top), and results of the ANOVA time course analysis (bottom). The selection times determined from these re-recorded signals are the same as from the original LFP signals (left – 142 ms, right – 105 ms). (C) The superimposed average .wav signals recorded through the Plexon LFP input channel for the two LFP signals (top), and results of the ANOVA time course analysis (bottom). The selection times determined from these re-recorded signals are nearly the same as from the original LFP signals (left – 144 ms, right – 104 ms). 2.3 Results Spiking activity and LFP responses were recorded concurrently on single electrodes inserted into the FEF of two monkeys in 43 separate recording sessions. The monkeys performed a memory-guided saccade task (Figure 2.1A) and one of two covert visual search tasks (Figure 2.1B). In the covert visual search tasks the monkeys made a manual lever turn as the behavioral report. Monkey S was required to report the location of the singleton target in the search array (20 recording sites) and monkey C was required to report the orientation of the C among Os in the search array (23 recording sites). Single neuron activity recorded with this task was described previously (Thompson et al., 2005b). For this study we combined the activity from simultaneously recorded single neurons into a single representation of spiking activity at each recording site. The primary aim of this study was to compare the times that a spatially selective response first appeared in the LFPs and spikes in the covert visual search task. We refer to this time as the selection time. For the data collected at a recording site to be included in the study, there must have been measurable visual response onset latencies in both the LFPs and spikes, and a measurable selection time in the visual search task for either the LFP response or the spiking activity. In addition, the visual response latencies and selection times must have occurred before the average reaction time of the session. Over all sessions, lever turn reaction times averaged 284 ms for monkey S and 297 ms for monkey C. There were strong correlations between the directional tuning of the spatially selective responses in the LFPs and spikes within and across the visual search task and the memory-guided saccade task, which is consistent with a functional relationship between the LFPs and spikes (see Figure 2.9 below). But first 21 we describe the results of the time course analysis which is blind to the preferred target directions of the two signals. Visual response latencies and spatial selection times of LFPs and spikes The spiking activity and LFP signals recorded simultaneously at each recording site were analyzed using the same methods to obtain the visual response onset latencies and the time of spatial selection measured from the time of search array presentation. The temporal relationship between initial visual response latencies measured in LFPs and spikes was the same across the two tasks, even though visual response latencies were measured using different visual stimuli and different measurement methods in the memory-guided saccade and visual search tasks (see METHODS). The initial visual response occurred earlier in the LFPs than in the spikes. For the memory-guided saccade task, the average ± SE selection time was 63.4 ± 3.2 ms for LFPs, and 72.8 ± 4.3 ms for spikes (paired t-test: p < 0.001) . For the visual search task, the average ± SE onset latency was 56.5 ± 2.4 ms for LFPs, and 71.8 ± 4.0 ms for spikes (p < 0.001). There were also strong correlations between the selection times obtained from the memory-guided saccade task and the visual response onset latencies obtained from the visual search task at each recording site (LFPs: r = 0.48, p = 0.001; spikes: r = 0.78, p < 0.001). Because we were interested comparing visual onset times to spatial selection times in visual search, in this study we will focus mostly on results obtained in the visual search tasks. Cumulative distributions of onset latencies and selection times measured in the visual search task are shown separately for the two monkeys in Figure 2.6A and B. Visual response latencies were obtained for the spiking activity and the LFP response from all 43 recording sites. For spiking activity, the average ± SE onset latency was 68.4 ± 3.5 ms for monkey S, and 74.7 ± 3.3 ms for monkey C. For the LFP response, the average ± SE onset latency was 53.6 ± 1.0 ms for monkey S, and 59.0 ± 0.9 ms for monkey C. An ANOVA that factored the monkey and response measure revealed a significant difference in response latencies between the 2 monkeys (p = 0.02), and between spiking activity and LFP response (p < 0.001) with no interaction between monkey and activity measure (p = 0.86). Selection times in the visual search task were obtained for spiking activity from 38 (88.4%) recording sites and for the LFP response from all 43 recording sites. For spiking activity, the average selection time was 124.6 ± 5.1 ms for monkey S, and 113.0 ± 6.2 ms for monkey C. For the LFP response, 22 the average selection time was 155.2 ± 6.3 ms for monkey S, and 133.3 ± 7.1 ms for monkey C. An ANOVA revealed a significant difference in the selection times in the visual search task between the 2 monkeys (p = 0.01), and between spiking activity and LFP response (p < 0.001) with no interaction between monkey and the activity measure (p = 0.43). The differences in visual response latencies and selection times between the two monkeys may be due to individual differences or to the different visual stimuli used in two different visual search tasks in the two monkeys. It has previously been shown that a search for a gap in a C among Os is very easy (Treisman and Gormican, 1988) and the visual system may be able to resolve a single gap in a circle faster than it can resolve a color difference in a search array. Nevertheless, the important result is the absence of significant interaction between monkeys performing different visual search tasks and the measured timing differences between LFPs and spikes. This means that in spite of the individual differences, the temporal relationships between LFPs and spikes were the same in the two monkeys. 23 Figure 2.6: Population results from the covert visual search tasks shown separately for the two monkeys. (A) Cumulative distributions of visual response latencies and spatial selection times for all recording sites in monkeys S performing the ‘location’ version of the covert visual search task. The average ± SE times from left to right were 53.6 ± 1.0 ms for LFP visual latencies (thin dotted line; median = 53.5), 68.4 ± 3.5 ms for spike visual latencies (thin solid line; median = 65), 124.6 ± 5.1 ms for spike selection times (thick solid line; median = 119), and 155.2 ± 6.3 ms for LFP selection times (thick dotted line; median = 152.5). (B) The same as (A) but for monkey C performing the ‘identity’ version of the covert visual search task. The average ± SE times from left to right were 59.0 ± 0.9 ms for LFP visual latencies (median = 59 ms), 74.7 ± 3.3 ms for spike visual latencies (median = 72 ms), 113.0 ± 6.2 ms for spike selection times (median = 102.5 ms), and 133.3 ± 7.1 ms for LFP selection times (median = 129 ms). (C and D) The percentage of recording sites showing significant modulation at each millisecond following the presentation of the search array in the monkey S (C), and monkey C (D). The plots were smoothed using a running window of 5 ms for easier viewing. The line types correspond to those in A and B. 24 To validate the results in Figures 2.6A and 2.6B, we plotted the percentage of recording sites showing significant modulation at each millisecond following the presentation of the search array separately for monkey S (Figure 2.6C) and monkey C (Figure 2.6D). For both monkeys, significant visual responses are evident in the LFPs before the spikes and significant spatially selective responses are evident in the spikes before the LFPs. Because the relationships between spiking activity and LFP responses were the same for both monkeys, the data from the two monkeys are combined in the following analyses. 25 Figure 2.7. Population results from the covert visual search task at each recording site combined across the two monkeys. (A) Visual response latencies in the covert visual search task of the LFP responses (open squares) and spikes (filled diamonds) at each recording site sorted by the visual response latency of the spikes. LFP and spike visual response latencies were obtained from all 43 recording sites. The histogram shows the distribution of LFP visual response latency relative to spike visual response latency obtained across all recording sites (LFP – spikes; mean = -15 ± 2.2 ms). Similar results were obtained from the selection times measured in the 26 memory-guided saccade task (see Figure 2.7C). (B) Selection times in the covert visual search task of the LFP responses (open and filled circles) and spikes (filled triangles) at each recording site sorted by the selection time of the spikes. LFP and spike selection times were obtained from 38 recording sites. The histogram shows the distribution of LFP selection time relative to spike selection time obtained across all recording sites (LFP – spikes; mean = 24.7 ± 5.0 ms). The filled circles in the scatter plot and filled bars in the histogram represent the 10 recording sites that the spatial tuning of the LFP and spikes differed by more than 40° of visual angle (see Fig. 2.9B). (C) Selection times in the memory-guided saccade task measured from the LFP responses (open squares) and spikes (filled diamonds) at each recording site sorted by the selection time of the spikes (N = 42). The histogram shows the distribution of LFP selection time relative to spike selection time across all recording sites (LFP – spikes; mean = -9.9 ± 2.5 ms). Compare to results in Figure 2.7A. We compared the response latencies and selection times measured from the spiking activity and LFP responses recorded simultaneously at individual recording sites during the visual search task (Figure 2.6 and 2.7). Significant positive correlations between spiking activity and LFP responses for onset latencies (r = 0.46; p = 0.002), and for selection times (r = 0.51, p = 0.001) support the claim that spiking activity and LFP responses are related. Spiking activity and LFP response onset latencies for each recording site are plotted in Figure 2.7A, and selection times are plotted in Figure 2.7B. In both plots, the times from each site are sorted according to the time measured in the spiking activity and a histogram shows the distribution of differences between the times obtained from the LFPs and spikes. For nearly all (41/43 = 95%) of the recording sites, the measured response onset latency was earlier in the LFP response than in the spiking activity. On average, the LFP visual response began 15.3 ± 2.2 ms earlier than the spike visual response. The visual latencies of the LFP responses varied less than the spike responses. As a consequence, the difference between visual onset latency measured in the spikes and in the LFP increased with increasing spike response latency. Nevertheless, even the recording sites with the earliest spike responses had LFP response latencies that were significantly earlier. For the quartile of recording sites with the earliest spike visual response (range: 48 – 60 ms), the LFP visual response began on average 2.3 ± 0.7 ms earlier than the spike visual response (paired t-test: p = 0.01). The earlier initial visual onsets in the LFP signal is consistent with the expected result that feedforward visual inputs in postsynaptic potentials precedes the visual evoked spiking activity (Schroeder et al., 1998). Selection times in the visual search tasks were obtained from all 43 sites for the LFP response and from 38 sites for spiking activity. For the 38 recording sites with selection times from both measures, selection times occurred later in the LFP response than in the spiking activity for 84% (32/38) of the 27 recording sites and differed, on average, by 24.7 ± 5.0 ms (Figure 2.7B). However, at 10 recording sites, the spatial tuning of the LFP response and spiking activity differed by more than 40° of visual angle (see Figure 2.9B); these are indicated in Figure 2.7B by the filled circles in the scatter plot and shaded bars in the histogram. It is possible that at these recording sites the LFP response and spiking activity were less related to each other than at the sites in which the spatial tuning of the two signals corresponds. When these 10 sessions were removed from the analysis, the selection times occurred later in the LFP response than in the spiking activity at 93% (26/28) of the recording sites and differed, on average, by 31.5 ± 5.1 ms. For comparison we also compared the selection times for LFPs and spiking activity in the memory-guided saccade task (Figure 2.7C). Selection time in the memory-guided saccade task measures visual response latency because it identifies the first time that the responses differed across target locations for a single visual stimulus presented alone. It corresponds to the visual response latency measured in the covert visual search task, and across the recording sites the two measures were strongly correlated for both spikes (Pearson’s r = 0.78, p < 0.001) and LFPs (r = 0.60, p < 0 001). Just like the visual response latencies measured in the visual search task (Figure 2.7A), the selection times measured in the memory-guided saccade task were earlier (9.9 ± 2.5 ms) for LFPs than for spikes (Figure 2.7C). The similarity in the results across the tasks and analysis methods is an indication that the later LFP selection times in the visual search task are not due to the quality of the LFP signal or the analysis methods is the result of the selection time analysis in the memory-guided saccade task. Overall, approximately 2.5 times more trials were included in the analysis of visual search data (an average of 290 trials per session) than were included in the analysis of the memory-guided saccade data (an average of 110 trials per session). Because variability affects an ANOVA more when there are fewer samples, it should have affected the selection time results from the memory-guided saccade task more than from the visual search tasks. But in the memory-guided saccade task, the selection times were on average 9.9 ± 2.5 ms earlier in the LFPs than in the spikes (Figure 2.7C). This shows that the later LFP selection times observed in the visual search task (Figure 2.7B) are unlikely due to the analysis method. 28 Figure 2.8. The relationship of selection time in LFPs and spikes to LFP visual response latency. The symbols representing the different times are the same as in Fig. 2.7. The visual response latencies and selection times across all recording sites are sorted by increasing LFP visual response latency. Each of the data points plot the average for a group of eight sorted recording sites. Consecutive data points represent the average of eight recording sites after shifting the averaging window by one. The statistical comparisons are shown at the top (large symbols). The averages ± SE of the response latencies and selection times are plotted after dividing the recording sites into 2 groups based on LFP visual response latency. The recording sites with LFP visual response latencies between 48-55 ms were assigned to the ‘early’ group (N = 22; LFP visual response latencies = 51.7 ± 0.5 ms; spike visual response latencies = 65.0 ± 3.3 ms; spike selection times = 109.5 ± 4.7 ms; LFP selection times = 145.4 ± 6.9 ms), and recording sites with visual response latencies between 56-67 ms to the ‘late’ group (N=21; LFP visual response latencies = 61.5 ± 0.8 ms; spike visual response latencies = 79.0 ± 2.9 ms; spike selection times = 126.3 ± 6.6 ms; LFP selection times = 141.5 ± 7.6 ms). Relationship of LFP visual response latency to selection time Studies have shown that the earliest visual response latencies of LFPs recorded in dorsal stream areas of visual cortex are in cortical layer 4 which corresponds to the feedforward projection of visual inputs (Chen et al., 2007; Schroeder et al., 1998). We hypothesized that if the inputs to FEF from visual 29 cortex were spatially selective, they would be evident first at the recording sites with the earliest LFP visual response latencies. Therefore, we examined whether LFP visual response latencies were related to times of spatial selection (Figure 2.8). It should be noted that this analysis does not establish the cortical layer of the recording sites, but it is motivated by the assumption that recording sites in FEF with earlier visually evoked LFP activity are functionally closer to the feedforward visual input from visual cortex. Surprisingly, this was not the case. To visualize the data we plotted how the spike visual response latencies and selection times, and LFP selection times changed with increasing LFP visual response latency (Figure 2.8). For statistical analysis, the recording sites were divided into two groups based on LFP visual response latency measured in the visual search task. The sites with LFP visual response latencies between 48 and 55 ms were assigned to the ‘early’ group (N = 22), and sites with latencies between 56 and 67 ms to the ‘late’ group (N=21). The large symbols in Figure 2.8 indicate the average ± SE of each group. The spike visual response latencies differed significantly across the ‘early’ (65.0 ± 3.3 ms) and ‘late’ groups (79.0 ± 2.9 ms) (t-test: p = 0.003). This is consistent with the result that LFP and spike visual latencies were positively correlated. LFP response selection times did not differ significantly between the ‘early’ (145.4 ± 6.9 ms) and ‘late’ (141.5 ± 7.6 ms) groups (p = 0.7). For the spiking activity, the selection times of the ‘early’ (109.5 ± 4.7 ms) and ‘late’ (126.3 ± 6.6 ms) groups were marginally different (p = 0.04). The surprising result that the recording sites with the earliest LFP visual response latencies exhibited the earliest spike selection times and the latest LFP selection times suggests that spatial selectivity first appears in the spiking output of neurons closest to the feedforward input from the visual cortex. The difference between spike and LFP selection times in the ‘early’ group was highly significant (paired t-test: p < 10-5). For the ‘late’ group, the difference between the LFP and spike selection times did not reach statistical significance (p = 0.07). We also divided the recording sessions into ‘early’ and ‘late’ groups based on the ‘selection times’ measured in the data collected from the memory-guided saccade task which were recorded in a separate block of trials in each session. Note that ‘selection time’ for the memory-guided saccade data is determined using the exact same analysis method as for visual search data but actually measures the visual onset latency to a single visual stimulus. The results were statistically identical to those shown in Figure 2.8. 30 Comparison of spatial tuning The variation of spatially selective LFP and spiking responses with target direction in the memory-guided saccade and visual search tasks was characterized with Gaussian functions (Figures 2.2E and 3E). The spatial parameters of the best-fit Gaussian curves provide estimates of the preferred direction and spatial extent of the LFP and spiking response fields. Details of the spatial tuning analysis are provided in the METHODS. There were no differences in the directional tuning measures between the two monkeys. The preferred direction was provided by the optimum direction (Φ) parameter. The preferred tuning directions of the spiking activity and LFP responses in the memory-guided saccade and visual search tasks were compared by taking the angle difference between the two measures. Angle differences can range from -180° to +180°. Figure 2.9 shows the distributions of angle differences between the preferred target directions of the LFPs and spikes for the memory-guided saccade and visual search tasks (Figures 2.9A and B), and between the preferred target directions obtained from the memory-guide saccade and visual search tasks for spikes and LFPs (Figures 2.9C and D). All the distributions are peaked near 0° (Rayleigh test; p < 0.001). An analysis that measures the correlation between two circular variables (Mardia and Jupp, 2000) showed that there were strong correlations between the preferred directions obtained from LFPs and spikes in the memory-guided saccade task (Figure 2.9A; p < 10-9) and the visual search task (Figure 2.9B; p = 0.001). There were also strong correlations between the preferred directions obtained across the two tasks for both spikes (Figure 2.9C; p < 10-8) and LFPs (Figure 2.9D, p = 0.001). In summary, there were overall strong correlations between the directional tuning of the LFP and spike response fields across the memory-guided saccade task in which a visual stimulus is presented alone and the covert visual search tasks in which the target must be identified among distractors. 31 Figure 2.9. Comparisons of spatial tuning in spiking activity and LFP responses recorded in the memory-guided saccade and covert visual search tasks. (A-D) The distributions of the differences in the preferred target directions measured from the spiking activity and LFP responses for the recording sites that exhibited significant spatial tuning in the spiking activity. LFP responses exhibited significant spatial tuning at all 43 recording sites in both the memory-guided saccade and covert visual search tasks. Angle differences can range from -180° to +180°. All of the distributions are peaked near 0° (Rayleigh test; p < 0.001) A circular correlation analysis (Mardia and Jupp, 2000) showed that the preferred target directions are significantly correlated between: (A) LFPs and spikes recorded in the memory-guided 32 saccade task (N = 42; p < 10-9). (B) LFPs and spikes recorded in the covert visual search tasks (N = 38; p = 0.001). (C) Spikes recorded in the memory-guided saccade task and spikes recorded in the covert visual search tasks (N = 37; p < 10-8). (D) LFPs recorded in the memory-guided saccade task and LFPs recorded in the covert visual search tasks (N = 43; p = 0.001). (E) The distribution of tuning widths of the LFP (open bars) and spike (filled bars) responses in the memory-guided saccade task. Tuning width was defined as the standard deviation (Tφ) parameter of the best fit Gaussian curves. The average response field width is 38.7 ± 3.7° for spiking activity, and 64.0 ± 3.0° for LFP responses; and the two distributions differ significantly (paired t-test; p < 10-7). (F) The distribution of tuning widths of the LFP and spike responses in the covert visual search tasks. The average response field width is 26.5 ± 2.6° for spiking activity, and 44.7 ± 3.3° for LFP responses (paired t-test; p < 10-4). The tuning widths of the LFP and spike response fields were estimated by the standard deviation (Tφ) parameter of the best fit Gaussian curves. The distributions of tuning widths (in polar angle coordinates) for the single visual stimulus in the memory-guided saccade task and for the target among distractors in the search tasks are shown in Figures 2.9E and 9F, respectively. For the memory guided saccade task, the average response field width is 38.7 ± 3.7° for spiking activity, and 64.0 ± 3.0° for LFP responses. For the visual search tasks, the average response field width is 26.5 ± 2.6° for spiking activity, and 44.7 ± 3.3° for LFP responses. The results of an ANOVA showed that LFP tuning widths were significantly larger than spike tuning widths (p < 0.001), and the tuning widths of responses in the memoryguided saccade task were significantly larger than in the visual search task (p < 0.001). When converted to visual field angles according to the law of cosines, the width of receptive fields for a single visual stimulus alone averaged 6.5 ± 0.4° for spikes and 10.4 ± 0.4° for LFPs; and receptive fields in the visual search task averaged 4.6 ± 0.4° for spikes and 7.4 ± 0.5° for LFPs. The sizes of receptive fields of the spiking activity and the narrower spatial tuning in the visual search task than for single targets are similar to previous reports that used comparable methods (Schall et al., 1995a; Schall et al., 2004). This is important because it shows that our data is consistent with previous reports and suggests that the results of this study are reproducible in slightly different visual search tasks. 2.4 Discussion We show for the first time that LFPs in FEF exhibit visually evoked responses that are spatially selective; they identify the location of a target presented alone in a memory-guided saccade task and 33 identify the location of a behaviorally important stimulus during covert visual search in the absence of eye movements. We compared the LFP responses to the single unit activity recorded concurrently on the same electrodes (Thompson et al., 2005b). In the covert visual search task, both the LFPs and spikes exhibited a short latency non-spatially selective visual response followed by a selective response that identified the location of the behaviorally relevant stimulus that instructed the monkey to manually turn a lever to the left or right. The spatial selectivity for the behaviorally relevant target in the visual search task appeared in the spiking activity before the LFP response. This result is especially intriguing because it suggests that a cognitive representation identifying the location of behaviorally important visual stimuli is computed in the FEF from non-spatially selective inputs (Thompson and Bichot, 2005; Thompson et al., 2005a). The spatial tuning for target location was consistent across tasks and across LFPs and spikes at each recording site, but was generally broader in the LFP signal than in the spikes. Previous spike vs. LFP comparisons either used full field visual stimulation (e.g. Chen et al., 2007; Logothetis et al. 2001) or placed visual stimuli based on the spatial extent of the spike receptive fields (Fries et al., 2001; Liu and Newsome, 2006; Pesaran et al., 2002). We are not aware of any study that compared the spatial extent of visual responses of LFPs to that of spikes recorded on the same electrode. But the broader spatial tuning in LFPs than in the spikes is consistent with the view that LFPs reflect synaptic activity over a larger area of cortex than is reflected in the spiking output of a few localized neurons (Kreiman et al., 2006; Liu and Newsome, 2006; Logothetis et al., 2007; Logothetis and Wandell, 2004; Mitzdorf, 1985; Mitzdorf, 1987). Nevertheless, the overall strong correlations of spatial tuning between the LFP responses and spiking activity when a target was presented alone and when presented among distractors indicate that the LFP and spike signals originate from the same region of FEF. In a recent study, Buschman and Miller (2007) compared the time course of spatially selective spiking activity recorded simultaneously in FEF and the lateral intraparietal area (LIP), an area that is interconnected with FEF, in monkeys performing visual search tasks. Their results suggest that spatial attention signals appear first in the FEF during top-down attention and first in LIP during bottom-up attention. The implication is that visually driven attention signals flow from LIP to FEF and cognitively driven attention signals flow from FEF to LIP. Although simultaneous spike recordings can be used to 34 compare signals in interconnected areas, this experimental method does not address whether or how different brain areas influence each other or how synaptic inputs are transformed into spiking outputs in a given area. In addition, the results of Buschman and Miller (2007) have been called into question mainly due the difficulty in knowing whether the neurons recorded in LIP and FEF in that study were those that received input from or influenced activity in the other brain area (Schall et al., 2007). The combined LFPspike analysis described in this study may be able to address some of these unresolved issues. Combined analysis of LFP and spiking activity can provide information about computations that cannot be obtained when these signals are considered separately (Kreiman et al., 2006; Nielsen et al., 2006). In the cerebral cortex, there is strong evidence that the LFP is a mass signal that is primarily influenced by the excitatory postsynaptic potentials of dendrites (Chen et al., 2007; Cruikshank et al., 2002; Juergens et al., 1999; Kaur et al., 2004; Kreiman et al., 2006; Logothetis and Wandell, 2004; Mitzdorf, 1985; Mitzdorf, 1987; Nielsen et al., 2006), and thus reflects inputs from other brain regions as well as local neural processes mediated by interneurons. Spiking activity reflects local processing and the outputs of neurons to other brain regions. Simultaneous LFP and spike recordings provide a way to compare the dendritic input to the spiking output, which is required to understand the transformation of neural signals from one processing stage to the next. In general, brain areas where cognitive functions are computed should show response modulations in the spiking activity of single units before they appear in the LFP. Whereas, the brain areas that receive this information from other areas should show response modulations first in the LFP, or simultaneously in the LFP and spiking activity (Nielsen et al., 2006). In this study, we specifically examined the transformation of a non-selective visual representation of items in a search array into a cognitive signal that identifies the location of the behaviorally relevant target stimulus. The FEF is an important site of convergence in the visual system (Jouve et al., 1998; Schall, 1997; Schall et al., 1995b; Vezoli et al., 2004). The FEF receives retinotopically organized input from dorsal stream visual areas MT, MST, and LIP; ventral stream visual areas V4, TEO, and TE; and from the supplementary eye field and prefrontal areas 46 and 12. The dorsal stream innervation is most likely responsible for the fast non-selective initial visual responses we measured in the LFP and spikes (Bisley et al., 2004; Chen et al., 2007; Pouget et al., 2005; Schmolesky et al., 1998). The latencies of the initial visually-evoked LFP and spike responses were correlated, appearing in the LFP signals about 15 ms before 35 the spikes in the visual search tasks, and about 10 ms before the spikes in the memory-guided saccade task. At the recording sites with the earliest spike latencies, the LFP latency was about 2 ms earlier. The earlier visually-evoked modulation in the LFP than in the spike response is consistent with studies in visual cortex (Logothetis et al., 2001; Schroeder et al., 1998), and with the hypothesis that the LFP signal reflects synaptic input and indicates that the initial visual response was relayed to the FEF from other brain areas. The reverse temporal relationship was found in the visual search data when we compared the time course of spatial selectivity in the LFP response and spiking activity. Following the initial non-selective visual response, a spatially selective signal identifying the location of the search array target emerged first in the spiking activity, and then in the LFP signal about 30 ms later. The earlier spatially selective signal in the spiking activity suggests that the representation of the location of the behaviorally relevant target stimulus is computed within the FEF rather than relayed from other brain areas. This temporal difference could not have been produced by LFP signal distortion (see METHODS). Our tests showed only a 1-2 ms delay due to our recording methods. Another indication that this was not a serious problem was the earlier LFP visual onset latencies measured in the visual search tasks (see Figure 2.7A) and earlier LFP selection times in the memory-guided saccade tasks (see Figure 2.7C). If signal filtering caused the delay of the selection times in the visual search LFP signals, it should have also caused delay of selection times in the memory-guided saccade LFP signals – but this was not the case. The alternative interpretation is that some modulations in synaptic activity cannot be detected in event related brain potentials using the methods we employed in this study. It is possible that FEF generates the strong spatially selective spiking signals by amplifying weak differences in the synaptic inputs. Although the exact nature of the input signals to FEF is currently unknown, they must contain information about the visual stimuli, and differences between them. Our results suggest that computations in FEF convert these differences into a strong categorical representation identifying the target location, regardless of the visual feature that differentiates the target from distractors. Consistent with this view, in our study we used two different classes of visual features, color and shape, and we obtained the same results. The recording sites with the earliest LFP visual response latencies tended to have the earliest spatial selection times in the spiking activity. In dorsal stream visual areas of monkey cortex, LFPs 36 recorded in lamina 4 have the shortest visual response latencies due to feedforward input from lower areas (Chen et al., 2007; Schroeder et al., 1998). We therefore made the reasonable assumption that the FEF recording sites with the earliest LFP visual response latencies were functionally closer to the feedforward inputs. Although we cannot identify the cortical layers we were recording from, the results depicted in Fig. 2.8 suggest that spatial selectivity in FEF originates first in neurons near the feedforward input and then is distributed to the functionally more distant regions in FEF via local connections or feedback from other areas. Consistent with this view, at the recording sites with the latest LFP visual response latencies, the selection times measured in the LFP and spikes did not differ significantly. This is the first evidence for such functional architecture and further studies are clearly needed test this hypothesis. Our results suggest that spatial selectivity during a pop-out covert visual task is generated in FEF from non-spatially selective inputs. A few studies have examined the relationships between LFP and spiking responses in other areas. In area MT, for example, Liu and Newsome (2006) found that tuning for motion direction and speed in LFP responses are highly correlated with that of spike activity. In inferotemporal (IT) cortex, Krieman et al. (2006) showed a simultaneous time course of object selectivity in LFP responses and spiking activity. A study by Nielsen et al. (2006) showed that spikes and LFPs in IT exhibited learned object selectivity and the modulation of LFP responses, but not spiking activity, grew stronger from posterior to anterior IT. Because LFP modulation reflects the synaptic input, they concluded that learned object selectivity was encoded first in posterior IT and then transmitted to anterior IT. Only one study, conducted in area V4, has compared the spatial selection process measured in LFPs and spikes during visual search (Bichot et al., 2005). In that study, spatially selective responses appeared in the LFP and spikes at the same time. Although it was not specifically addressed in that study, the simultaneous modulation in LFP and spikes suggests that the spatial selectivity was present in the inputs. The combined analysis of LFPs and spikes promises to provide useful information for understanding computations in the brain. Also, LFPs recorded in monkeys can be an important link between monkey single unit data and human EEG and imaging data (Logothetis and Wandell, 2004; Woodman et al., 2007). For example, the spatially selective LFP response we report could be related to the attention-related modulations observed in human EEG recordings during visual search (Luck and Hillyard, 1994). Single units, LFPs and EEG recordings provide high temporal resolution. It is more difficult, 37 however, to localize the source of the computations reflected in EEG recordings than in the other two signals. EEGs recorded from scalp electrodes reflect the post-synaptic potentials summed over a large region of the brain that could include many areas that are related to spatial vision. The FEF is just one of the potential sources of the spatially selective signals necessary for spatial attention (Pessoa et al., 2003; Serences and Yantis, 2006). Further work is needed to determine the relationships between LFPs and spikes within and between the many regions of the brain involved in spatial attention. Chapter 3 Spatial selective signals in the frontal eye fields mediate spatial attention and predict accuracy in a covert visual search task that requires object identification Relating spatial attention to neural activity requires an ability to create behavioral tasks that allow the separation of spatial attention from other cognitive factors and a potential target area in the brain from which to obtain the neuronal data. In this chapter, we explore evidence from our experiments that combine neurophysiology and behavior in order to demonstrate that FEF is one of the sources of the spatially selective signal in the brain. We explore the time course of the spatial selection in FEF in local field potentials (LFP) and neural spiking activity. 3.1 Introduction Visual spatial attention is a critical component of normal vision and is necessary for the recognition of objects in natural environments (Sheinberg and Logothetis, 2001; Rensink, 2002). It enhances the representation of visual information at selected peripheral locations thereby improving detection and discrimination (Maunsell and Cook, 2002). It is widely accepted that the spatially selective signals in the frontal eye field (FEF) are associated with the planning and execution of saccadic eye movements (Goldberg and Segraves, 1989; Hanes et al., 1998; Schall and Thompson, 1999; Tehovnik et al., 2000). A growing body of evidence suggests that FEF also plays a causal role in directing covert spatial attention (Awh et al., 2006). Human fMRI studies show that the FEF is active during the allocation of attention with and without eye movements (Beauchamp et al., 2001; Corbetta and Shulman, 2002; Kincade et al., 2005; Bressler et al., 2008; Kelley et al., 2008). Transcranial magnetic stimulation over FEF modulates perceptual 38 39 performance in covert attention tasks (Grosbras and Paus, 2002; Muggleton et al., 2003; Smith et al., 2005), and also modulates visual activity in extrastriate visual cortex (Silvanto et al., 2006; Taylor et al., 2007; Morishima et al., 2009). In monkeys, electrical microstimulation of FEF enhances perception (Moore and Fallah, 2001, , 2004; Schafer and Moore, 2007) and produces enhanced visual responses in extrastriate visual cortex that resemble the effects of covert spatial attention (Moore and Armstrong, 2003; Armstrong and Moore, 2007; Ekstrom et al., 2008). Inactivation of FEF disrupts target detection during covert visual search (Wardak et al., 2006) Neuron recordings in monkeys have shown that the activity of visually responsive FEF neurons identifies the location of an attended visual stimulus, even in the absence of eye movements (Kodaka et al., 1997; Murthy et al., 2001; Sato and Schall, 2003; Thompson et al., 2005a). Conspicuously absent from the evidence linking FEF to covert spatial attention, however, is data that directly link neuronal activity in FEF to behavioral measures of attention. In this study we compare the activity of neurons during trials with differing behavioral outcomes to determine whether and when neural signals in FEF predict the monkeys’ accuracy and speed of identifying a target object during visual search. The results support the hypothesis that FEF activity plays a causal role in directing covert spatial attention and enhances the processing of a stimulus to be identified as it is being processed by the visual system. 3.2 Methods Data collection The data were collected from two male monkeys (Macaca mulatta) weighing 6.5 kg (monkey C) and 7.8 kg (monkey B). All surgical and experimental protocols were approved by the National Eye Institute Animal Care and Use Committee and complied with the National Institutes of Health Guide for the Care and Use of Laboratory Animals. Sterile surgery was performed under ketamine and isofluorane anesthesia to place a head-holding device, a plastic recording chamber over the frontal eye field, and a scleral search coil. Frontal eye field was localized within the recording chamber using low current microstimulation (< 50 µA) to evoke saccades, and by the presence of saccade-related movement neurons (Bruce and Goldberg, 1985). Neuronal recordings were made in the left FEF in monkey C and in the right 40 FEF in monkey B. Recording sites were confirmed to be in the rostral bank of the arcuate sulcus by MRI in both monkeys and histologically in monkey C. Visual stimulation and behavioral control was done by a computer running REX (Hays et al., 1982). Visual stimuli were presented on a computer monitor (26 cm X 21 cm, 1024 X 768 pixel resolution, 60 Hz frame rate) viewed at a distance of 57 cm. In each recording session, a single tungsten microelectrode (Frederick Haer, Bowdoin, ME) was inserted into the FEF by a motorized microdrive under computer control. Action potential waveforms were digitized, and saved using a computer-based data acquisition system (Plexon). Often two or three units were recorded simultaneously. Offline spike sorting separated single units based on the size and shape of the spike waveforms. Analog eye position and lever position signals were digitized and sampled at 1 kHz. Behavioral training and tasks Monkeys were seated in a primate chair with head fixed. Using operant conditioning with positive reinforcement, the monkeys were trained to perform a memory-guided saccade task and a covert visual search task. The two tasks were run in separate blocks of trials. The memory-guided saccade task was used to map the spatial extent of each neuron’s response field (Bruce and Goldberg, 1985). After the monkey fixated a 0.3° diameter grey spot on a black background for 400-800 ms, an identical spot was flashed for 50 ms at a peripheral location. The monkeys were required to maintain fixation on the central spot for a random interval ranging from 800 to 1400 ms. After the central spot disappeared, the monkeys were rewarded for making a saccade to the remembered location of the target. Once gaze shifted, the target reappeared to provide feedback and a fixation target for the monkeys. After mapping the neuron’s response field with the memory-guided saccade task, the monkeys performed the cued covert visual search task illustrated in Figure 3.1a. A lever that could be turned left or right of vertical was attached to the front of the chair within easy reach of the monkey. When no force was applied to the lever, a spring automatically returned it to the vertical position. Although they were free to use either hand to grasp and turn the lever, both monkeys were exclusively right-handed. After grasping the lever and positioning it within 10° of vertical, a small (0.3°) central yellow fixation cross appeared on a black background. The monkeys were required to maintain eye position 41 within 2° of the central fixation cross until the reward. After fixating the central cross for a random interval (400-800 ms), a cue array was presented that was comprised of eight 2° diameter isoluminant rings spaced equally around the fixation cross. The rings were isoeccentric and adjusted so that at least one ring was inside the neuron’s response field. Response field eccentricities of recorded neurons ranged between 8° and 12° of visual angle. On about 75% of trials one of the rings was red and the other seven rings were green. On the remaining 25% of trials, all the rings were green. The position of the red ring was randomized from trial to trial and served as a spatial cue for the most probable location of the visual search target to be identified. After a cue-target onset asynchrony (CTOA) lasting between 0 and 470 ms one item of the search array appeared simultaneously inside each of the colored rings. All items of the search array were gray in color and 1° in diameter. The target of the search array was a leftward or rightward oriented Landolt C target with a 0.5° gap on the left or right. The Landolt C target appeared on about 75% of trials (range: 66% to 83% across sessions). The monkeys’ task was to report the presence and identity of a Landolt C target with a leftward or rightward lever turn that corresponded to the location of the gap in the Landolt C target. The monkeys were rewarded for making the correct lever turn (> 15° from vertical) within 2 seconds following search array presentation; in practice, the monkeys nearly always turned the lever to the limit of 35° from vertical. If the monkey broke fixation at any time during the trial, released the lever, or made an incorrect lever turn, the trial was immediately aborted. In about 25% of trials (ranging from 17% to 34% across sessions) no target and only distractors were presented and the monkeys were rewarded for releasing the lever. Overall, about 46% of target present trials were valid cue trials; the Landolt C target appeared inside the red ring (range from 31% to 55%). About 29% of target present trials were invalid cue trials; the Landolt C target appeared inside one of the three green rings opposite the red ring (range from 14% to 39%). Therefore, on invalid cue trials each of the three target locations opposite the red ring had about a 10% probability of containing the C target. The remaining 25% of target present trials were neutral cue trials; all the rings in the cue array were green. Combining the probabilities of cue or neutral cue, target or no target trial, and valid or invalid cue, the approximate percentages of each trial condition shown in Figure 3.1a within a recording session are as follows: valid cue trials – 34%, invalid cue trials – 22%, neutral cue target trials – 19%, cue no target 42 trials – 19%, and neutral cue no target trials – 6%. Although the task was the same for the two monkeys, there were some differences in the visual stimuli. Data was collected first from monkey C. In this monkey, we tested various CTOAs between 0 and 470 ms. We found that reaction time was longest for 0 ms CTOAs, and reaction times did not differ across trials with CTOAs greater than 100ms (Fig. 3.1b). For monkey B we used only three CTOAs: a short CTOA of 33 ms, a medium CTOA of 167 ms, and a long CTOA of 300 ms. The luminance of the search array and were also different between the two monkeys. Luminance was measured using a Minolta CA100 spectrophotometer. For monkey C, the luminance of all the visual stimuli was 3.7 cd/m2 on a dark gray background of 0.5 cd//m2. During practice sessions with monkey B it became evident that the task was too easy. Therefore, to generate enough errors for analysis it was necessary to make the task more difficult. We did this by decreasing the luminance of the search array stimuli to 0.2 cd/m2 on a black background of 0.02 cd/m2. In addition, to make the task more difficult for monkey B, we altered the shape of the distractors. For monkey C the distractors of the search array were gray rings with no gaps, for monkey B they were upward or downward oriented Landolt Cs as illustrated in Figure 3.1. Finally, to provide further encouragement to monkey B to direct attention to the red ring, on about 70% of valid cue trial the search array stimuli were presented for only 50 ms. The behavioral and neurophysiological results were the same from trials in which the search array was presented briefly and from trials in which the search array remained on and were combined for the behavioral and neurophysiological analyses. In all invalid cue and neutral cue trials the search array remained on the screen for the entire trial. The results from 30 recording sessions are included in this study (monkey C: 10 sessions; monkey B: 20 sessions). Data analysis A lever turn was defined as turn > 15° from vertical. The beginning and end of each lever turn was defined as the beginning and end of the monotonic change in lever position before and after the 15° threshold was reached. The time of the beginning of the lever turn on each trial was used as the reaction time for that trial. The only trials included in the analysis where those in which the monkeys maintained fixation on the central fixation cross until after the behavioral report of lever turn or lever release. Spike density functions were calculated for each trial by convolving spike times with a Gaussian filter (σ = 10 ms). For the analysis of neural activity, we compared activity on trials when the target of the 43 search array was in the response field to activity on trials when only distractors were in the response field. For invalid cue trials, distractor related activity was from trials in which the red cue stimulus appeared in the response field. To be included in any of the analyses in this study a neuron had to exhibit significant spatially selective activity before the lever turn on correct trials and have at least 4 trials for each trial condition being compared. The time course and magnitude of spatially selective activity was quantified by calculating the area under the receiver operating characteristic (ROC) curve at each millisecond from activity aligned on the time of search array presentation. This method has been described in detail in previous reports (Thompson et al., 1996; Thompson et al., 2005b). The ROC area measures the separation of 2 distributions normalized to values between 0 and 1 and corresponds to the probability of an ideal observer correctly identifying which distribution a sample belongs (Green and Swets, 1966). An ROC area of 0.5 indicates that the 2 distributions of activity are completely indistinguishable. ROC areas of 0 and 1 are equivalent statistically; both indicate that the 2 distributions are completely separate. For each neuron in this study, the ROC analysis was used to compare the activity during trials in which the search array target was presented in the RF was compared to the activity during trials in which the distractor was presented in the RF. As a convention, the analysis was structures so that ROC area values > 0.5 indicate that the activity during trials that the target was in the RF was greater than the activity during trials that distractors were in the RF. Only those neurons that exhibited spatially selective activity in the cued covert search task were included in this study. A neuron was determined to exhibit spatially selective activity if the average activity during correct valid cue trials was significantly greater in trials in which the target was in the RF than in trials in which a distractor was in the RF during the time interval between 150 ms following the appearance of the search array and the median reaction time. Statistical significance was determined using a t-test (p < 0.05). To be included in an analysis, there must have been at least 4 trials for each of the trial conditions being compared. In addition, the ROC area comparing target and distractor related activity on correct valid cue trials had to reach a value of 0.6. 44 3.3 Results We recorded single unit activity in the FEF of two monkeys performing a cued covert visual search task (Fig. 3.1a). The monkeys’ task was to maintain fixation on the central cross and report the presence and identity of a peripheral leftward or rightward oriented gray Landolt C in the search array with a leftward or rightward lever turn, respectively, or to report target absence by releasing the lever In about 75% of trials a popout red ring among green rings in a cue array identified the most probable target location. In the remaining neutral cue trials, all the rings in the cue array were green. Figure 3.1: Task and behavior. a, The cued covert visual search task. After the monkey grasped the lever in the vertical position and fixated a small central fixation stimulus a cue array appeared. Cue trials – In about 75% of trials one of the rings was red and the rest were green. Neutral cue trials –In about 25% of trials all the rings were green. After a variable cue-target onset asynchrony (CTOA) ranging from 0 to 470 ms a search array appeared. The target of the search array was a leftward or rightward oriented Landolt C. The location of the red ring identified most probable target location, but in a minority of trials the cue was invalid – the target C appeared in one of the 3 green rings directly opposite the red ring. The monkeys were required to maintain fixation at the center and were rewarded for turning the lever in the same direction as the leftward or rightward oriented C target and for releasing the lever if there was no target. Reaction time (RT) was measured from search array presentation to the time of the monkeys’ behavioral report. The temporal structure of the task is illustrated at the bottom. The approximate percentages of each trial condition are indicated in parentheses. b, Cue-related behavior. Average RT and percent correct ± SEM across recording sessions is shown for monkey C (left) and for monkey B (right) from valid cue trials (red bars), invalid cue trials (green bars), and neutral cue trials (white bars). Percent correct performance from 45 no target trials is shown for each monkey (gray bars). For Monkey C we grouped trials into 4 groups based on similar CTOAs. For Monkey B all recording sessions included the same three CTOAs: 33 ms, 167 ms, and 300 ms. For each monkey, the average RT and percent correct across all trials is shown at the far right. c, Average reaction times of monkey C (top) and monkey B (bottom) in correct and error trials for each trial condition. The color conventions are the same as in b. Errors in target present trials are separated by behavior into lever turn errors and lever release errors. The critical manipulations in this study were whether or not there was a red cue ring and the spatial relationship between the red cue ring and the Landolt C target. For each monkey we determined whether reaction time and performance accuracy varied with whether or not there was a spatial cue, with cue validity, and with duration of the cue-target onset asynchrony (CTOA)(Fig. 3.1b). The behavioral results were similar across CTOAs. The only exception was that reaction times were longer for the shortest CTOAs (0 ms for monkey C and 33 ms for monkey B). For all CTOAs, reaction times were shorter and performance accuracy was better on valid cue trials compared to invalid cue trials. For reaction time, an ANOVA that factored the type of trial (valid cue trial vs. invalid cue trial) and CTOA group revealed a significant difference in reaction times between valid and invalid cue trials (p = 0.005 for Monkey C, p<.001 for Monkey B), and across CTOA groups (p<.001 for both monkeys) due to the longer reaction times for the shortest CTOAs. When the shortest CTOA trials were removed, there was no significant difference in reaction time across the remaining CTOA groups for either monkey. There was a significant difference in accuracy across valid cue and invalid cue trials (monkey C: p = 0.04; monkey B: p<.00001), with no difference across CTOA groups for either monkey. There were no significant interactions between CTOA and trial type for reaction time or performance accuracy. The behavioral results from neutral cue trials tended to be intermediate between valid and invalid cue trials for monkey C, and similar to the results from invalid cue trials for monkey B. Performance accuracy on no target trials is included in Figure 3.1b. Performance accuracy did not differ significantly across CTOAs for either monkey. For all trial types the performance accuracy was well above chance for a three alternative choice task (33%) which indicates that the monkeys were using the stimuli in the search array to guide their behavioral reports. In addition, the improvement of performance on valid cue trials relative to invalid and neutral cue trials indicates that the monkeys used the red ring as a spatial cue to guide attention. Average reaction times from correct and error trials across all recording sessions and CTOAs are 46 compared in Figure 3.1c. There were two types of errors in trials in which a target was present. The monkey could turn the lever in the wrong direction indicating that the C target was present but incorrectly identifying its direction or the monkey could release the lever indicating that the target was not present. On no target trials, an error was a lever turn indicating the presence of a target. There are two main results when comparing reaction times from correct and error trials. First, reaction times from error trials tended to be slightly longer than from correct trials suggesting that errors were not due to a speedaccuracy tradeoff. Second, reaction times from correct and error lever release trials, in which the monkeys reported the absence of a target, were approximately double that of lever turn trials in which the monkeys reported the presence of a target. This doubling of reaction times in trials that the target was not found is indicative of an inefficient, serial, self-terminating visual search (Treisman, 1988; Chun and Wolfe, 1996). Neuron activity The focus of this study is the relationship between spatially selective activity in FEF and visual perception as indexed by the monkeys’ behavioral report of target identity. Therefore, we included only those neurons that exhibited significant spatially selective activity for the location of the C target before the lever turn in correct valid cue trials. We recorded enough trials from a total of 49 FEF neurons that met this criterion (19 from monkey C and 30 from monkey B). As in a previous study, all of the neurons with spatially selective activity during the covert search task were visually responsive (Thompson et al., 2005a). Figure 3.2 shows the activity during correct target present trials from a representative FEF neuron recorded in monkey C performing the cued covert visual search task with three CTOAs. This neuron exhibited an initial visual response at about 80 ms following the cue array presentation that did not discriminate the red cue ring from the green rings. After the initial non-selective response, the neuron’s activity identified the location of the red ring by exhibiting greater activity when a red ring was in the response field (RF) than when a green ring was in the RF. During valid cue trials (left column), after the appearance of the search array the neuron maintained spatial selectivity for the C target until the monkey’s behavioral report. During invalid cue trials (middle column), after the presentation of the search array, the spatial selectivity switched to identify the location of the C target inside a green ring in 47 the RF before the monkey’s behavioral report. During neutral cue trials (right column) the spatially selective response emerged after the appearance of the search array and identified the location of the C target. The time course and magnitude of spatial selectivity in the cued covert search task were quantified for each FEF neuron using an ROC analysis (Thompson et al., 1996; Thompson et al., 2005b). Examples of this analysis are shown in Figure 3.2b for the activity recorded on 235 ms CTOA trials (last row of Figure 3.2a). The area under the ROC curve quantifies the difference between two distributions normalized to values between 0 and 1 with 0.5 indicating completely overlapping distributions. Values of 0 and 1 indicate completely non-overlapping distributions. The magnitude of spatial selectivity is indexed in the distance the ROC area is from 0.5; an ROC value of n is equivalent in magnitude to 1 – n. The analysis was structured so that ROC values greater than 0.5 indicate spatial selectivity toward the target location and ROC values less than 0.5 indicate spatial selectivity away from the target location. For convenience we will refer to the ROC area as the selectivity index (SI). Before the visual response to the cue array, the SI was close to 0.5 indicating equivalent baseline activity. Following the visual response to the cue array the activity for the red ring stimulus during valid and invalid cue trials grew to higher levels than the activity for the green ring stimuli. This emerging difference in activity is reflected in a change in the SI that begins around 100 ms following the presentation of the cue array. Due to the way the ROC analysis was structured, the initial change in the SI is toward 1 on valid cue trials indicating that the spatial selectivity was directed toward the future target location and toward 0 on invalid cue trials indicating that the spatial selectivity was directed away from the future target location. In the neutral cue trials all 8 rings were green. The probability of the target to be identified appearing in anyone of the green rings was 1/8. Therefore, during these trials the SI remained around 0.5 indicating equivalent activity before the presentation of the search array. After the presentation of the search array, but before the behavioral report, the SI across all trial conditions became greater than 0.5 reflecting the selection of the C target location. The switch in spatial selectivity during invalid cue trials from the cue location to the target location is indexed by the changing SI in time from values less than 0.5 following the cue array to values greater than 0.5 following the target array. The pattern of activity exhibited by the example neuron shown in Figure 3.2 was present in the activity of all 48 49 neurons (see Figs 3.4-3.6) included in this study. Figure 3.2: Activity of a single FEF neuron during correct target present trials. a, Average activity during valid cue trials (left column), invalid cue trials (middle column) and neutral cue trials (right column) separated by cue-target onset asynchrony (CTOA) in different rows. The CTOAs of the trials in each row are indicated at the left. Activity is aligned on the time of the search array presentation and ends at the median reaction time of each trial condition. The time of cue array presentation is marked by a black triangle below each plot. The task illustrations at the top of the figure provide a key to the color and line types of the neuronal activity traces. The color corresponds to the color of the cue array stimulus in the response field (RF) (red: red ring, green: green ring). The line type differentiates activity from trials in which the target C (solid line) or a distractor stimulus (dotted line) appeared in the RF. b, Analysis of the change in spatial selectivity in time for activity from trials with 235 ms CTOAs (bottom row in a). The selectivity index (SI) is the area under the receiver operating characteristic (ROC) curve computed at each millisecond from the activity occurring before the lever turn on individual trials. SI values greater than 0.5 indicate selectivity toward the target location and SI values less than 0.5 indicate selectivity away from the target location. A recent study has shown that some FEF neurons can exhibit shape selective responses (Peng et al., 2008). Therefore we examined whether there was any evidence of a preference for a leftward or rightward oriented C targets. For each neuron we compared the firing rates from valid cue trials with leftward C target in the receptive field to those with rightward C targets in the receptive field. The average firing rate measured between 100ms after search array onset to median reaction time did not 49 differ significantly for 94% (46/49) of neurons (t-test, p > 0.05). Although 3 neurons exhibited a slight significant difference in firing rate (p < 0.05), the difference never exceeded 10%. The key result was that all 49 neurons exhibited a strong spatially selective response for both leftward and rightward oriented C targets. Therefore, we conclude that the neurons recorded in this study did not exhibit object-related selectivity for the C target, but instead represented the location of the target to be identified. Relationship of cue-related spatial selectivity to behavioral measures of spatial attention The relationship between neuronal activity and perceptual performance averaged across recording sessions is shown in Figure 3.3. For each neuron we quantified the cue-related selectivity during valid, invalid, and neutral cue trials as the average SI measured from 50 ms before to 50 ms after the appearance of the search array. This period of time captures the activation related specifically to the cue array during the 100 ms just prior to the beginning of activity related to the search array. Only trials with CTOAs greater than 100 ms were included in this analysis because shorter CTOAs do not allow for the opportunity for cue related responses to develop before the search arrays appears (see Fig. 3.2a, top row). Figure 3.3a plots, for the two monkeys separately, average performance accuracy across recording sessions as a function of average SI for each trial condition from all neurons. Error trials were included in the SI calculations because performance accuracy is a measure that includes correct and incorrect behavioral reports. Figure 3.3b plots reaction time from correct trials averaged across recording sessions as a function of average SI calculated from correct trials. Performance accuracy was best and reaction time was fastest when the cue related spatial selectivity was directed toward the future target location during valid cue trials. The worst performance accuracy and slowest reaction times were from trials when the cue-related selectivity was directed away from the future target location during invalid cue trials. During neutral cue trials there was not a spatially selective response for the cue array and the behavioral measures were intermediate. There was a weak spatial bias in correct neutral cue trials (Fig. 3.3b) that will be addressed below (see Fig. 3.6). A Pearson correlation analysis using the ROC areas and behavioral measures obtained from individual recording sessions revealed significant positive correlations for accuracy (monkey C: r = 0.30, p = 0.02; monkey B: r = 0.50, p < 0.001) and significant negative correlations for reaction time (monkey C: r = -0.32, p = 0.02; monkey B: r = -0.51, p < 0.001). 50 Figure 3.3: Relationship between cue-related spatial selectivity and behavioral measures of spatial attention for monkey C (open symbols) and for monkey B (solid symbols). For each neuron the cue-related SI was calculated for each trial condition as the average SI between 50ms before to 50ms after the appearance of the search array. Only trials in which the CTOA was > 100ms were included. a, Each point plots the average percent correct performance across recording sessions as a function of average SI from each trial condition (circles: valid cue trials, triangles: neutral cue trials, squares: invalid cue trials. The error bars are SEM across recording sessions for percent correct performance and SEM across neurons for SI. Correct and error trials were included in the analysis. The correlation across individual neurons and recording sessions is significant for both monkeys (monkey C: r = 0.30, p = 0.02; monkey B: r = 0.50, p < 0.001). b, Each point plots the average of median reaction times in correct trials across recording sessions as a function of average SI in correct trials across trial conditions. Conventions are the same as in a. The correlation across individual neurons and recording sessions is significant for both monkeys (monkey C: r = -0.32, p = 0.02; monkey B: r = -0.51, p < 0.001). These results are consistent with the hypothesis that FEF neurons track the monkey’s allocation of spatial attention in the task. The activity first identifies the location of the attentional cue, and this activity is generally predictive of overall behavioral tendencies across trial conditions. In addition, following the appearance of the target array, the spatially selective activation identifies the location of the target object to be identified. Notably, during invalid cue trials the spatially selective response shifts 51 away from the popout cue stimulus and towards the target before the monkey identifies the target with a lever turn. These observations, however, do not demonstrate a direct relationship between spatially selective activity in FEF and an enhancement of perceptual processing on a trial-by-trial basis. In the following analyses we directly examine the relationship between FEF activity and the monkeys’ ability to detect and identify the target of the search array as indexed by their behavioral report. Accuracy of target identification Only trials with CTOAs greater than 100 ms were included in this analysis because shorter CTOAs do not allow for the analysis of cue related responses. Because the monkeys’ behavior across the remaining longer CTOAs were not different they are combined. Also, the neuronal activity results from the two monkeys were not different, so they are combined. Valid cue trials. Figure 3.4 shows an analysis of the population activity during valid cue trials. The C target appeared inside the red ring. In Figures 3.4a-c the population average activity during trials in which the red cue ring and the C target were presented in the RF is compared to the population average activity during trials in which a green ring and distractor were presented in the RF. Shown in Figure 3.4 are plots of the average activity in correct trials (Fig. 3.4a), error trials in which the monkeys made an incorrect lever turn (Fig. 3.4b), and error trials in which the monkeys released the lever (Fig. 3.4c) from the 17 neurons that had enough trials for the analysis. 52 Figure 3.4: Population analysis of activity during correct and error valid cue trials. Color and line conventions are the same as in Figure 3.2. a-c, Pooled average activity in (a) correct trials, (b) lever turn error trials, and (c) lever release error trials for the 17 neurons with enough trials for the analysis of all three trial conditions. Activity is aligned on the time of the search array appearance and ends at the average of median reaction times across the included sessions. The range of median reaction times across recording sessions from correct valid cue trials is indicated by the bracket above the activity plots. d, Pooled average SI from correct (solid line) and error trials (dotted line). All error trials were combined to obtain a greater number of neurons with enough trials to analyze (N = 29). SEM at each millisecond is indicated by the width of the magenta (correct trials) and cyan (error trials) shading. The horizontal black bar above the time axis marks periods of significantly greater spatial selectivity in correct trials than in error trials (paired t-test, p < 0.05). e, Average activity between 150 ms after search array presentation and median reaction time in correct trials from the 17 neurons shown in a-c. The approximate time range of the activity analysis is shown by the gray bars in a-c. Median reaction time varies across recording sessions; the vertical line within the RT bracket represents the median reaction time on correct trials across recording sessions. Error bars in e indicate SEM across neurons. Solid red bars show the average activity during trials in which the red cue ring and target C were in the neurons’ RF. Dotted green bars show the activity during trials in which the green ring and distractors were in the neurons’ RF. The average activity on correct trials, lever turn error trials (turn) and lever release error trials (release) are shown separately. The magnitude of target related activity varied significantly across the three trial types (Friedman test for related samples, p = 0.014). Distractorrelated activity did not vary significantly across the three trial types (Friedman test, p = 0.33). Before the appearance of the search array there is greater activity for the red cue ring than for the green rings in both correct and error trials. After search array presentation, the spatially selective response continues in both correct and error trials. During the error trials in which the monkeys released the lever, thereby reporting that no target was present, the spatially selective response eventually disappears before 53 the behavioral report (Fig. 3.4c). Reaction times were faster in correct trials than in error trials (Fig. 3.1c). Reaction times from correct trials, therefore, provide an endpoint time to look for difference in FEF activity that is predictive of the monkeys’ accuracy of reporting target identity. In other words, to influence target identification, modulations of activity must occur before the monkeys typically would make a correct report of target identity. For each neuron, the time course of the SI was calculated from correct and error trials up to the median reaction time from correct trials. To include the maximum number of neurons in the analysis, the two types of error trials were combined. Two main points are evident in Figure 3.4d, which compares the population average SI from correct and error valid cue trials. First, the spatially selective responses for the red cue in correct and error trials occurring before the appearance of the search array do not differ. Second, the spatially selective responses in correct and error trials become significantly different between 143 ms and 271 ms after the search array appears. The two types of error trials are qualitatively different. Lever turn errors indicate that the monkey detected a target C but incorrectly identified its direction and lever release errors indicate that the monkey did not detect the presence of a target C in the search array. We examined whether there are differences of activity for the two types of errors. Figure 3.4e compares the magnitude of neuronal activity in correct and error valid cue trials before the time the monkeys’ made a correct behavioral report. For each neuron we calculated the average activity for each trial type from 150 ms after the presentation of the search array to the median reaction time from correct trials. The neurons exhibited significantly greater activity for the target C in the RF than for distractors in the RF during correct and error trials (paired t-test, p < 0.001). However, the magnitude of target related activity varied significantly across the three trial types (Friedman test for related samples, p = 0.014) and was ranked according to the correctness of the behavioral report. The highest activity was during correct trials, intermediate activity was during the partially correct trials in which the monkeys correctly reported that the C was present but incorrectly reported its direction, and the lowest activity was during error trials in which the monkey reported that the target was absent. Distractorrelated activity did not vary significantly across the three trial types (Friedman test, p = 0.33). 54 Invalid cue trials Figure 3.5: Population analysis of activity during correct and error invalid cue trials. Figure conventions are the same as in Figure 3.4. Color and line conventions are the same as in Figure 3.2. a-c, Pooled average activity in (a) correct trials, (b) lever turn error trials, and (c) lever release error trials for the 22 neurons with enough trials for the analysis of all three trial conditions. d, Pooled average SI from correct (solid line) and error trials (dotted line). The horizontal black bar above the time axis marks periods of significantly greater spatial selectivity in correct trials than in error trials (paired ttest, p < 0.05). e, Average activity between 150 ms after search array presentation and median reaction time in correct trials from the 22 neurons shown in a-c. Error bars in indicate SEM across neurons. The magnitude of target related activity (solid green bars) varied significantly across the three trial types (Friedman test for related samples, p < 0.001). Distractor related activity (dotted red bars) did not vary significantly across the three trial types (Friedman test, p = 0.14). Figure 3.5 shows the analysis of the population activity during invalid cue trials. The target appeared in one of the three green rings opposite the red ring. Activity before the appearance of the search array was selective for the location of the red cue ring in correct and in error trials (Figs. 3.5a-c). The magnitude of the cue-related selectivity is the same as in valid cue trials because before the appearance of the search array the monkeys have the same expectation of target location. Also, the cue-related spatially selective activity in correct trials was not different from that in error trials. After search array presentation, the spatially selective responses in correct and error trials become markedly different. In correct trials, 55 there is a clear switch of spatial selectivity from the location of the red ring to the location of the C target. This switch in spatial selectivity during correct trials is quantified Figure 3.5d. The SI shifts from values less than 0.5 before and immediately after search array onset to values greater than 0.5 after about 150 ms following search array onset. In error trials, the shift of spatial selectivity away from the red cue was sluggish and by the time that the monkeys made a lever turn in correct trials, the activity in error trials only weakly identified the location of the C target. The SI from correct trials became significantly greater than the SI from error trials at 136 ms following search array presentation and continued past the median reaction time of correct trials. Figure 3.5e compares the magnitude of neuronal activity during correct trials, lever turn error trials and lever release error trials in the time period between 150 ms following the presentation of the search array to the median RT of correct trials. During this time the neurons exhibited greater activity for the C target in the RF than for distractors in the RF in correct trials (paired t-test, p = 0.003), but not in error trials. As in valid cue trials, the magnitude of target related activity varied significantly across the three trial types (Friedman test for related samples, p < 0.001) and was ranked according to the correctness of the behavioral report. Distractor-related activity did not vary significantly across the three trial types (Friedman test, p = 0.14). Neutral cue trials 56 Figure 3.6: Population analysis of activity during correct and error neutral cue trials. Figure conventions are the same as in Figure 3.4. Color and line conventions are the same as in Figure 3.2. a-c, Pooled average activity in (a) correct trials, (b) lever turn error trials, and (c) lever release error trials for the 32 neurons with enough trials for the analysis of all three trial conditions. d, Pooled average SI from correct (solid line) and error trials (dotted line). The horizontal black bar above the time axis marks periods of significantly greater spatial selectivity in correct trials than in error trials (paired ttest, p < 0.05). e, Average activity between 150 ms after search array presentation and median reaction time in correct trials from the 32 neurons shown in a-c. Error bars in indicate SEM across neurons. The magnitude of target related activity (solid green bars) varied significantly across the three trial types (Friedman test for related samples, p < 0.001). Distractor related activity (dotted green bars) did not vary significantly across the three trial types (Friedman test, p = 0.7). Figure 3.6 shows the analysis of the population activity during neutral cue trials in which there was no spatial cue, all the rings in the cue array were green. There was not a red cue ring in the cue array and before the appearance of the search array the activity was not spatially selective (Figs. 3.6a-c). Spatial selectivity emerged after the presentation of the search array and identified the location of the C target. Like the results from valid and invalid cue trials the SI from correct trials was greater than the SI from error trials during the time in which the search array was present (Fig. 3.6d). However, unlike the results from valid and invalid cue trials, the SI before and around the appearance of the search array was slightly yet significantly (p < 0.05) greater during correct trials than during error trials. This result suggests that in the absence of an exogenous spatial cue the monkeys anticipate the location of the target and this anticipation 57 is reflected in FEF activity. When he guesses correctly, as indexed by a slightly higher SI, he is more likely to be correct. It should be noted, however, that this weak anticipatory bias was associated with performance accuracy but not with reaction time differences across trials (see Fig. 3.8). Figure 3.6e compares the magnitude of neuronal activity during correct trials, lever turn error trials and lever release error trials in the time period between 150 ms following the presentation of the search array to the median RT of correct trials. During this time in correct neutral cue trials the neurons exhibited greater activity for the C target in the RF than for distractors in the RF (Fig. 3.6a; paired t-test, p < 0.001). During error trials in which the monkey made an incorrect lever turn the difference was marginally significant (Fig. 3.6b; p = 0.05) and during error trials in which the monkey released the lever the activity did not differ significantly (Fig. 3.6c; p = 0.4). As in valid and invalid cue trials, the magnitude of target related activity varied across the three trial types (Friedman test for related samples, p = 0.001) and was ranked according to the correctness of the behavioral report. Distractor-related activity did not vary significantly across the three trial types (Friedman test, p = 0.7). 58 Target absent trials Figure 3.7: A comparison of population average activity during (a) correct rejection trials and (b) invalid cue miss trials. In both types of trials the monkey reported target absence by releasing the lever and reaction times (RT) did not differ across trial conditions (Wilcoxon rank sum test; p= 0.9). a, Pooled average activity during correct rejection trials. The activity during trials in which the red ring and a distractor were presented in the RF (red dotted line) is compared to the activity in which a green ring and a distractor were presented in the RF (green dotted line). Activity is aligned on the time of search array presentation. The brackets above the plot show the range of median RTs during individual recording sessions. The vertical line within the RT bracket indicates the median RT on correct rejection trials across recording sessions. b, Pooled average activity during invalid cue miss trials. Conventions are the same as in Figure 3.5c. c, A comparison of the pooled average SI in correct rejection trials (solid line, magenta shading) and invalid cue miss trials (dotted line, cyan shading). SI values less than 0.5 indicate selection of the cue stimulus and SI values greater than 0.5 from invalid cue trials indicate selection of the target stimulus. No target appeared in approximately 25% of trials, all the search array stimuli were distractors. The monkeys were rewarded for reporting target absence by releasing the lever. We identify these trials as correct rejections (Green and Swets, 1966). The average reaction time of correct rejection trials was approximately double that of correct target present trials (see Fig. 3.1c). There is a spatially selective 59 response for the location of the red cue stimulus during correct rejection trials Figure 3.7a. Following the presentation of the search array the spatially selective response weakens and eventually disappears before the monkeys release the lever to report target absence. We hypothesize that the reduction of the spatially selective response corresponds to the monkeys shifting attention away from the red cue stimulus to find the C target. To examine the relationship between spatially selective activity in FEF and the monkeys’ perceptual judgment of target presence, we compared the activity from correct rejections trials to that from invalid cue misses – trials in which the target appeared but was not identified or detected (Fig. 3.7b). The monkeys’ behavioral report was the same; the monkeys released the lever reporting target absence. Reaction times of misses were not statistically different from reaction times of correct rejections (Wilcoxon rank sum test; p= 0.9) (see Fig. 3.1c). The similar reaction times lend confidence to the assumption that when a monkey incorrectly released the lever on a target present trial he was accurately reporting his percept that the target was absent. The activity during invalid cue miss trials is similar to that during correct rejection trials. Following the appearance of the search array, the spatial selectivity for the cue location disappears and the SI returns to values near 0.5. However, during miss trials the SI moves toward 0.5 at a faster rate and ends at slightly higher values than during correct rejection trials. The average SI from correct rejection trials and miss trials measured between 150 ms following the search array and the median reaction time of correct rejections differ significantly (p = 0.01). Therefore it appears that the presence of a target stimulus influences the shift in the spatially selective response away from the red cue stimulus. But the most important result is that after the appearance of the search array a spatially selective response for the target location never develops. This result provides neurophysiological evidence that a visual spatial selection process must occur before an object can be detected and identified. Speed of target identification Manual reaction times have classically been used to measure the benefits and costs of allocating spatial attention (Posner, 1980; Pashler, 1998). The idea behind this practice is that directed attention speeds perceptual processing at the cued location thus leading to faster reaction times. Therefore, we reasoned that if FEF activity corresponds to the allocation of attention, then some aspect of the spatially selective response should be related to the time it takes for the monkey to report the identity of the target 60 stimulus. To test this hypothesis we separated correct trials from each recording session into fast reaction time groups (fastest 35% of the trials) and a slow reaction time groups (slowest 35%) for each trial condition and calculated the SI as before to quantify the time course and magnitude of spatial selectivity. Figure 3.8: The relationship between the SI and the speed of target identification. a-c, Pooled average SI from correct valid cue trials (a), invalid cue trials (b), and neutral cue trials (c) separated in fast reaction time (fastest 35% of trials – solid line, magenta shading) and slow reaction time (slowest 35% of trials – dotted line, cyan shading) trial groups. All plots end at the average of the median RTs across the included sessions. The ranges of median reaction times from the fast RT groups are shown by the brackets above each plot and the vertical line represents the median RT across recording sessions. The shading around the average SI traces represents the SEM across neurons. The horizontal black bars above the time axis indicates periods in which the SI was significantly different between fast and slow RT groups (paired t-test, p < 0.05). d and e, The relationship between the time of spatial selection and the speed of target identification during invalid cue trials. The selectivity switch time is defined as the time the SI crosses 0.5 during invalid cue trials – the time that the spatially selective activity switches from the red cue ring to the C target. d, Each point of the scatter plot compares the selectivity switch time from fast RT and slow RT trials from a single neuron (N = 24). The average selectivity switch time from monkey C neurons (open symbols) is 127 ± 7 ms for fast RT trials and 170 ± 18 ms for slow RT trials. The average selectivity switch time from monkey B neurons (solid symbols) is 186 ± 11 ms for fast RT trials and 227 ± 13 ms for slow RT trials. e, Distribution of the percentage of change in reaction time accounted for by the change in selectivity switch time for single neurons. The average across all neurons is 21 ± 4 %. Figures 3.8a, b and c plot the average SI from the fast and slow reaction time groups for correct 61 valid cue trials (Fig. 3.8a), correct invalid cue trials (Fig. 3.8b), and neutral cue trials (Fig. 3.8c). The critical question is whether there are differences in the SI before the monkey reports the identity of the target in fast reaction time trials that could account for the difference in reaction times across the two groups of trials. The consistent result for all three trial conditions is that the selectivity for the C target location, as indexed by the SI, is greater in fast reaction time trials than in slow reaction time trials. The greatest difference in magnitude of spatial selectivity between fast and slow reaction time trials occurs during the time in which the visual system is processing the target of the search array. During valid cue (Fig. 3.8a) and neutral cue (Fig. 3.8c) trials there were a few brief significant differences in the cue related activity prior to the appearance of the search array. But these differences, although significant statistically, were not consistently greater for fast reaction time trials and, therefore, cannot account for the differences in reaction times across trials. Across all three trial conditions there was a greater spatially selective response in fast reaction time trials beginning around 150 ms and ending around 250 ms following the presentation of the search array (reaction times of fast trials averaged around 300 ms). This result suggests that the magnitude of the spatial selection in FEF influences the speed of the visual processing leading to target identification. Some of the difference in reaction time across trials could be due to differences in the speed of the visual selection process rather than the magnitude of selection. We probed this relationship by examining the time of the switch of spatial selectivity during invalid cue trials from the location of red cue ring to the location of the C target. For each neuron, we first smoothed the SI values in time with a 100 ms boxcar filter. Then we identified the time that the smoothed ROC area crossed 0.5 after the presentation of the target search array on invalid cue trials. We termed this time the selectivity switch time and it is indicated Figure 3.8b. The scatter plot in Figure 3.8d compares the selectivity switch times in fast reaction time trials and slow reaction time trials from all neurons with enough correct invalid cue trials for the analysis (N=24). For monkey C, the average ± SEM selectivity switch time was 127 ± 7 ms in fast trials and 170 ± 18 ms in slow trials. For monkey B, the average selectivity switch time was 186 ± 11 ms in fast trials and 227 ± 13 ms in slow trials. For both monkeys the selectivity switch time in fast and slow trials differed significantly (paired t-test; p < 0.001). If the variability in reaction time is simply related to when the target is selected, then the increase 62 in target switch time should be the same as the increase in reaction time. To test this, for each neuron we calculated the percentage change in reaction time accounted for by the change in selectivity switch time. On average, the percentage of the increase of reaction time accounted for by the increase of selectivity switch time was 21 ± 4 %. Therefore, about 20% of the variability in reaction time to accurately report the identity of the target during invalid cue trials can be attributed to the time of visual selection as measured in FEF. Other contributing factors include the difference in the magnitude of spatial selection as well as variability in motor processes that occur after target identification. 3.4 Discussion The fundamental goal of this study was to directly link neuronal activity to subjective perception as indexed by a behavioral report. This was done by determining whether the spatially selective responses of FEF neurons are related to behavioral measures of spatial attention in a cued object identification task. We used a color popout stimulus as an exogenous spatial cue to inform the monkey of the most probable location of a target to be identified. If the target was not at the cue location (invalid cue trials), or if there was no spatial cue (neutral cue trials), the monkeys had to covertly search for the target by endogenously (i.e., willfully) shifting attention to non-cued locations. The doubling of reaction time in no target trials to report target absence indicates that the search for the target was inefficient and effortful (Treisman, 1988; Chun and Wolfe, 1996). The results of this study show that the magnitude of the spatially selective response in FEF occurring specifically during the time that the target stimulus is being processed by the visual system predicts the accuracy and speed of target identification. In a previous report we showed that the activity of FEF visually responsive neurons identifies the location of the visual search target in monkeys performing easy popout visual search tasks in the absence of eye movements (Thompson et al., 2005a). We hypothesized that the spatially selective activity corresponded to the allocation of covert spatial attention. But this was an inferred relationship based on psychophysical reports showing that the oddball targets of popout search arrays automatically capture attention (Egeth and Yantis, 1997; Turatto and Galfano, 2000). In the current study we confirm that there is a relationship between the spatially selective activity in FEF for a color popout singleton stimulus and behavioral measures of attention obtained simultaneously in the same monkeys. Overall, the highest 63 proportion of correct trials and the fastest reaction times were observed in valid cue trials in which the spatially selective responses to the singleton target of the popout cue array matched the location of the target to be identified (see Fig. 3.3). This pattern of results is consistent with behavioral studies of spatial attention showing improvements in perceptual performance at the attended location at a cost to perceptual performance at non-attended locations (Posner, 1980; Pashler, 1998). This correlation, however, only demonstrates an overall relationship between neuronal activity and perceptual performance and it is entirely predictable based on the results of the previously cited psychophysical and neurophysiological studies. In other words, the same relationship would be observed if the behavioral data and neurophysiological data were obtained at different times or from different subjects. To directly examine the link between FEF and the efficacy of visual processing, in the current study we correlated the activity of FEF neurons with the perceptual performance of the monkeys on a trialby trial basis. We asked whether and when in the course of a covert visual search task do differences in neural activity correspond to differences in perception as indexed by a behavioral report. The results demonstrate a clear relationship between the strength of the spatially selective signal in FEF and the accuracy and speed of reporting the identity of a target in a cued covert visual search task in the absence of eye movements. The correlation between neural activity and the behavioral measures of perceptual performance occurred predominantly between 100 ms following the presentation of the target to be identified and the initiation of the behavioral report. This is the period of time in which the processing related to target detection and identification is taking place in the visual system. The task used in this study requires the monkey to detect, recognize and identify a visual object. As in a previous report (Thompson et al., 2005a), we found no evidence that FEF neurons could identify whether the C target was leftward or rightward. In the primate visual system, there is strong evidence the representation and analysis of visual objects takes place primarily in the temporal lobes along the ventral visual processing stream (Logothetis and Sheinberg, 1996; Tanaka, 1997). Studies have shown that spatial attention is necessary for object recognition in natural scenes (Rensink, 2002), and for object selective responses of neurons in inferior temporal cortex during visual search which begin about 150 ms after search array appearance (Chelazzi et al., 1998). Sheinberg and Logothetis (2001) showed that during active scanning of complex visual scenes with saccadic eye movements, object selective neurons in inferior 64 temporal cortex become active specifically before the saccades used to fixate the target object of the search. These object selective responses in IT cortex that occur specifically before saccades that bring the object to the fovea are likely related to the shift of spatial attention that precedes saccades (Hoffman and Subramaniam, 1995; Kowler et al., 1995; Deubel and Schneider, 1996). There is growing support for the premotor theory of attention (Rizzolatti et al., 1987) that hypothesizes that presaccadic signals from areas like the frontal eye field are a source of the spatial attention signals that enhance visual processing (Awh et al., 2006). Hamker (2005), specifically proposed a model in which spatially selective signals from FEF play an important role in object recognition. In a previous study (Thompson et al., 2005a) and in this study we show that visually responsive FEF neurons exhibit spatially selective responses in the absence of eye movements and, therefore, are a potential source covert spatial attention. In an elegant series of experiments, Moore and colleagues have shown that weak electrical microstimulation of FEF using currents below the threshold to evoke saccades increases the monkeys’ perceptual sensitivity at the location in visual space represented by the neurons at the stimulated site (Moore and Fallah, 2001, , 2004; Schafer and Moore, 2007), and modulates visual responses in extrastriate visual cortex in a manner similar to the effects of directed visual attention (Moore and Armstrong, 2003; Armstrong et al., 2006; Armstrong and Moore, 2007). A particularly relevant result from the study by Moore and Armstrong (2003) is that FEF stimulation modulates activity in area V4 only when the neuron in the visual cortex is being driven by a stimulus in its receptive field. A recent fMRI study in monkeys showed that FEF stimulation modulates activity throughout the visual cortex (Ekstrom et al., 2008). In the current study we demonstrate that spatially selective activity in FEF is correlated with the accuracy and reaction time of object recognition during covert visual search specifically during the period of time in which neurons in the visual cortex are processing the object to be identified. Together these results provide strong converging evidence that FEF activity plays an important role in ongoing visual processing by providing a spatial signal that selects and enhances the representation of behaviorally relevant objects. Human psychophysical experiments have manipulated perceptual task difficulty to demonstrate that visual processing efficiency is directly related to the amount of attentional effort (Lavie and Tsal, 1994; Urbach and Spitzer, 1995). Single-unit recording studies have shown that representations of visual stimuli 65 in the visual cortex of monkeys are stronger with increased task demands (Spitzer et al., 1988; Spitzer and Richmond, 1991; Boudreau et al., 2006). Boudreau et al. (2006) specifically showed that neuronal modulations related to task difficulty in area V4 are correlated with the magnitude of spatial attention as measured with changes in the ability of the monkeys to detect small changes in a visual stimulus. Therefore, a neuronal source of spatial attention should exhibit activity that is related to different degrees of attention as measured by the ability to process visual stimuli. Additionally, the results of recent psychophysical experiments in humans performing covert visual search tasks suggest that spatial attentional selection is a necessary prerequisite for object identification (Evans and Treisman, 2005; Zenon et al., 2008). Therefore, the level of neural activity in an area that directs spatial attention should predict whether or not an object is identified. Our results show that FEF activity exhibits these properties. The magnitude of target related activity in FEF is correlated with the efficiency of processing the target stimulus. For all 3 trials conditions – valid, invalid, and neutral cue trials – target location related activity in FEF was correlated with the accuracy of the behavioral report. The highest level of activity was associated with correct object identification. Activity below this level can support partial processing leading to target detection, but not identification. And when the spatially selective activity in FEF does not reach a minimum threshold level, the monkey does not detect the target. In summary, there are two main results of this study. First, FEF exhibits spatially selective activity for an exogenous spatial cue in the absence of eye movements that is correlated with the overall behavioral benefits and costs of directed covert spatial attention across trial conditions that vary in cue validity. This result is consistent with the view that FEF is a part of a distributed spatial attention network that includes LIP and the superior colliculus, brain structures that exhibit similar patterns of spatially selective responses for the target in popout visual search arrays (McPeek and Keller, 2002; Constantinidis and Steinmetz, 2005; Thomas and Pare, 2007). But trial-to-trial variability in cue-related spatial selection before the appearance of the target is not correlated with the efficacy of visual processing within trial conditions. The second main result is that trial-to trial variability in the accuracy and reaction time of visual object recognition is correlated with the strength of the spatially selective signal after the appearance of the search array, primarily during the time in which the processing related to target detection and identification is taking place along the ventral visual processing stream. This result indicates that FEF 66 activity exerts a top-down modulatory influence on ongoing visual processing and is consistent with the hypothesis that it is a source of covert spatial attention needed for object perception (Hamker, 2005). Chapter 4 Summary and Conclusions The world is ever-changing and infinitely complex. Our brain’s sensory systems are designed to rapidly detect relevant cues, which then are used to guide our behavior. In a covert visual search task, I showed that neuronal activity in the frontal eye fields selects the locations of salient targets that need to be identified to guide the subject’s behavior to obtain reward. The magnitude of spatial selection in the frontal eye fields for the location of the target to be identified predicts accuracy and the speed of identification of that target. The explicit spatial signal observed in the activity of FEF neurons is likely computed from spatially nonselective inputs at the level of FEF and is one of the sources of top-down spatial attention. 4.1 Overview The main goal of the thesis was to establish a link between spatial selection in FEF and spatial attention, and to explore the origin of spatial selection in the brain during covert visual search. In Chapter 2, we compared the timing of spatial selection in LFPs and spiking activity while the monkeys covertly searched for the task relevant target. We found that the timing of spatial selection in spiking activity preceded the timing of spatial selection in LFPs by approximately 30 milliseconds; we argued that this suggests that FEF is one of the sources of spatial selection in the brain. Though until this thesis, the relationship between LFPs and spiking responses has not been studied in FEF, it has been studied in other brain areas. For example, in inferotemporal (IT) cortex the time course of object selectivity in spiking activity and LFPs has been shown to be nearly simultaneous (Krieman et al., 2006). A study by Nielsen et al. (2006) showed that spikes and LFPs in IT exhibited learned object selectivity, and that the modulation of LFP responses, but not spiking activity, grew stronger from posterior to anterior IT. Because LFPs likely contain inputs from other brain areas, they concluded that the learned object selectivity was encoded first in posterior IT and then transmitted to anterior IT. In the experiments outlined in Chapter 2, we showed that 67 68 non-selective stimulus evoked responses in FEF are first detectable in LFPs and then in the spiking activity. We then showed that at the same recording sites, during the same recording sessions, spatial selectivity emerges first in the spiking activity and 30ms later in the LFPs. This approach allowed us to verify that 1) input from the visual cortex is strongly represented in LFPs and 2) to show that FEF is likely one of the cortical sources for spatial selectivity during covert visual search. The experiments in Chapter 3 show that spatial selectivity in FEF is related to behavioral measures of attention in a cued object identification task. Behaviorally, spatial attention is measured as a speeding of reaction time and an improvement in performance accuracy in visual processing tasks at the attended location (Pashler 1998). We showed that these behavioral measures are correlated to the variability in the magnitude of spatial selection in FEF. First, we reconfirmed the results of previous studies demonstrating that spatially selective activity corresponds to the location of spatial attention (e.g. Thompson et al., 2005, Zhou and Thompson, 2008). However, previous studies have only inferred a general relationship between attention and spatial selection. Instead of inferring the relationship between measures of attention and the overall levels of neuronal activity across different task conditions, we correlated neuronal activity to measures of attention on a trial by trial basis. We tested whether and when differences in neural activity correspond to differences in perception as indexed by the subjects’ behavioral report. We showed that the correlation between neural activity and the behavioral measures of perceptual performance occurred predominantly between 100 ms following the presentation of the target to be identified and the initiation of the behavioral report. The magnitude of cue related selectivity did not differ on correct and error trials within each trial condition. The findings presented in Chapter 3 therefore suggest that FEF spatial selectivity is helpful to visual processing in the period of time in which the processing related to target detection and identification is taking place in the visual system. Additionally, when there was no target related selection the monkeys reported that the target was absent. A larger magnitude of selection resulted in a report of target’s presence, though even a greater magnitude of selection was correlated with a correct identification of a target. These results suggest that there is a threshold of spatial selection that is required for object identification. The results presented in this thesis demonstrate that FEF is one of the cortical sources of spatial attention. 69 4.2 Future Directions One of the biggest mysteries in cognitive science is how the brain locates and identifies objects in a complex and ever-changing world. One alternative is that it relies on a low-level feature analysis to guide attention to the most probable location of a target and then commits resources to the identification of the objects at that location. Another alternative is that the brain gathers information about the possible identity of the objects in the visual scene and then guides attention to the location of the most behaviorally relevant object. The results of Chapter 2 and 3 suggest that FEF is one of the sources of spatial attention and that its activity plays a direct role in visual processing. While the results of this thesis add to our general understanding of cortical processing and the role of FEF activity in the CNS, we did not directly record the activity of neurons in a brain area that is involved in processing the features of the target (e.g. V2, MT, V4, IT, etc) to directly ascertain the effect of FEF activity on visual processing. To (1) directly assess the relationship between FEF activity and visual processing and (2) to test if spatial selection precedes object identification, we are currently simultaneously recording single neurons in FEF and inferior temporal cortex (IT), an area crucial for object identification. Thus far, our findings are consistent with the results of Chapter 3 and models of attention (Hamker, 2005) that propose that spatial selection is required to bind features of an object to create a representation of the object's identity (recognition). Many areas in the brain exhibit spatially selective properties (e.g. lateral intraparietal area (LIP), 7a, superior colliculus (SC), and dorsolateral prefrontal cortex (dlPFC)). What are the different contributions these brain regions make to our ability to attend? Schaffer and Moore (2007) electrically stimulated FEF in an experiment that tested the contribution of FEF activity to eye movement preparation and to the enhancement of saliency of visual stimuli. They showed that artificial FEF activation influences perception of a stimulus and not the eye movement plan to the location of that same stimulus. A study by Buschman and Miller (2007) suggests that FEF is more involved in top-down (or cognitively driven) spatial attention and LIP is more involved in bottom-up (or stimulus driven) attention. In line with the hypothesis that FEF is a source of top-down spatial selection, Hanes and Wurtz (2001) showed that information flows from FEF to SC and from SC to the pre-motor neurons in the brainstem (Hanes and Wurtz, 2001) and that pre-saccadic spatial selection in FEF does not originate from the SC (Berman, et al., 2009). Simultaneous recording of spikes and LFPs in LIP, SC, and FEF would help to quantify the time course of spatial 70 selection across multiple brain regions and therefore shed light on the complex multi-area interactions that take place during spatial selection of targets for eye movements and visual processing. One possibility is that in a top-down attention task, spatial selection in LIP (or SC) would first be detectable in LFPs rather than spikes, while in FEF spatial selection would be first detectable in spikes and later in the LFPs, as reported in Chapter 2 of this thesis. Such a finding would strengthen the hypothesis that FEF is one of the sources of spatial selection in the brain. But regardless of the outcome, the proposed experiment would add to our understanding of the neural mechanisms of spatial selection. In general, a combination of methods such as direct manipulation (e.g. electrical and optical stimulation, inactivation), optical imaging, neural recording, and fMRI will be required to unravel the complex circuitry that is responsible for the primate’s ability to attend to stimuli and select targets for visual processing. Bibliography Allport A (1987) Selection for action: Some behavioral and neurophysiological considerations of attention and action. In: Perspectives on Perception and Action (Heuer H, Sanders AF, eds), pp 395-419. Hillsdale, NJ: Lawrence Erlbaum Associates. Armstrong KM, Moore T (2007) Rapid enhancement of visual cortical response discriminability by microstimulation of the frontal eye field. Proc Natl Acad Sci U S A 104:9499-9504. Armstrong KM, Fitzgerald JK, Moore T (2006) Changes in visual receptive fields with microstimulation of frontal cortex. Neuron 50:791-798. Awh E, Armstrong KM, Moore T (2006) Visual and oculomotor selection: links, causes and implications for spatial attention. Trends Cogn Sci 10:124-130. Beauchamp MS, Petit L, Ellmore TM, Ingeholm J, Haxby JV (2001) A parametric fMRI study of overt and covert shifts of visuospatial attention. Neuroimage 14:310-321. Bichot NP, Rossi AF, Desimone R (2005) Parallel and serial neural mechanisms for visual search in macaque area V4. Science 308:529-534. Bisley JW, Krishna BS, Goldberg ME (2004) A rapid and precise on-response in posterior parietal cortex. Journal of Neuroscience 24:1833-1838. Boudreau CE, Williford TH, Maunsell JH (2006) Effects of task difficulty and target likelihood in area V4 of macaque monkeys. J Neurophysiol 96:2377-2387. Bressler SL, Tang W, Sylvester CM, Shulman GL, Corbetta M (2008) Top-down control of human visual cortex by frontal and parietal cortex in anticipatory visual spatial attention. J Neurosci 28:1005610061. Bruce CJ, Goldberg ME (1985) Primate frontal eye fields: I. Single neurons discharging before saccades. J Neurophysiol 53:603-635. Buschman TJ, Miller EK (2007) Top-down versus bottom-up control of attention in the prefrontal and posterior parietal cortices. Science 315:1860-1862. Chelazzi L, Duncan J, Miller EK, Desimone R (1998) Responses of neurons in inferior temporal cortex during memory-guided visual search. J Neurophysiol 80:2918-2940. Chen CM, Lakatos P, Shah AS, Mehta AD, Givre SJ, Javitt DC, Schroeder CE (2007) Functional Anatomy and Interaction of Fast and Slow Visual Pathways in Macaque Monkeys. Cereb Cortex 17:15611569. Chun MM, Wolfe JM (1996) Just say no: how are visual searches terminated when there is no target present? Cognit Psychol 30:39-78. Constantinidis C, Steinmetz MA (2005) Posterior parietal cortex automatically encodes the location of salient stimuli. J Neurosci 25:233-238. Corbetta M, Shulman GL (2002) Control of goal-directed and stimulus-driven attention in the brain. Nat Rev Neurosci 3:201-215. Cruikshank SJ, Rose HJ, Metherate R (2002) Auditory thalamocortical synaptic transmission in vitro. J Neurophysiol 87:361-384. Deubel H, Schneider WX (1996) Saccade target selection and object recognition: evidence for a common attentional mechanism. Vision Res 36:1827-1837. Egeth HE, Yantis S (1997) Visual attention: control, representation, and time course. Annu Rev Psychol 48:269-297. Ekstrom LB, Roelfsema PR, Arsenault JT, Bonmassar G, Vanduffel W (2008) Bottom-up dependent gating of frontal signals in early visual cortex. Science 321:414-417. Evans KK, Treisman A (2005) Perception of objects in natural scenes: is it really attention free? J Exp Psychol Hum Percept Perform 31:1476-1492. Fries P, Reynolds JH, Rorie AE, Desimone R (2001) Modulation of oscillatory neuronal synchronization by selective visual attention. Science 291:1560-1563. Goldberg ME, Segraves MA (1989) The visual and frontal cortices. In: The Neurobiology of Saccadic Eye Movements, Reviews of Oculomotor Research, Vol. III (Wurtz RH, Goldberg ME, eds), pp 283313. Amsterdam: Elsevier. 71 72 Green DM, Swets JA (1966) Signal Detection Theory and Psychophysics. New York: Wiley. Grosbras MH, Paus T (2002) Transcranial magnetic stimulation of the human frontal eye field: Effects on visual perception and attention. Journal of Cognitive Neuroscience 14:1109-1120. Hamker FH (2005) The reentry hypothesis: the putative interaction of the frontal eye field, ventrolateral prefrontal cortex, and areas V4, IT for attention and eye movement. Cereb Cortex 15:431-447. Hanes DP, Patterson WF, Schall JD (1998) Role of frontal eye fields in countermanding saccades: Visual, movement, and fixation activity. Journal of Neurophysiology 79:817-834. Hays AV, Richmond BJ, Optican LM (1982) A UNIX-based multiple process system for real-time data acquisition and control. WESCON Conference Proceedings 2:1-10. Hillyard SA, Anllo-Vento L (1998) Event-related brain potentials in the study of visual selective attention. Proc Natl Acad Sci U S A 95:781-787. Hoffman JE, Subramaniam B (1995) The role of visual attention in saccadic eye movements. Percept Psychophys 57:787-795. Ipata AE, Gee AL, Goldberg ME, Bisley JW (2006) Activity in the lateral intraparietal area predicts the goal and latency of saccades in a free-viewing visual search task. J Neurosci 26:3656-3661. Jouve B, Rosenstiehl P, Imbert M (1998) A mathematical approach to the connectivity between the cortical visual areas of the macaque monkey. Cereb Cortex 8:28-39. Juergens E, Guettler A, Eckhorn R (1999) Visual stimulation elicits locked and induced gamma oscillations in monkey intracortical- and EEG-potentials, but not in human EEG. Exp Brain Res 129:247-259. Kaur S, Lazar R, Metherate R (2004) Intracortical pathways determine breadth of subthreshold frequency receptive fields in primary auditory cortex. J Neurophysiol 91:2551-2567. Kelley TA, Serences JT, Giesbrecht B, Yantis S (2008) Cortical mechanisms for shifting and holding visuospatial attention. Cereb Cortex 18:114-125. Kincade JM, Abrams RA, Astafiev SV, Shulman GL, Corbetta M (2005) An event-related functional magnetic resonance imaging study of voluntary and stimulus-driven orienting of attention. J Neurosci 25:4593-4604. Kodaka Y, Mikami A, Kubota K (1997) Neuronal activity in the frontal eye field of the monkey is modulated while attention is focused on to a stimulus in the peripheral visual field, irrespective of eye movement. Neurosci Res 28:291-298. Kowler E, Anderson E, Dosher B, Blaser E (1995) The role of attention in the programming of saccades. Vision Res 35:1897-1916. Kreiman G, Hung CP, Kraskov A, Quiroga RQ, Poggio T, DiCarlo JJ (2006) Object selectivity of local field potentials and spikes in the macaque inferior temporal cortex. Neuron 49:433-445. Lavie N, Tsal Y (1994) Perceptual load as a major determinant of the locus of selection in visual attention. Percept Psychophys 56:183-197. Liu J, Newsome WT (2006) Local field potential in cortical area MT: stimulus tuning and behavioral correlations. J Neurosci 26:7779-7790. Logothetis NK, Sheinberg DL (1996) Visual object recognition. Annu Rev Neurosci 19:577-621. Logothetis NK, Wandell BA (2004) Interpreting the BOLD signal. Annu Rev Physiol 66:735-769. Logothetis NK, Kayser C, Oeltermann A (2007) In vivo measurement of cortical impedance spectrum in monkeys: implications for signal propagation. Neuron 55:809-823. Logothetis NK, Pauls J, Augath M, Trinath T, Oeltermann A (2001) Neurophysiological investigation of the basis of the fMRI signal. Nature 412:150-157. Luck SJ, Hillyard SA (1994) Electrophysiological correlates of feature analysis during visual search. Psychophysiology 31:291-308. Luck SJ, Woodman GF, Vogel EK (2000) Event-related potential studies of attention. Trends Cogn Sci 4:432-440. Mardia KV, Jupp PE (2000) Directional Statistics. Chichester: John Wiley & Sons Ltd. Maunsell JH, Cook EP (2002) The role of attention in visual processing. Philos Trans R Soc Lond B Biol Sci 357:1063-1072. McPeek RM, Keller EL (2002) Saccade target selection in the superior colliculus during a visual search task. Journal of Neurophysiology 88:2019-2034. Mitzdorf U (1985) Current source-density method and application in cat cerebral cortex: investigation of evoked potentials and EEG phenomena. Physiol Rev 65:37-100. Mitzdorf U (1987) Properties of the evoked potential generators: current source-density analysis of visually evoked potentials in the cat cortex. Int J Neurosci 33:33-59. 73 Monosov IE, Trageser JC, Thompson KG (2006) A comparison of the spatial tuning and time course of attention-related signals in the primate frontal eye field. Soc Neurosci Abstr 32:439.415. Monosov IE, Trageser JC, Thompson KG (2008) Measurements of simultaneously recorded spiking activity and local field potentials suggest that spatial selection emerges in the frontal eye field. Neuron 57(4):614-25. Moore T, Fallah M (2001) Control of eye movements and spatial attention. Proc Natl Acad Sci U S A 98:1273-1276. Moore T, Armstrong KM (2003) Selective gating of visual signals by microstimulation of frontal cortex. Nature 421:370-373. Moore T, Fallah M (2004) Microstimulation of the frontal eye field and its effects on covert spatial attention. Journal of Neurophysiology 91:152-162. Morishima Y, Akaishi R, Yamada Y, Okuda J, Toma K, Sakai K (2009) Task-specific signal transmission from prefrontal cortex in visual selective attention. Nat Neurosci 12:85-91. Muggleton NG, Juan CH, Cowey A, Walsh V (2003) Human frontal eye fields and visual search. Journal of Neurophysiology 89:3340-3343. Murthy A, Thompson KG, Schall JD (2001) Dynamic dissociation of visual selection from saccade programming in frontal eye field. J Neurophysiol 86:2634-2637. Nelson MJ, Pouget P, Schall JD (2006) Effect of electrode impedence on measurement of local field potentials in the supplementary eye field. Soc for Neurosci Abstr 32:835.814. Nielsen KJ, Logothetis NK, Rainer G (2006) Dissociation between local field potentials and spiking activity in macaque inferior temporal cortex reveals diagnosticity-based encoding of complex objects. J Neurosci 26:9639-9645. Pashler HE (1998) The Psychology of Attention. Cambridge, MA: MIT Press. Peng X, Sereno ME, Silva AK, Lehky SR, Sereno AB (2008) Shape selectivity in primate frontal eye field. J Neurophysiol 100:796-814. Pesaran B, Pezaris JS, Sahani M, Mitra PP, Andersen RA (2002) Temporal structure in neuronal activity during working memory in macaque parietal cortex. Nature Neuroscience 5:805-811. Pessoa L, Kastner S, Ungerleider LG (2003) Neuroimaging studies of attention: From modulation of sensory processing to top-down control. Journal of Neuroscience 23:3990-3998. Posner MI (1980) Orienting of attention. Q J Exp Psychol 32:3-25. Pouget P, Emeric EE, Stuphorn V, Reis K, Schall JD (2005) Chronometry of visual responses in frontal eye field, supplementary eye field, and anterior cingulate cortex. J Neurophysiol 94:2086-2092. Rensink RA (2002) Change detection. Annual Review of Psychology 53:245-277. Reynolds JH, Desimone R (2003) Interacting roles of attention and visual salience in V4. Neuron 37(5):853-63. Reynolds JH, Pasternak T, Desimone R (2000) Attention increases sensitivity of V4 neurons.Neuron 26(3):703-14. Reynolds JH, Chelazzi L (2004). Attentional modulation of visual processing. Annu Rev Neurosci 27:611-47. Rizzolatti G, Riggio L, Dascola I, Umilta C (1987) Reorienting attention across the horizontal and vertical meridians: evidence in favor of a premotor theory of attention. Neuropsychologia 25:31-40. Sato T, Murthy A, Thompson KG, Schall JD (2001) Search efficiency but not response interference affects visual selection in frontal eye field. Neuron 30:583-591. Sato TR, Schall JD (2003) Effects of stimulus-response compatibility on neural selection in frontal eye field. Neuron 38:637-648. Schafer RJ, Moore T (2007) Attention governs action in the primate frontal eye field. Neuron 56:541-551. Schall JD (1997) Visuomotor areas of the frontal lobe. In: Cerebral Cortex (Rockland K, Kaas JH, Peters A, eds), pp 527-638. New York: Plenum Press. Schall JD, Thompson KG (1999) Neural selection and control of visually guided eye movements. Ann Rev Neurosci 22:241-259. Schall JD, Pare M, Woodman GF (2007) Comment on "Top-down versus bottom-up control of attention in the prefrontal and posterior parietal cortices". Science 318:44; author reply 44. Schall JD, Morel A, King DJ, Bullier J (1995a) Topography of visual cortex connections with frontal eye field in macaque: Convergence and segregation of processing streams. J Neurosci 15:4464-4487. Schall JD, Hanes DP, Thompson KG, King DJ (1995b) Saccade target selection in frontal eye field of macaque. I. Visual and premovement activation. J Neurosci 15:6905-6918. 74 Schall JD, Sato TR, Thompson KG, Vaughn AA, Juan CH (2004) Effects of search efficiency on surround suppression during visual selection in frontal eye field. J Neurophysiol 91:2765-2769. Schmolesky MT, Wang Y, Hanes DP, Thompson KG, Leutgeb S, Schall JD, Leventhal AG (1998) Signal timing across the macaque visual system. J Neurophysiol 79:3272-3278. Schroeder CE, Mehta AD, Givre SJ (1998) A spatiotemporal profile of visual system activation revealed by current source density analysis in the awake macaque. Cereb Cortex 8:575-592. Serences JT, Yantis S (2006) Selective visual attention and perceptual coherence. Trends Cogn Sci 10:3845. Serences JT, Yantis S (2007) Spatially Selective Representations of Voluntary and Stimulus-Driven Attentional Priority in Human Occipital, Parietal, and Frontal Cortex. Cereb Cortex 17:284-293. Sheinberg DL, Logothetis NK (2001) Noticing familiar objects in real world scenes: the role of temporal cortical neurons in natural vision. J Neurosci 21:1340-1350. Silvanto J, Lavie N, Walsh V (2006) Stimulation of the human frontal eye fields modulates sensitivity of extrastriate visual cortex. J Neurophysiol 96:941-945. Smith DT, Jackson SR, Rorden C (2005) Transcranial magnetic stimulation of the left human frontal eye fields eliminates the cost of invalid endogenous cues. Neuropsychologia 43:1288-1296. Spitzer H, Richmond BJ (1991) Task difficulty: ignoring, attending to, and discriminating a visual stimulus yield progressively more activity in inferior temporal neurons. Exp Brain Res 83:340-348. Spitzer H, Desimone R, Moran J (1988) Increased attention enhances both behavioral and neuronal performance. Science 240:338-340. Tanaka K (1997) Mechanisms of visual object recognition: monkey and human studies. Curr Opin Neurobiol 7:523-529. Taylor PC, Nobre AC, Rushworth MF (2007) FEF TMS affects visual cortical activity. Cereb Cortex 17:391-399. Tehovnik EJ, Sommer MA, Chou IH, Slocum WM, Schiller PH (2000) Eye fields in the frontal lobes of primates. Brain Res Brain Res Rev 32:413-448. Thomas NW, Pare M (2007) Temporal Processing of Saccade Targets in Parietal Cortex Area LIP During Visual Search. J Neurophysiol 97:942-947. Thompson KG, Bichot NP (2005) A visual salience map in the primate frontal eye field. Prog Brain Res 147:251-262. Thompson KG, Biscoe KL, Sato TR (2005a) Neuronal basis of covert spatial attention in the frontal eye field. J Neurosci 25:9479-9487. Thompson KG, Bichot NP, Sato TR (2005b) Frontal eye field activity before visual search errors reveals the integration of bottom-up and top-down salience. J Neurophysiol 93:337-351. Thompson KG, Hanes DP, Bichot NP, Schall JD (1996) Perceptual and motor processing stages identified in the activity of macaque frontal eye field neurons during visual search. J Neurophysiol 76:40404055. Treisman A (1988) Features and objects: the fourteenth Bartlett memorial lecture. Q J Exp Psychol A 40:201-237. Treisman A, Gormican S (1988) Feature analysis in early vision: evidence from search asymmetries. Psychol Rev 95:15-48. Turatto M, Galfano G (2000) Color, form and luminance capture attention in visual search. Vision Res 40:1639-1643. Urbach D, Spitzer H (1995) Attentional effort modulated by task difficulty. Vision Res 35:2169-2177. Vezoli J, Falchier A, Jouve B, Knoblauch K, Young M, Kennedy H (2004) Quantitative analysis of connectivity in the visual cortex: extracting function from structure. Neuroscientist 10:476-482. Wardak C, Ibos G, Duhamel JR, Olivier E (2006) Contribution of the monkey frontal eye field to covert visual attention. J Neurosci 26:4228-4235. Woodman GF, Kang MS, Rossi AF, Schall JD (2007) Nonhuman primate event-related potentials indexing covert shifts of attention. Proc Natl Acad Sci U S A 104:15111-15116. Zenon A, Ben Hamed S, Duhamel JR, Olivier E (2008) Spatial and temporal dynamics of attentional guidance during inefficient visual search. PLoS ONE 3:e2219.