Introduction

Memory is arguably a central ability that enables humans and other species to maintain a sense of identity in that it connects the present with the past, and therefore permits us to anticipate the future. Humans are distinguishable from other species by maintaining traces of memory beyond a current generation in history and folklore. While it has been a topic of investigation since the dawn of psychology, more recently conceptual advances have been made based on neuroscience research in animals and humans. A comprehensive review of memory is beyond the scope of a single paper, and the interested reader can find several recent publications that delve into specific topics. Here we will only outline the main historical currents that are at the basis of our scientific understanding of memory, and proceed to describe major distinctions made in the present memory literature. These formulations have benefitted from incorporating neuroscience, initially by using clinical-pathological correlations and more recently with neuroimaging. We will conclude by focusing on memory disturbances associated with psychosis, and describe some of our work and others' on memory functioning in healthy people and in psychosis spectrum disorders. Importantly, sex differences in memory are underemphasized in the cognitive psychology literature, but because of their prominence in psychosis they will be highlighted. Furthermore, as schizophrenia spectrum disorders are increasingly recognized as developmental in nature, we will highlight developmental features.

The scientific roots of memory research

Scientific work on memory was sparked by the seminal studies of the Russian physiologist and Nobel prize winner Ivan Pavlov (1849-1936), the German psychologist Hermann Ebbinghaus (1850-1909) and the American psychologist Burrhus Frederic “B. F.” Skinner (19041990). Pavlov demonstrated that a reflexive response can be produced in the absence of its natural (unconditioned) trigger if the latter has been previously paired with another (conditioned) trigger. Subsequent work in animals and humans has used this “classical conditioning” paradigm to gain insights into acquisition of knowledge, generalization, and discrimination principles. Principles of reinforcement have been investigated through examining the acquisition and retention phases with varying degrees of delay between presentation of the conditioned and unconditioned stimuli. Physiological measures ranged from skin conductance levels to heart rate, pupillary changes, and other indicators of autonomic arousal, and conditioned stimuli have ranged from aversive to pleasurable. This work on classical conditioning has informed us systematically about processes leading to habit formation, and continues to shed light on memory processes to this day.

Skinner proposed an alternative paradigm, which he termed “operant conditioning.” He distinguished two kinds of behavior: respondent and operant. The former is under direct control of the stimulus, whereas the relation of operant behavior to stimulation is more nuanced. Some behavior appears to be emitted or spontaneous, such as grabbing a leaf off of a bush that one passes by. The behavior may have been triggered by the sight of the leaf, but the person didn't have to grab it. Similarly, when I answer my ringing or vibrating cellphone, compelling as the stimulus can be, it is a discriminating signal, letting me know that someone is trying to reach me. My response to it exemplifies operant and not respondent behavior. Operant behavior is much more varied and less predictable, it operates on the environment to produce some effects (hence it is sometimes also referred to as “instrumental” behavior). The ensuing paradigms differed from classical conditioning studies in multiple ways. To produce operant conditioning in the laboratory, a hungry animal would be placed in a box equipped with a protruding bar (“Skinner box”). A light above the bar can be controlled by the experimenter.

The animal would be ambulating in the box and may occasionally press the bar. The rate of occasional pressing is used to calculate operant level of bar pressing. One can then begin to reward the animal for bar pressing and see that measure climb as an index of learning. Placing the animal back in that box after a delay and examining how quickly it relearns to press the bar is an index of memory. The light can be used for discrimination learning (eg, reward bar presses only when the light is on).

Research using the operant conditioning paradigm has unraveled many principles of learning and memory. It has related the rate of learning a wide range of operant behaviors across animals and humans, discovering markedly analogous laws of learning and memory that operate across species. For example, reinforcement strength, frequency, and predictability have similar effects in worms, pigeons, rodents, and humans. Even some “paradoxical” effects in animals have found immediate translation into human research on learning and memory. For example, in 1908 Yerkes and Dodson published a study in mice, where they related the strength of a negatively reinforcing stimulus (electric shock) to the speed of avoidance learning.1 They found, as expected, that mice will learn more quickly to avoid moderately strong shocks than mild shocks. Indeed, learning to avoid a moderate shock took between one and two trials. Counterintuitively, however, the strongest shock did not further improve speed of learning but instead slowed it down. This inverted-U relationship between intensity of reinforcement stimulus and what Yerkes and Dodson called “rapidity of habit formation” motivated multiple studies in humans, establishing an inverted-U relationship between anxiety and rate of learning and memory.2-5

This work has led to many insights on the nature of learning and memory. For example, it was discovered quite early that sensorimotor skills are learned differently from more complex cognitive functions (see Bell, 1950),6 and quantitative models have been proposed that integrate classical and operant conditioning parameters to account for learning and memory (eg, the Hull-Spence theoryrelating excitatory potential to drive and habit strength and the Estes stimulus sampling theory).

Ebbinghaus and his followers pioneered the experimental study of human memory, and his own early studies have already established learning and forgetting curves and spacing effects. Among his seminal contributions to setting research paradigms in memory, he introduced what would later be called “nonsense syllables”; today they are often termed consonant-vowelconsonant (CVC) trigrams. This methodology helped identify language-free learning characteristics (although it was later found that people do assign meaning to nonsense syllables). The work carried out in the tradition of Ebbinghaus (see 7) has helped shape our understanding of memory, especially more complex verbal and associative memory. For example, it has established the distinction between recognition and recall8 indices of memory, the former relating to the ability to judge whether the present stimulus (or experience) is “old” or “new” and the latter to the ability to recount the details of the stimulus. They investigated the veracity and distortions of memory traces and uncovered interference effects such as retroactive and proactive inhibition of memory associated with interfering events.

Dimensions of memory

The study of memory has been characterized by evolving conceptions and methodologies in which competing distinctions have been emphasized over the years. The initial emphasis has been on associative learning and memory and the competing and, eventually, complementary paradigms of classical and operant conditioning have yielded information on the conditions and time course of the ability of organisms to learn new associations and retrieve acquired information.

These initial efforts were developed without much attention to the neurological or neuropsychological literature on memory. The young and ambitious science of behavior was firmly convinced that behavioral science could be erected without reference to its physical organ, the brain. Physiological measures were obtained as proxies for arousal, but theories did not consider neuroscience evidence as being of much relevance to the theoretical articulation of memory. This situation has changed with advances in clinical and basic neuroscience. Clinical neuroscience has presented a series of highly informative cases and increased sophistication in documenting clinical-pathological correlations in routine cases. For example, the case of HM, who underwent bi-hippocampal dissection and lost the ability to learn new information while retaining memories acquired prior to surgery, brought into sharp focus the role of the hippocampus in new learning.9, 10 Studies of patient populations with memory deficits related to seizure disorders, dementing disorders, and substance-use disorders have further identified distinct networks related to aspects of memory. At the same time, the advent of neuroimaging has opened up new avenues for probing memory processes in healthy and clinical populations. Functional neuroimaging studies in healthy research participants could be safely conducted with functional magnetic resonance imaging (fMRI), and this methodology was applied extensively to probe aspects of memory processing.

It is beyond our scope even to attempt a summary of this expanding literature. Instead, we will present some of the main dimensions of memory that this work has highlighted as having potential for understanding neural mechanisms.

Procedural memory

Both clinical and functional neuroimaging studies have supported distinct neural systems involved in “procedural” learning and memory, which differ from those regulating “declarative” memory. Procedural memory refers to retention of skills acquired by repeated practice, while declarative memory refers to knowledge of content of previously experienced situations. As established by the case of HM and confirmed in functional neuroimaging studies, declarative memory involves the hippocampus and associated temporal and frontal cortices. Procedural memory is different, as has been established by psychologists long ago, in multiple respects. It is mostly unconscious and its learning depends more on rote repetition than on insight. Furthermore, once learned, it is retained after the passage of years. Examples are riding a bicycle, skiing, driving a car, and swimming. If trained in these skills at an early age, one can retain these skills and, while rusty after prolonged lack of practice, would show evidence of prior learning by rapid return to earlier skill levels. This learning takes place in regions outside the hippocampus, predominantly in cerebellum, basal ganglia, and sensorimotor cortex.

Declarative memory

Declarative memory refers to specific autobiographical events that can be recounted by an individual. For example, a memory of your birthday when you received a bicycle as a gift. One is able to place the event in time and context. Further distinctions have been made within declarative memory, between episodic and semantic memory. Semantic memory is factual, whereas episodic memory contains representations of past experience that include sensory, perceptual, conceptual, and affective features.11 Episodic memory can fade rapidly and is subject to distortions.12 All forms of declarative memory appear to recruit medial temporal structures. However, there is evidence that prefrontal regions are involved in the encoding of new episodic memories.

More recently an important distinction within episodic memory has been proposed—between recollection and familiarity.13 Recollection reflects the retrieval of information, whereas familiarity reflects the passing of a threshold where one recognizes a stimulus or event as having been experienced. There is much current research attempting to identify distinct neural systems related to recollection and familiarity.14

Note that another domain, working memory, is often discussed in functional neuroimaging studies. Although that construct described by Baddeley15 encompasses mnemonic processes as it relates to events immediately preceding the present, its emphasis is on the rapid replacement of information by new input entering our mental scratch pad. Therefore, working memory properly belongs to the executive control domain.

Sex differences in memory

As a first step toward profiling the neurocognitive deficits in schizophrenia we needed to establish a normative database of individuals who received the entire battery. While individual tests in the battery have each been standardized on differing normative samples, a rigorous characterization of any clinical population requires that both patients and controls be given the entire instrument under the same test configuration. When we compared the profile of men and women, we noted similar performance in the executive domain of abstraction and mental flexibility and attention, but in verbal memory females outperformed males by a substantial margin (Figure 1).

Figure 1.
Figure 1. Sex differences. A. Sex differences in neurocognitive profile. ABF, abstraction and mental flexibility; ATT, attention; VMEM, verbal memory; SMEM, spatial memory; LAN, language reasoning; SPA, spatial processing; SEN, sensory; MOT, motor speed. B. Sex differences on the California Verbal Learning Test (CVLT). The CVLT presents 16 words and the participant is asked to repeat them in any order during the recall phase. This is repeated for 5 trials. Then a new list of 16 words is presented, followed by immediate and delayed recall with and without cuing. Recognition is determined at the end by showing words that were and were not included in the first list, and the participant is asked to classify them into old and new.

The advantage of females in verbal memory was clearly evident at the rate at which they learned a new word list. They remembered more words after the first exposure to the list, and this advantage was maintained after repeated exposures and throughout efforts to elicit recollection. Notably, item recognition at the conclusion of testing was nearly identical for males and females. The males correctly recognized the words that they were exposed to when asked to pick them out from new words. However, they had less access to them, compared with females, when attempting to recollect the word list. In contrast to the better performance on word memory, females did not differ from males in spatial memory. This could relate to their poorer performance in spatial tasks. For example, as can be seen in Figure 1 , their performance on a spatial processing test was below that of males.

The advent of functional neuroimaging has enabled a systematic investigation of neural substrates for these sex differences in verbal memory. Initial studies of regional cerebral blood flow have revealed that, in addition to better verbal memory, women have higher rates of resting regional cerebral blood flow. We examined whether there are sex differences in the relationship between verbal episodic memory and resting cerebral blood flow.16 Twenty-eight healthy right-handed participants (14 male, 14 female) underwent a neuropsychological evaluation and a positron emission tomography (PET) [15]0-water study. Immediate and delayed recall was measured on the logical memory subtest of the Wechsler Memory Scale - Revised, and on the California Verbal Learning Test. Resting cerebral blood flow (mL/100 g/min) was calculated for four frontal, four temporal, and four limbic regions of interest. Women had better immediate recall on both tasks. Sex differences in cerebral blood flow were found for temporal lobe regions. Women had greater bilateral blood flow in a mid-temporal brain region. There were also sex differences in cerebral blood flow correlations with performance. Women showed positive correlations of verbal memory performance with cerebral blood flow laterality in the temporal pole. Greater relative cerebral blood flow in the left temporal pole was associated with better Wechsler Memory Scale immediate and delayed recall in women only. These results suggest that trait differences in temporal pole brain-behavior relationships may relate to sex differences in verbal episodic memory.

Studies with cerebral blood flow measures during the application of neurobehavioral probes have proliferated with the development of procedures for measuring blood flow changes using fMRI.17 These efforts have resulted in an accumulation of tasks that showed reliable activation pattern in specific brain systems. We have assembled such tasks and adapted them to be used for measuring individual differences in performance. The computerized format of these tests made them more efficient and accurate than the traditional neuropsychological batteries. The Penn Computerized Neurocognitive Battery (CNB) was developed and validated in healthy individuals.18,19 It currently measures the domains of executive functioning, episodic memory, complex cognition, social cognition, and sensorimotor speed. Episodic memory is measured for words, faces, and shapes. To validate the original CNB, we administered it along with a traditional battery to a sample of 92 healthy individuals (44 men, 48 women) in a counterbalanced order. Both approaches showed a significant “sex-typical” gradient, with women outperforming men in verbal memory relative to spatial tasks. Both methods also yielded similar profiles of sex differences, with the additional computerized measure of face memory showing better performance in women.18

The CNB proved immensely useful in collecting largescale data because the computerized format and the simplified response requirements necessitated by the neuroimaging environment resulted in ease and efficiency of administration. New tests were added to the domains and the battery was standardized on large samples and applied in multisite clinical and genomic studies. The expanded version of the CNB included tests of episodic memory for words, faces, and shapes. The results in a normative sample replicated the sex differences in memory. As can be seen in Figure 2, females performed more accurately for word and face memory, but did not differ from males in shape memory. Notably, other sex differences including better performance for males on spatial and motor speed measures were replicated, buttressing the construct validity of the CNB.

Figure 2.
Figure 2. Mean (±SEM) of men (light blue bars) and women (dark blue bars) on the tests included in the battery. Note that no accuracy measures are available for the sensorimotor test because no errors were made and for the motor speed test because no errors were possible. *, P<.01 ; **, P<.001 . Dark blue stars indicate that women had better performance, while light blue stars indicate better performance in men. Note that accuracy and speed provide complementary information, and while performance is similar overall for men and women, some well-known sex differences are replicated (better performance for women in memory and affect recognition, and for men in spatial and motor domains), and some novel ones revealed (better performance for men on CPT and working memory). Reproduced from ref 19: Gur RC, Richard J, Hughett P, et al. A cognitive neuroscience-based computerized battery for efficient measurement of individual differences: standardization and initial construct validation. J Neurosci Methods. 2010;187:254-462. Copyright© Elsevier 2010

Memory across the lifespan

Memory decline with aging has been a focus of investigation because of the memory loss in dementing disorders. Considerable work has also been done on the development of episodic memory.12 It has been well established that memory abilities decline even with healthy aging and it is important to characterize the extent and the nature of this decline in order to establish a baseline against which effects of brain pathology can be detected. The CNB permits evaluation of age effects on memory compared with other neurocognitive domains. Furthermore, because of its computerized format it allows separate measures of accuracy and speed. As can be seen in Figure 3, within the age range of 18 to 84, older age was associated with poorer memory performance. The decline was evident both in accuracy and in speed (longer response times), although some modality-specific effects are noticeable. For example, for word memory accuracy is less affected than speed.

Figure 3.
Figure 3. Correlations of age with accuracy (black bars) and response time (RT; gray bars) indices of performance on the tests. Error bars indicate 95% confidence intervals based on 1 000 bootstraps. As seen, the effects of age are stronger for speed than for accuracy, and more pronounced for abstraction/flexibility and episodic memory than for attention and working memory. ABF, abstraction and mental flexibility; ATT, attention; WM, working memory; Imm, immediate; Del, delayed; LAN, language reasoning; SPA, spatial processing; SM, sensorimotor speed; MOT, motor speed; EMO, emotion processing Reproduced from ref 19: Gur RC, Richard J, Hughett P, et al. A cognitive neuroscience-based computerized battery for efficient measurement of individual differences: standardization and initial construct validation. J Neurosci Methods. 2010;187:254-462. Copyright © Elsevier 2010

A recent application of the CNB in the Philadelphia Neurodevelopmental Cohort study of youths aged 8 to 21 years permitted us to examine developmental age effects on episodic memory in the context of other domains.20 In this study we evaluated clinical phenotypic measures and assessed neurocognitive performance with the CNB in genotyped individuals. As can be seen in Figure 4, age-related increase in memory performance was more evident for speed than for accuracy. For verbal and spatial memory, accuracy changes were minimal between ages 8 to 21; only for face memory was there an effect size exceeding 1 standard deviation.

Figure 4.
Figure 4. (Opposite) Age-related increase in memory performance. A. Means (+ SE) of z-scores for accuracy (top panel) and speed (bottom panel) for females (dark blue bars) and males (light blue bars) across the sample on each behavioral domain. ABF, abstraction and mental flexibility; ATT, attention; WM, working memory; VME, verbal memory; FME, face memory; SME, spatial memory; LAN, language reasoning; NVR, nonverbal reasoning; SPA, spatial processing; EMI, emotion identification; EMD, emotion differentiation; AGD, age differentiation; SM, sensorimotor speed; MOT, motor speed. B. Means (+ SE) of z-scores for accuracy (top panel) and speed (bottom panel) for females (dark blue bars) and males (light blue bars) on episodic memory tasks. The results are shown by age groups and correlations with age are provided by sex (bold correlations are significant at P<.001). Reproduced from ref 20: Gur RC, Richard J, Calkins ME, et al. Age group and sex differences in performance on a computerized neurocognitive battery in children age 8-21. Neuropsychology. 2012;26:251-265. Copyright © American Psychological Association 2012

Memory in schizophrenia

Cognitive deficits in schizophrenia have been traditionally investigated by measuring specific abilities. While impairments were documented in multiple domains, their relative magnitude and their relations to brain systems were not established until neuropsychology and neuropsychiatry began to exert influence. In our first neuropsychological characterization of schizophrenia, we and the field were surprised that memory deficits had the largest effect sizes after controlling for relevant factors.21 Spatial and verbal memory and verbal learning showed effect sizes nearing 3 standard deviations below normal, compared with abstraction and mental flexibility that had an effect size approaching 1 SD (Figure 5). Subsequent work already performed in the neuropsychological framework that considers schizophrenia as a complex brain disorder replicated the prominence of memory deficit22,23 and further delineated its nature and characteristics. For example, a recent quantitative review confirmed the prevalence of recollection memory deficits in schizophrenia and its important role in functional outcome. This review examined, in addition, the distinction between recollection and familiarity. Contrary to earlier reports that only recollection is impaired in schizophrenia, the authors found evidence that both recollection and familiarity deficits can be documented. However, the effect sizes are smaller for familiarity than for recollection deficits, suggesting that the former uses a compensatory ability while the latter could serve as a treatment target. These findings implicate multifocal medial temporal lobe and prefrontal cortex dysfunction.24

Figure 5.
Figure 5. Neuropsychological profile (±SEM) for patients with schizophrenia (n=36) relative to controls (n=36) whose performance is set to zero (±1 SD). Functions are abstraction (ABS), verbal cognitive (VBL), spatial organization (SPT), semantic memory (SME), visual memory (VME), verbal learning (LRN), language (LNG), visual-motor processing and attention (VSM), auditory processing and attention (AUD), and motor speed and sequencing (MOT). Reproduced from ref 21: Saykin AJ, Gur RC, Gur RE, et al. Neuropsychological function in schizophrenia. Selective impairment in memory and learning. Arch Gen Psychiatry. 1991;48:618-624. Copyright © American Medical Association 1991

Such studies, and others like them, have delineated the cognitive deficits in schizophrenia and led to the recognition that these deficits are core features of schizophrenia spectrum disorders.25,26 Thus, an extensive literature has well documented the deficits associated with the disorder and has evolved with advances in cognitive neuroscience and in functional neuroimaging. While we focus here on episodic memory, it is important to emphasize that diffuse deficits have been noted in schizophrenia across neurocognitive domains.

Among the domains investigated in schizophrenia, social cognition has been a relatively more recent addition that has attained considerable interest, and has been related to negative symptoms and poor functional outcome.27,28 Studies have shown deficits in the ability to identify facial and vocal expressions of emotions, and these deficits have been related to abnormalities in brain activation in the temporo-limbic network. For example, abnormally increased activation in the amygdala to the appearance of a face expressing threat-related emotion, such as anger or fear, was associated with greater likelihood of performance error and more severe symptoms of flat affect.29-30 To examine the relationship between flat affect and neurocognitive profile we compared patients with flat affect with those with normal affect, based on a standard clinical rating scale. The results indicated that patients with flat affect indeed performed more poorly on facial emotion identification tests. However, they did not differ from their counterparts without flat affect on any of the neurocognitive measures except for word memory Figure 6). This suggests that the memory and emotion processing abnormalities are linked, implicating the medial temporal structures such as hippocampus and amygdala.31

Figure 6.
Figure 6. Neurocognitive and emotion processing performance (z-score mean ± SEM for efficiency) in flat affect (FA) and non-flat affect (NFA) groups. ABF, abstraction/flexibility; ATT, attention; VME, verbal memory; SME, spatial memory; FME, facial memory; LAN, language; SPA, spatial; SM, sensor-motor; EMO, average emotion processing tests. Reproduced from ref 31: Gur RE, Kohler CG, Ragland JD, et al. Flat affect in schizophrenia: relation to emotion processing and neurocognitive measures. Schizophr Bull. 2006;32:279-287. Copyright © Oxford University Press 2006

Summary

Memory is an important capacity of humans and animals that operates along similar principles across species. Memory has been studied extensively by behavioral investigators and by neuroscientists, and there are sophisticated models accounting for its characteristics. The neuroscience of memory has benefited from the confluence of data obtained with animal investigations, clinical-pathological correlations, and, more recently, neuroimaging. Thus, memory offers a uniquely wellsuited construct for examining mechanistic processes giving rise to important behavioral phenomena. The investigation of neural substrates of memory has led to identifying aspects of memory that can be linked to distinguishable brain systems. Thus, declarative episodic memory is importantly hinged upon the operation of hippocampal-centered networks that involve frontal encoding strategies, whereas procedural learning does not require hippocampal integrity and relates instead to cerebellar and sensorimotor components of the supratentorial brain.

To understand the effects of neuropsychiatric disorders such as schizophrenia on memory it is important to note individual differences in memory that can be observed in healthy people. Among the most salient demographic effects are sex differences, with females having better verbal and face memory, and age. Children do not showmuch improvement in memory accuracy between the age of 8 and 21, and memory declines from adulthood to old age, especially for speed of recollection. Patients with schizophrenia show pronounced deficit in memory, compared with most other domains, and these deficits are strongly related to their functional outcome. In particular, patients with negative symptoms, especially flat affect, have more severe memory deficits and this is associated with poorer quality of life and functional adjustment. This condition is more prevalent in males with schizophrenia, and may relate to the greater prevalence and severity of negative symptoms. Given its centrality, memory should be a major target for intervention.