Tracing Recurrent Activity in Cognitive Elements (TRACE):
A Model of Temporal Dynamics in a Cell Assembly

Running head: Activity in Cognitive Elements

Introduction 4

The Cell Assembly: Neurophysiological and Psychological Perspectives 5

Building the TRACE Model 15

Simulating the TRACE Model 30

Conclusion 44

Keywords: Neural networks, connectionism, cell assembly, neural fatigue, perception, simulation, peak activation, consolidation

In 1949, Hebb presented a cautious yet revolutionary theory to account for the psychology and the physiology of perception as it was understood at the time. Central to Hebb's theory was the cell assembly construct. Despite the venerable age and admitted incompleteness of cell assembly theory, this construct provides a cognitive foundation for connectionism not possessed by other models. In a cognitive connectionism the cell assembly serves as a symbol-like unit of thought. Unlike other symbol systems (and most connectionist systems) the cell assembly is grounded in an explicit neural mechanism. Taken a step further, the cell assembly provides automatic mechanisms for learning and structure generation.

Despite the potential power of the cell assembly concept, the theory is, as Hebb himself emphasized repeatedly, incomplete. Further theoretical development is called for to harness the potential power of this deceptively simple concept. Over the years since the Organization of Behavior was written, it has became increasingly clear that several new theoretical components are needed. It has also become apparent that the resulting system of interacting constructs would necessarily be a reformulation rather than a simple extension of Hebb's original model.

The purpose of this paper is to present such a reformulation. The cell assembly provides the cognitive system with flexibility far beyond the simple activation of concepts. Instead of viewing the assembly as simply active or latent we see the activation of the assembly as coming in a series of phases. Each phase of activity serves a different purpose, giving the theory the power and flexibility to handle a wide range of psychological data.

This time course of activity in the cell assembly is due to the introduction of new control mechanisms to the basic model. Milner (1957) added one control mechanism, inhibition, critical to the theory. We postulate two additional control mechanisms, fatigue and short term connection strength, that not only are important functionally in controlling activity, but also serve the role of breaking the time course of activity into distinct phases.

After examining the cell assembly concept from physiological and psychological perspectives, we focus on what we see as a key conceptual issue, namely the potential roles of persisting activity in the cell assembly. We then turn our attention to the development of the model. TRACE is a mathematical model and computer simulation of a single Hebbian cell assembly. In keeping with our conceptual focus we have decided upon what might be called a "molar strategy." The cell assembly is represented in our simulation not as a network, but as a complex temporal process.

We then report on a series of simulation experiments in which the model is found to substantially agree with several theoretical predictions. The newly introduced factors not only serve their required functions as control mechanisms, but provide the cell assembly construct with additional information processing capabilities.

A cell assembly is a highly interconnected group of neurons that form a reverberatory circuit which can sustain activity. This reverberatory activity, through the action of the learning rule, is the mechanism by which cell assemblies are formed and strengthened. These two processes are mutually reinforcing: the learning rule operates through the action of reverberatory activity, and the operation of the learning rule provides for enhanced reverberatory activity in the future. Storing information in memory is achieved by the strengthening of appropriate cell assemblies; retrieval from memory is the reactivation of these same assemblies.

Hebb assumed that long term memory is realized by changes in synaptic strength and that this synaptic strengthening is in turn determined by contiguity and repetition of neural activity. Hebb (1949) expressed this assumption as an explicit, qualitative rule:

Unlike previous theories invoking the cell assembly mechanism for dynamic memory storage, which have been refuted by empirical evidence (Hilgard & Marquis, 1940), Hebb's cell assembly resolves the apparent contradiction between dynamic and structural theories of memory. Dynamic memory stores information for a relatively short period of time. It also serves as a mechanism for laying down long term memory, which is structural. It is, in other words, a mechanism to implement the consolidation process, first proposed by Müller and Pilzecker (1900). The dynamic activity creates the structural changes which the data indicate are necessary for the durable character of long term memory.

Much more is known today about cortical structure and function than was available to Hebb in 1949. None of the new information contradicts the cell assembly hypothesis, and some of it provides, if not conclusive empirical support, then at least strong encouragement for the notion.

The assumption that long term memory is implemented by an increase in synaptic efficacy is now widely accepted. An effect which operates similarly to Hebb's contiguity learning rule, known as long term potentiation (LTP) has been demonstrated and studied extensively in the hippocampus and to some extent in the cortex as well (Lynch, 1986). A likely candidate for the mechanism of this change has been found in the action of NMDA receptors, which seem to play a role in use-dependent synaptic strengthening during development and learning (Cotman & Iverson, 1987).

Detailed studies of the source and destination of axonal projections between cortical processing areas in primates have revealed that whenever an area receives input from another, it usually has a reciprocal influence on that area through feedback projections (Van Essen, 1985). Such recurrent connections are required for the self-excitatory loops which support developing cell assemblies.

A major shortcoming in Hebb's theory, which he recognized, is the need for some negative influence to prevent a rapid spread of neural activity through the strong positive synapses. Hebb cautiously omitted the inhibitory synapses necessary to correct this problem because of a lack of empirical evidence for them. Milner (1957), with Hebb's encouragement, added inhibition to the theory. Today several types of inhibitory interneurons are known to be abundant throughout the cortex (Shepherd, 1988), returning Hebb's proposal to a solid neurophysiological foundation.

Few research efforts have been undertaken to identify cell assemblies per se. There is a strong tendency to study the detailed molecular workings of neurons and synapses before proceeding to explore networks of neurons. In addition, there are technical difficulties facing any search for cell assemblies. While the components of cell assemblies near sensory cortex may be localized, the neurons composing other cell assemblies may be scattered throughout the cortex and possibly subcortical regions as well (Mishkin and Appenzeller, 1987). Thus it is far from clear where to look for the individual cells that comprise a single assembly.

Single cell recordings have identified extremely selective neurons in the inferior temporal cortex of monkeys (reviewed in (Perrett et al., 1987)). Mishkin and Appenzeller (1987) suggest that these neurons are the extreme end of a processing pathway along which cells respond to more of the visual field and more complex combinations of features until a complete representation of an object is achieved. This suggests the possibility that recordings are being made from a single cell of an extensive cell assembly.

Measuring the concurrent activity of multiple cells constitutes yet another challenge to any search for cell assemblies. Simultaneous in vivo multi-cell recording, once limited to monitoring less than fifty neurons (Gerstein et al., 1983), can now monitor a few hundred neurons, but this number is still short of the potentially thousands of neurons which might comprise a cell assembly. However, information revealed by extracellular measurements, although it represents an average of many neurons' firings, may be useful in making inferences about the underlying neural activity.

Early evidence for self-excitation within recurrently connected groups of cortical neurons was found by Burns (1951). He isolated an area of a cat's cerebral cortex from all connecting neural tissue while maintaining its blood supply. He then measured changes in electrical potential on the surface of the isolated cortex in response to brief point stimulations. Because the response was a long lasting, asynchronous, repetitive pattern which spread across the entire cortical area before ending abruptly after 2 - 4s, he concluded that it resulted from excitation travelling around closed axodendritic loops.

Reports of widely separated neurons in cat visual cortex which fire in synchrony when responding to features from the same object (Gray et al., 1989) suggest that groups of neurons can act in concert and might be a part of the object recognition process.

Work done by Walter Freeman's group (Freeman, 1991) provides the most convincing evidence to date that neurons act in concert rather than individually. They believe that this cooperative activity can be explained by appealing to the existence of cell assemblies. They have studied the processing of scents in rabbits by monitoring EEG signals in an array of locations across the olfactory bulb and olfactory cortex. The EEG reading at each point in the array provides a time series of average activity values for a localized group of neurons rather than a single neuron. Taken together, the EEG readings yield a picture of the collective activity within the regions of study over time. The data suggest that unique individual scents are represented in the olfactory bulb by a unique spatial pattern of activity amplitudes across the array. Thus for a given scent, some locations are highly active while others are not; for a different scent a different set of locations are highly active. These patterns appear in a "burst" in response to olfactory input, during the period between inhalation and exhalation, and last less than a second. Such a concerted, unitary response is very much what might be expected if cell assemblies were responsible for the activity.

During the period when Milner was modifying Hebb's theory, a set of computer simulations were conducted to investigate whether cell assemblies would form under the conditions Hebb assumed (Rochester et al., 1956). This work is one of the earliest examples of modern connectionism. Currently, connectionist interest is focused on pattern classification tasks, following the direction pioneered by other early work (Rosenblatt, 1962). The simulation of cell assembly like aggregations has attracted little attention since the Rochester et al. study. This is unfortunate, since we believe the cell assembly construct could provide the critical link between pattern classification networks and networks which model higher cognitive processes.

Modern connectionist studies of higher cognitive processes (e.g. Rumelhart and McClelland (1986)) exhibit a peculiar discordance between the representational power assigned to a single unit and the neurophysiological basis for the unit. The units of the network are either assigned, if they are input or output units, or come to represent, if they are hidden units, specific semantic features. In other words, the unit represents a feature at a very high level of neural processing. Yet the function of a unit is loosely based on the computations that might be performed by a single, highly simplified neuron. The evidence suggests that, on the contrary, an extensive set of neurons contributes to the recognition of high level features. An even greater disparity is evident in the propositional models of classical artificial intelligence (AI), which dispense with any attempt to provide a grounding in neural mechanisms and begin their analysis at the level of the symbol.

It seems there may be a question which is not addressed in existing models of higher cognition, be they classical or connectionist. What exactly are the units of which thought is composed? They are probably not single neurons, yet the units in connectionist models function like simplified neurons. They may be symbols, but classical AI so far has revealed nothing of their origins. Hofstadter (1985, chap. 26) refers to this problem when he criticizes the passive symbols employed in AI models. Harnad (1990) has called this the symbol grounding problem.

Hebb's cell assembly is ideally suited to play the role of the unit of thought. Its development and function can be ascribed to explicit neural mechanisms. It is composed of an extensive set of neurons, and so it has properties beyond those of a single neuron. Once formed, a cell assembly acts as a unit. Its strong recurrent connections cause the entire assembly to become active if enough of its constituent neurons are activated, giving it the ability to "fill in" features missing from its input. This provides a mechanism for what Bruner (1957) referred to as "Going beyond the information given" and is closely akin to Gibson's (1979) concept of "affordances." Since a cell assembly is built by perceptual learning it represents an object of experience. This means it is a symbol which is firmly grounded in the environment and represents something meaningful in that environment. Margolis (1987) presents a useful analysis of why perception and pattern recognition are central to cognitive theory.

We suggest that the unit used in connectionist models of higher cognition ought to to be modelled after the cell assembly rather than the single neuron. The problems and benefits of using the cell assembly as an "active symbol" of something in external reality which can be combined with and influence other symbols in a connectionist cognitive system are presented in detail in Kaplan et al. (1990).

There is, then, a sound basis for considering the cell assembly or recurrent circuit to be a central construct in a connectionist model of mind. A key aspect of the Hebbian synthesis is the insight that the amount of activity among the elements making up the cell assembly determines the role the assembly plays. To oversimplify, an inactive assembly represents latent structure, while an active assembly is a cognitive element participating in ongoing cognition. In simplest terms, activity signifies presence as opposed to possibility. This distinction is a crucial one in human cognition, since the domain of "possibility" is very large while what can be considered "present" is severely limited. The human cognitive system, in other words, is characterized by a vast capacity for inactive or latent structure. The portions of this structure that may be active concurrently is orders of magnitude smaller.

This limited capacity for what can be thought of and/or perceived at any one time may seem at a first glance to be an unfortunate bottleneck in an otherwise powerful system. Further consideration, however, suggests a quite different conclusion. Unlike many other animals, humans have the capacity to consider multiple alternatives, even possibilities that are hypothetical or contrary to fact. Such a system could all too readily become lost in thought. Indeed, early steps toward the development of a cognitively oriented learning theory by Tolman (1932) received exactly this criticism (Guthrie, 1935). The constraint of a limited capacity for activity means that the number of alternatives considered at one time will be modest, thus reducing the gulf between cognition and action.

The role of activity in distinguishing presence as opposed to possibility does not, however, exhaust the potential of this concept. Activity is a powerful and flexible code, capable of playing roles more differentiated and more varied than this simple binary distinction would suggest. Four roles are of particular interest in the present context:

1. Activation and perception. A complex organism presumably has a great many tightly knit associative structures, or, in the language of our framework, latent cell assemblies. Only a few of these can be actively experienced at any one time. Activity provides an appropriate means for distinguishing these few from the rest. In our model perception occurs when an environmental event leads to a high level of activity in a corresponding cell assembly.

2. Peak activation and dominance. Connectionist models tend to view perception as a competitive process. Those potential percepts receiving the greatest support, both external and internal, are presumed most likely to inhibit alternative interpretations of the same stimulus configuration. In this way a relatively small number of cell assemblies will ultimately become substantially more active than any others, and thus dominate. This same competitive process presumably applies to thought as well; the crucial differences are the distance of the cell assembly in question from the sensory interface and the degree to which its activation was due to the activity of other cell assemblies rather than to input from the environment.

The state of dominance characteristic of cell assemblies undergoing peak activation has important functional significance, since these highly active cell assemblies are the most likely to transmit information to other cell assemblies and/or to generate behavioral output. This mechanism provides an orderly way in which a small subset of the total stored information in the system can control the output of the system at a given time.

3. Activation and primary memory. James (1892) described primary memory as memory whose retrieval required no effort. Primary memory immediately follows perception. We propose that a lower level of activation than that required for perception serves as a code for a "just perceived" state. In this state the retrieval potential is high, but the stimulus in question is not experienced as being present.

4. Activation and consolidation. Considerable evidence points to a brief post perceptual process that is crucial for information to be stored in long term memory (Weingartner and Parker, 1984). The intensity and duration of this process impact whether information is stored, and how strongly it is stored. This post perceptual activity, called the consolidation process, can be viewed as an internal form of repetition that allows for a learning process considerably more efficient than one completely dependent on external repetition. While the function is different from primary memory, the same intermediate level of activation is assumed to provide the mechanism for both.

To fulfill these four roles then, the proposed model should include at least three distinct levels of activity. The lowest level corresponds to the latent, or inactive state. The highest level represents peak activation and perception; a middle level embodies primary memory and consolidation.

Each of these levels might be expected to have a distinctive time course. These characteristic temporal patterns, in turn, must also reflect the properties of the cell assembly as a whole. The most basic of these properties is that patterns must occur in a tightly knit structure. The danger in such a configuration is that activity has the potential of spreading exponentially. Such a tendency toward positive feedback necessarily requires a negative feedback control.

Activity would be expected to display initial positive feedback effects, followed by the influence of negative feedback controls. Furthermore, the precise form of this function might be expected to be susceptible to influence by a number of key parameters beyond the duration and amplitude of the stimulus. In this way, what Hebb (1963) has called the "semi-autonomous process" has a semi-autonomous time course that is by no means identical to the time course of the stimulus that initiated it.

Since the cell assembly has the potential to play such a pivotal role in a theory of cognition, it seems promising to explore this construct more fully. At the time of the Rochester et al. (1956) studies, computer simulation was the method of choice. In vivo neuron recording was in its infancy and formal analytic methods could be applied only to very simple networks. Little has happened since then to favor another approach. While single cell recording technology is quite sophisticated, and extracellular recording techniques have proved useful for monitoring aggregate neuronal activity, many problems stand in the way of using multi-cell recording techniques to investigate cell assemblies (Gerstein et al., 1989). Formal methods have been extended to the analysis of multi-layer networks, but only in certain restricted cases, e.g. fully symmetric networks (Hopfield, 1982) and back propagation networks (Rumelhart et al., 1986). Thus computer simulation is still the preferred method. Computer simulation makes it possible to study phenomena too complex to analyze completely and too elusive to capture in the real world (Wolfram, 1984).

Since most of the important questions about the anatomy and physiology of cell assemblies remain unanswered, how do we go about constructing a model of one? Ecologists have faced a similar problem in their study of population dynamics. The behaviors of individuals are highly variable and yet their combined effect is often predictable. Ecologists have employed dynamic systems models which capture the essential interactions between a selected set of aggregate state variables. We pursue a similar course in modelling the global dynamic behavior of a cell assembly without specifying the detailed interactions between each of its component neurons.

The construction of a molar model affords us an opportunity to gather and organize the many constraints applicable to cell assembly functioning. These constraints can in the future guide construction of a network model as well. Furthermore, the molar model can specify quantitative behavior which future network models can strive to duplicate.

In order to simulate the workings of this complex system it is necessary to identify the key factors at work. These become state variables in a mathematical description of how these factors influence themselves and each other. The resulting description takes the form of a system of simultaneous equations. The focus of these equations is, of course, activity. This construct is designated P, in memory of perseveration, a term for persisting neural activity used by Müller and Pilzecker in 1900. P stands for the level of activity in a single cell assembly.

The variables that influence activity can be grouped into four primary conceptual domains: Input, Prior experience, Competition, and Control. Let us examine each of these in turn.

An input pulse in this simulation is designated I. Input provides the initial impetus for activity, but in a well learned cell assembly the many strong interconnections assure the spread of activity after that activity has reached some critical level. This allows a subset of features to trigger the perception of an object, making available to the organism more information about the object than was available in the stimulus configuration.

The connection strength between elements in a cell assembly reflects the corresponding prior experience of the organism. This latent long term structure is expressed as L in the simulation. Long term structure brings with it a potential hazard. Consider a stimulus configuration for which the individual possesses only a weak cell assembly, or in the extreme case, none at all. In such a situation the activity generated by sensory input from the stimulus is likely to be drawn off into other, better learned cell assemblies. This tendency of a cell assembly to function as an attractor, while adaptive under many circumstances, thus constitutes a potential drawback as well. In the extreme case, such a tendency would dictate that only preexisting cell assemblies could be activated and hence new ones could never be learned.

Consider a bird-watcher-in-training who starts out with firm knowledge of only two birds, the robin and the blue jay. Then our inexperienced protagonist sees a kingfisher. This new stimulus clearly fits the blue jay category far better than the robin category. It is not an excellent fit, but if there is not some way for temporary structure to arise rapidly, the pattern of activity generated by the stimulus will not be supported. Instead, the overlap with the preexisting category, plus the positive feedback capacity of the underlying structure, will lead to a shift in the center of gravity of the resulting pattern. The kingfisher, in other words, would be categorized as a blue jay every time.

In a cell assembly that is not yet well learned, some factor is required that could substitute, albeit partially and temporarily, for the stable connections that will come to define the cell assembly and make positive feedback possible. A temporary connection change would be sufficient to permit the temporary operation of new cell assemblies. A similar construct has been proposed by Peterson (1966) and by Hinton and Plaut (1987). This hypothetical short term connection strength (STCS) is designated S.

The context for the cell assembly we are simulating is a larger neurocognitive system. This larger system necessarily has, as we have seen, a limited capacity as far as the number of active cell assemblies is concerned. The concrete expression of this limited capacity is in terms of competition from other active cell assemblies. The mechanism for this competition is the activity of inhibitory neurons such as those described by Milner (1957). These neurons are assumed to be sensitive to nearby activity, and to respond by generating nonspecific inhibitory activity in the region. This inhibitory activity serves to protect the very cell assemblies that engendered it, making it difficult for competing assemblies to become active. Thus, as long as P is large, there will be a correspondingly high level of regional inhibition, which will tend to restrict competition. Conversely, as P declines, there will be less inhibitory protection and a greater likelihood of competing activity. This competing activity, in turn, generates inhibition, which will tend to further undermine P.

A system so well endowed with positive feedback requires comparably powerful controls. Paralleling other biological systems that tend to expend resources and build up deficits, it seems appropriate to employ a fatigue factor (designated F) for this purpose. Such a component would make it impossible for the activity of a given cell assembly to persist beyond reasonable bounds, thus providing an effective control on duration of activity. A highly active cell assembly is in a position of competitive advantage. It is in a dominant role with respect to the system as a whole, and it generates inhibition that constrains the activity of potentially competing cell assemblies. It is for this reason that fatigue plays such a vital role. Without it a dominant percept or thought could remain dominant indefinitely.

The upper bound on the amount of activity in a cell assembly is provided by the fact that it is comprised of a finite number of neurons. Another control on the amount of activity is the stochastic tendency of units to drop out over time. This is assumed to occur most often at high levels of P, since the neurons brought into play then are the most tenuously connected and the most likely to drop out.

Together these various considerations suggest a general form for the expected activity function. It begins to rise gradually until the short term connection factor builds to the point of aiding the process, and until there are enough active units to help stimulate other units, producing a snowballing effect. There is then a rapid rise as more and more elements are caught up in the positive feedback. After a certain point, however, there are fewer and fewer units left in the cell assembly that are not already activated, and the rise gradually flattens out. By this point some of the initial elements would be beginning to show the effect of fatigue, and a gradual decline in activity begins. The activity in the cell assembly is protected from interference by the inhibition it generates. However, as the activity declines, the inhibition lessens, and there is an ever increasing likelihood that competing activity will arise. This newly active competition, itself now generating inhibitory protection, will finally terminate the initial activity.

While these psychological and theoretical considerations provide important constraints in developing the model, further input is needed to bridge the gap between these rather broad considerations and the specification required for a mathematical description. This additional input is largely empirical. Its primary contribution is to help structure the temporal characteristics of key theoretical constructs.

Not only is activity a central theme in this model; it is also an important code. The highest levels of activity indicate perception, while a somewhat lower level of activity denotes primary memory. Thus, in order to determine the general form one might expect of the curve of activity over time, it is necessary to assign approximate durations to each of these codes.

The duration of perception was a central concern to Hebb in The Organization of Behavior; his comment on the value he chose is instructive:

What are experienced as percepts of longer duration might then be the expression of what Hebb (1963) called "a serially ordered activity of mediating processes or cell assemblies" (p. 26). James (1892) took a similar position. He pointed out that the focus of sustained interest "is not an identical object in the psychological sense, but a succession of mutually related objects forming an identical topic only, upon which the attention is fixed" (p. 92).

To this estimate of 0.5 s for perception it is possible to add a bit of further specification based on short term visual storage data. The transition to positive feedback should come at approximately 20 ms and the beginning of perception at approximately 50 ms after the onset of stimulation.

The duration of primary memory can be approximated in two different ways. One approach is to assume that it is equatable with the memory span, as that construct is usually measured. The traditional measurement procedure involves presenting digits at a rate of one per second. Thus, since the memory span has been determined to encompass 5 ± 2 items (Mandler, 1975), the approximate duration of primary memory can be estimated to be in the five s range.

A second approach to estimating the duration of primary memory depends upon the equation of primary memory with the period after the presentation of an item during which the information may be consolidating. Since consolidation and primary memory are both post-perceptual active processes, this would seem to be an appropriate assumption. Miller and Marlin (1984) have noted that the best controlled studies of the consolidation process also arrive at a maximum value of approximately five s.

Much of the rationale for our assumptions about the behavior of STCS and fatigue depend upon a line of research begun almost thirty years ago by Kleinsmith and Kaplan (1963, 1964). The findings of these and other studies using what we shall refer to as the "Kleinsmith paradigm" were so radical that they have not yet been successfully incorporated into the traditional theoretical frameworks in psychology. Since we feel that these findings can be readily understood in terms of a connectionist model, and since TRACE was greatly influenced by this line of research, it may be useful to describe this work in some detail.

The Kleinsmith paradigm emerged from what began as a straightforward investigation of the relationship between learning and arousal. The intention had been simply to measure differences in a subject's learning at high and low arousal levels. The learning task involved incidental learning of a paired associate list presented only a single time. Arousal was measured by item rather than by subject. Based on skin resistance reactions, each subject's items were divided into high and low arousal groupings. Although the initial expectation had been that high arousal material would be better recalled, it only required a few pretests to discover that quite the opposite was the case.

Initial puzzlement gave way to the insight that the intense neural activity resulting from high arousal might be producing a temporary fatigue effect that was leading to an apparent failure of learning. Thus, if there were indeed a facilitating effect of arousal on learning, it would only be apparent at a later time, after the fatigue had dissipated. Based on this new set of expectations the experiment was redesigned with recall measured not only immediately after learning, but at various points thereafter (20 minutes, 45 minutes, one day, and one week in the first experiment in the series). To avoid contamination, a different group of subjects was tested at each recall interval.

For the low arousal items the recall was as would be expected. Recall of these items was relatively high soon after the learning trial, but declined steadily over time. The high arousal results were dramatically different, confirming the hypothesis that a fatigue effect was involved. Kleinsmith and Kaplan (1963) found that for items where arousal was high, the recall started out low and slowly improved. Thus subjects would be more likely to recall an item 20 min after seeing it than 2 min after seeing it. In addition to the surprising fact that the recall curve actually rose in the high arousal case these results were striking in that for a period of 10 to 20 min after the presentation of the material, recall was actually better for low arousal items than for high arousal items.

Because these results directly contradicted what any known model would have predicted, Kleinsmith and Kaplan (1964) repeated the experiment using nonsense syllables instead of words. This second experiment generated essentially identical results. A number of other attempts have been made to replicate these findings (Walker & Tarte, 1963; McLean, 1969; Butter, 1970). These studies have also obtained comparable results.

In order to make inferences about the time courses of STCS and fatigue based on these findings, it is necessary to assign some theoretical linkages. It is our assumption that the "low arousal" curves reflect some sort of short term memory, since recall for that material steadily declines over time. It cannot be a curve of primary memory, since that process appears to have a much briefer duration. Rather it seems to fit far better the STCS construct, posited to provide a basis for positive feedback in cases where the requisite long term structure, L, is not present. Based on the Kleinsmith data, STCS would be expected to persist for a considerable period, say on the order of 10 to 20 min.

The "high arousal" curves obtained using the Kleinsmith paradigm reflect what on theoretical grounds would be expected to be a fatigue effect, since the high arousal would cause intense activity with a concomitant rise in fatigue. It is evident that fatigue must not build up too rapidly, lest it destroy the trace activity before perception occurs. On the other hand, it must decay reasonably rapidly, since the Kleinsmith results show recovery of previously inaccessible material by 5 or 6 min after presentation. At its peak, of course, it must be able to overcome both L and S combined, since these same studies show below chance performance where fatigue would be expected to be operative.

The cell assembly is an internal structure that corresponds to some class of events in the environment. When the corresponding environmental event occurs, the cell assembly should become active. Eventually its activity should cease. Thus one would expect P to rise and fall over time in response to an input. It is possible to separate those factors contributing to the rise of activity from those contributing to its fall, such that

The sensitivity variable V represents a gathering together of all the factors that make a latent cell assembly more readily activated. This can be represented by

V(t) = (3)

are essentially similar: they both depend on the activity level, P; they are both limited by their maximum value of unity; they both decay gradually. Each incorporates constants which modify the rates of growth and decline. Adjustments of the constants is such as to make fatigue rise more slowly and decay more rapidly than short term connection strength, a decision based on logical and empirical considerations, to be discussed subsequently.

In the set of experiments presented here, we simplify by modelling a single cell assembly without learning, i.e.

In TRACE simple equations are used to model the aggregate effect of large numbers of processes. The input equation is no different. Further, the equation is not designed to model a particular input, but instead a prototypic input. The input function

I(t) = (10)

In our discussion of TRACE we have focused on the constraints that have influenced the generation of the model. These constraints range from the neurophysiological to the cognitive. A series of experiments have been carried out to explore the implications and capabilities of the proposed model with regard to these constraints. These experiments vary in complexity and tone from the simple but critical verification of the model's ability to generate the desired time course of activity, to an examination of the model's potential to support distinctive modes of functioning.

Conceptually the experiments are divided into three categories (Table I). The first and most important focuses on the model's ability to generate the general form of the activity function. The second category consists of experiments two through five. These experiments deal with the key modulators of activity, fatigue, and STCS. While fatigue and STCS are supposed to play well specified roles in the model, it is not clear in a system as complex as TRACE, especially where timing is so critical, whether or not the

Table I. Summary of simulation experiments

interactions of such factors will have unforeseen consequences and whether the individual components will play their expected roles. The first two of these experiments examine the role that fatigue plays; they focus on the growth and decline parameters, respectively. The next two experiments view STCS in two contexts, one where the percept in question is already well learned and one where it is not. The final category of experiments, consisting of experiments six and seven, explores the potential of the model to be generalized to meet the special requirements of differing ends of the neurocognitive spectrum, specifically high level functioning on the one hand and near the sensory interface on the other.

The TRACE difference equations were simulated as shown in Table II. Since the reader has by now seen the derivation of these equations in some detail, we simply summarize the variables and parameters in Table II.

In interpreting the solution curves, we make a convenient assignment of 10 ms as the interval between time steps. Perception is considered to occur when the activity, P, rises above 0.55. This level is somewhat arbitrary in that the perceptual level of a cell assembly in this context cannot be directly determined. The proposed value, however, represents a useful rule of thumb based on extensive experience with the simulation.

Table II. The TRACE difference equation system as simulated

I(t) =

Variable Interpretation Initial condition

P(t) perseverance, activity P(0) = 0.01

F(t) fatigue F(0) = 0.01

S(t) short term connection strength S(0) = 0.01

I(t) external input I(0) = 0.0

Parameter Interpretation Unmodified value

Let us begin with the basic curve of activity over time that is the focus of the proposed model.

The results of Experiment 1, shown in Figure 1, show that TRACE is indeed capable of modelling the general theoretical case. However, generating one curve is only a beginning in supporting TRACE's claim to be an interesting and useful cognitive model. TRACE was constructed conceptually with each component playing a well defined role. It is important that each of these components fill those roles as specified. Further, it is important that each of these additional components is free of unforeseen side effects. In addition the basic form of activity trace generated in Experiment 1 should be the rule rather than the exception. That is, the TRACE model should tend to generate activity curves essentially similar to the one in the basic model. That way the modest variations in parameters that will occur from individual to individual would not be expected to result in dramatic behavioral differences.

In this context the first experiment serves as only the first step in testing TRACE. The following experiments were designed to further test TRACE under a wide variety of conditions.

The Rochester et al. (1956) results suggest that activity based systems must have negative controls to hold the spread of activity in check. In most connectionist networks negative feedback is expressed in terms of inhibition. While the TRACE model uses inhibition as a method of selecting between competing percepts, an additional factor, fatigue, has been postulated that is not only complimentary to inhibition, but equally critical in holding activity in check. Competition between cell assemblies will mean that only the strongest circuits will become percepts. However, once an assembly becomes strongly active an additional mechanism is necessary to ensure that it does not stay active indefinitely. The role of fatigue therefore should not be so much in preventing percepts from forming, but in providing a kind of shutoff mechanism once perception has occurred.

Table III. Effects of fatigue parameter changes

Experiment 2

Experiment 3

The left column identifies the parameter manipulated and the values tested. The right three columns give the values of three dependent variables: the height of the peak of the TRACE curve, the time steps until the onset of perception, and primary memory as measured by duration in time steps. (A time step equals 10 msec.)

The depletion of resources within the cell assembly leads to the growth of fatigue. As fatigue grows the cell assembly loses its ability to sustain activity. Therefore the rate at which fatigue grows should impact how long the assembly can remain active. The next experiment was designed to test how changing the growth rate affected the durations of the perceptual and primary memory phases. The major goal of this experiment is to show that fatigue plays a critical role in shutting off the activity in a cell assembly. However, fatigue should also play a decisive role in the duration of perception while only having a minimal impact on its onset.

The results of Experiment 2 were conclusive in showing the important role that fatigue plays in the cell assembly. They did not necessarily, however, tell the whole story. Fatigue is brought on by the depletion of resources. If the cell assembly is to recover from fatigue then those resources must be replenished. Just as the activity trace has a characteristic time course so does the buildup of fatigue. Fatigue is a consequence of activity and therefore should not have an effect on activity until later in the time course. Experiment 2 showed that by varying the rate at which fatigue builds up, it is possible to change the time that fatigue begins to take effect. Recovery from fatigue is a different issue, however. Just as activity must precede fatigue so must fatigue precede recovery from fatigue. It follows that changing the recovery rate of fatigue should only impact the activity trace late in its time course. Experiment 3 was designed to test this hypothesis. The object of the experiment was to determine at what point in the time course of activity recovery from fatigue becomes a factor.

Fatigue serves a definite function in shaping the activity trace. Once reverberation begins, a mechanism is needed to eventually shut it back down. Fatigue, even at widely varying parameter settings, serves this purpose. It dampens activity sufficiently to end perception fairly quickly and then later, in conjunction with inhibition, it shuts down primary memory. The parameters controlling fatigue can be used to vary these durations, and small changes in the parameters lead to similarly small changes in these durations.

Experiment 3 provides an interesting example of the tendency of the system to maintain stability. The effects of changing the recovery rate are asymmetrical around the base rate. A drop in the recovery rate will have a much smaller effect than a corresponding rise. In the context of a system consuming resources this is a reasonable reaction. It is unlikely that resources will ever suddenly become more readily available. However, it is easy to conceive of situations where resources are scarcer than usual. The results show that a drop in the incoming resources will have only minor effects.

The previous two experiments examined the critical role that fatigue plays in determining the length of the various phases of activity. Conceptually fatigue's counterpart is STCS. Fatigue serves to dampen activity, ultimately even to extinguish it, but only after activity becomes strong. STCS provides temporary positive support to permit activity to get started. It should be critical during the early rise of activity.

We have not called this modulator "short term memory" because of our conviction that there are in fact several types of short term memory, each with its own mechanism and characteristic range of durations. Primary memory and short term connection strength are, for example, distinct in both respects; yet both are appropriately included under the broad heading, "short term memory." The special role of STCS is, as its name suggests, to provide a transient increment in connection strength. This is particularly critical in poorly learned or loosely connected assemblies where the capacity for reverberatory feedback is small. In these cases STCS can provide the necessary extra impetus to keep the assembly active. In well learned assemblies, however, where the capacity for feedback is large, the impact of STCS should be minimal.

Basic TRACE simulates a well learned cell assembly. In such an assembly the impact of STCS should be small. The feedback in the assembly should be enough to drive perception on its own, and STCS should serve only to strengthen and speed the process. Once activity is at high levels the combined effect of feedback and fatigue should virtually nullify any impact of STCS.

The results in Table IV show that STCS has the predicted effect for a well learned cell assembly. The major impact of short term growth rate comes during the rise of activity. However, the durations of perception and primary memory are virtually unchanged. Peak activity varies from .659 to .782, a small change considering the large variation of the parameter. The minimal impact of short term decline rate is even more striking. Large variations in the decline rate have virtually no impact on the system. It is only at a level more than 100 times the normal level that any noticeable, though still minor, change is seen.

Short term growth, sg

Short term decline, sd

High long-term connection strength (l = 0.5)

Short term growth, sg

Low long-term connection strength (l = 0.2)

Short term growth, sg

Short term decline, sd

Although not shown in the table, the experiment also established that the growth of STCS is related to the speed of perception. To take two examples, when short term growth, sg, was set to 0.3 the onset of perception took 0.18 s (slow compared with the well learned case). At 0.6 this was speeded up to 0.1 s. It is worth noting here that in Experiment 3 manipulation of fatigue growth rate yielded only negligible effects on this onset rate.

Experiment 6 was designed to test the ability of TRACE to generate behavior associated with high level assemblies. Consider a higher level cognitive activity such as planning. The landmark treatment of this topic in Plans and the Structure of Behavior (Miller et al., 1960) made it clear that the circuitry involved in planning should be capable of activity that is persistent and highly resistant to fatigue. As Tolman (1932) pointed out in his classic analysis of purposive behavior, the plan must continue to guide behavior as the organism moves through the environment, encountering obstacles and impediments of various kinds along the way. The nature of the experiment intended to capture this essential property of high level assemblies was simple; the idea was to reduce fatigue with the expected result that activity should be more persistent. Resistance to fatigue implies that the available resources are used more efficiently. Therefore fatigue decline rate, fd, was raised, signifying that resources were replenished more quickly.

The experiment proved successful. By increasing fatigue decline, fd, from the base rate of 0.0001 it was possible to extend the length of the activity curve indefinitely. A fatigue decline rate equal to 0.00014 generates a persistent activity trace appropriate for the high level assembly being modelled. These results can be seen in Figure 2.

A more complex set of issues was raised by an experiment dealing with behavior at the lower end of the processing continuum, as might be exhibited at the sensory interface. The target behavior calls for relatively short activity traces with STCS and fatigue also clearing rapidly. At the perceptual interface there can be little or no long term structure. After all, the elements at this end of the hierarchy are shared components that temporarily participate in the link between information from the environment and cognitive activities at a higher level. For this reason, the constant value of long term connection strength, L, was lowered to 0.2. In addition there is the issue of competition. As James (1892) points out, perception is of "the probable and the definite." Perception is characteristically experienced as clear and definitive despite uncertainties and ambiguities. The system is biased towards a single dominant percept as opposed to several alternate conceptions of what one might be looking at. For this reason the inhibitory competition parameter, qc, was reduced to 4, thereby raising the level of competition. In addition the amplitude of the input, a, was increased to 1.0 to reflect the fact that these elements are receiving direct stimulation.

In the absence of long term structure the role of STCS becomes more vital in providing the impetus for activity. With this in mind short term growth, sg, was set at 1.0 to maximize the onset rate. Even with short term connection strength at its maximum, without long term structure the sensitivity is particularly susceptible to fatigue. For this reason the growth rate of fatigue, fg, was set only at 0.1. Both fatigue and short term connection strength should recover rapidly since the system must be ready for another input. For this reason their decline rates, sd and fd, were set at high levels (0.2 and 0.01 respectively).