Gabora, L. (2007). Revenge of the ‘neurds’: Characterizing creative thought in terms of the structure and dynamics of memory. Creativity Research Journal.
Revenge of the ‘Neurds’:
Characterizing Creative Thought in terms of the Structure and Dynamics of Memory
University of British Columbia
Address for Correspondence:
University of British Columbia
Okanagan campus, 3333 University Way
Kelowna BC, V1V 1V7, CANADA
Phone: (250) 807-9849
Fax: (250) 470-6001
ABSTRACT: Empirical results suggest that defocusing attention results in primary process or associative thought, conducive to finding unusual connections, while focusing attention results in secondary process or analytic thought, conducive to rule-based operations. Creativity appears to involve both. It is widely believed that it is possible to escape mental fixation by spontaneously and temporarily engaging in a more divergent or associative mode of thought. The resulting insight (if found) may be refined in a more analytic mode of thought. The question addressed here is: how does the architecture of memory support these two modes of thought, and what is happening at the neural level when one shifts between them? Recent advances in neuroscience shed light on this. It was demonstrated that activated cell assemblies are composed of multiple ‘neural cliques’, groups of neurons that respond differentially to general or context-specific aspects of a situation. I refer to neural cliques that would not be included in the assembly if one were in an analytic mode, but would be if one were in an associative mode, as ‘neurds’. It is posited that the shift to a more associative mode of thought conducive to insight is accomplished by recruiting neurds that respond to abstract or atypical subsymbolic microfeatures of the problem or situation. Since memory is distributed and content-addressable this fosters remindings and the forging of creative connections to potentially relevant items previously encoded in those neurons. Thus it is proposed that creative thought involves neither randomness, nor search through a space of predefined alternatives, but emerges naturally through the recruitment of neurds. It is suggested this occurs when there is a need to resolve conceptual gaps in ones’ internal model of the world, and resolution involves context-driven actualization of the potentiality afforded by its fine-grained associative structure.
What is happening in the brain when one engages in creative thought? The complex nature of creativity has made the task of achieving a biological account of creativity somewhat elusive (Runco, 2004). Advances have been made, however. Hoppe and Kyle (1991) studied Sperry’s (1964) infamous commissurotomy patients (who underwent surgical bisection of the corpus callosum to inhibit epileptic seizures) and found they lacked the capacity for integrated thought and affect-laden interpretation of experience, which they speculated was related to these patients’ “impoverished fantasy life”. Their work suggested that interaction between the two hemispheres of the brain is important for creativity. Dietrich (2004) hypothesized that different kinds of creativity (deliberate versus spontaneous, and emotional versus cognitive) involve different neural circuits. Vandervert’s work (in press) suggested that the cerebellum plays a key role.
This paper synthesizes findings that together provide a tenative explanation for what is taking place at the neural level when one puts information together in a new and useful, creative manner. The paper begins by reviewing evidence for the hypothesis that creativity involves the capacity to spontaneously shift back and forth between analytic and associative modes of thought according to the situation (Finke, Ward, & Smith, 1992; Gabora, 2000, 2002a, b, 2003; Howard-Jones & Murray, 2003; Martindale, 1995). Next we examine how mental representations are encoded as cell assemblies composed of ‘neural cliques’ in a sparse, distributed, content-addressable memory, and how this cognitive architecture is navigated in a stream of thought (Hinton, McClelland, & Rummelhart, 1986; Hopfield, 1982; Kanerva, 1988; Lin et al., 2006). Note that memory is not just a storehouse of previous experiences (Goldman-Rakie, 1992; Miyake & Shah, 1999; Vandervert, in press). As we will see, the capacity to generate and refine creative ideas relies on the distributed, content-addressable manner in which items are encoded in memory. Finally the research on creativity at the cognitive level is interpreted in terms of research findings in neuroscience and computer models of memory to arrive at an (in principle) testable hypothesis as to what is going on in the brain when one comes up with a creative idea. Specifically, this article connects brain research to creativity by posited that the shift to an associative mode of thought conducive to creative insight is accomplished by recruiting neurds: neural cliques that respond to abstract or atypical aspects of a particular problem or situation. Because memory is distributed and content-addressable, this fosters the forging of creative connections to potentially relevant items previously encoded in these neurds. The paper concludes with a concrete example of what this theory predicts is happening at both the cognitive level and the neural level during a creative problem solving situation.
The theory that creativity involves the capacity to shift between focused and defocused attention is sometimes referred to as contextual focus, because this shifting is thought to be brought about by the situation or context. It is largely inspired by studies of characteristics associated with high creativity. Martindale (1999) identified a cluster of such attributes. One is defocused attention: the tendency not to focus exclusively on the relevant aspects of a situation, but notice also seemingly irrelevant aspects (Dewing & Battye, 1971; Dykes & McGhie, 1976; Mendelsohn, 1976). A related attribute is high sensitivity (Martindale, 1977, 1999; Martindale & Armstrong, 1974), including sensitivity to subliminal impressions; stimuli that are perceived but of which one is not conscious of having perceived (Smith & Van de Meer, 1994). Creative individuals also tend to have flat associative hierarchies (Mednick, 1962). The steepness of one’s associative hierarchy is measured by comparing the number of words generated in response to stimulus words on a word association test. Those who generate few words for each stimulus have a steep associative hierarchy, whereas those who generate many have a flat associative hierarchy. Thus, once such an individual has run out of the more usual associations (e.g. chair in response to table), unusual ones (e.g. elbow in response to table) come to mind.
The evidence that creativity is associated with both defocused attention and flat associative hierarchies suggests that creative individuals not only notice details others miss, but these details get stored in memory and are available later on. Ones’ encoding of a situation includes aspects that are less central to the particular concept that best categorizes it, features that may in fact make it defy straightforward classification as strictly an instance of one thing or another.
However, a considerable body of research suggests that creativity involves not just the ability to defocus and free-associate, but also the ability to focus and concentrate (Eysenck, 1995; Feist, 1999; Fodor, 1995; Richards et al., 1988; Russ, 1993)1. To quote Feist (1999): “It is not unbridled psychoticism that is most strongly associated with creativity, but psychoticism tempered by high ego strength or ego control. Paradoxically, creative people appear to be simultaneously very labile and mutable and yet can be rather controlled and stable” (p. 288). He noted that, as Barron (1963) put it: “The creative genius may be at once naïve and knowledgeable, being at home equally to primitive symbolism and rigorous logic. He is both more primitive and more cultured, more destructive and more constructive, occasionally crazier yet adamantly saner than the average person” (p. 224). How do we make sense of this seemingly contradictory depiction of the creative individual?
This paradox can be reconciled. There is an enduring notion that there are two kinds of thought, or that thought varies along a continuum between two extremes (Arieti, 1976; Ashby & Ell, 2002; Freud 1949; James, 1890/1950; Johnson-Laird, 1983; Kris, 1952; Neisser, 1963; Piaget, 1926; Rips, 2001; Sloman, 1996; Werner, 1948; Wundt, 1896). Thought is believed to vary along a continuum from an intuitive, ‘primary process’ or associative mode to a rule-based, ‘secondary process’ or analytic mode. Several researchers (Finke, Ward, & Smith, 1992; Gabora, 2000, 2002a, b; Howard-Jones & Murray, 2003; Martindale, 1995) appear to be converging on the following general picture. Associative thought is conducive to unearthing remote or subtle connections between items that share features or are correlated but not necessarily causally related. It may yield an idea or problem solution, though perhaps in a vague, unpolished form. Analytic thought, on the other hand, is conducive to hammering out relationships of cause and effect between items already believed to be related, as well as to the fine-tuning and manifestation of the creative work. Noting that a creative idea is generally considered to possess two main qualities—appropriateness and originality—Howard-Jones and Murray (2003) suggest that associative thought ensures originality while analytic thought ensures appropriateness. Finke et al. (1992) suggest that broadening the focus of attention might improve creativity and help overcome fixation—the mental state of individuals who are unable to move beyond a known problem solving approach to generate a new one (Jansson & Smith, 1991; Smith, 1995).
Note that to be constantly in a state of defocused attention, in which relevant dimensions of a situation do not stand out strongly from irrelevant ones, would be clearly impractical. It is only when one does not yet know what are the relevant dimensions—or when those assumed to be relevant turn out not to be—that defocused attention is of use. After the relevant dimensions have been found it is most efficient to focus on them exclusively. Indeed it has been shown that in stimulus classification tasks, psychological space is stretched along dimensions that are useful for distinguishing members of different categories, and shrunk along nonpredictive dimensions (Nosofsky, 1987; Kruschke, 1992). In ALCOVE, a computer model of category learning, only when activation of each input unit was multiplied by an attentional gain factor did the output match the behavior of human subjects (Kruschke, 1992; Nosofsky & Kruschke, 1992). Thus learning and creative problem solving involve not just (1) associating stimuli with outcomes, but also (2) shifts in attention that determine how one ‘parses’ conceptual space. A reparsing of conceptual space can be conducive to noticing connections between items not previously known to be related, as occurs in associative thought. However since associative thought is of little value in many of the routine tasks of daily life, and since it can lead one’s attention away from the ‘here and now’, it would be dangerous to have the capacity to enter it unless one had the ability to stop it. Associative thought would be of adaptive value only if the capacity for it were to have evolved side-by-side with the capacity to revert to a more analytic mode of thought if needed (i.e. if some pressing situation were to arise that demanded logical analysis and a quick response). Once the capacity to shift between analytic and associative modes of thought arose as needed, however, the capacity for creativity would be unprecedented. The explosion of creativity in the Middle/Upper Paleolithic has led to speculation that the capacity for contextual focus at this time (Gabora, 2003).
Martindale (1995) points out that something very much like a shift between associative and analytic modes of thought takes place in a Hopfield neural network (Hopfield, 1982). The concept of energy minimization from physics is applied to refer to the mutual constraint satisfaction amongst the nodes of a neural network. The extent to which nodes activate one another depends on the weights of the links connecting them, which varies according to a probabilistic function. The term temperature is used to refer to the degree of randomness in the weights. At a low temperature, nodes tend to activate only their adjacent neighbors, and to a predictable degree, while at a high temperature they behave more erratically. High temperature thus simulates a cognitive state conducive to associative thought whereas low temperature simulates a cognitive state conducive to analytic thought. A similar procedure was employed by Mitchell in an analogy making program (Mitchell & Hofstadter, 1989; Mitchell, 1993). However here temperature refers not to the degree of randomness but to the degree to which not just typical but atypical associations are evoked. The difference in approach is that the choice of nodes activated at a high temperature does not come from ‘out of the blue’ but reflects a genuine overlap of features between concepts, just not the most obvious ones. This is closer to the behavior of creative individuals in Mednick’s (1962) associative hierarchies studies, who gave ‘elbow’ in response to ‘table’, which has some relevance to ‘table’, just not as much as ‘chair’. The point is that the escape from fixation or ‘breaking out of a rut’ is accomplished not by injecting randomness but by capitalizing on subtleties in the associative structure of the network.
In sum, there is convergent evidence that creativity involves contextual focus: the ability to funnel or expand the field of attention, and thereby match where one’s mode of thought lies on the spectrum from associative to analytic to the situation one is in.
What is happening differently in the brain in associative thought versus analytic thought? To address this question we turn to the architecture of cognition. We take as a starting point some fairly well established characteristics of memory. Human memories are encoded in neurons that are sensitive to ranges (or values) of subsymbolic microfeatures (Churchland & Sejnowski, 1992; Smolensky, 1988). For example, one might respond to a particular shade of blue, or the quality of being ‘honorable’, or quite likely, something that does not exactly match an established term (Mikkulainen, 1997). Although each neuron responds maximally to a particular microfeature, it responds to a lesser extent to related microfeatures, an organization referred to as coarse coding. Not only does a given neuron participate in the encoding of many memories, but each memory is encoded in many neurons. For example, neuron A may respond preferentially to lines of a certain angle (say 90 degrees), while its neighbor B responds preferentially to lines of a slightly different angle (say 91 degrees), and so forth. However, although A responds maximally to lines of 90 degrees, it responds somewhat to lines of 91 degrees. The upshot is that storage of an item is distributed across a cell assembly that contains many neurons, and likewise, each neuron participates in the storage of many items (Hinton, McClelland, & Rummelhart, 1986). Thus, the same neurons get used and re-used in different capacities, a phenomenon referred to as neural re-entrance (Edelman, 1987). Items stored in overlapping regions are correlated, or share features. Memory is said to be content addressable; there is a systematic relationship between the state of an input and the place it gets encoded. As a result, episodes stored in memory can thereafter be evoked by stimuli that are similar or ‘resonant’ (Hebb, 1949; Marr, 1969).
This kind of distributed, content-addressable memory architecture is schematically illustrated in Figure One. Each vertex represents a possible microfeature, and vertices that are close together respond to microfeatures that are similar or related. Only a fraction of these possible vertices, those with circles on them, is maximally responded to by an actual neuron. Circles with black dots in them have at one point or another been the epicenter of some episode encoded in the memory. The large, diffuse region of whiteness indicates the region of memory activated by the current thought or experience. Note that even if a brain does not possess a neuron that would respond maximally to the particular microfeature that is most salient in some experience, because its representations are distributed across many neurons, the brain is still able to encode that experience.
Figure 1. A highly schematized drawing of a portion of a distributed memory. Each vertex represents a possible microfeature. Each small black ring represents a microfeature that is maximally responded to by an actual neuron. Rings with circles inside represent the epicenter of where a particular memory item was previously encoded. Large dashed gray circles represent groupings of neurons that commonly function as neural cliques. The most activated clique is indicated by long dashes. There are no activated neurds. White region indicates cell assembly activated by the current thought. In this simple illustration, the cell assembly is composed of only one neural clique containing two neurons, only one of which has previously been the epicenter of an encoded memory. For more details see text.
The fact that memory distributed and content-addressable is of critical importance for creativity. If it were not distributed, there would be no overlap between items that share subsymmbolic microfeatures, and thus no means of forging a connection between them. If it were not content-addressable, connections forged between items would not be semantically meaningful. The more detail with which items have been encoded in memory, the greater their potential overlap with other items, and the more retrieval routes for creatively forging relationships between what is currently experienced and what has been experienced in the past. Note that a computer memory is also content-addressable, but the one-to-one correspondence in a computer arises simply because each possible input is stored in a unique address in memory. Retrieval is thus simply a matter of looking at the address in the address register and fetching the item at the specified location. This is quite different from human memory, where content addressability is achieved through a combination of course coding and distributed representation. Since in a one-to-one content-addressable memory there is no overlap of representations, there is no means of creatively forging new associations based on newly perceived similarities.
An interesting question is: how much overlap must there be with respect to which microfeatures a particular neuron responds to, in order for creative connections to be made? Another way of asking this question is: how coarse coded must the memory be? At one extreme it could be not coarse coded at all, like a typical computer memory. If the mind stored each item in just one location as a computer does, then in order for an experience to evoke a reminding of a previous experience, it would have to be identical to that previous experience. And since the space of possible experiences is so vast that no two ever are exactly identical, this kind of organization would be fairly useless. But at the other extreme, in a fully distributed memory, where each item is stored in every location, the stored items interfere with one another. This phenomenon goes by many names: ‘crosstalk’, ‘superposition catastrophe’, ‘false memories’, ‘spurious memories’ or ‘ghosts’ (Feldman & Ballard, 1982; Hopfield, 1982; Hopfield, Feinstein, & Palmer, 1983).
The problem of crosstalk is solved by constraining the distribution region. One way to do this in neural networks is to use a radial basis function, or RBF (Hancock, Smith, & Phillips, 1991; Holden & Niranjan, 1997; Lu, Sundararajan, & Saratchandran, 1997; Willshaw & Dayan, 1990). Each input activates a hypersphere (sphere with more than three dimensions) of locations, such that activation is maximal at the center k of the RBF and tapers off in all directions according to a (usually) Gaussian distribution of width s. The result is that one part of the network can be modified without affecting the capacity of other parts to store other patterns. A spiky activation function means that s is small. Therefore only those locations closest to k get activated, but they are activated a lot. A flat activation function means that s is large. Therefore locations relatively far from k still get activated, but no location gets very activated.
This distinction between flat and spiky activation functions in neural networks is reminiscent of Mednick’s (1962) flat and steep associative hierarchies. Indeed it has long been known that items in memory are distributed across assemblies of nerve cells (Hebb, 1949; Marr, 1969) but the distributions are constrained. A given experience activates not just one neuron, nor every neuron to an equal degree, but activation is distributed across the members of an assembly. Recently it has been found that the cell assembly involved in the encoding of a particular experience is made up of multiple groups of collectively co-spiking neurons referred to as neural cliques (Lin et al., 2005, 2006). New techniques enabling their patterns of activation to be mathematically described, directly visualized, and dynamically deciphered, reveal that some cliques respond to situation-specific elements of an experience (e.g. where it took place), while others respond to characteristics of varying degrees of generality. These range from the type of experience (e.g. being dropped) to characteristics common to many types of experience (e.g. anything that is dangerous). Lin et al. depict this as a pyramid in which cliques that respond to the most context-specific elements are at the top of the pyramid, and those that respond to the most general elements are at the bottom.
This can be seen in Figure One. The degree to which any given region of memory is activated by the current thought or experience is indicated by the degree of whiteness. The white area thus defines the active cell assembly that is composed of one or more neural cliques, indicated by dashed gray circles. The neuron for which activation is maximal we call k, and activation decreases with distance from k. The further a neuron is from k, the less activation it not only receives from the current instant of experience but in turn contributes to the next instant of experience, and the more likely its contribution is cancelled out by that of other simultaneously active locations. Following neural network terminology, we say the broader the region affected by a given stimulus, the flatter the activation function, and the narrower the affected region, the spikier the activation function. It is possible that the hierarchical control proffered by the pyramidal structure of working memory acts in concert with the hierarchical control structure of the cerebellum (see Vandervert, in press).
Content addressability ensures that one naturally retrieves items that are related to the current experience. This is why the entire memory does not have to be searched or randomly sampled in order for one item to evoke another. It also means that there are as many routes to a reminding event as there are microfeatures by which they overlap; i.e. there is plenty of room for typical as well as atypical associations. It is because the size of the region of activated memory locations falls midway between the two extremes—not distributed and fully distributed—that one can generate a stream of coherent yet potentially creative thought. When one encounters a situation, or realizes something, that does not entirely fit or make sense with respect to ones’ internal model of the world, representations are recursively re-viewed in light of one another. Items previously encoded to neurons activated by the current thought provide ‘ingredients’ for the next thought. This next thought is slightly different, so it activates and retrieves from a slightly different region, and so forth, recursively. In Figure One, a single neuron in the activated region has been the epicenter of a previously encoded item, and it is only marginally activated. Therefore, the present thought is unlikely to evoke a reminding or retrieval of whatever was encoded there, and initiate a stream of thought.
Having reviewed the evidence for contextual focus and some of the basic characteristics of human memory, contextual focus will now be interpreted in terms of the architecture and dynamics of memory. Recall that the contextual focus hypothesis posits that creativity requires not just the capacity for both associative and analytic modes of thought, but the capacity to adjust the mode of thought to match the demands of the problem at a given instant. Knowing that creativity is associated with both conceptual fluidity on the one hand, and focus or control on the other, puts us in a good position to posit an underlying mechanism: the capacity to spontaneously and subconsciously adjust the spikiness of the activation function in response to the situation. Each successive instant of thought activates recruitment of more or fewer neuronal cliques, depending on the nature of the problem, and how far along one is in the process of solving it.
In a state of defocused attention, more aspects of a situation get processed, and activation function is flat, as in Figure 2. Unlike Figure 1, many previously encoded items are now stored in neurons near k. Given Lin et al.’s (2005, 2006) data, it seems very reasonable that what is going on here is neural cliques that respond to general or abstract aspects of this particular situation have been recruited. Since more neural cliques are active, another way to see this is that working memory is expanded.
Figure 2. Schematic representation of region of memory activated and retrieved from during the associative thought. The activated cell assembly is now composed of several neural cliques containing many neurons, a few of which have previously been the epicenter of an encoded memory. The most activated clique is indicated by long dashes. Neurds are indicated by alternating short and medium length dashes. For more details see text.
Note that some neural cliques do not fall within the activated region in Figure One but do fall within the activated region in Figure Two. They are the cliques that would not be included in a cell assembly if one were in an everyday, analytic mode, but would be included if one were in an associative mode. We will refer to them as ‘neurds’. Neurds respond to subsymbolic microfeatures that are marginally relevant to the current thought. It is important to note that there is no particular portion of memory where neurds reside. The subset of neural cliques that count as neurds is defined by context, and changes constantly. For each different situation one might encounter, from a first glimpse of a sunset to being midway through a calculus problem, a different group of neurds is involved.
A flat activation function increases the likelihood of retrieving or reconstructing items that were previously encoded to a given region, and thereby of ‘catching’ a representation that isn’t usually associated with the experience that evoked it. Because a neurd responds to the less superficial, more abstract aspects of the current thought, it spreads activation to items in memory that share its abstract conceptual structure, and can therefore ‘pull’ the next thought quite far from the one that preceded it while retaining a thread of continuity. So the network of concepts is not only penetrated more deeply, but also traversed more quickly, and there is greater potential for representations to ‘bleed’ into one another in ways they never have before.
Our examination of the architecture of memory suggests a simple and natural means by which insight could arise in a conceptual network. Before delving into it, it is prudent to look briefly at two other well-known hypotheses for how insight comes about.
A first possibility is that the process is random (Campbell, 1960; Simonton, 1999a, 1999b, in press). This view is debated at length elsewhere (Dasguptas, 2004; Gabora, 2005, in press; Weisberg in press) but it is worth mentioning that statements by creators to the effect that ‘the idea came from out of the blue’ are often interpreted as meaning that the idea was random. But the two are not equivalent; in fact, due to our having imperfect knowledge of all things including the workings of our own minds, a purely deterministic process could yield an outcome that appears to have come ‘out of the blue’. The idea that creativity is stochastic or random or ‘quasi-random’ (whatever that might mean) stems from an effort to describe creative thought as an evolutionary process, a worthy goal, for it would put us on the road toward a theoretical framework that unifies the psychological and social sciences as did Darwinism for the biological sciences. However the endeavor to apply natural selection to creative thought is fundamentally flawed because (amongst other things) natural selection is inapplicable to the description of change of state in processes where there is non-negligible inheritance of acquired characteristics. Indeed because of this, even the evolution of early life itself cannot be described by natural selection (Gabora, 2006; Vetsigian et al., 2006). (Elsewhere I have proposed that it is not ideas or artifacts but worldviews that evolve through culture, and that they evolve not through a Darwinian process but through a process akin to that of the very earliest forms of life (Gabora, 2004).
Natural selection was the inspiration for the genetic algorithm, a technique computer scientists use for searching through a space of possibilities. If natural selection is not the right algorithm, than perhaps creative thought involves some other kind of search algorithm for choosing from amongst a set of predefined alternatives (Simon, 1973; Weisberg, 1993, 2006a,b). However, creativity involves the manifestation of representations that were in no sense predefined, that come into existence through the interaction between a conceptual network and a context or situation. It is because of the difficulty of formally modeling such an interaction that many interested in creativity have been led to the research topic of how concepts adapt to different contexts, and gain or lose properties when they combine, as this topic appears to lie at the heart of the creative process, yet is manageable enough in scope that it lends itself to laboratory experiments and formal modeling.
One branch of this work (Aerts & Gabora, 2005a, 2005b; Bruza & Cole, 2005; Gabora & Aerts, 2002) has led to a third possible explanation of how insight arises: through the actualization of cognitive potential when a particular conceptual network interacts with a particular situation. Since this too is presented in (Gabora, 2005), the explanation here is restricted to those aspects that are crucial for the present argument. We saw how content addressability ensures that the entire memory does not have to be searched or randomly sampled in order for one item to evoke another. But does content addressability help out when one item evokes another that is not just similar to it, but related to it in a rarely noticed but profoundly useful or appealing way? Unless the experimental and theoretical evidence pointing to the existence of neurds is in error, the answer is yes. Recall how if the regions in memory where two distributed representations are encoded overlap then they share one or more microfeatures. They may have been encoded at different times, under different circumstances, and the correlation between them never explicitly noticed. But the fact that their distributions overlap means that some context could come along for which this overlap would be relevant or useful, and cause one to evoke the other.
If a state of a concept is not affected by a particular context it is said to be an eigenstate for this context. Otherwise it is a potentiality (superposition-like) state for this context, reflecting its susceptibility to change. When a particular concept (such as ‘zebra’) is in no way part of one’s current conscious experience (such as, most likely, your concept of ‘zebra’ before you read this sentence), it is in its ground state. A concept is always evoked in some context; one never experiences it in its raw or undisturbed ground state. The ground state is a state of potentiality in that there exists the possibility for it to manifest different ways given the various (internally or externally generated) contexts that could elicit it. (If not just one concept but the mind as a whole were in its ground state then one would be thinking of nothing at all.) A context actualizes a concept in such a way that it may accentuate or draw out some features and de-emphasize others. Whether a feature is accentuated or de-emphasized depends on the constellation of other concepts actualized at that instant. Thus the process of actualization through interaction with a context involves both a manifestation of potential and a loss of potential. How the mind manifests some aspects of its potentiality as a conscious thought reflects the particularities of the activated cell assembly at that instant, the context at that instant, and the interaction between them.
In the course of routine life, neurds are excluded from the activated cell assembly. Their time in the limelight comes when one has to break out of a rut. In an associative mode of thought, the context―whether it be a concrete stimulus or an abstract thought―activates a larger assembly, one that includes neurds. Recall that whether a particular neural clique is a neurd or not depends on how directly relevant the microfeature it responds to is to the situation at that instant. Neurds are specifically activated by the current context (although in a conventional mode of thought they would not be), but are not the elements of memory most activated by it. As usual, whatever has been previously encoded to all the activated neurons including the neurds merges with the context to generate the next instant of thought. But because whatever has been previously encoded to neurds can be quite different from the context that activated them, their potential to unite previously disparate clusters of concepts is exceptionally high.
We have examined the relationship between contextual focus and the structure of human memory. This synthesis will now be applied to the analysis of a genuine creative act. In keeping with the view that everyone is creative (Runco, in press), the creative act that we analyze is not an earthshaking achievement but a simple event in the life of an everyday person.
The situation that motivates the creative act is the following. Jane, a ski instructor, wants to build a fence so that she will feel safer and can let her dog run around in the yard. However, she cannot afford a new fence. We say that her worldview has a ‘gap’ because her model of reality does not present a solution to this problem. The gap would be filled by a free fence, but this does not exist (it is inconsistent with other elements of the world as represented by her worldview).
Jane paces the yard trying to solve the situation through a straightforward deductive process. Neural cliques that have encoded memories of particular fences, and that respond to the concept ‘fence’, are activated. Her inability to solve the problem rationally eventually leads to a spontaneous and subconscious defocusing of attention. She enters an associative mode of thought, and her activation function becomes flat, such that the associative structure of her memory is more widely probed. Characteristics of fence posts, such as ‘tall’, ‘skinny’, and ‘sturdy’, are still strongly activated, but now they become less tied to fences. As neurds get recruited, her memory begins to respond to not just the context-specific aspects but also the abstract, conceptual aspects of her situation i.e. moving further down Lin et al.’s (2006) feature-encoding pyramid. She starts thinking not just about different kinds of fences but different ways of safeguarding her property, and things that are fence-like.
Because memory is content-addressable (which as we saw earlier means that there is a systematic relationship between the content of an item and the locations in memory it activates), representations other than the concept ‘fence’ or memories of fences that have been encoded to these locations in the past come to mind. The neural cliques that respond to ‘tall’, ‘skinny’ and ‘sturdy’, now activated in the context of needing to build a fence, had previously encoded numerous memories of skis. These neural cliques spread activation to cliques that respond to more specific items that are ‘tall’, ‘skinny’ and ‘sturdy’, such as skis, resulting in a combining of the concepts ‘skis’ and ‘fence’ to give a new concept: a fence made of skis. ‘Ski fence’ has some properties unique to skis (e.g. bindings and curved tips) and some properties unique to fences (e.g. surrounds and protects property). This new ‘connection’ between ‘skis’ and ‘fence’ is not literally a connection but a distributed hypersphere of microfeatures that have never been activated together before as an ensemble.
Jane goes to a shed crammed full of old skis. Having hit upon this idea of using skis to build a fence, she must determine if it would really work in practice. Although in the short run a flat activation function is conducive to creativity, maintaining it would be impractical since the relationship between one thought and the next may be remote; thus a stream of thought lacks continuity. Access to obscure associations would at this point be a distraction. Thus, now that the framework of her idea has been painted in the broad strokes, she enters a more analytic mode by ‘decruiting’ the neurds, thereby narrowing the region of memory that gets activated. Thought becomes more logical in character because the activation function becomes spikier, thereby affording finer control; fewer locations release their contents to participate in the formation of the next thought. By focusing attention on the promising aspects of the idea (such as that skis are long and skinny and available) and ignoring irrelevant aspects (such as that skis have bindings) Jane figures out things like what to use as crossbeams and how to drive the skis into the ground. She thus settles on a workable solution to her problem.
In fact the situation is slightly more complex, because some aspects of adapting the idea of building a fence to using skis instead of fence posts probably require or lend themselves to a slightly more associative mode of thought. For example, perhaps white rocks at the base of the skis would not only help stabilize the skis but be suggestive of snow. By shifting back and forth along the spectrum from associative to analytic, the fruits of associative thought become ingredients for analytic thought, and vice versa. Notice how it was the nature of the gap (the unattainable ‘free fence’) that guided the entire process.
This paper has attempted to synthesize experimental and theoretical work on creativity and memory into a reasonable account of how the brain generates novel ideas. A long-held view is that there are two modes of thought: (1) primary process, or associative thought, which is conducive to unearthing similarity or relationships of correlation between items not previously thought to be related, and (2) secondary process or analytic thought, which is conducive to hammering out causal relationships between items already thought to be related. It has been suggested that analytic thought requires a state of focused attention, and associative thought requires a state of defocused attention. Creativity involves not just the capacity for both, but the capacity to spontaneously shift between them according to the situation. The hypothesis put forward here is based on recent neurophysiological work which indicates that the cell assembly is made up of neural cliques, some of which respond to situation-specific aspects of an episode, and others of which respond to general aspects. Neural cliques activated exclusively if the given situation is processed in an associative mode of thought were referred to as neurds. It is proposed that the shift between analytic and associative forms of thought occurs by spontaneously and subconsciously making the activation function flatter or spikier, thereby recruiting more or fewer neural cliques into the activated cell assembly. The contents of the activated cell assembly merge in the generation of the next instant of thought. Because what has been encoded previously to the neurds may be superficially quite different with the present situation but share some aspect of its deep structure, their contribution to the next instant of thought may yield an insightful solution to a problem. The shift to a more associative mode of thought conducive to insight is accomplished by recruiting neurds that respond to abstract or atypical subsymbolic microfeatures of problem or situation. Since more neural cliques are active, working memory is expanded. Since memory is distributed and content-addressable, this fosters remindings and the forging of creative connections to potentially relevant items previously encoded in those neurons. This may occur in collaboration with the hierarchical control structure of the cerebellum (see Vandervert, in press). One an insight has been found, one finds a way for it to be realized (given the constraints of the world as represented by one’s internal model of it) by focusing attention, increasing the spikiness of the activation function, and dropping the neurds from the activated cell assembly.
It is interesting to speculate as to the long-term consequences of the proclivity to readily shift into a defocused state of attention. Since more aspects of attended stimuli participate in the process of activating and evoking from memory, more neural cliques participate in the encoding of an instant of experience and release of ‘ingredients’ for the next. The more cliques activated, the more they can in turn activate, and so on; thus streams of thought last longer. In individuals who are inclined toward defocused attention and associative thought, if something does manage to attract attention, it will tend to be more thoroughly assimilated into the conceptual network, and more time taken to settle into any particular interpretation of it. New stimuli are therefore less able to compete with what has been set in motion by previous stimuli (which may possibly explain the stereotype of the absent-minded professor). Moreover, the fruits of this associative process can later serve as ingredients for an analytic process. Thus, it is predicted that, over time, individuals who are inclined to shift between associative and analytic thought will end up with a more multifaceted, fine-grained memory structure, although empirical research suggests that this may not be a domain-general effect but limited to the domain(s) they devote themselves to (Runco et al., 2006). Therefore gaps in the human knowledge base waiting to be connected are more apparent. The richer and more vulnerable (prone to change) a conceptual network is, the more likely it is to hit upon a problem in need of a solution, and the more potential to be creative.
Lin et al. (2005) claim “Conversion of activation patterns of these coding unit assemblies into a set of real-time digital codes permits concise and universal representation and categorization of discrete behavioral episodes across different individual brains.” Unfortunately, the large-scale recording technique they used with mice would probably not be suitable for human subjects. If it were, by comparing recordings of particularly creative and less creative subjects on creatively demanding tasks, the existence and behavior of neurds could in principle be scientifically verified and investigated. Neurds are generally withheld from participating in a stream of thought, but when they do come out they come out with a vengeance, and their contribution is sometimes not readily forgotten.
I would like to thank Mark Runco and the reviewers for suggestions on the paper, and acknowledge the support of Foundation for the Future.
Aerts, D. & Gabora, L. (2005). A state-context-property model of concepts and their combinations I: The structure of the sets of contexts and properties. Kybernetes, 34(1&2), 167-191. [quant-ph/0402207]
Aerts, D. & Gabora, L. (2005). A state-context-property model of concepts and their combinations II: A Hilbert space representation. Kybernetes, 34(1&2), 192-221. [quant-ph/0402205]
Arieti, S. (1976). Creativity: The magic synthesis. New York: Basic Books.
Ashby, F. G., & Ell, S. W. (2002). Single versus multiple systems of learning and memory. In J. Wixted & H. Pashler (Eds.) Stevens’ handbook of experimental psychology: Volume 4 Methodology in experimental psychology. New York: Wiley.
Barron, F. (1963). Creativity and psychological health. Princeton, NJ: Van Nostrand.
Bowers, K. S. & Keeling, K. R. (1971). Heart-rate variability in creative functioning. Psychological Reports, 29, 160-162.
Bruza, P. & Cole, R. (2005). Quantum logic of semantic space: An exploratory investigation of context effects in practical reasoning. In (S. Artemov, H. Barringer, A. S. d'Avila, Garcez and L. C. Lamb and J. Woods, Eds.) We will show them: Essays in honour of Dov Gabbay. College Publications, 1, 339-361.
Churchland, P. S. & Sejnowski, T. (1992). The computational brain. Cambridge MA: MIT Press.
Dasguptas, S. (2004). Is creativity a Darwinian process? Creativity Research Journal 16(4), 403-413.
Dewing, K. & Battye, G. (1971). Attentional deployment and non-verbal fluency. Journal of Personality and Social Psychology, 17, 214-218.
Dietrich, A. (2004). The cognitive neuroscience of creativity. Psychonomic Bulletin & Review, 11(6), 1011-1026.
Dykes, M. & McGhie, A. (1976). A comparative study of attentional strategies in schizophrenics and highly creative normal subjects. British Journal of Psychiatry, 128, 50-56.
Edelman, G. (1987). Neural darwinism: The theory of neuronal group selection. New York: Basic Books.
Eysenck, H. (1995). Genius: The natural history of creativity. Cambridge: Cambridge University Press.
Feist, G. J. (1999). The influence of personality on artistic and scientific creativity. In R. J. Sternberg (Ed.) Handbook of creativity. Cambridge UK: Cambridge University Press.
Feldman, J. & Ballard, D. (1982). Connectionist models and their properties. Cognitive Science, 6, 204-254.
Finke, R. A., Ward, T. B. & Smith, S. M. (1992). Creative cognition: Theory, research and applications. Cambridge MA: MIT Press.
Fodor, E. M. (1995). Subclinical manifestations of psychosis-proneness, ego-strength, and creativity. Personality and Individual Differences, 18, 635-642.
Freud, S. (1949). An outline of psychoanalysis. New York: Norton.
Gabora, L. (2000). Toward a theory of creative inklings. In (R. Ascott, Ed.) Art, Technology, and Consciousness, Intellect Press, Bristol, UK, 159-164.
Gabora, L. (2002a). Cognitive mechanisms underlying the creative process. In (T. Hewett & T. Kavanagh, Eds.) Proceedings of the Fourth International Conference on Creativity and Cognition, October 13-16, Loughborough University, UK, 126-133.
Gabora, L. (2002b). The beer can theory of creativity. In (P. Bentley & D. Corne, Eds.) Creative Evolutionary Systems, Morgan Kauffman, San Francisco CA, 147-161.
Gabora, L. (2004). Ideas are not replicators but minds are. Biology & Philosophy, 19(1), 127-143.
Gabora, L. (2005). Creative thought as a non-Darwinian evolutionary process. Journal of Creative Behavior, 39(4), 65-87. [nlin.AO/0411057]
Gabora, L. (2006). Self-other organization: Why early life did not evolve through natural selection. Journal of Theoretical Biology. (DOI: 10.1016/j.jtbi.2005.12.007)
Gabora, L. & Aerts, D. (2002). Contextualizing concepts using a mathematical generalization of the quantum formalism. Journal of Experimental and Theoretical Artificial Intelligence, 14(4), 327-358. [quant-ph/0205161]
Goldman-Rakic, P.S. (1992, September). Working memory and the mind. Scientific American, 267, 111-117.
Hancock, P. J. B., Smith, L. S. & Phillips, W. A. (1991). A biologically supported error-correcting learning rule. Neural Computation, (2), p. 201-212.
Hebb, D. (1949). The organization of behavior. New York: Wiley.
Hinton, G., McClelland, J. & Rummelhart, D. (1986). Distributed representations. In D. Rummelhart, & J.McClelland (Eds.) Parallel distributed processing. Cambridge MA: MIT Press.
Holden, S.B. & Niranjan, M. (1997). Average-case learning curves for radial basis function networks. Neural Computation, 9(2), p. 441-460.
Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences (Biophysics), 79(8), 2554-2558.
Hoppe, K. & Kyle, N. (1991). Dual brain, creativity, and health. Creativity Research Journal, 3, 150-157.
Hopfield, J. J., Feinstein, D. I. & Palmer, R. D. (1983). 'Unlearning' has a stabilizing effect in collective memories. Nature, 304, 159-160.
Howard-Jones, P. A. & Murray, S. (2003). Ideational productivity, focus of attention, and context. Creativity Research Journal, 15(2&3), 153-166.
James, W. (1890/1950). The principles of psychology. New York: Dover.
Jansson, D. G. & Smith, S. M. (1991). Design fixation. Design Studies, 12, 3-11.
Jausovec, N. & Bakrecevic, K. (1995). What can heart rate tell us about the creative process? Creativity Research Journal, 8(1), 11-24.
Johnson-Laird, P. N. (1983). Mental models. Cambridge MA: Harvard University Press.
Kanerva, P. (1988). Sparse, distributed memory. MIT Press.
Kris, E. (1952). Psychoanalytic explorations in art. New York: International Universities Press.
Kruschke, J. (1992). ALCOVE: an exemplar-based connectionist model of category learning. Psychological Review, 99, 22-44.
Lin, L., Osan, R., Shoham, S., Jin, W., Zuo, Wenjun & Tsien, Joe Z. (2005). Identification of network-level coding units for real-time representation of episodic experiences in the hippocampus, Proceedings of the National Academy of Sciences, 102, 6125-6130.
Lin, L., Osan, R., & Tsien, J. Z. (2006). Organizing principles of real-time memory encoding: Neural clique assemblies and universal neural codes. Trends in Neurosciences, 29(1), 48-57.
Lu, Y.W., Sundararajan, N. & Saratchandran, P. (1997). A sequential learning scheme for function approximation using minimal radial basis function neural networks. Neural Computation, 9(2), p. 461-478.
Marr, D. A. (1969). Theory of the cerebellar cortex. Journal of Physiology, 202, 437-470.
Martindale, C. (1977). Creativity, consciousness, and cortical arousal. Journal of Altered States of Consciousness, 3, 69-87.
Martindale, C. (1995). Creativity and connectionism. In S. M. Smith & T. B. Ward, & R. A. Finke (Eds.) The creative cognition approach (pp. 249-268). Cambridge MA: MIT Press.
Martindale, C. (1999). Biological bases of creativity. In R. J. Sternberg (Ed.) Handbook of creativity. Cambridge UK: Cambridge University Press.
Martindale, C. & Armstrong, J. (1974). The relationship of creativity to cortical activation and its operant control. Journal of Genetic Psychology, 124, 311-320.
Martindale, C. & Hasenfus, N. (1978). EEG differences as a function of creativity, stage of the creative process, and effort to be original. Biological Psychology, 6, 157-167.
Mednick, S. (1962). The associative basis of the creative process. Psychological Review, 69, 220-232.
Mendelsohn, G. A. (1976). Associative and attentional processes in creative performance. Journal of Personality, 44, 341-369.
Mikkulainen (1997). In (A Brown, Ed.) Natural language processing with subsymbolic neural networks. Neural network perspectives on cognition and adaptive robotics. Bristol UK, Philadelphia: Institute of Physics Press.
Mitchell, M. and Hofstadter, D. R. (1989). The role of computational temperature in a computer model of concepts and analogy-making. In Proceedings of the Eleventh Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Lawrence Erlbaum Associates.
Mitchell, M. (1993). Analogy-making as perception: A computer model. Cambridge, MA: MIT Press.
Miyake, A., & Shah, P. (Eds.). (1999). Models of working memory: Mechanisms of active maintenance and executive control. New York: Cambridge University Press.
Neisser, U. (1963). The multiplicity of thought. British Journal of Psychology, 54, 1-14.
Nosofsky, R. (1987). Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychlogy: Learning, Memory and Cognition, 13, 87-108.
Nosofsky, R. & Kruschke, J. (1992). Investigations of an exemplar-based connectionist model of category learning. In D. L. Medin (ed.) The psychology of learning and motivation, 28, 207-250.
Piaget, J.(1926) The Language and Thought of the Child. Harcourt Brace, Kent UK.
Richards, R. Kinney, D., Lunde, I., Benet, M., & Merzel, A. (1988). Creativity in manic depressives, cyclothymes, their normal relatives, and control subjects. Journal of Abnormal Psychology, 97, 281-289.
Rips, L. (2001). Necessity and natural categories. Psychological Bulletin, 127(6), 827-852.
Runco, M. A. (2004). Creativity. Annual Review of Psychology, 55, 657-687.
Runco, M. A. (in press). Everyone is creative. In R. S. Sternberg, E. Grigorenko, & J. Singer (Eds.), Who is creative? Washington, DC: American Psychological Association.
Runco, M. A., Dow, G. & Smith, W. (2006). Information, experience, and divergent thinking: An empirical test. Creativity Research Journal 18(3), 269-277.
Russ, S. W. (1993). Affect and creativity. Hillsdale NJ: Erlbaum.
Simon, H. A. (1973). Does scientific discovery have a logic? Philosophy of Science, 40 (December), 471-480.
Simonton, D. K. 1999a. Origins of genius: Darwinian perspectives on creativity. New York: Oxford.
Simonton, D. K. 1999b. Creativity as blind variation and selective retention: Is the creative process Darwinian? Psychological Inquiry 10, 309-328.
Simonton, D. K. (in press). The creative imagination in Picasso’s Guernica sketches: Monotonic improvements or nonmonotonic variants? Creativity Research Journal.
Sloman, S. (1996). The empirical case for two systems of Reasoning. Psychological Bulletin, 9(1), 3-22.
Smith, S. M. (1995). Fixation, incubation, and insight in memory and creative thinking. In S. M. Smith & T. B. Ward, & R. A. Finke (Eds.) The creative cognition approach (pp. 135-156). Cambridge MA: MIT Press.
Smith, G. J. W. & Van de Meer, G. (1994). Creativity through psychosomatics. Creativity Research Journal, 7, 159-170.
Smolensky, P. (1988). On the proper treatment of connectionism. Behavioral and Brain Sciences, 11, 1-43.
Weisberg, R. W. (in press). We are all partly right: Comment on Simonton. Creativity Research Journal.
Vandervert, L. R., Schimpf, P. H. & Liu, H. (in press). How working memory and the cerebellum collaborate to produce creativity and innovation. Creativity Research Journal.
Vetsigian, K., C. Woese, Goldenfeld, N. (2006). Collective evolution and the genetic code. Proceedings of the National Academy of Science USA, 103, 10696-10701.
Weisberg, R. W. (1993). Creativity: Beyond the myth of genius. New York: Freeman.
Weisberg, R. W. (2006a). Creativity: Understanding innovation in problem solving, science, invention, and the arts. Hoboken, NJ: John Wiley.
Weisberg, R. W. (2006b). Expertise and reason in creative thinking: Evidence from case studies and the laboratory. In J. C. Kauffman & J. Baer (Eds.). Creativity and reason in cognitive development. New York: Cambridge.
Werner, H. (1948). Comparative psychology of mental development. New York: International Universities Press.
Willshaw, D. & Dayan, P. (1990). Optimal plasticity from matrix memories: What goes up must come down. Neural Computation, 2(1), 85-93.
Wundt, W. (1896). Lectures on human and animal psychology. New York: MacMillan.
1. Interestingly, there is also evidence of an association between creativity and high variability in physiological measures of arousal such as heart rate (Bowers & Keeling, 1971; Jausovec & Bacrecevic 1995), spontaneous galvanic skin response (Martindale, 1977), cortical activity (Martindale & Armstrong, 1974) and EEG alpha amplitude (Hoppe & Kyle, 1991; Martindale & Hasenfus, 1978; Martindale, 1999).