What is Cognition?

snoopy2In 1981 I was an eager post-doctoral fellow, learning to record place cell’s in Jim Ranck’s lab and beginning to understand John O’Keefe and Lynn Nadel’s “Cognitive Map” theory of the hippocampus. One afternoon, while I had a rat in the maze and watched traces of action potentials sweep by on the oscilloscope, Jim Ranck looked over my should and said …

“This is terrific! Place cells are the gateway to understanding how the brain produces cognition.”1

This was both inspirational and opaque. On the one hand, I understood that our project was new and exciting. On the other hand, I wasn’t sure what the word “cognition” really meant or why hippocampal place cells were the path to understanding cognition. Why the hippocampus and not, for example, the visual cortex?

Looking up conventional definitions of “cognition” was of little help. Most are not definitions; rather, a list of properties or functions. For example, wikipedia says …

“Cognition is a group of mental processes that includes the attention of working memory, producing and comprehending language, learning, reasoning, problem solving, and decision making.”2

snoopy1Eminently unsatisfying. For decades the issue festered. About a month ago I decided to re-evaluate the question. What follows is a proposed re-definition of “cognition.” I start by separating cognition into two components: pure cognition and embodied cognition. I find this approach helpful in several ways. First, it is a definition, not a list of attributes. Second, it separates the work of the brain into categories that seem helpful. The definitions are followed by a rough-draft of a cognitive theory.

Pure Cognition

Definition: Cognitive processes are brain-based mental operations capable of operation without feedback from the world3

In other words, cognitive operations are high-level mental operations that do not require immediate sensory input or motor output.

Processes that qualify as “pure cognitive” include imagination, episodic memory, what-if scenarios of imagined behavior, planning, creative artistic processes, day dreaming, dreams, language, math, and hallucinations.

Processes that are not purely cognitive include perception, motor action, and (especially) sensory-motor flow.

In that pure cognitive processes cannot use real-world feedback as a crutch in representation, they are totally dependent on internal representations. Rather than relying on a sensory-motor assistance, a representation of the world in a pure cognitive task must be completely brain-generated. The cost of producing a complete, internal representation results in a large benefit: the internal representation is a model of the world (or self-world relations). Having a virtual world permits playing out imagined action scenarios, without the downsides of real-world timing constraints or failures.

Let’s contrast two navigational problem-solving scenarios. Imagine an animal sitting at a location and attempting to go to a goal location.

  • Non-cognitive navigation. The animal can see features of the goal, and begins movement towards the goal. A slight modification of this scenario is if the animal takes a route marked by a series of seen familiar landmarks that ultimately lead to the goal.
  • Cognitive Navigation. The animal cannot see the goal, but has a brain-based representation of the layout of the environment. This representation is similar to a map-like overhead view, and does not represent immediate sensory input or anything the animal has ever seen. Using this representation, the animal sits in its current location, imagines the goal location, and computes an optimal route to the goal. Finally, the animal transforms the map representation into its current view of the world and initiates a path towards the goal.

One could argue that the above second example is not pure cognition, that an animal or human may use a combination of internal representation and immediate sensory information to plan an optimal route. To make the cognitive process pure, imagine planning a route with your eyes closed, with or without a map-like representation. Clearly doable. The point is that pure cognitive operations exist and require internal representations.

Pure Cognition Versus Embodied Cognition

The concept of embodied cognition has been laid out by Andy Clarke, Alva Noë and others4. In embodied cognition, the sensory and physical constraints of the world aid an animal (or human) in optimizing behavior. These real-world aids are termed “affordances”.5 Affordances are action aids produced by the environment. They are effective because the environment is organized and predictable. Noë, for example, argues that “consciousness” is not in our heads, it is in the interaction of our brains, ourselves and the world. Note the contrast with “pure cognition” where external support mechanism are not essential; in place of action aids, the brain must substitute a complete representation. My sense is that embodied cognition and pure cognition are complementary operations. A second notion is that during action the brain does not need a complete, internal representation of the environment: the constraints of self-and-environment are part of the representation. JJ Gibson’s cover picture from his 1979 book captures some of the essence. During flight, the bird’s brain need not compute all pieces of sensory data and derive optimal flight instructions. Rather, the birds body and brain interact with simplified patterns, such as optic flow, to maintain smooth flight. Indeed, many flight adjustments are made in bird-wing accommodations to shifting air currents, and are not processed by the brain at all.

Embodied cognition roughly corresponds to “flow”, the almost thoughtless, smooth process of self-world interaction. For animals I think of Gibson’s drawing of a bird’s smooth flight. At the human level I think of a high-level athlete, such as a downhill skier, operating in complete control. Or a highly-trained surgeon smoothly operating on the heart. In the human examples, consciousness is engaged, but only at the highest level. Many critical behavioral elements are highly practiced, efficient, fast and “automatic”. While embodied cognition involves both perception and action, the clearest examples have active body movement.

Rough Draft of a Theory

In contrast, Pure Cognition is characterized by stasis. The body is doing almost nothing. In place of sensory and motor constraints on cognition, frameworks, or representations, must be internally generated. Pure Cognition is thought, absent of immediate sensation or action.

Pure Cognition is useful because it is NOT connected to action. Imagine yourself at a choice-point, a split in the road where there are four possible routes. You have a goal, but you haven’t been at this point on the road before and are unsure which route to take. You could make an arbitrary selection of one of the routes. On the positive side, there is a one-in-four chance that this will work. On the other hand, on the high likelihood of failure, you will return to the choice point and continue making arbitrary choices until you find a working route. Pretty inefficient.

Pure Cognition gives you the option of standing still and thinking about it. If you have the mental capacity to imagine what you will see as you walk along one of the paths, without actually walking on the path, you may, in your mind, see that it does or does not lead to the goal. If it fails, you could continue this process until an optimal path is found. This second mechanism is faster than the first, energy efficient and doesn’t expose you to potential danger. It’s termed vicarious-trial-and-error8. Vicarious (imagined) perception and action is the domain of pure cognitive systems; it is dependent on complete internal representations; although characterized by inaction, it ultimately produces efficient action.

Origins of Pure Cognition

If Pure Cognition has such advantages, do all animals have it? What are the costs? Embodied Cognition is evident in virtually all animals including small-brained creatures. A house fly that jumps and flies quickly to avoid a slap is exhibiting embodied cognition. In contrast, it’s difficult to find examples of Pure Cognition in small brained creatures. Perhaps it requires a large brain. While clearly common in humans, other mammals appear to use pure cognition less frequently. Outside of birds and mammals, clear examples are hard to find. I propose the following, highly simplistic evolutionary scenario.

Pure cognition evolved from embodied cognition. Initially, there are special purpose embodied processes, such as the fly avoiding the slap, fixed and inflexible. These responses, similar to the affordances of a bird wing in flight, do not require a central representation, and may be the product of separate, discrete processors. Later, central representations begin to develop, which permit singular decisions, along with generalized, learned and flexible responses. These early central representations still rely on immediate sensory inputs and behavioral responses. Still later, the central representations reach a capacity where they can operate with, or without sensory input, and with or without, immediate behavioral responses. Our primate ancestors had modest pure cognitive capacity, largely used for cooperative game hunting. As brains evolved and grew, pure cognitive capacity enlarged6. Further, while pure cognition may have originally be restricted to certain domains, perhaps spatial, expanded cognitive capacity released pure cognition from domain specificity. While pure gestural communication may not depend on pure cognition, symbolic language does. If the utterance “dog” represents an unseen dog, producing and understanding the utterance requires pure cognition. The sound is a symbol for the unseen animal. Furthermore, if the sound represents a group of animals, rather than a specific dog (“dog” rather than “Rufus”) an abstraction is being represented.

Cognitive Frameworks. Each cognitive operation, whether embedded or pure, occurs within a framework — or context. An example of a framework is the visual field. An object can be seen in the upper left portion of the visual field if it is real or imagined — as in “imagine an apple in the upper left of your visual field”. Kant proposed that there were two innate frameworks, space and time; termed a priori intuitions. As with cognition, cognitive frameworks can be embedded in the real or abstract. Abstract frameworks, (described briefly in footnote 8) give added power to cognition and are critical to some of the functions listed below. 9

Let’s return to the list of pure cognitive functions, and discuss core functions and specific examples briefly.

Core Functions:

  • What-if scenarios are the core of problem solving and creativity. Exploring imagining behaviors, in series, for an optimal solution to a problem. I believe this process is the core of all creative acts, and the greatest benefit provided by pure cognition.
  • Creativity is produced by the “what-if” scenarios described above. The specifics of creativity derive from the specific associations that produce each imagined behavior. (The nature of associations is a separate topic. The “next association” in a series will be determined, at least in part, by the experience and endowment of the individual.)
  • Symbolic representations may be a stretch, but a critical one. The notion: a symbolic representation is laid out in a framework that does not correspond to a real-world framework. For example, a graph is laid out in a virtual space. We have a cognitive capacity for space for behavioral and navigational purposes. Although a graph does not correspond to a behavioral space, the capacity to graph derives emerges from brain structures that use real-world space to construct effective spatial behavior. (perhaps this is why graphs above 3d are ineffective).
  • Language is a particular type of symbolic representation. The basic structure of language is a linear, temporal structure, the sentence. The sentence likely derives from “natural” time-lines of behavioral sequences, but can be used for radically enlarged purposes. A symbolic representation requires a virtual structure. Once there is the basic sentence structure, any sorts of objects, unconstrained by real things, can be placed in the structure.
  • Mathematical representations (algebraic, graphic) are abstract and symbolic. Algebraic representations are cousins to the sentence, laid out in linear time. Graphical representations are described above.

Specific Functions (built from core functions):

  • Imagination. If I “imagine a beach”, no current sensory data nor motor output support my mental image.
  • episodic memory The mental image of memory of an event (e.g. first day in school) does not require sensory input or motor output.
  • Planning creative artistic projects see “epilog” below.
  • Personal Timeline The concept that your life has a beginning (birth), landmarks (episodic memories) and an end (grim reaper).
  • Historical Timelines Cosmological, geological, life-on-earth, civilization, biblical, national, etc.
  • Day dreaming
  • Hallucinations

Is the Hippocampus a Cognitive Map?

This post started with a quote from Jim Ranck, my post-doctoral advisor. Jim asserted that the study of place cells was the gateway to understanding cognition. The proposition was strong at the time and is stronger today. Although, during normal navigation hippocampal place cell firing reflects the animal’s current environment, the affordances of the current environment are not critical. We now know that place cell firing occurs in the dark, during sleep, during moments at rest, and appears to exhibit the vicarious-trial-and-error processing indicative of pure cognition. Although the process is not finished, I believe Jim was correct. Hippocampal place cell firing is a gateway to understanding the mechanisms of cognition.


I’ll conclude with with an ancient description of pure cognition, as translated by Stephen Mitchell in The Second Book of the Tao:

Ch’ing the master woodcutter carved a bell stand so intricately graceful that all who saw it were astonished. They thought that a god must have made it

The Marquis of Lu asked, “How did your art achieve something of such unearthly beauty?”

“My Lord,” Ch’ing said, ‘I’m just a simple wood-worker—I don’t know anything about art. But here’s what I can tell you. Whenever I begin to carve a bell stand, I concentrate my mind. After three days of meditating, I no longer have any thoughts of praise of blame. After five days, I no longer have any thoughts of success for failure. After seven days I’m not identified with a body. All of my powers are focused on the task; there are no distractions. At that point, I enter the mountain forest. I examine the trees until exactly the right one appears. If I can see a bell stand inside it, the real work is done, and all I have to do is get started. Thus, I harmonize inner and outer. That’s why people think that my work must be superhuman”10

1Jim’s quote is a paraphrase, but close enough. Date is also an estimate.
2Checking for definitions from more reputable people and sources not helpful. For example, William James, Edward Tolman or definitions of cognitive psychology.
3From hereon out, the word “cognition” or the words “pure cognition” will assume this definition.
4Andy Clark Whatever next? Predictive brains, situated agents, and the future of cognitive science 
A. Clark and D. Chalmers, The Extended Mind 1998 
Alva No&euml: Out of our Heads 
5Affordances, JJ Gibson, The Ecological Approach to Visual Perception, 1979
6Pure cognition may have been a driving forced for brain enlargement, a consequence (a spandrel), or a combination of the two.
7 O’Keefe and Nadel’s “cognitive map” is derived from Kant, but specialized, a subset of Kant’s spatial a priori. Kant’s space includes visual fields, egocentric space and personal space; O’Keefe and Nadel’s does not; theirs is restricted to “allocentric space”.
8“Vicarious Trial and Error” (VTE) is a term that Muenzinger and Tolman in the 1930s used to describe the rat’s conflict-like behavior before responding to choice. More recently, Redish and colleagues have investigated VTE behavior in parallel with recording hippocampal neurons, and reported that hippocampal neurons fired as if proceeding along imagined paths. Muenzinger KF (1938). (Vicarious trial and error at a point of choice. I. A general survey of its relation to learning efficiency. Journal of Genetic Psychology 53: 75– 86. Redish, A. David (2013-06-19). The Mind within the Brain: How We Make Decisions and How those Decisions Go Wrong (Kindle Locations 8350-8351). Oxford University Press, USA.)
9 Real and Abstract Frameworks. In a manner similar to the cognitive dichotomy described above, I propose that frameworks can be divided into two categories, real and abstract. Initially, the time and space frameworks are “real” in that each can map onto immediate sensory-motor features of the world-and-self. An example of the “real” use of the spatial framework is the current visual field. A real temporal framework is current working memory (the set of events that transpired in the previous few moments). The proposal is that “real” each of the core “real” frameworks can be abstracted for non-real applications. A graph is an abstraction of the spatial framework. A historical timeline, a narrative, or a sentence are abstractions of the temporal framework.
10The Second Book of the Tao, translated by Stephen Mitchell. 2009, Penguin Press, NY.


15 thoughts on “What is Cognition?

  1. Hi John – thank you for an engaging read. If I may I would like to challenge your dichotomy between ‘purely cognitive’ and ‘not purely cognitive’ processes.

    I agree with you on the former – it appears that cognitive processes can be engaged without immediate sensory input. However I’m not clear on your position for the latter. In your ‘non-cognitive navigation’ example, are you suggesting that the animal is only utilising sensory input and no longer engaging internal cognitive processes/representations?

    I don’t think this is the case: this task appears to involve the interface between sensory input and multiple cognitive processes/representations. Firstly, the animal has a goal, suggesting they have some internal motivations. Secondly, they identify features of their goal, suggesting they have directed their attention to relevant information in their environment and compared their perceptions to some internal representation of that goal. Finally, the animal proceeds to the goal by utilising ‘familiar landmarks’ – which clearly suggests a role for memory.

    Whilst the animal has certainly been using immediate sensory input here (which is the key difference with your ‘cognitive navigation’ example), they must have been using it in conjunction with cognitive processes and representations. How could the external input be useful to the animal otherwise? If sensory inputs did not interact with internal representations, the animal would just be faced with “one great blooming, buzzing confusion” (James) and be unable to reach the goal.

    What do you think of this (less eloquent) distinction: ‘cognition without immediate sensory input’ and ‘cognition with immediate sensory input’ ? The implication being that similar cognitive processes/representations are engaged in both situations and the only difference being the additional role of immediate sensory input. In other words, whereas you appear to characterise the key difference between your two navigation scenarios by the absence/presence of cognition, I would instead say they are only differentiated by the absence/presence of environmental input.

    I hope I have interpreted your article correctly and I look forward to hearing your thoughts.

    Best wishes,

    • Hi Tom,
      Thanks for your engaged read. I’ll have to take time to digest it further and give more complete feedback. Although I mostly agree, there are two reasons I would hesitate to use the dichotomy “cognition with sensory” and “cognition without sensory” (although I agree it would be easier to understand). First, there are a numerous people interested in “embodied cognition”. The term is theirs. Some seem to feel that embodied cognition is all there is to cognition. I’d like to hear their responses. Second, “embodied cognition” is more than sensory; minimally, it’s sensory-motor. Its about the person (agent) interacting with the world. This is much of the strength of the idea.

      More comments later.

      — John

  2. How would you describe it when a pilot is called (while on vacation at a secluded spot in the country – no tech equipment at all other than a phone) to talk a non-pilot through landing a complicated plane such as a Boeing 777?

    • Interesting scenario. The pilot on vacation is mostly using “Pure Cognition”, but in an interesting way: to explore what may normally be a process of “flow” when he’s flying a 777. A bit similar to a retired athlete remembering great moments. Or more similar to the retired athlete using memories of great moments to coach a novice. Since the pilot (or coach) is relying entirely on internally generated information, its “Pure Cognition”.

      • Just to try to make the boundaries of stuff more fuzzy, can you see this pilot, with his very strong skills and knowledge of the 777, operating it while blindfolded if there is a co-pilot to read the gauges to him, but nothing else? This is such that all visual queues for the pilot come via voice of the copilot.

        If you can imagine the pilot working blind, how much of the ‘pure cognition’ is just cognition. If an interface were built so the pilot’s thoughts would operate the 777’s equipment such that the pilot need not move a muscle from take off to landing. Is it pure cognition?

        The nex t small step is to see our senses and muscles as the interface to the 777. When then does pure cognition happen? When we can see our senses as simple a remote interface to the world for our brains, which are in the real world but not part of it beyond being entrapped in the bodies we see, then what becomes of pure cognition?

    • Bill,
      Yes, dreaming fits. Not far from Hallucinating. But the “rational operator” is asleep on the job. But the specific contents of dreams are brain creations.

      • So that, combined with other considerations, pretty much implies that pure cognition occurs in virtually all mammals. The next question would be whether it occurs in any other groups, for example birds.

      • yes, I’d guess all mammals. I read recently that humans spend half their waking hours day-dreaming (I’ll check out the reference). My guess is that humans are not unique in having “pure cognition”, but the list of “pure cognitive” activities in people is extensive and unique. One thing I’ve wondered about is whether our-type-of-consciousness requires both pure cognition and a self-awareness about pure cognitive tasks. For its only during pure cognitive tasks that our mind’s reality is separate from reality. A kind of dualism. So, for example, when a dog dreams, does it confuse dream-reality with reality, or, during conscious states, does it say to itself “I was only dreaming”?

  3. John,

    Reading this reminded me of a moment last week when I had been landscaping below a raised deck, thinking while I worked about the NEURO.tv episode with the Mosers talking about navigation. I needed a tool, so I stood up and looked at the deck. It seemed I was observing my own brain map from outside my body, with a vivid sense of the place where I was (tight quarters surrounded by trees and shrubs and rocks) and the place I wanted to be (a couple of feet higher, on a narrow ledge with lots of glass behind it). The term “next-place cells” popped into my head, and I spent an obtrusive amount of time mentally moving my identification between where I was (pondering the “next place”) and where I wanted to be (remembering the previous place). Making the move would require a “leap” between the two points, during which I could only hope my planning had been correct and I would land gracefully.

    So I’m trying to fit that experience into your framework:
    Processes that qualify as “pure cognitive” include imagination, episodic memory, what-if scenarios of imagined behavior, planning,

    Processes that are not purely cognitive include perception, motor action, and (especially) sensory-motor flow.

    In that pure cognitive processes cannot use real-world feedback as a crutch in representation, they are totally dependent on internal representations.

    Having a virtual world permits playing out imagined action scenarios, without the downsides of real-world timing constraints or failures.

    I had direct visual perception of both locations, but the motor action and feedback were purely internal (until the leap). The real-world constraints and possible failures were vividly imagined… Perhaps this was not “navigation” in your sense, or maybe it was the smallest step in a navigation process. I could “see features of the goal”, but I did not begin movement until I’d imagined it successfully.

    Are you suggesting non-cognitive navigation is performed via embodied cognition? That learning to ski involves transitioning from a pure cognitive representation of the slopes to an embodied, non-cognitive representation?

    My “leap” was the first time I had ever made the transition between those two places. After performing it once, I can’t get back to the feeling I had while imagining it. My body knows I can do it successfully (though not yet confidently). So I’d say there is both a dichotomy between pure cognitive and embodied cognitive, and a gradual progression toward deeper embodiment once the body has made the leap once.

    The very real threats of body vs. wood and glass made perception a critical part of my example. I can now imagine making the leap with my eyes closed, but without the reassurance of precise visual orientation it would be much scarier. It is the pre-planning orientation that is essential there, not the visual feedback during the move.

    So has anyone recorded activity in the “next-“place cell an animal is about to transition to? Or do they only light up when actually occupied?

    • I’m a bit confused about the situation you describe, but there are many situations where “pure cognition” and “embedded cognition” interact and, perhaps, flicker back-and-forth.

      the closest is “next place cells” may be the “teleportation” study in the Moser lab:

      Wired magazine coverage:

      Original Paper:
      Jezek K, Henriksen EJ, Treves A, Moser EI, Moser MB. Theta-paced flickering between place-cell maps in the hippocampus. Nature. 2011 Sep 28;478(7368):246-9.

      Also: Bob Muller and I have a paper from the late 1980s, where we show that place cells actually seem to anticipate the rat’s location by a few 10ths of a second. 2. David Redish and colleagues have recorded from rats at choice-points and find place cells “jump ahead” of the rat, in a series of directions.

      • My perceptual “situations” are indeed confusing, even to me. I’m always happy to try to explain further, but the lack of shared higher-level concepts makes explanations idiosyncratic and speculative. Hence my efforts to adopt insights from more objective neuroscience and stretch them to subjective experience.

        Thank you for the links! Some unusually stimulating ideas. Despite an elaborate inner system of place and direction awareness and attention routing, I’m often confronted with the “teleportation” experience – suddenly feeling the room has a dramatically different shape despite knowing logically I’m in a known space. I’d go along with the idea that my place cells can resolve the change in milliseconds, but the effects on visual perception and sometimes balance can last seconds.

        “Thus, the ‘correct’ headlight placement might be at the rat’s nose rather than between its ears.” Did you resolve this? I found papers using a two-spot system, but none that discussed this reference positioning issue. I can’t know how it would translate into a rat’s brain, but for me the correct position would be between the centers of my eyeballs. Mentally it would be the point around which the blind spots of my retinas appear to swing as they project to my enclosure boundaries when I turn my head side-to-side. The angle to either side of my head from which perspective recedes equally to front and back.

        I guess in your round enclosure there is no obvious rectangular perspective, but presumably two-point rules apply. In a narrow arm of a maze, the strong forward one-point perspective dominates. I can just barely imagine the transition to two-point at ninety degrees to either side, but I can’t imagine at all the one-point perspective receding behind me. I’d expect my sense of my location would move forward toward the bridge of my nose in a narrow hallway, relative to being in an open room.

  4. Hmmm… do you think your idea of pure cognition maps (so to speak) to John Locke’s view of ideas? Essentially, he argued that ideas come from either sensations, or internal reflections. The latter were “the notice the mind takes of its own operations”, and seem similar to your pure cognitions.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s