The function of learning is clear: modifying behavior through experience. Memory, the storage of information that supports learning, is clearly necessary and valuable. Current psychology and neuroscience tell us that there are two memory systems enabled by separate neural systems. Procedural memory relies on reward circuitry and trial-and-error processes to mold efficient behaviors. Episodic memory stores specifc events in the life of the individual — but for what purpose?
Imagine you are a brilliant technician, and that you have the tools to design a wonderful robot, Ralph. You give Ralph a body much like ours, capable of sensing the world, locomotion, and interaction. You also give your robot a brain. You “hard-wire” Ralph’s circuits with behavior patterns including walking, grasping, avoiding cliffs and language. You give Ralph a “value system” that helps it make decisions to attain a set of goals. You give Ralph the capacity to learn from experience. Ralph, through trial-and-error, improves its behavioral efficiency in attaining goals. Moreover, not only can Ralph learn from experience, it can learn from the experience of others, by watching robots or humans attain goal states. Finally, it can learn rules by instruction, by rules and facts told by others (think of Hal and his instructor in the movie 2001 a Space Odyssey). Although you act as Ralph’s tutor, you also give the it 20 years of exploration and experience, to learn about the world.
After 20 years, what would Ralph be like? How would it resemble a person? How would it be different? I’d imagine that Ralph would be terrific at performing many tasks of everyday life: dressing itself, protecting itself, moving objects from one location to another, driving a car, helping around the house, etc. Virtually any task experienced frequently and directly related to its pre-established values.
Note that, in the description of Ralph’s brain, I didn’t specify episodic memory, the ability to record experienced events, independent of rewards and punishments. I’ll leave it to you, reader, to imagine Ralph’s behavior. Minimally, in conversation, Ralph could not tell you the events of its life, such as what it had for breakfast.
Although Ralph is clearly a limited conversationalist, the question I’d like to address is how limited would Ralph be?
Ralph’s learning, largely procedural, would leave no historical record. Consider examples of procedural learning, such as learning to shooting a basketball, ride a bike or the algorithm for solving a mathematical problem, improvement is in an incremental. As performance improves, the record of previous stages, the pathway to improvement, is lost. There is no mechanism to step back, to use prior-state information for alternate solutions or to solve different problems.
Episodic memory is quite different. Episodes are memory records that are not coupled to specific behavioral tasks. Episodic memories can be configured and reconfigured to construct behavioral solutions. Let’s consider two situations, then add other categories of learning.
Situation #1. Me, a couple of weeks ago.
I arrived home after work and, through habit, emptied my pockets. My set of work keys was missing. This has happened before, due to my bad habit of leaving keys in the door of a room I’m entering and plan on leaving soon. Thinking back on the day’s events elicited a fairly good set of incidental episodic memories: going to meetings, having lunch, working at my desk, attending a class. The last time I remembered using my keys was early afternoon, entering an equipment storage room near my office. Although I did not remember leaving my keys in the door, I traced the events that followed; I had not been to many places and none were likely locations where I would use my keys. I had identified a likely spot. Next day, I went to work and found the keys in the suspected door.
Commentary. This was situation where incidental episodic memory was very useful.1 Thinking back on the day’s events elicited a fairly good set of incidental episodic memories. Note that the memories were, roughly, accurate, none was associated with “learning” and none were associated with rewards. Nonetheless, the utility in storing these memories for several hours is evident. Without them I would not have been able to locate the keys. Note that this several-hour store of episodic memory would not last.2 Today, two weeks later, the only specific memory I can recall from that day is the surprise of the missing keys when I emptied my pockets. Longer-term episodic memory is certainly not all-encompassing; it is less “incidental” in that it is biased by reward and surprise. We tend to remember “important” events.
Situation #2. Hypothetical.
Fred has been your friend/acquaintance for many years. Fred has loaned you money on three occasions, helped you move, and been available to help around the house when you were sick. He is a good friend and trustworthy. In addition, when you’ve been together you’ve watched him spend money carefully. You think he’s competent with money among other things. Another friend/acquaintance, Bill, tells you that Fred, on occasion, steals money and he thinks he’s stolen money he’s found in your house. He also says that Fred has overspent and on the verge of bankruptcy. You have a problem. You have come to trust and depend on Fred, but should you continue to do so? In order to make a strategic decision you must re-evaluate your history with both Fred and Bill, to see if there is a coherent solution to your dilemma. Perhaps you’ve misjudged Fred? Perhaps Bill is a deceitful liar? Perhaps Bill has is simply wrong on the facts? Whatever your conclusion, you must re-evaluate past history. You need a clear record of previous events.
Commentary. The general role of episodic memory: the storage of unbiased, incidental knowledge that permits reworking and reconfiguration. If you had simply stored facts, such as “Fred is trustworthy”, “Fred is competent with money” and “bill says Fred is a liar” you’d be at an impasse. With the aid of episodic memory, conclusions (facts) can be reconfigured.
Condition #3. Autobiography. It seems self-evident that creating a personal narrative of one’s life depends on episodic memory. Sense of “self” depends, in part, on a personal narrative. The value of the sense of self is important, but beyond the scope of this discussion.
General Speculation: Episodic memory plays a central role in the development of the facts and world-models. Recall that Ralph the Robot incorporated facts via unexamined acceptance from an instructor. Humans are different. While we may receive facts and models from trusted sources, we also have mechanisms for for re-evaluating and updating accepted “facts” as well as creating our own. Nadel and Moscowitch have proposed a “multiple trace theory” which suggests a mechanism where episodic memories, by repetition and generalization, can be come facts3; this is consistent with the finding that humans with hippocampal damage have intact procedural learning, but are deficient in recalling episodes and learning facts.
The general proposition is that episodic memory permits the storage of events in an individual’s life history, without linking the events to behaviors or conclusions. By storing the raw data, the facts and events can be reworked. Recofiguring event data to draw novel conclusions or hypotheses is a complex cognitive process, done offline. Although heavy in computational load, there appear to be major benefits.
1The concept of “incidental knowledge” was suggested to me in a recent talk by Richard Morris. It is almost certainly in his publications. Help finding a reference will be appreciated. Episodic and incidental knowledge overlap, but are not identical.
2 Highly likely there is experimental data on the differences in quantity and quality between episodic recall after a few hours and several days. As with incidental knowledge, I’d appreciate help in finding relevant literature.
3 Nadel, L and Moscovitch, 1997. Memory consolidation, retrograde amnesia and the hippocampal complex. Curr Opin Neurobiol. Apr; 7(2):217-227.
Note: “Ralph” is “HAL” spelled a different way.
Thank you for very interesting post.
Actually there are also many arguments for episodic memory.
I do not fully understand the phrase: “Ralph could not tell you the events of its life.” Does this mean Ralph does not have historical record? I would like to comment carefully. At this point the assumption might not be realistic. However again I would like you to clarify the assumption.
There are some phrase in this page:
You give Ralph a “value system”.
Ralph learn from experience.
On the other hand,
Ralph could not tell you the events of its life.
Ralph’s learning would leave no historical record.
I have some questions:
Ralph has episodic memory?
Ralph has procedural memory?
Situation #1 and #2 seems the case there is only episodic memory. (I withhold additional discussion here at this point.)
And there are two more questions:
We may receive facts and models from trusted sources only? Human can not receive gems and stones as information?
Episodic memory is “without linking the events to behaviors or conclusions”? I would like further explanation.
Ralph has procedural memory, but not episodic memory. Procedural memory, based on reinforcement learning, leaves no clear historical record. One could get to the endpoint (riding a bike) via a wide variety of paths There is no record of the particular learning path taken.
Trusted sources. Our trust in the source has great influence about whether the information is valid, or held in question. Trust can be cased on competency or honest.
Episodic Memory “without linking to behaviors or conclusion”. Means that an episode is raw material, a fact in historical record. How it is used (behaviors or conclusions) is flexible and may change in time.
Thank you for the response.
I misunderstood Ralph has episodic memory. I re-write comments again.
Mambo: to be clear. Ralph has NO episodic memory.
I understood Ralph had procedural memory, but not episodic memory. I would like to comment carefully. It was difficult for me to understand no-episodic memory robot. If so, Ralph could not tell you the events of its life actually.
I understood Situation #2 that human could understand difficult problems with thinking historical record, because human has episodic memory.
It was difficult for me to understand no-episodic memory robot, because my hypothesis is consciousness is based on Declarative memory (including Semantic memory and Episodic memory). And my understanding is these two are seamless connection.
Though it is also important to understand simple Semantic memory only comparing to input for consciousness (and feel “Oh”), historical record should be also important for understand rich experience. It would also be effective for comparing trusted sources to “not trusted” ones.
Unfortunately I did not understand the phrase “without linking the events to behaviors or conclusions”, because I understand Semantic memory and Episodic memory are seamless connection.
I like this robot thought experiment. I share much of the spirit of your ideas, particularly “Episodic memories can be configured and reconfigured to construct behavioral solutions” – but I think about them in a different way. So here are a couple of speculative ideas I’d be interested to hear your thoughts on:
1. There is no such thing as an episodic memory. We constantly record information about our lives in the brain, but not as discrete episodic units (e.g. a memory for yesterday’s breakfast) or as ‘raw data’ as you put it – instead new information is being merged and concatenated with an existing network of information. We have the *illusion* of episodic memories because we have powerful (re)constructive processes that operate on this network. The constructive processes involve the combination of information in the network (you might call it semantic memory) that we have acquired over a lifetime, and recently encoded new information (which remains particularly salient for a short time). This gives rise to ‘schema-distortion’ effects and the like, where our recall of autobiographical events is littered with intrusions from our semantic memory – false memories that never actually happened.
2. The “purpose” (in terms of survival advantage) of this ability to construct ‘memories’ on-the-fly is mainly about generating hypothetical scenarios (I think you use the term ‘world-models’) that could occur in the future, which allows us to prepare our actions appropriately. Perhaps the partial ability to recall autobiographic events from our lives is a happy by-product of this (re)constructive ability. This could explain why we are not actually very good at it (assuming some variance in diet, try recalling what you had for breakfast two days ago), and on occasions where we appear to be good at it (that spectacular holiday in Timbuktu), the events are particularly unique (and thus less susceptible to internal interference from other information in the network), and we have often been heavily assisted by external cues (conversations with others, photographs etc) – which itself leads to all kinds of distortions.
And so I might say that you have given Ralph all of the information *storage* capabilities available to a human, but you have not given him a crucial memory *process* that would allow him to operate on that information in the creative and flexible manner that is characteristic of human behaviour.
I should stress that I am largely just speculating off the top of my head here in the interest of debate!
thanks for the thoughtful and provocative comment. My reactions:
1. While I agree that episodic memories are not veridical, I hesitate going to the other extreme and saying they have no historical data content. In addition, I suspect it’s useful to keep historical events somewhat separate and discrete. As I argued in situation #2 (trust of Fred) keeping them separate permits reshuffling. Perhaps one event is tainted by something you didn’t know at the time. Recoding this event can only be done if remains a discrete entity.
2. My guess is that “unusualness” is part of what makes specific events memorable. But I wouldn’t rule out unexpected rewards and punishments. dopamine.
Pingback: Designing a Robot – Signal Surgeon