UNIVERSIDADE DE LISBOA
FACULDADE DE CIÊNCIAS FACULDADE DE LETRAS FACULDADE DE MEDICINA FACULDADE DE PSICOLOGIA
Modelling Cognitive Offload and its Different Purposes
Pedro Neves
Mestrado em Ciência Cognitiva
Dissertação orientada por:
Prof. Doutor Luís Miguel Parreira e Correia
Prof. Doutor Emanuel Pedro Viana Barbas de Albuquerque
Index
1. Introduction 1
2. State of the Art 3
2.1. Cognitive Offload 3
2.2. External-Internal Equivalence 5
2.3. Why do Cognitive Agents Offload? 8
3. Methods 10
3.1. Overview 10
3.2. The Task 11
3.3. The Robots 13
3.4. Reactive Controllers 14
3.5. Cognitive Controllers 16
3.6. The Evolutionary Process 18
3.7. Testing Conditions 20
4. Results 21
5. Discussion 36
6. Conclusion 38
7. Bibliography 39
List of Figures
Figure 3.1: Screenshot of the double-T maze. 12
Figure 3.2: Screenshot of the virtual Robot1 in Webots. 13 Figure 3.3: Screenshot of the virtual Robot2 in Webots. 14 Figure 3.3: Schema of the neural network of the reactive controllers. 16 Figure 3.4: Schema of the neural network of the cognitive controllers with pen. 17 Figure 3.5: Schema of the neural network of the cognitive controllers without pen. 21 Figure 4.1: Progression of fitness across generations of population RP1. 22
Figure 4.2: Average performance scores of the best RP1 robot across the four
different scenarios in conditions identical to those it had evolved in. 23 Figure 4.3: Average performance scores of the best RP1 agent for each initial
orientation in the interval [-15;15], across the four different scenarios. 23 Figure 4.4: Average performance scores of the best RP1 agent across the
different test conditions. 24
Figure 4.5: Average fitness of the three evolved populations of Robot1. 25 Figure 4.6: Progression of fitness across generations of population R1. 26 Figure 4.7: Average performance scores of the best R1 agent across the four
different scenarios in conditions identical to those it had evolved in. 27
Figure 4.8: Performance scores of the best R1 robot across the four different
scenarios for each initial orientation it could experience in its evolutionary conditions. 27 Figure 4.9: Average performance scores of the best R1 agent across the different
test conditions. 28
Figure 4.10: Progression of fitness across generations of population C1. 28 Figure 4.11: Average performance scores of the best C1 agent across the
four different scenarios in conditions identical to those it had evolved in. 29 Figure 4.12: Performance scores of the best C1 robot across the four different
scenarios for each initial orientation it could experience in its evolutionary conditions. 30 Figure 4.13: Average performance scores of the best C1 agent across the
different test conditions. 30
Figure 4.14: Progression of fitness across generations of population Cpen1. 31 Figure 4.15: Average performance scores of the best Cpen1 agent across the
four different scenarios in conditions identical to those it had evolved in. 32 Figure 4.16: Performance scores of the best Cpen1 robot across the four different
scenarios for each initial orientation it could experience in its evolutionary conditions. 32 Figure 4.17: Average performance scores of the best Cpen1 agent across the
different test conditions. 33
Figure 4.18: Progression of fitness across generations of population Cpen2. 33
Figure 4.19: Average fitness of the two evolved populations of cognitive agents
with pen. 34
Figure 4.21: Performance scores of the best Cpen2 robot across the four different
scenarios for each initial orientation it could experience in its evolutionary conditions. 35 Figure 4.22: Average performance scores of the best Cpen2 agent across the
different test conditions. 35
List of Acronyms
PCT: Pattern Copy Task. 6
PFC: Prefrontal Cortex. 7
IR: Infra-Red. 13
Acknowledgements
I would like to thank both my supervisors who, from the beginning, showed interest and trust in this project and since then have always been available to provide guidance. Without them, this project would not have been possible.
I would like to thank all professors in this master programme, who have provided a stimulating environment in class and also put together the amazing program of the course.
I would like to thank my colleagues and friends, who have motivated me both through their interest in my own ideas but also through their own interests, which led me to think about new and exciting topics.
I would also like to thank my family, that beyond motivating me, was always supportive and sustained my education.
Finally, I would like to thank my girlfriend, who stood by my side every step of the way, always with new insights and ideas, and making me see things clearer in the harder moments.
Thank you all!
Resumo
Um agente é uma entidade que perceciona o seu ambiente através de sensores e atua sobre este por uso de efetores. Os organismos vivos podem então ser entendidos como agentes da vida real no sentido em que percecionam o seu ambiente através dos seus órgãos sensoriais e, após um processamento variável desta informação, um comportamento irá resultar e o organismo atuará sobre o seu ambiente. Os seres vivos evoluíram de modo a serem capazes de manter um comportamento funcional na presença de incerteza sobre o seu ambiente e sobre o impacto das suas ações. Desta pressão evolucionária, resultaram processos cognitivos complexos, como a memória, que permitiriam aos organismos distinguir entre estímulos pouco informativos ou, de outro modo, indistinguíveis. Isto seria possível através de integração de informação sensório-motora ao longo do tempo, formando representações internas. A utilização de representações internas é característica de cognição complexa e comportamentos ou estratégias que as empreguem são denominados de cognitivos. Agentes com a capacidade de manter representações internas também são chamados de “agentes cognitivos”. De qualquer modo, tanto o ambiente circundante como o próprio corpo de um agente apresentam uma estrutura e um contexto com potencial de serem usados para regular o comportamento do organismo de modo que este atue apropriadamente. Isto é de tal modo relevante que alguns problemas podem ser resolvidos de forma puramente reativa, ou seja, sem empregar representação interna. Um “agente reativo” é um agente que age apenas por reflexo, tendo sempre a mesma resposta para um mesmo estímulo sensorial. Este tipo de agentes não considera para a determinação do seu comportamento informação além da presentemente disponível externamente sem usarem memória.
Uma estratégia que ocupa um papel interessante entre representação interna e externa é o offload cognitivo. O offload cognitivo é definido como agir de modo a alterar as necessidades de processamento de informação de um problema tendo em vista a diminuição das exigências cognitivas que este problema possa impor. Uma das várias formas que o offload cognitivo pode tomar é de alterar o ambiente de modo a codificar nele mais informação relevante, tornando o ambiente um cenário mais rico e melhor regulando o comportamento do próprio agente. Contudo, o que a investigação neste processo sugere é que, quando organismos complexos capazes de representação interna - entre os quais os humanos - empregam offload cognitivo, parecem não ser formadas representações internas. Por outro lado, organismos muito simples incapazes de representação interna conseguem adquirir comportamentos inesperadamente complexos, como se tivessem memória, através do emprego de offload cognitivo.
Parece então que codificar informação externamente ou codificar informação internamente são duas estratégias alternativas e funcionalmente equivalentes. Porém, nem tudo fica explicado por esta perspetiva. Se codificar informação interna e externamente é funcionalmente equivalente fica sem resposta o porquê de agentes cognitivos complexos, como nós humanos, utilizarem offload cognitivo e ambos os tipos de representação. Investigação em humanos revela que tarefas mais difíceis incentivam o uso de offload cognitivo, enquanto tarefas mais fáceis tendem a ser resolvidas utilizando apenas capacidades cognitivas internas. Outra descoberta interessante é de que a metacognição preenche um papel importante na decisão de utilizar ou não utilizar offload cognitivo. Este papel é visível, por exemplo, em dados que sugerem que participantes humanos são enviesados para evitar esforço cognitivo. Também já foi demonstrado que, além da utilização do offload nem sempre ser acompanhada de benefícios, participantes descrevem que utilizam offload cognitivo com o objetivo de maximizar o seu desempenho e não de minimizar o esforço da tarefa, o que até parece contradizer a própria definição de offload cognitivo. Após experimentação com agentes artificiais chegou a ser sugerido que codificar
a mesma informação tanto interna como externamente poderia resultar em benefícios para os agentes que o fizessem.
Para o presente estudo, foram utilizados, em simulação, diferentes tipos de agentes artificiais com o propósito de entender qual a relação entre representação interna e representação externa e se haveria diferenças fundamentais entre estas. Os agentes artificiais utilizados divergiam não só na sua habilidade de codificar informação externamente através de offload cognitivo como também na sua habilidade de manter representações internas. Um destes tipos de agentes seria reativo com capacidade de fazer offload cognitivo, mas não teria capacidade de manter representações internas. Outro agente seria cognitivo e teria a sua capacidade de fazer offload cognitivo limitada, mas teria capacidade de manter representações internas. Por fim, um terceiro e último tipo de agente seria também cognitivo e teria capacidade de fazer offload cognitivo tal como a capacidade de manter representações internas.
Cada um destes agentes foi encarregue de resolver uma tarefa num labirinto em duplo-T que envolvia memória prospetiva: No início da tarefa o agente seria exposto a um de quatro estímulos e de acordo com este estímulo teria de navegar até uma localização específica no labirinto, e depois teria de regressar novamente ao local onde a tarefa começou. Esta tarefa requer não só que o agente codifique a informação relativa a qual estímulo lhe foi apresentado inicialmente, mas também a informação relativa a qual o caminho que foi percorrido, de modo a ter como regressar. A codificação desta informação poderia ser feita internamente ou externamente. Cada agente era composto por um robot virtual equipado com uma rede neuronal. As redes neuronais foram treinadas para esta tarefa utilizando um algoritmo evolucionário. Os agentes equipados com redes neuronais evoluídas foram depois submetidos a diferentes condições experimentais em que teriam de resolver o labirinto com o seu acesso às suas representações externas dificultado.
O produto das evoluções levadas a cabo neste estudo ficou aquém das expectativas, mostrando que os agentes equipados com redes neuronais evoluídas manifestaram um baixo sucesso para a resolução do labirinto. Métodos idênticos já foram anteriormente utilizados para treinar redes neuronais para tarefas semelhantes e soluções de bom desempenho foram encontradas. Seria, assim, esperado que o desempenho dos nossos agentes fosse mais elevado que o verificado. Mesmo lidando com agentes subótimos, foi possível observar que os nossos agentes reativos evoluíram um comportamento
“especialista”. Estes desenvolveram uma resposta de grande desempenho quando confrontados com um estímulo específico. No entanto, esta resposta era sempre desencadeada, independentemente do estímulo inicial apresentado no início da tarefa, e nestes casos o desempenho não seria tão bom. Mesmo assim este comportamento permitiu-lhes ter o melhor desempenho observado durante este trabalho, mas, no entanto, estes agentes estavam longe de resolver toda a tarefa. Esta estratégia permitiu aos agentes reativos necessitar de codificar pouca informação externamente e deste modo ter um comportamento robusto. Isto permitiu aos nossos agentes reativos não perder desempenho quando expostos às nossas condições experimentais. Os nossos agentes cognitivos com capacidade limitada de fazer offload mostraram ser os piores a resolver o labirinto, mas o seu comportamento mostrou que estes codificam, do seu modo limitado, alguma informação externamente. No entanto, quando expostos a condições experimentais, o comportamento destes agentes demonstrou-se frágil e mostraram um desempenho reduzido. Os nossos agentes cognitivos com total capacidade de fazer offload cognitivo e de manter representações internas pareceram mostrar um comportamento generalista em que tentavam encontrar todos os objetivos no labirinto. O facto destes agentes serem consideravelmente melhores que os outros agentes cognitivos apenas por terem evoluído com a capacidade total de fazer offload cognitivo e mesmo assim manterem um comportamento robusto quando expostos às nossas condições experimentais, sugere que estes agentes utilizam os dois tipos de representação: interna e externa. Os nossos resultados são consistentes com a ideia de que representação interna e representação externa interagem de modo
sinergístico, que resulta em acrescida funcionalidade que não poderia ser obtida empregando somente um destes tipos de representação.
Palavras-Chave: Vida Artificial, Offload Cognitivo, Representação Externa, Robótica Evolutiva, Redes Neuronais.
Abstract
Living organisms evolved to cope with uncertainty about their environment and the impact of their actions. From this selective pressure resulted complex cognitive processes, such as memory, that would allow living beings to tell apart otherwise undistinguishable stimuli. This would be possible through the integration of sensorimotor information over time, forming internal representations.
Nonetheless, both the environment and the agent’s body provide an exploitable structure and context to act appropriately, and some tasks can be solved through solely reactive means. Furthermore, through cognitive offload, an agent may externally encode the necessary information to regulate its behaviour, making the environment an even more informative scenario. However, research suggests that when agents with cognitive capabilities, such as humans, employ cognitive offload, internal representations seem to not be formed. It would then appear that cognitive strategies that rely on internal processing and fully embodied reactive strategies represent two alternative solutions. Yet, some results remain unexplained, and it has been suggested that encoding the very same information both internally and externally might yield benefit. In this work, by evolving in simulation different types of agents differing in their capacity both to maintain internal representations and deploy cognitive offload, we have set out to understand the relationship these two strategies pose toward one another. Our results do seem to support the idea that internal and external representation interact synergically, leading to increased functionality that could not be achieved with only one type of representation.
Keywords: Artificial Life, Cognitive Offload, External Representation, Evolutionary Robotics, Neural Networks.
1. Introduction
Despite its incredible power and flexibility, human mental capacities have well known limits.
For example, our working memory, the “online” workspace where information is processed, has a short duration of 10 to 15 seconds (Goldstein, 2011) and can only handle around four items (Cowan, 2001) at a given time. One way in which we can compensate for these shortcomings is by actively using our bodies and manipulating our environment to our aid. This is known as cognitive offload and is formally defined as ‘the use of a physical action to transform the information processing requirements of a task to reduce its cognitive demands’ (Risko & Gilbert, 2016) and can take many forms: from simply tilting one’s head to read rotated text to using object tokens as aids for calculus. Interestingly, one can also offload cognition to other people, forming, for example, a transactive memory system where one does not remember some information but knows someone who knows it and can therefore also access it (Sparrow et al., 2011). Cognitive offload has been shown to be an effective method to maximize performance in a large variety of real-life problems involving arithmetic (Carlson et al., 2007; Goldin- Meadow et al., 2001), reading (Risko et al., 2014), prospective memory (Gilbert, 2015), navigation (Fenech et al., 2010) and spatial reasoning (Armitage et al., 2020; Chu & Kita, 2011). Cognitive offload is understood to release internal resources, which would otherwise be occupied in internally maintaining representations, by replacing them with an external structure or device, an external representation.
Representation is understood as mental states and processes with an information-bearing structure which carries intentionality: referring or being about other things in the world. Such mental states are commonly identified as thoughts, beliefs, desires, perceptions and imagings. Cognitive offload could theoretically replace any internal cognitive process with external representation as long as the environment is exploitable in a suitable way. For example, a written note could represent the intention to do a certain task in the future and ease the load on working memory that no longer has to store this information; in the case of frequently relying on search engines to access information, it could be interpreted that long-term memory representations are replaced; while using a calculator, the internal reasoning required for this task is mitigated (Sparrow et al., 2011).
But is replacing internal processes all that cognitive offload accomplishes? Do these externalized processes fulfil the exact same role as when carried out internally? Arguably not. For example, (Kirsh, 2010) hypothesized seven ways in which external representations enhance cognitive power: by 1) changing the cost structure of the inferential landscape; 2) serving as a shareable object of thought; 3) facilitating representation; 4) often being a more natural representation of structure than internal representations; 5) facilitating the computation of more explicit encoding of information; 6) enabling the construction of arbitrarily complex structures; and 7) aiding in coordinating thought. All these seven examples seem to come from the same property of external representations: that they can be handled with increased precision as they are consistency-preserving and persistent in time due to being physically implemented (Kirsh, 2010). However, all these seven operations seem to also be carried out by internal representations, just with less precision or amplitude. If the human species had more time to evolve its cognition, wouldn’t it be able to handle internal representations with this same level of performance? Are there no fundamental differences between internal and external representations?
Interestingly, Carvalho & Nolfi’s experimentation with artificial agents (Carvalho & Nolfi, 2016) suggested that encoding the same representation internally and externally might translate into advantages for the agent that does so. This supports the idea that perhaps there are functions and processes exclusive to one kind of representation. It is then important to understand the implications of both types of encoding if we are to understand the purpose of cognitive offload in complex organisms
such as us humans. In this thesis, inspired by the work of (Carvalho & Nolfi, 2016), we shall focus on how artificial agents with the ability to handle both internal and external representations employ cognitive offload, with the aim of unravelling the relationship between these two types of encoding and figure out if these solutions represent mutually exclusive functions or if instead, they interact and compensate one another (Kirsh, 2010). For this we will compare agents built to have full capacity of using both types of representation with 1) agents built to only be capable of external representation and with 2) agents built to have the capacity to use both but with an hindered capacity to use external representation. If the agents that can fully exploit both types of representation show an increased functionality, for example by being more resistant to environmental noise than the others, we should be able to tell that indeed internal and external representations complement one another. If, however, these agents perform similarly to the others or without showing any added functionality, this means that all functionality was accounted by one type of representation and the possibility of using both types did not have an impact because these were functionally equivalent.
This is a relevant question since, conceptually, offloading cognitive processes to the body or the world could be interpreted as actively extending the mind beyond the boundaries of the brain. Following from the Parity Principle postulated by (Clark & Chalmers, 1998), if one’s own body or even environment is to be considered part of its mind, then it is expected that the offloaded mental functions carried out by these external elements are the same as those carried out ‘in the head’ when offload is not employed. To claim that the external world can indeed be considered part of the mind, this must be verified by studying the relations internal and externalized processes pose towards one another. If internal cognitive processes are indeed replaced by externalized ones upon cognitive offload, then we can assume the externalized process is mental. If, however, externalized processes have a different function, then we cannot guarantee they are mental (Clark & Chalmers, 1998).
Wherever we draw the line between what is constituent of mind and what is not, it is undeniable that cognitive offload and externally encoded information play a crucial role in the proper functioning of the mind. It was hypothesized that, because the cognitive architecture required to handle internal representations is usually complex, the first organisms to evolve intelligence did so through perfecting strategies that involved only manipulation of their environment to encode externally the information required to regulate behaviour. Internal representations would only appear later. Complex cognition would evolve later through the internalization of the relevant information. Thus, understanding the properties of externally encoded information would then be understanding the properties of the first means used to regulate intelligent behaviour in evolutionary history and an insight into the origin of cognition and mind (Carvalho & Nolfi, 2016; Chung & Choe, 2009; Pezzulo & Nolfi, 2019).
Also, as Kirsh & Maglio pointed out, ‘not all actions performed by well-adapted agents are best understood as useful physical steps’ (Kirsh & Maglio, 1994). Some actions, such as offloading cognition, have an epistemic rather than a pragmatic role. However, the epistemic value of action only received attention in the academic domain more recently than the pragmatic value (Kirsh & Maglio, 1994). As a consequence of having allowed artificial agents to exploit the informational structure of their environment, the internal structure they require to act appropriately turned out to be simpler than what was hypothesized through disembodied perspectives of cognition. Cognitive offload is an ability that can potentially make artificial intelligence agents not only simpler but also more robust by allowing agents to retrieve relevant information from the very problem they are tasked to solve (Carvalho & Nolfi, 2016; Kirsh & Maglio, 1994).
Another issue is that our dependency on computer technologies with the capacity to fulfil cognitive tasks in our stead is growing every day, making cognitive offload all the more common. This sort of technology immediately impacts on our cognition, unlocking new possibilities for cognitive actions, which (Salomon, 1990) defines as ‘effects with technology’. Research suggests that interfaces displaying too much information lead to less internalization and retention of this information, which might be particularly problematic when the goal is learning itself (Nimwegen & Oostendorp, 2009;
Waldron et al., 2007). These ‘effects with technology’ are ultimately linked to the considerably less well understood ‘effects of technology’, the manifestation of a cognitive skill outside the technology-bound context in which it was acquired or developed (Salomon, 1990). This raises the concern that there might be detrimental ‘effects of technology’, hindering a user’s internal abilities in the long-term.
Understanding the properties of external representations would allow the development of better technologies and possibly the mitigation or removal of potential negative long-term cognitive side effects of technology use (Nimwegen & Oostendorp, 2009; Salomon, 1990; Waldron et al., 2007).
In this section, we hope to have properly raised attention to the relevance of the topic and the need for a fundamental, bottom-up approach, such as ours is, to uncover the fundamental differences between internal and external representations. In the next section we will start by reviewing the state of the art on cognitive offload and external representation, consider the implications of current understanding towards the functional equivalence between these two types of representation and finally highlight the gap in the knowledge pertaining to the purpose of cognitive offload in agents capable of making use of both types of representation. The following section will describe the methods used in this thesis in the attempt to close the mentioned gap. We will first briefly layout the structure of the whole project and then describe each of its components in detail. With the methods presented we will follow with another section describing the results obtained, which are discussed on the section after. We will then conclude with a final section where we evaluate the progress made with this work towards understanding if there are fundamental differences between internal and external representations and then we consider prospects for future work on this topic.
2. State of the Art
2.1. Cognitive Offload
An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through effectors (Russell et al., 2010). Organisms can be understood as biological real-life agents in the sense that they possess a sensorial apparatus that receives, as input, information from the environment and, after a variable amount of processing of this information, an action or behaviour will result, as output, and the organism will have an impact in the world. To persist, living organisms evolved to cope with uncertainty about their environment and the consequences of their actions. This eventually resulted both in innate automatic instincts and complex cognitive processes that inform decision making and allow organisms to act in an adequate way that attempts to maximize their fitness (Pezzulo & Nolfi, 2019; Russell et al., 2010).
To act this way, organisms can rely on reactive or cognitive strategies. Reactive strategies consist of mapping situations to action, resulting in action policies where a response is acquired for each
sensorial stimulus. These strategies utilize fully situated cognition dependent on agent-environment interactions and consider only the agent’s current state, defined by its present sensorial input. Agents can carry out these strategies with simple cognitive architectures. While allowing fast action taking, once the situation-to-action mapping is done, a particular input is always met with the same output. This makes reactive strategies insufficient in complex environments where stimulus might be aliased or otherwise insufficiently informative. In contrast, cognitive strategies are defined by utilizing internal representations/states in addition to the current state to regulate the agent’s behaviour. These internal states consist of sensory-motor information that was integrated over time. This dynamic processing is a fundamental requirement for higher order cognitive capabilities such as memory, attention, reasoning, language, etc.… As such, cognitive strategies allow for a more flexible and context appropriate action selection and better cope with aliased or uninformative stimuli than reactive strategies as more information is available. To employ cognitive strategies, however, agents with more complex cognitive architectures are often required. A cognitive agent is thus an agent not only with the capacity to observe its current state but also the capacity to maintain and consider internal states, being capable of deploying both reactive and cognitive strategies. A reactive agent can solely observe its current state and has no ability to internally store information over time, being limited to reactive behaviour. Commonly, even simple organisms rely on both types of strategies. Spiders are a good example of this. These tiny predators are known to bear internal representations as they have the ability to plan routes in advance, have a sense of numerosity and are keen learners. During web-building, spiders make many measurements and have to memorize distances travelled to fix the new threads at correct positions. Yet, the spider also relies on the threads already placed and does not need to retain an internal representation of the entire web. As a new spiral segment is woven, the distance measured from the previous spiral segment can be forgotten as the newly fixed thread will cue the next step in building the web. It is clear that cognitive and reactive strategies coexist, but if these two strategies actively compensate one another or if they are two exclusive or even competing alternatives is still unknown (Carvalho & Nolfi, 2016;
Japyassú & Laland, 2017; Pezzulo & Nolfi, 2019).
Evidencing a dichotomy in our understanding of the paradigm between reactive and cognitive strategies, there are two computationally distinct approaches to design artificial agents that cope with uncertainty: an agent can be sensorimotor enriched or cognitively enriched. An agent that is sensorimotor enriched has the capacity to act in a way that injects more information in future stimuli, resulting in an increased ability to exploit reactive strategies. An example of sensorimotor enrichment is an agent capable of communicating through stigmergy. Stigmergy is an indirect coordination mechanism of emergent systems where the individual parts communicate with each other through modifying the environment (Korosec & Silc, 2009). In nature, stigmergy is characteristic of ants that communicate with each other by differentially depositing pheromones in the world as they navigate in search of food, which results in ants being able to form and follow trails connecting their nest to food sources without having a complex cognitive apparatus. An agent is cognitively enriched if it can integrate information about how the world works and about the impact of its own actions to form an internal model. This allows the employment of cognitive strategies. These model-based agents not only directly observe their current state, fully defined by active sensorial information, but also utilize information about past states and actions to guess the state of the world and act accordingly. However, each new context corresponds to a set of possible states of the world. The agent needs to decide the most probable scenario and then act accordingly, making the decision process intractable for large state spaces. There are fundamental differences in these two types of enrichment as sensorimotor enrichment focuses on rendering stimuli more informative and cognitive enrichment focuses on rendering internal representations more informative (Chung & Choe, 2009; Korosec & Silc, 2009; Pezzulo & Nolfi, 2019;
Russell et al., 2010).
Fulfilling an interesting role in the interplay between reactive and cognitive strategies is cognitive offload. Cognitive offload can be divided into two categories: ‘thinking with the body’ and
‘putting cognition into the world’ (Risko & Gilbert, 2016).
The concept of offloading as ‘thinking with the body’ refers to using one’s own body to reduce cognitive demand, the most common example of this being external normalization as when one physically tilts its head to perceive rotated text. This concept is closely related to another concept from artificial intelligence research: morphological computation, the notion that some body structures can be exploited to replace brain or centralized processing. An example often put forward as representative of morphological computation is the case of ‘cheap grasping’, when a robotic hand, by virtue of the soft materials it is built from, allows for grasping with minimal control involved. A simple ‘close’ scheme is applied to a humanoid shaped hand, and as the fingers come together, the soft materials of the hand will self-adapt to the shape of the object being grasped. This allows for grasping of a multitude of objects without the need to generate an internal representation of the hand conformation prior to action taking.
It is interesting to note that (Müller & Hoffmann, 2017) have called out this example for not performing computation per se and as being more accurately described as an example of ‘morphology that facilitates control’. In that paper Müller & Hoffmann restrict morphological computation to a much more select group of cases and also distinguish ‘morphology that facilitates perception’. Nonetheless, all these cases are cases of cognitive offload where information processing demands imposed on a centralized system (be it the brain or a robotic controller) are mitigated (Müller & Hoffmann, 2017; Pfeifer & Gómez, 2009;
Risko & Gilbert, 2016).
Cognitive offloading of the type ‘putting cognition into the world’ refers to externally encoding the states that regulate an agent’s behaviour. This would happen, for example, when writing down information on a piece of paper. This latter type of cognitive offload plays a particularly interesting role in the cognitive-reactive paradigm as it suggests that the internal states characteristic of cognitive strategies can be replaced by external elements. External elements are only known to an agent while they happen to be perceived by its sensorial apparatus. This is also required for these states to be considered by a reactive agent. Thus, reactive and cognitive strategies appear alternative, yet functionally equivalent, solutions. The main body of research on the use of cognitive offload by humans seems to support this claim. However, recent findings bring new controversy to the matter (Morrison &
Richmond, 2020; Pezzulo & Nolfi, 2019; Risko & Gilbert, 2016).
2.2. External-Internal Equivalence
While offloading usually results in an immediate performance increase in memory tasks, a bulk of evidence is accumulating that it impairs memory and retrieval for larger time scales, which indicates an absence of an internal representation. In a Short Term Memory task where participants had to recall items from lists, (Kelly & Risko, 2019a) found that participants that employed cognitive offload, when compared to participants that had to rely on internal memory, recalled less items when the external aids were removed. However, these participants showed reduced primacy but typical recency effects for these items, similar to what happens with directed forgetting. Similarities with intentional forgetting were also found by (Eskritt & Ma, 2014) in a task where participants had to play the game Concentration.
In Concentration, a player is required to remember both identity and location information about face- down cards. In (Eskritt & Ma, 2014), half the participants had the chance to study the cards before playing, while the other half were allowed to take notes instead, which were unexpectedly taken away before playing. Memory for identity information was similar for both conditions but participants that
offloaded had significantly worse memory for location information and showed no proactive interference effects. These results suggest that internal memory can be strategically forgotten and replaced completely or to a great extent by external memory without impairing behaviour while the external aid is present. This forgetting induced by offloading doesn't seem to be intentional: (Soares &
Storm, 2018) have shown that subjects touring a museum poorly remember artifacts they were allowed to photograph compared to those they were only allowed to observe. This persisted even when subjects were informed they would not have access to the photographs later. There is also evidence which indicates that the use of cognitive offload negatively impacts subject’s internal capabilities for even larger timescales. For example, (Moritz et al., 2020) have shown that participants with the opportunity to train for a task with resource to cognitive offload, performed at the level of untrained baseline condition in a subsequent equivalent task held up to 24 hours later where offload was not available. This result however did not hold for a near transfer task (Eskritt & Ma, 2014; Grinschgl et al., 2021; Kelly &
Risko, 2019a; Moritz et al., 2020; Morrison & Richmond, 2020; Pezzulo & Nolfi, 2019; Risko & Gilbert, 2016; Soares & Storm, 2018; Sparrow et al., 2011).
Interestingly, this detrimental impact on memory seems to have the potential to be strategically exploited as seen with the saving-enhanced memory and saving-enhanced performance effects. The saving-enhanced memory effect was described by (Storm & Stone, 2015) who showed that participants that saved one file before studying a new one had significantly improved memory for the new file, when compared to those that didn’t save the first file. (Runge et al., 2019) replicated this saving-enhanced memory effect and also described the saving-enhanced performance effect. This was visible when their participants solved more subsequent modular arithmetic problems after they’ve offloaded a word list relevant to a prior task than when they did not offload said list. These effects are consistent with the idea that cognitive offload replaces internal representation, freeing up resources that may be allocated to other tasks (Runge et al., 2019; Storm & Stone, 2015).
Other methodologies which employ the Pattern Copy Task (PCT) point towards a similar trade- off. In the PCT, participants copy a pattern displayed in a window on one half of the screen to another window on the other half of the screen. Usually, one window is covered with a mask while the other is visible and participants may alternate at will which window is uncovered. This allows free adaptation of offloading behaviour and choice of the preferred strategy. The usage of offloading is visible by the number of times participants choose to uncover the window with the to be copied pattern. Research with the PCT shows that increased externalization leads to higher task processing speed but reduces memory for the processed visuo-spatial information with participants scoring lower in immediately posterior unexpected recall tests. Also, participants that more heavily relied on offloading and were interrupted mid-task performed poorly when asked to resume the task from memory, as opposed to participants that relied more on internal memory. Interestingly, (Grinschgl et al., 2021) showed through a delayed recall test that cognitive offload is also detrimental for retention past working memory maintenance but participants aware of the upcoming memory test, therefore with the explicit goal to form memory representations, were able to counteract this effect. Still, participants do not always minimize the usage of cognitive offload to achieve this but those that tend to offload less usually perform better (Grinschgl et al., 2021; Morgan et al., 2009, 2013; Waldron et al., 2007).
The long-term impact of frequent Internet use in human cognition has been investigated by (Sparrow et al., 2011). Beside replicating the finding of believed to be externally available information being less remembered by participants, results showed that participants were primed to think about computers when faced with difficult questions. Also, when participants didn’t expect to have limited access to information, they had better recall for where the information was stored than for the
information itself. The authors then concluded that the Internet became an external memory storage coupled with human individuals, forming a transactive memory system on which the human is ever more dependent. This dependence would come from the reduced motivation to form internal memory representations as their contents are externally available (Sparrow et al., 2011).
This negative impact of cognitive offload in memory has been associated with reduced employment of top-down mnemonic strategies such as rehearsal for offloaded items, much like what happens in intentional forgetting with to-be-forgotten items. This hypothesis was tested by (Kelly &
Risko, 2019b) who searched for an isolation effect for offloaded items. The isolation effect refers to a better recall for distinct items processed amongst generic ones, such as a word belonging to a certain category processed amongst other words of another single category. This effect is not a consequence of top-down mechanism and thus should still be visible for offloaded items. Their findings supported the hypothesis with cognitive offload not having a relevant impact on recallability of distinct items. Thus, despite the impact of cognitive offload on memory being possibly explained by a lack of effortful attempts at memorizing, there are scenarios in which external representation doesn’t seem to replace internal representation (Kelly & Risko, 2019b).
A unique consequence of externalizing cognition is that offloaded items become vulnerable to external manipulation. The was investigated by (Risko et al., 2019) who had participants perform a memory task in which they could offload a list of to-be-remembered word items into a computer file.
Then the participant would perform a distractor task without access to the file and access to the file would be restored for a recall test. The interval between encoding and test phases was inferior to one minute. Only in a final trial would the file be altered during the distractor task by introducing in it a new word not meaningfully related to the others in the list. Participants rarely noticed the alteration and often internalized this new information in their biological memory, creating false memories. Still, inserted items were experienced differently by participants with them broadly reporting a lower confidence that these items were present in the original list. Interestingly, these results are indicative of a general tendency to have internal and external representations fulfil independent functions: the lower reported confidence shows that the quality of externalized information can be questioned, however, participants behaviour in the recall test is indicative of a “blind” reliance on the offloaded items and little intervention of internal mechanisms (Risko et al., 2019).
Neuroimaging data about cognitive offload was obtained by (Landsiedel & Gilbert, 2015) from a prospective memory task. Previous research (see (Burgess et al., 2011) for a review) had already associated intention maintenance with increased signal in lateral rostral prefrontal cortex (PFC) and decreased signal in medial rostral PFC and characterized these as task-positive and task- negative/default-mode regions, respectively. However, it was unknown whether these had an inverse, but equivalent, or a dissociable function. In (Landsiedel & Gilbert, 2015) paradigm, participants under an functional magnetic resonance imaging scan that could externally encode delayed intentions were compared to those that could only rely on their own internal resources. Their results showed that offloading intentions is accompanied by a reduced deactivation in the medial rostral PFC and a slightly, yet not statistically relevant, increased activity in the lateral rostral PFC, thus showing a qualitative difference in the activation profile of these two regions. Lateral rostral PFC is thought to play, at least in some cases, a content-free role in prospective memory, being associated with the stimuli-independent preparedness to interrupt ongoing tasks in order to fulfil delayed intentions (Friston et al., 2002). This would explain the little impact of offloading in these region’s activity. Medial rostral PFC was proposed (Gilbert et al., 2009) to be associated with externally-cued rather than self-initiated realization of delayed intentions, which would explain its increased activity paired with cognitive offload. This would also
suggest that its baseline deactivation is correlated with maintaining internal representations of the intended behaviour in working memory, as when such content is externally accounted for, the deactivation is reduced. Medial rostral PFC activity patterns suggest that brain activity accompanies cognitive offload, most likely because some internal functions were replaced by external equivalents.
However, the intact activity of the lateral rostral PFC doesn’t rule out that there are cognitive functions that can only be internally carried out, opening the possibility that the inverse may also be true (Burgess et al., 2011; Friston et al., 2002; Gilbert et al., 2009; Landsiedel & Gilbert, 2015).
Stepping away from humans, it was shown that offloading may mimic complex cognitive capabilities without resorting to internal processing, further highlighting the reactive-cognitive equivalence. This mimicry can be seen in nature when memory-like behaviour emerges in ants through the usage of chemical cues/ pheromones left in the environment. This has been replicated in the laboratory by (Chung et al., 2009; Chung & Choe, 2009, 2011) through use of reactive artificial ants that were able to mark with scent cues their nest-to-food and food-to-nest routes as they travelled them.
This allowed these artificial ants to retrace their steps and solve the task of harvesting food, which would be impossible without any kind of memory. The authors hypothesize that the evolution of internal memory could have happened through the internalization of the offloaded states. In an intermediate stage, a transition from external to internal memory would have occurred by replacing externally placed pheromones for neuromodulators placed analogously but within the brain. Their hypothesis is supported by the close relatedness of the olfactory system, an evolutionarily primitive way to perceive the environment and communicate, and the hippocampus, responsible for spatial memory (Chung et al., 2009; Chung & Choe, 2009, 2011; Pezzulo & Nolfi, 2019).
In another setting, (Reid et al., 2012) have shown that the slime mold Physarum polycephalum, a unicellular and reactive organism, can use its extracellular slime secretion to mark already explored areas. While searching for food, P. polycephalum avoids marked areas and has a preference to search unmarked areas. With this simple offload the slime mould is able to efficiently escape a U-trap and find a chemotactic food source on the other side of the trap. This ability is greatly impaired if the environment is manipulated to be all covered by the extracellular slime, rendering this cue uninformative. This result also suggests that primitive navigation could have appeared first through reactive means and only later did internalized spatial memory appear (Reid et al., 2012).
2.3. Why do Cognitive Agents Offload?
Despite the advantages in employing cognitive offload, it is still unknown why these advantages are not obtained solely through internal representations. It is therefore also unknown why cognitive agents with access to both internal and external encoding still make use of both. Still, it is well established that, in humans, both internal and external factors play a role in determining offloading behaviour, that is, both play a role in determining when humans opt for one strategy or the other.
Regarding external factors, research with human participants consistently shows that increasing the cost or effort associated with offloading or decreasing the internal cognitive demands imposed by the task, participants tend to offload less often. Also, the availability of more intuitive tools seems to increase offloading behaviour. Individual differences in offloading behaviour across participants subject to similar conditions raise attention to the importance of internal factors. However, in the work of (Meyerhoff et al., 2021), individual offloading behaviour in one task was shown not to be related to individual differences in offloading behaviour in another task. In light of this, the tendency to offload information does not seem to reflect a general habit that consistently emerges across different tasks.
Still, working memory capacity seems to be an important factor as results from several sources (Ball et al., 2022; Meyerhoff et al., 2021; Risko & Dunn, 2015; Risko & Gilbert, 2016) show that participants with poorer capacity tend to offload more often. However, (Morrison & Richmond, 2020) failed to replicate these results or even observe that those with poorer memory capacity had greater benefits from offloading. This suggests instead that cognitive offloading may be an effective compensatory mechanism for individuals with a wide range of internal capabilities (Ball et al., 2022; Meyerhoff et al., 2021; Morrison & Richmond, 2020; Risko & Dunn, 2015; Risko & Gilbert, 2016).
A much more solid finding regarding internal factors guiding offloading behaviour pertains to the role of metacognition in the matter. What research suggests is that simple cost-benefit evaluations are not enough to explain offloading behaviour as offload is better predicted by participants' reported confidence of solving the task without external aids than by cognitive capability measures such as working memory capacity (Morrison & Richmond, 2020). Offloading behaviour can also be manipulated to some extent by manipulating the feedback offered to participants about their performance. This shows a reliance also on the subject's own beliefs about their internal capabilities, which might not be accurate. Also, explicit goals imposed on participants appear to influence offloading behaviour, with participants changing their behaviour to match task demands (Grinschgl et al., 2021;
Weis & Wiese, 2019). On top of this, research suggests that humans are unconsciously biased to avoid cognitive effort, this is evident as participants often offload more than what would be optimal and a bias towards underconfidence in one’s own internal capabilities doesn’t account for results obtained from settings where cognitive effort is rewarded (Kool et al., 2010; Sachdeva & Gilbert, 2020). The role of metacognition in cognitive offloading is interesting as it implies a complex internal architecture to handle this process of externalization (Boldt & Gilbert, 2019; Dunn & Risko, 2016; Gilbert et al., 2020;
Grinschgl et al., 2021; Kool et al., 2010; Malik & Annu, 2020; Morrison & Richmond, 2020; Risko &
Gilbert, 2016; Sachdeva & Gilbert, 2020; Weis & Wiese, 2019).
By looking at the developmental origins of cognitive offload, (Armitage et al., 2020) found that children aged 4-5 already employ physical rotation to offload mental rotation. This is surprising as this is the same age in which mental rotation emerges, suggesting that at least some skills might be externalizable right from the moment of acquisition. However, 4-5 year olds often performed indiscriminate offload, showing an intention to offload but not always exploiting the environment in a suitable way and not accomplishing any performance benefit. Tendency to offload was also shown to increase with age alongside the selectiveness of the offload, with 10-11 year olds offloading very often and mostly in an appropriate manner. Similar findings were reported by (Bulley et al., 2020) in a memory task requiring children to recall the location of hidden targets. In one experiment, children were made aware of the option to mark the target locations during the hiding period. At least 30% of children from all age groups offloaded cognition selectively but children over 8 years old almost always did so. Across ages, children offloaded more often when task difficulty was higher, with more to-be-remembered target locations. In a second experiment, children were to devise their own offloading strategy. Very few younger children could spontaneously come up with an offloading strategy by themselves but there was a significant increase of the rate of offloading after being prompted. By the ages of 10 and 11 the majority of children were able to devise the offloading strategy by themselves. In this experiment however, there was no noticeable correlation in offloading behaviour and task difficulty. The authors attribute this difference to an increased task difficulty as the time between the hiding and test phases was higher for this second experiment. Children benefited from offloading even in the easier conditions of the second experiment but not in the easier conditions of the first. These results suggest that a metacognitive awareness of the difficulties of thinking arises early in development and until 10-11 years of age arises the ability to mitigate these difficulties with external representation, which requires an awareness of how
to change the environment to compensate for cognitive limitations (Armitage et al., 2020; Bulley et al., 2020).
Despite the immediate task performance enhancement of externalization, internal encoding seems to be more easily accessed and impervious to environmental noise. Thus, offloading in agents with access to both internal and external encoding, is often interpreted as a simpler but suboptimal strategy employed only as a way to push past internal limitations. Since reactive behaviour that uses cognitive offload requires simpler cognitive architectures, it is expected to appear first during evolution.
Due to the challenge of evolving robot controllers with complex internal capabilities such as memory and learning, it was hypothesized that cognitive offload would be a local-optimum in the fitness landscape and thus moving on towards higher cognition would impose loss of fitness, blocking or retarding its emergence. This would be because selective pressures favouring the completion of tasks that require cognitive capabilities would not necessarily favour the intermediate stages of cognitive controllers. This theory was supported by results showing easier evolution of cognitive controllers with multi-objective selective pressures (Ollion et al., 2012) and novelty driven as opposed to objective driven search algorithms (Lehman & Miikkulainen, n.d.). This is also consistent with the idea that offloading replaces internal representations, leaving no benefit in evolving them when they are already externally accounted for. However, (Carvalho & Nolfi, 2016) utilized evolving artificial agents that had to navigate a maze towards a predefined destination according to a variable stimulus presented at the start of the task. These agents had to encode the information transmitted by the initial stimulus so as to know where to travel to and could do so internally or externally. Interestingly, their employment of cognitive offload, which consisted in following a particular wall of the maze according to the initial stimulus rather than remembering the stimulus to turn correctly at the maze’s bifurcations, was shown to be an important component of effective strategies that allowed agents to fulfil their goals and maximize performance. This coexisted with and promoted the evolution of complementary cognitive capabilities, contradicting the idea that it represented a local optima during evolution. Furthermore, their results suggested that encoding the same state both internally and externally might have brought some benefit. It was hypothesized that cognitive agents would employ offload to opt for a reactive solution with a simpler decision process paradigm, reducing the state space and consequently computational effort. Still, cognitive offload may be used to facilitate cognitive solutions without changing the decision process. For example, when one writes down a shopping list, information is externally encoded, however the task still can’t be solved completely through reactive means. Another interesting result obtained by (Risko & Dunn, 2015) showed that, in humans, offloading may be perceived as more effortful than relying solely on internal mechanisms, being employed as an attempt to maximize performance and not to alleviate task effort. However, participants still employed cognitive offload even when it presented no benefit. This seems to contradict the idea that cognitive offload releases internal resources and alleviates cognitive demands. These results together seem to indicate that internal and external representations might not be truly equivalent. It is then unclear what are the benefits of offloading and when do they outweigh its costs (Carvalho & Nolfi, 2016; Lehman & Miikkulainen, n.d.; Ollion et al., 2012; Pezzulo & Nolfi, 2019; Risko & Dunn, 2015).
However, in the work of (Carvalho & Nolfi, 2016), the decision to offload was implicit and relied mainly on exploiting features already present in the environment rather than changing the environment to actively encode information in it. Our current work shall expand on (Carvalho & Nolfi, 2016) by focusing on how cognitive agents employ offload in a different, but still similar, task where the action to offload will require an explicit decision and active manipulation of the agent’s surroundings. Enabling the agents to perform cognitive offload in this other way will also allow us to better observe if the cognitive agents will rely on a greater, a smaller or an equal extent on cognitive
offload than reactive agents and understand if indeed externally encoded states replace internally encoded ones or if the use of the two types of encoding complement each other (Carvalho & Nolfi, 2016).
3. Methods
3.1. Overview
The present study was an extension of (Carvalho & Nolfi, 2016) and features artificial agents differing orthogonally in two dimensions: a) internal cognitive capabilities, and b) access to cognitive offload. Cognitive capability and ability to offload were the independent variables. These variables were measured by their impact on the agent’s ability to solve a navigation task involving prospective memory in a double-T maze, making the agent’s performance the dependent variable.
Cognitive capabilities were manipulated by using different types of agents: cognitive agents and reactive agents. Access to cognitive offload was manipulated by providing or not the ability to control an on-board pen to the agents. This pen could mark the floor underneath the robot when activated and could potentially be used to mark the path a robot travels. This sort of marking represented a form of cognitive offload distinct from relying on the maze’s walls, observed in (Carvalho & Nolfi, 2016), as it implies a more active decision process and manipulation of the environment. By making use of the on- board pen, the agent has the control to place a stable mark when and where it chooses, so it needs a perfected decision process in order to make this an informative cue and make its environment richer than it initially was. This is a relevant difference between our study and (Carvalho & Nolfi, 2016).
Two types of cognitive agents were used. One of these had full access to cognitive offload. The other had its access to offload restricted by not having the ability to control the introduced on-board pen, which was locked in a ‘not-writing’ state, but these agents were still able to exploit their environment’s structure. Only one type of reactive agents with the ability to control the on-board pen, thus full access to cognitive offload, was used. Each of these types was defined by a robot controller in the form of a specific neural network that was trained for our task using an evolutionary algorithm. We have opted to train our neural networks through an evolutionary algorithm because it allows our agents to freely determine how they should solve the task with minimal constraints imposed by us (Gigliotta & Nolfi, 2008). This was already shown to be a feasible approach to train neural networks for this sort of task (Carvalho & Nolfi, 2016; Ollion et al., 2012). The genes encoded the weights, biases and time-constants of time-continuous neurons of the neural network of the robot controller. How evolved agents performed and how they relied on cognitive offload was analysed. The results were then performance measures for each evolved neural network under several different test conditions. The testing conditions involved submitting the evolved agents to conditions where their access to cognitive offload differed from their evolutionary setting, either by blocking their ability to control the pen (when applicable) or by introducing different noise regarding their initial position. Our rational was that the greater the noise, the greater would be the difficulty in relying on its environment to deploy cognitive offload. If, for example, an agent would have an hindered performance when dealing with increased noise, this would be because it could no longer deploy cognitive offload effectively, and it relied on cognitive offload to achieve its previous performance. It could instead merely mean that the agent’s network was overfitting to the input data and its behaviour then suffered from novel initial orientations. However, reactive agents
resistant to this condition will have to imply they robustly deploy cognitive offload, as these will always depend on environmental conditions. Cognitive agents resistant to initial orientation noise could either be because of their use of cognitive offload or because they adaptively employ internal representations.
A similar logic should be valid for the manipulation of pen access: if an agent suffered from no longer being able to deploy this form of cognitive offload, it was because it relied on this ability. If the agent would resist the removal of the pen, then it would be because it only required internal representation.
For this latter case, overfitting does not seem to be a confounding factor as the disabling of the pen is not an alteration to input data itself.
Our environment and the robots used in our project were simulated using the software Webots.
Webots is a robust and deterministic robotics simulator that utilizes a physics engine. Webots is realistic enough so it can be used to train robot controllers to be used in real robots (Michel, 2004). It also comes with a library of premade assets, which significantly reduced the time required to implement our project.
Simulation world files and the python scripts used to implement this project are available here:
https://drive.google.com/drive/folders/1KQ1B1hQR-XmX8aNBPMrGNSf6X0463UWA?usp=sharing
3.2. The Task
The task the agents were required to complete was divided in two parts. In the first part the agents were sent out to find a particular goal destination in a double-T shaped maze, akin to the task already employed by (Carvalho & Nolfi, 2016). The double-T maze is a straight corridor that in the end bifurcates in two T-shaped mazes. At the start of the task, in position S at the beginning of the first corridor (shown in Figure 3.1), the agents were presented with one of four stimuli. This stimulus had the form of a coloured patch of floor at the robot’s initial position S. The colour of the stimulus then defines the four possible scenarios the agent could be subject to which we labelled from 1 to 4.
According to the colour presented, the agent had to travel either to the bottom-left terminal of the labyrinth, in case of scenario 1; the top-left terminal in the case of scenario 2; the top-right terminal, in case of scenario 3; or the bottom-right terminal in case of scenario 4. Each of these four terminals of the maze is also coloured in a way such that when the robot reached the correct destination it perceived the same-coloured stimulus as the one presented at the start of the trial. This setting forces an agent to somehow retain information about what was the presented stimulus in order to turn in the correct direction when it later reaches the first and then the second bifurcation of the maze. This is then a task which implies the use of prospective memory to fulfil a delayed intention. The agent could either retain this information through use of internal memory or through use of cognitive offload. In (Carvalho &
Nolfi, 2016), cognitive offload happened through exploitation of the maze’s environmental structure, when the agent, upon perceiving the initial stimulus, approached a particular wall of the maze and then simply followed it until the end of the labyrinth.
Figure 3.1: Screenshot of the double-T maze. The starting position of the robot, marked as S, will be near the centre of the white coloured square. This white square, at the start of each trial, will change its colour to present the colour of a square located at one of the maze’s terminals (top-right, top-left, bottom-right, or bottom left), indicating where the robot should travel to. These terminals are labelled from 1 to 4. The robot never perceives the initial stimulus as white since the square changes its colour before the robot’s controller is initiated.
After reaching the goal destination the second part of the task begins and the agent is then required to return to the starting position, S. This creates the need to also retain information about how to return to this starting position, which may be done internally or externally. The on-board pen was implemented to aid the agents in this second part of the task: by strategically using the paint markers the agent would be able to record in the environment the path travelled towards the goal destination and retrace it to the initial position. Neither this second part nor the ability to mark in the environment were present in (Carvalho & Nolfi, 2016) and it was added as a way to better observe cognitive offload.
Performance in this task is measured as the distance an agent travels towards its destinations within the maze.
3.3. The Robot
We used two different robots in this study, Robot1 and Robot2. Both were differential two- wheeled robots. Regarding Robot1: At the front of the robot is a bumper-type collision sensor and 4 infra-red (IR) distance sensors, each of which shoots two 0.3m rays with an aperture of 1 mm between them. For the robot to perceive the markings on the maze's floor, on its bottom surface there are an additional 2 IR sensors facing the ground, each shooting a single 0.3m ray. A pen was also present at the centre of the robot’s body and when activated it painted a white circle with a radius of 0.1m on the floor underneath the robot. Robot2 differed from Robot1 in that, instead of four frontal IR sensors it had, instead, 24 IR sensors on the centre of its top surface. Each of these IR sensors shot a 0.5m ray and was displaced 15º from its neighbours, granting the robot perception of its surroundings in a 360º radius.
Figure 3.2: Screenshot of the virtual Robot1 in Webots. A is a screenshot taken from a top-view, B from a frontal view and C from a bottom view. The red cubes represent the IR sensors, two are facing the ground while the other 4 are facing onwards from the front of the robot. The green parallelepiped is the bumper sensor. The dark blue circle at the centre of the robot, visible only from below, is the tip of the on-board pen.
Figure 3.3: Screenshot of the virtual Robot2 in Webots. A is a screenshot taken from a top-view, B from a frontal view and C from a bottom view. The red cubes represent the IR sensors facing the ground. The green parallelepiped is the bumper sensor. The crown of rays on top of the robot are the rays shot by the 24 IR sensors placed on its top, the sensors themselves are invisible. The dark blue circle at the centre of the robot, visible only from below, is the tip of the on-board pen.
3.4. Reactive Controllers
Two reactive controllers were designed for this study. Both reactive controllers were simple feedforward neural networks that always responded similarly to the same sensory state. These networks consisted only of an input layer fully connected in a feedforward fashion to an output layer. Although these controllers were conceptually equal, they differed in the quantity of input neurons, to accommodate the different sensors of each robot.
