Work plan

The project is composed of 4 work packages (WP). The WP0 is dedicated to the administrative management of the project, while the 3 others are centered around our scientific objectives. Each workpackage is divided into different tasks. The general organization of work packages and tasks is summarized in Table 1.

Study and implementation of an SNN (Objective 1), CerCo, IRIT.

This workpackage will be mainly managed by the partner CerCo (assisted by IRIT for certain tasks), and will cover Objective 1. It concerns the study and the implementation of an SNN.

Task T1.1

The state of the art is advancing rapidly in this area, and we are planning 4 months in order to deepen our knowledge of the latest models and work carried out on the different learning strategies of these networks (unsupervised, by translation, by error backpropagation, etc.). Not being parallel programming specialists, we are also planning 2 months to learn the basics from IRIT colleagues (team STORM). This task will be carried out jointly by the two partners and will give rise to a report.

Task T1.2

This task (6 months, led by the doctoral student, assisted by IRIT and CerCo) will be dedicated to the optimized implementation of a large SNN (of the order of a billion neurons). It will be assumed that a stimulus excites the input neurons sequentially (to take into account the order of the excited concepts), and only the output neurons concerned will have to check, at each excitation, whether they have recognized a pattern or no. Each output neuron will work in a time window of size fixed by a parameter in order to reset its state if it has received insuffcient excitations during a given period of time. The implementation will be highly parallelized on a GPU. The development of the implementation can be done on an example of word recognition for example. The doctoral student will be the driving force behind this task, assisted by the two partners. The code, as well as a short report presenting the model and its use will be the subject of a deliverable (D1.1).

Task T1.3

This task (6 months, carried out by the doctoral student and CerCo) will consist in extending the previous model in order that the network can learn automatically during a specific phase, and according to the JAST algorithm of which we spoke above. The tests will be carried out on the same type of example, and by the same partners. The code, as well as an additional report on the learning model and its use will be the subject of a deliverable (D1.2). Not being specialists in parallel programming and optimization, we will call on a subcontractor (3 months) to optimize the code developed previously (D1.3).

Task T1.4

This task (3 months) will consist of implementing the model from T1.3 by providing it with a graphical interface allowing network configuration (size, parallel algorithm or not, initialization input concepts, etc.) and view results in an interactive and user-friendly way. As this task is not in the field of research but of engineering, it will be entrusted to a subcontracting computer engineer. The goal is to provide a program that can be used by non-specialists in neural networks (D1.4). This will help disseminate (under an open source license) somewhat generic results of the project and promote it.

Task T1.5

This task (8 months, carried out by the doctoral student, IRIT for the knowledge of the laws of action, and CerCo for the structuring of the SNN) will also start from the result from T1.3 and will specialize the network for action recognition. We will start by providing the network with simple actions such as: if we have the goal of drinking or giving something to drink (in accordance with the description of Objective 1, the goal will make it possible to restrict all of the relevant knowledge of agent to provide to the SNN), and that you are thirsty, that you have an empty glass and a bottle, then you can help yourself to a drink. If the bottle is empty, you can go fill it, or buy one, or request one, etc.. The various situations will be used to train the network. We can encode more specialized situations (compared to more generic ones), by playing on dierent thresholds among the output neurons. The more specialized neurons will require more excited connecting neurons. Eventually, we will rely on corpora from robot competitions to have diversified lists of descriptions of actions. We will develop a description of how this module works and how to use it (D1.5).

Cognitive architectures and NNs (objective 2), IRIT

Task T2.1

During this task (9 months, led by IRIT), we will define an epistemic logical language whose semantics will be defined with belief bases. The point that will deserve the most attention is to be able, on the one hand, to associate a set of concepts with each formula (3 months), and on the other hand to define thematic sets of formulas in depending on the current goal (6 months). In the description of Objective 2 we have given some leads on this subject. First, we will use meta-processing on the formulas (excluding the object language of logic). Depending on the level of diffculty, we may consider including them in the language. A technical report will be produced at the end of this task (D2.1).

Task T2.2

This task (9 months, led by IRIT) will consider (while waiting for T1.5 to be completed) that a function external to the logic plays the role of an oracle to predict the result (s) of the SNN for a given set of beliefs. We will extend the language defined in the previous task in order to make the agent’s belief base evolve according to the result of the actions carried out. Partner IRIT has already investigated the case of a belief base expansion by a new information, but not yet the case of a belief revision. The introduction of temporal indices could be helpful to describe the base at different temporal steps (agent memory).

Moreover, the whole point of a neural network is that it is capable of making non-monotonic inferences… if it is well trained and functioning. Wethink that it will be diffcult to obtain a perfect learning of the network in the case of the laws of action (it is very complex), and this is why we want to endow the network with the ability to learn “on the fly” (see T2.3).
One consequence is that the network can provide none (and here we won’t be able to do anything other than resubmit the case so that the network learns it) or several answers. An important part will be to choose which action to take in the event of a multiple response. For example, if a “more specialized” neuron has triggered a spike, then the neuron corresponding to the “default situation” will also have triggered a spike. However, these actions can be contradictory. For example, if I have an empty glass and water (the trigger threshold of the neuron is equal to 2) I can help myself to drink; if I have an empty glass which is dirty and water (here, the threshold is 3) I must rather wash the glass before: if the current situation validates the context 2, then it also validates the 1. Several strategies are possible (but it is necessary to determine which it is the best according to the cases): either one draws at random, or one takes the action of which the threshold of release is the highest (the action is more specialized). A third strategy might be to sort and test them. A technical report will describe this extension (D2.2).

Task T2.3

This task (15 months, carried out by the post-doctoral fellow) will consist of studying the update in the SNNs. The first step is to determine how to update an SNN locally (12 months). In particular, to achieve Objective 2, it will be necessary to determine whether JAST allowsnewlearning (this assumes that the output neurons are not already “frozen”), or whether it is necessary to work specifically on the structure of the network (rearrange “by hand” the connection neurons of certain output neurons.


This task will be concomitant with the second half of T1.3. (On his/her arrival, we allow 3 months for the post-doctoral fellow to be able to familiarize himself/herself, in general terms, with the formal tools used by IRIT as well as 3 months to become familiar with the SNNs, which shifts the starting of this task.) This is a delicate task which will require starting with very simple examples, and going towards more complex examples (especially in terms of the number of concepts to update). It will be a question of defining a formal model (D2.3). In a second phase (3 months), it will be necessary to proceed, as far as possible, to the update of the implementation (D2.4). (By counting 3 months for the implementation, and 3 months for the publication of its results, this task will occupy almost exclusively the post-doctoral fellow.)

Hybrid architecture (objective 3), both CerCO & IRIT

Task T3.1

The first phase of the assembly will be to implement the logical reasoning kernel defined in WP2 (12 months, PhD student, helped by partners). At the end of this task, we will have a module (D3.1) written in epistemic modal logic, having a translation into propositional logic, and capable of using a SAT solver to check the satisfiability or the validity of a formula. For this purpose, we will make the output language of the module compatible with the input language of TouIST because this one will automatically take care of the conjunctive normal formatting of a formula, then of its translation into DMACS language that it will then submit to an SAT solver. We are optimistic about this task, insofar as the IRIT partner will be able to benefit from the expertise it has developed in this area during the ANR project CoPains. The start of this task has been postponed to T0+19 so that the doctoral student can work on the T1.3 task.

Task T3.2

Following the completion of the previous task, it will be necessary to implement a module to

review / update the knowledge base of the artificial agent (9 months, thesis student, helped by partners). We are thinking of using a module that will be external to the logic, which makes it relatively easy to implement on a knowledge base. Due to T2.2, we will be able to relate the logical implementation to the implementation of SNN: in T2.2, a function played the role of an oracle giving the SNN’s response. Here, it will really be the response(s) from the SNN that will be provided to the logic module. The critical part will be to implement the strategies studied in T2.2 in order to choose an answer in case of multiple answers. The new module will be our deliverable (D3.2).

Task T3.3

In the last three months of the project, a subcontracting engineer will develop an application (D3.3) from the proof of concept resulting from the previous task so as to be able to distribute it in open access and make it usable by as many people as possible. This will help to disseminate and enhance the results of our project.
This period will be devoted, for the partners, to writing an article summarizing the results obtained or can be used to finalize the results in the event of delay. The doctoral student will be in the final phase of writing their manuscript.
The goal of this project is to provide a proof of concept. As said above, the greatest diculty for a scaling up in a real domain will be linked to having sufficient and varied data concerning the actions. This will eventually constitute the continuation of this project, which will require working with linguists or ergonomists in particular.