From Deep Learning to Deep University: Cognitive Development of Intelligent Systems

. Search is not only an instrument to find intended information. Ability to search is a basic cognitive skill helping people to explore the world. It is largely based on personal intuition and creativity. However, due to the emerged big data challenge, people require new forms of training to develop or improve this ability. Current developments within Cognitive Computing and Deep Learning enable artificial systems to learn and gain human-like cognitive abilities. This means that the skill how to search efficiently and creatively within huge data spaces becomes one of the most important ones for the cognitive systems aiming at autonomy. This skill cannot be pre-programmed, it requires learning. We offer to use the collective search expertise to train creative associ-ation-driven navigation across heterogeneous information spaces. We argue that artificial cognitive systems, as well as humans, need special environments, like universities, to train skills of autonomy and creativity.


Introduction
Current research in Artificial Intelligence emphasizes the role of human-like cognition modeling for artificial systems to enable automatic handling of complex situations in realistic environments by using such traditionally human capabilities, as learning, adapting, interacting, and understanding [1].Cognitive systems can already recognize image and speech, translate texts, identify diagnosis, filter content, make user-oriented recommendations.They are becoming more accurate in predictions, analytics and decision making due to the ability to process huge amounts of raw data and get deeper insights leveraging on deep non-linear learning models configured by people [2].The next step is to ensure the real depth of cognition and autonomy of intelligent systems: a system should be aware of its current features and abilities, and capable of self-development, self-configuration and self-maintenance which means to have and understand its own desires, intelligently form novel ideas, theories, inventions, do research, set up own goals, reach them and understand why and how [3].This is addressed by increase of the systems` creativity and efficiency [4,5,6].
In this research we focus on the techniques and architectures enabling cognitive development of intelligent systems.An intelligent system (either human or artificial one) cannot handle the complexity and the infinity of the open world at once as an input.It applies a combination of focused sensing, searching, asking, querying and browsing strategies, makes on-the-fly selections and navigations, studies and assesses search results, compares and integrates the obtained information, and finally develops new knowledge and abilities.Such exploratory search is used for exploration of the information space and satisfaction of the system`s informational needs [7].It is an iterative process comprising sequential refinement of search activities and update of search objectives by ongoing shift of search focus (specified, e.g., by keywords in Web search) of the system.Ability to switch the focus consciously is a required feature of the intelligent autonomous system.In this research we demonstrate which data models and techniques can be applied to train this ability in cognitive systems.
Information retrieval in Web is traditionally performed by use of keyword-based models.In [8] we showed a model of exploratory search in big data based on an association-driven navigation in the space of collective experience created by sets of keywords used for search queries.In this paper we argue that the same model in a more abstract form can be used for more generic creative association-driven navigation in heterogeneous information spaces.
We introduce a concept of the university for everything or "deep university", which can be launched as a creative, evolving and collaborative training environment for artificial and hybrid (Collective Intelligence) cognitive systems, which guides the course of cognitive processes and provides a cognitive system (a "student") with the information resources for the increase of the self-awareness and autonomy by deeplearning-driven self-development of capabilities to be curious, to ask questions (formulate queries) intelligently and creatively, find answers and make decisions.

2
Education for everything.Universities for cognitive systems

Growing fully developed cognitive systems
The old-fashioned view at an Artificial Intelligence system has become obviously too primitive.According to the problem-solving paradigm, an intelligent system is capable of getting a problem (from a human) and solving it by breaking into a set of intelligent tasks characterized as either a machine-learning problem, a planning problem, a theorem-proving problem, etc., autonomously or in collaboration with humans.However, the point is to build an intelligent system implementing human-like intellectual capacity to full extent: beginning from the ability to really want to do something or being curious to explore something, generate ideas, discover and formulate problems, define tasks intelligently, handle them creatively and consciously, and impact the environment synchronously with the self-management.Current separate machine learning techniques, even those called deep ones, cannot ensure all these, espe-cially for heterogeneous-multi-task handling systems.Advanced machine learning must ensure that a computational cognitive system meets all functional criteria, defined by Newell for cognitive architectures [9], such as, exhibiting rational, effective adaptive behavior, using vast amounts of knowledge about the environment, ability to integrate diverse knowledge, learning from the environment, acquiring capabilities through development, exhibiting self-awareness and self-consciousness, and others.
In the human world these issues are addressed by education which is an important enabler of human development implying expansion of capabilities (also creative ones) and freedoms.Schools and universities serve as centers for intellectual capacity building, verified information share and exchange.Likewise, a modern artificial cognitive system must be well-trained.We argue that (deep) learning for a machine is a dynamic, evolutionary process, very similar to traditional higher education, however, with new challenges and features.It facilitates comprehensive acquisition of different skills at all the major cognitive levels, leveraging on collaboration in creative, dynamically changing ecosystems, similar to those built around universities.
In order to address arising motivations and, therefore, survive and be effective, a system must have some basic instincts of curiosity, imagination and intuition.These three contribute to a system`s creativity and proactivity, enabling anticipatory, change-oriented and self-initiated behavior.Such preconditions for self-awareness and consciousness of a system will drive further its self-organization, selfconfiguration and self-maintenance.
Learning of cognitive systems as "students" in future universities is deeper than can be provided according to any traditional approach to education due to a multilayered comprehensive structure of the study.Basic features of deep universities are: ─ generation of dynamic artificial curricula on demand: to satisfy a system`s ongoing needs and curiosity; ─ a (deep) structure of the generated curricula consisting of a set of training courses for different cognitive functions of a system; ─ a deep structure of the "training courses" (smart syllabi) providing the hierarchical increase of the cognitive complexity of learning outcomes, according to the revised Bloom's taxonomy [10] and Webb's depth-of-knowledge levels [11].Within a "training course" a system learns remembering facts, understanding them, applying them, analyzing, evaluating and, finally, creatively developing new entities; ─ application of different machine learning methods for different tasks: supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning.

Implementation of developing cognitive architectures
System development is a process of self-configuration and self-maintenance, initiated by a system itself.A system finds a problem of an absolutely new type and needs configurational upgrades to be able to solve it.It specifies its learning requests and expectations (in terms of learning outcomes) and learns from the collaborative training environment, increasing its useful knowledge and enhancing its selfconfigurational abilities, i.e., skills how to choose cognitive models and adjust configurational hyperparameters of each model for different tasks.By this interpretation a cognitive system plays both roles: a "learner" and a "university".
This idea brings us to the basic (executive) structure of a cognitive system consisting of three main components: (i) a creative idea generator, (ii) a work planner, which defines a problem as a cascade of specific tasks according to emerged ideas, and (iii) problem solvers.On the top we have a learning subsystem (deep university) training each component of the executive subsystem to perform intelligently and creatively according to the artificial curricula developed for the system (see Fig. 1).

Learning from collective creativity
In the era of digitalization, the cognitive mobility of the society is a new mission and a new challenge for the universities.Such mobility is achieved due to the creativity of cognition: curiosity and independent search, free and smart orientation and selfnavigation in big data, the ability to ask new questions and find non-trivial tasks.Creativity and curiosity are recognized as the main 21st-Century Skills [12].
A classical university is a place of accelerated human development.One of the most important functions of a "deep university" is to train creative (exploratory) search skills based on the awareness of relevant collective experiences.Similarly to learning intelligent problem-solving skills in a human-like manner (i.e., extracting feature hierarchies with deep learning techniques), a cognitive system needs also to learn how to satisfy its curiosity and develop imagination and intuition.One possible approach is to leverage on collective creativity experience.
We use a TB network data model for collective experience representation.It is an evolution of a specific tree-based data structure called a TB (There-and-Back) structure [13], which we have previously used for modeling collective experience of a keyword search in [8].We focus on the sequential, interactive and associative nature of creativity which can be viewed as exploratory search in the collaborative creative space of states.Such search is driven by a smart iterative focusing where the focus (e.g., set of keywords for search or set of queries to sensors, etc.) at each stage is defined by the system itself.Creative space is organized as a network of concepts or cognitive states (e.g., objects, events and ideas) interconnected by links or pathways reflecting the strength of association between pairs of concepts (see Fig. 2).The input layer of the network contains nodes denoted by a set of possible initial states, the output layer keeps final statescuriosity satisfaction and creative ideas generation.Creativity process is therefore described as a chain of cognitive states   , from motivation to satisfaction.We call them trails because we use swarm learning algorithms for machine training on top of this data model:   ≔  1 ,  2 , . .,   , where  1motivational state,  terminal state of satisfaction.Links in the trails are creator`s associations representing a transfer from the output of one cognitive state to the input of another.The trails are combined into an associative network.The space is constructed by the algorithm of TB feeding described in details in [8].It is an iterative process of incorporation of new trails into an existing structure.New implicit relationships, thus new creative trails, can be automatically inferred over the basic structure.
In this case creativity learning is based on exploratory search in the previously formed creativity space.
Formally, a machine interacts with the creative space in discrete time steps.At each time step, a machine observes the environmental state   and applies an action according to the association   .Depending on   and   , the system transfers itself into a new state and receives a real-valued reward  ∈ .The system's goal is to maximize its expected cumulative reward, called return .The result is a map of cognitive states that generates an action for any given state.We suggest using swarm intelligence methods, e.g., the ant colony optimization algorithms, for finding good paths in the creative space by laying down pheromone trails and calculating reward obtained by a system in each state based on it.

Conclusions and future work
Nearly magic capabilities of cognitive computing and deep learning are really impressive.Despite this obvious success, there is still a great amount of limitations and restrictions in both cognitive systems and deep learning algorithms.We believe that the most challenging, interesting and demanding issue in cognitive computing is building cognitive architectures for artificial systems with capabilities of self-exploration and self-development.Big data around and within such systems requires smart (driven by best collected practices) iterative focusing to manage it autonomously.We find it reasonable to draw a parallel between a system and a human development and argue about the need of special universities for artificial systems` comprehensive cognitive development.Within such universities a system will consciously develop its cognitive skills to explore (search, sense and understand what to search, why and how) the external physical and abstract world together with itself smartly and creatively.

Fig. 2 .
Fig. 2.An example of a creative space construction