Simulated Consciousness in Automatic Pattern Recognition

This paper will be presented in Stream 1 (Outer and inner empiricism in consciousness research) at the Mind-4 conference, Dublin City University, Dublin, Ireland, on August 17, 1999.

Consciousness is discussed in terms of local and global aspects of sensory data. It is shown that it is similar to progress in natural science in general and to some phenomena in the training of automatic pattern recognition systems in particular.

Keywords: Consciousness, local and global awareness, automatic pattern recognition, artificial neural networks

[02] Once consciousness is defined, maybe even partially, in a recognizable and acceptable way, we can try to simulate it in a machine, a robot or a computer. The purpose of this effort is to achieve a better understanding of the similarities and differences of the natures of man and machine. Even the attempt to define and build a simulator serves this purpose, also if the state of a real experiment is never reached. For a good definition and design, consciousness has to be studied sufficiently well, which already yields a growth of knowledge. Observing the results of a running simulator is equivalent to observing human behavior. Both inform us just indirectly on what is happening with respect to, or as a result of their consciousness.

[03] As consciousness is a very fundamental aspect of each human ability, it is to be expected that aspects of it can be recognized in the more complicated systems that simulate human behavior or abilities. It is the goal of this paper is to show that this is the case for automatic pattern recognition systems. These systems are able to generalize from examples in recognizing printed or written characters, can find particular types of blood cells, can recognize objects for robot vision, etcetera. Moreover, it will be argued that progress made in natural science is similar to consciousness.

[04] In the next sections we will present our own description of consciousness, discuss the implications for simulators and analyze some examples from the area of automatic pattern recognition.

[05] One of the basic distinctions we can make is that we have two types of experiences, outer and inner ones. The first are related to the outside world and are caused by objects, light, sounds that live in our common environment. The ways they sound, reflect, taste can be traced back to the objects themselves. Consequently, they can be discussed and compared among the observers. The second group is related to our inner world. They mainly depend on our body, on our private past and memories and on our individual character. So we see the same needle, but we don't necessarily feel the same pain when it sticks our finger.

[06] For the discussion of consciousness this distinction is not sufficient. In particular the often described needle-pain example is too thin. It is not just a matter of objects, senses and awareness. The area of our thoughts has to be included. At first glance thoughts belong to the inner world. However, at the moment they refer to clear concepts, physical objects as well as abstract ideas, they can be shared by others. This follows from the fact that we can discuss objects, even if they are not available for the senses at that moment, we can discuss with others our ideas and arguments for decisions and actions. We even can express some abstract ideas in mathematical symbols and define computational rules for them without much confusion with other people. So the basis for our thoughts, the spiritual world of ideas, is shared. It is common for a comity of experts as good as the world of objects is common for a group of observers. In fact, science in general is only possible because there is a common world of ideas.

[07] On the other sides, our inner feelings for the beauty we observe in the rose, for the joy we hear in the music, for enthusiasm caused by the thoughts, are not a part of these shared worlds. These inner experiences can in no way be shared, although they may be caused by shared observations and thoughts. We will call this private world the soul. So there is a physical world of objects outside us that includes our body, senses and nerves and there is a common spiritual world of ideas and there is our private soul connecting the two.

[08] Our senses observe the physical world just locally, limited in time and place and restricted by various circumstances like light and position. The world of ideas is global, it covers objects, object relations and laws of physics that are not or cannot be observed directly at that moment, at that place. Our momentary observations have to be integrated in this global structure. This local to global transformation is an act of, or is caused by our consciousness. It is a unique operation that is entirely dependent on the time and the place of the observations by the senses on one side and by our thoughts on the other side. It is therefor, by our definition, an act of the soul.

[09] Consciousness is here defined as the integration and interpretation of the local observations into a global structure. At the moment it fits, the observer becomes conscious: he sees and understands his position in space and time.

[10] We defined the soul as an inner state not belonging to the outside worlds. As a consequence it cannot be observed externally. This holds also for consciousness as it is an act of the soul. The direct consequences of consciousness, however, the way observations are integrated into the global framework of understanding, are accessible for investigations. So directly we know nothing of the consciousness of another person, but indirectly, by studying his behavior, we can relate it to our own consciousness.

[11] The given definitions of soul and consciousness can be applied to machines if these machines bridge the gap between the two worlds. So, they have to be instrumented with sensors and the sensor signals should be matched by the machine with an existing model of the world observed by these sensors. Most machines have not-observable inner states due to sensor noise and not exactly computable processes due to its never infinitely exactly known physical structure. Therefore the resulting operation of such machines cannot be predicted exactly. In practice this is only of importance in case of highly nonlinear or chaotic machines, see also Penrose (1994) .

[12] Digital computers don't have hidden, inner states. Everything inside them can be exactly measured and predicted, for instance by a copy of the computer, or sometimes even by a different computer running the same program. The consequence of having no inner states is that a digital computer with digital inputs (so we assume sensors and digital-analog conversion to be external) has no soul and thereby no consciousness.

[13] Briefly formulated: a computer has no hidden states, is not unique, and consequently has no consciousness. This lack of identity makes the computer transparent and thereby perfectly suitable for simulating and studying other processes, including consciousness. This transparency is heavily used in the field of automatic pattern recognition, in which researchers try to integrate sensor data with existing global knowledge. Below some examples are given.

[14] In automatic pattern recognition (Gose, Johnsonbaugh, & Jost, 1996) one tries to learn from sensor-based measurements of objects (like handwritten characters, chromosomes, weather patterns, traffic participants) given some global knowledge of the objects (the most relevant features), the sensors (sensor physics) and some example objects. The goal is usually to find in an automatic way a classifier that discriminates between certain object types. In this field ones tries to find an automatic solution (in fact to learn it by a machine) for an ever returning scientific problem: how to increase the knowledge using new incoming measurements. This knowledge is global: not related to a particular sensor signal. So here we have an attempt to automize a process that is related to consciousness as described above. What we like to find here in studying this process is what exactly happens if local observations are transformed into global knowledge.

[15] The analysis of sensor data conditioned by the global effect of the results is really a problem as will be illustrated by the following example. Traditionally pattern recognition systems were designed with as few parameters as possible. The more complex a system, the more parameters have to be estimated using the data, the larger the total estimation error, the less accurate the final result for a fixed data size. With the reintroduction of artificial neural networks in the pattern recognition field between 1985 and 1990 this traditional rule was broken (Duin, 1994). Neural networks are complex systems with many parameters to be estimated. What happens with the classification error during training of a neural network is shown in Figure 1.

[16] Two curves are drawn. One is the error on the training data, the apparent error e_A. This is roughly the error that the neural network training procedure tries to minimize. Consequently this is a monotone decreasing function. The longer the data is fed to the neural network, the better its description is. The other curve is the error e_T on an independent testset. This is an estimate of the true error, that is, of how well the neural network describes data of the same nature that were not used for training. The true error shows a minimum. Because of the overly great complexity of the neural network, it adapts itself too much at a certain moment to the (noise in the) training data and loses generalization performance. Various techniques exist to deal with this overtraining phenomenon.

[17] What we see in this figure is the conflict between local and global performance. A system focussing too heavily on its primary task is not "aware" of its global performance and thereby neglects the effects of its efforts on the world outside, not used for training. One possible way to deal with this is to give the training system senses for global performance as well. In that case an independent second data set is used only for monitoring e_T, as in the figure. At the moment this error increases further, training is stopped. So now we have a system that optimizes in its own way the global effects of its local efforts. This can be considered as a simulation of consciousness as defined above. The system does the same thing we do if we look up from our work to inspect what the result of our efforts is for the rest of the world.

[18] In Figure 2 an example is given from image analysis. Left a microscope image is given of some gold particles embedded in glass. This image contains noise. If it is thresholded in order to retrieve the particles errors are made (middle). The objects show some holes that these particles never have in reality. Using this knowledge the incorrectly classified regions can be corrected (right). Again we see here that the local operation, thresholding, has to be corrected after the results are confronted with global knowledge.

[19] The two examples have in common that they consist of just two steps: one focussing on the measurement data, one on the global world. Many attempts have been made to do this iteratively, possibly aiming at a final total integration. See for instance the efforts that are necessary to combine and use the two dual representations in geometric algebra (Hestenes & Sobczyk, 1992). Until now, this has seemed almost impossible in the field of pattern recognition, because of two effects. First, if measurement data are analyzed using a heavy component of knowledge of how they should be, of what can be expected, it is very difficult to learn something new from them. Second, if global and local observations are analyzed in the same process, the global ones gradually lose their universal meaning for the analysis process, which becomes focussed on its details.

[20] Understanding the world from given knowledge and new observations is the general problem in science. This understanding, bridging the abyss between low level sensor signals and the high level knowledge is similar to reaching consciousness. At the moment one understands new observations in terms of the ideas and relations obtained from the past one reaches a new level of consciousness, one awakes in a richer world. Both worlds, the one defined by the direct, local awareness of the senses, as well as the world of global knowledge, are shared between observers. These observers may disagree, because of different points of view, but they are able to discuss them as they have a mutual understanding of the worlds themselves. What happens in between is hidden from the outside. Here, in the soul of man, or in the soul of the machine, a strictly private jump is made over the abyss between the two worlds: from local sensors to global understanding, see also (Perlovsky, 1998). This jump is entirely determined by the unique parts of the physical structure, e.g. as described by Penrose (1994). Computers don't have such a unique part. Consequently their jump can be defined and investigated externally. So they are perfectly suited for simulation studies

[21] What we observe there is the following. If we want to increase our knowledge in a certain field, we may focus on new observations. The analysis of these observations can best be done with just a minor use of the original knowledge, so in isolation. If we afterwards return to the original field, we become conscious of the differences, of what is just locally valid and of what is supported by the global picture. At this moment of integration, we become conscious of the relation between original field of interest and the isolated area in which the observations are made.

Block, N., Flanagan, O., & Güzeldere, G. (1997). The nature of consciousness. Cambridge, MA: MIT Press.

Duin, R. P. W. (1994). Superlearning and neural network magic (IAPR discussion pages). Pattern Recognition Letters, 15, 215-217.

Gose, E., Johnsonbaugh, R., & Jost, S. (1996). Pattern recognition and image analysis. Englewood Cliffs, NJ: Prentice-Hall.

Hestenes, D., & Sobczyk, G. (1992). Clifford algebra to geometric calculus. Dordrecht: Kluwer.

Penrose, R. (1994). Shadows of the mind. Oxford: Oxford University Press.

Perlovsky, L. I. (1998). Conundrum of combinatorial complexity. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20, 666-670.

Searle, J. (1997). The mystery of consciousness. New York: New York Review Press.

Taylor, J.G. & Freeman, W. (Eds.). (1997). Neural networks for consciousness. Neural Networks (special issue), 10(7).