当前位置：文档库 › 神经-符号系统

神经-符号系统

60Garcez and Zaverucha

employed in symbolic machine learning is to?nd hy-potheses that are consistent with a background knowl-edge to explain a given set of examples.In general, these hypotheses are de?nitions of concepts described in some logical language.The examples are descrip-tions of instances and non-instances of the concept to be learned,and the background knowledge provides additional information about the examples and the con-cepts’domain knowledge[1].

In contrast to symbolic learning systems,neural net-works’learning implicitly encodes patterns and their generalizations in the networks’weights,so re?ect-ing the statistical properties of the trained data[5].It has been indicated that neural networks can outper-form symbolic learning systems,especially when data are noisy[3].This result,due also to the massively parallel architecture of neural networks,contributed decisively to the growing interest in combining,and possibly integrating,neural and symbolic learning sys-tems(see[6]for a clarifying treatment on the suitability of neural networks for the representation of symbolic knowledge).

Pinkas[7,8]and Holldobler[9]have made important contributions to the subject of neural-symbolic integra-tion,showing the capabilities and limitations of neu-ral networks for performing logical inference.Pinkas de?ned a bi-directional mapping between symmetric neural networks and mathematical logic[10].He pre-sented a theorem showing a weak equivalence between the problem of satis?ability of propositional logic and minimizing energy functions;in the sense that for every well-formed formula(wff)a quadratic energy function can ef?ciently be found,and for every energy func-tion there exists a wff(inef?ciently found),such that the global minima of the function are exactly equal to the satisfying models of the formula.Holldobler pre-sented a parallel uni?cation algorithm and an auto-mated reasoning system for?rst order Horn clauses, implemented in a feedforward neural network,called the Connectionist Horn Clause Logic(CHCL)System. In[11],Holldobler and Kalinke presented a method for inserting propositional general logic programs[12] into three-layer feedforward arti?cial neural networks. They have shown that for each program P,there exists a three-layer feedforward neural network N with binary threshold units that computes T P,the program’s?xed point operator.If N is transformed into a recurrent network by linking the units in the output layer to the corresponding units in the input layer,it always settles down in a unique stable state when P is an acceptable program1[13,14].This stable state is the least?xed point of T P,that is identical to the unique stable model of P,so providing a declarative semantics for P(see [15]for the stable model semantics of general logic programs).

In[16],Towell and Shavlik presented KBANN (Knowledge-based Arti?cial Neural Network),a sys-tem for rules’insertion,re?nement and extraction from neural networks.They have empirically shown that knowledge-based neural networks’training based on the backpropagation learning algorithm[17]is a very ef?cient way to learn from examples and back-ground knowledge.They have done so by comparing KBANN’s performance with other hybrid,neural and purely symbolic inductive learning systems’(see[1] and[18]for a comprehensive description of symbolic inductive learning systems,including Inductive Logic Programming).

The Connectionist Inductive Learning and Logic Programming(C-IL2P)system is a massively parallel computational model based on a feedforward arti?cial neural network that integrates inductive learning from examples and background knowledge,with deductive learning from Logic Programming.Starting with the background knowledge represented by a propositional general logic program,a translation algorithm is ap-plied(see Fig.1(1))generating a neural network that can be trained with examples(2).Furthermore,that neural network computes the stable model of the pro-gram inserted in it or learned with the examples,so functioning as a massively parallel system for Logic Programming(3).The result of re?ning the background knowledge with the training examples can be explained by extracting a revised logic program from the network (4).Finally,the knowledge extracted can be more eas-ily analyzed by an expert that decides if it should feed the system once more,closing the learning cycle(5). The extraction step of C-IL2P(4)is beyond the scope of this paper,and the interested reader is referred to [19].

In Section2,we present a new translation algorithm from general logic programs(P)to arti?cial neural networks(N)with bipolar semi-linear neurons.The algorithm is based on Holldobler and Kalinke’s trans-lation algorithm from general logic programs to neu-ral networks with binary threshold neurons[11].We also present a theorem showing that N computes the ?xed-point operator(T P)of P.The theorem ensures that the translation algorithm is sound.In Section3, we show that the result obtained in[11]still holds,i.e.,

Connectionist Inductive Learning and Logic

Figure1.The connectionist inductive learning and logic programming system.

N is a massively parallel model for Logic Program-ming.However,N can also perform inductive learning from examples ef?ciently,assuming P as background knowledge and using the standard backpropagation learning algorithm as in[16].We outline the steps for performing both deduction and induction in the neu-ral network.In Section4,we successfully apply the system to two real-world problems of DNA classi?-cation,which have become benchmark data sets for testing machine learning systems’accuracy.We com-pare the results with other neural,symbolic,and hybrid inductive learning systems.Brie?y,C-IL2P’s test-set performance is at least as good as KBANN’s and better than any other system investigated,while its training-set performance is considerably better than KBANN’s. In Section5,we conclude and discuss directions for future work.

2.From Logic Programs to Neural Networks

It has been suggested that the merging of theory(back-ground knowledge)and data learning(learning from examples)in neural networks may provide a more ef-fective learning system[16,20].In order to achieve this objective,one might?rst translate the background knowledge into a neural network initial architecture, and then train it with examples using some neural learning algorithm like backpropagation.To do so,the C-IL2P system provides a translation algorithm from propositional(or grounded)general logic programs to feedforward neural networks with semi-linear neurons.

A theorem then shows that the network obtained is equivalent to the original program,in the sense that what is computed by the program is computed by the network and vice-versa.De?nition1.A general clause is a rule of the form A←L1,...,L k,where A is an atom and L i(1≤i≤k)is a literal(an atom or the negation of an atom).A general logic program is a?nite set of general clauses.

To insert the background knowledge,described by a general logic program(P),in the neural network(N), we use an approach similar to Holldobler and Kalinke’s [11].Each general clause(C l)of P is mapped from the input layer to the output layer of N through one neuron (N l)in the single hidden layer of N.Intuitively,the translation algorithm from P to N has to implement the following conditions:(1)The input potential of a hidden neuron(N l)can only exceed N l’s threshold (θl),activating N l,when all the positive antecedents of C l are assigned the truth-value“true”while all the negative antecedents of C l are assigned“false”;and(2) The input potential of an output neuron(A)can only exceed A’s threshold(θA),activating A,when at least one hidden neuron N l that is connected to A is activated.

Example2.Consider the logic program P={A←B,C,not D;A←E,F;B←}.The translation al-gorithm should derive the network N of Fig.2,setting weights(W s)and thresholds(θ s)in such a way that conditions(1)and(2)above are satis?ed.Note that, if N ought to be fully-connected,any other link(not shown in Fig.2)should receive weight zero initially.

Note that,in Example2,we have labelled each input and output neuron as an atom appearing,respectively, in the body and in the head of a clause of P.This allows us to refer to neurons and propositional vari-ables interchangeably and to regard each network’s in-put vector i=(i1,...,i m)(i j(1≤j≤m)∈[?1,1])as