SKEDSOFT

The traditional approach to building knowledge-based systems is to use rules articulated by experts and to build an expert system. If only this approach is used, some problems arise:

• Experts develop their own internal representation of reality, which is only one aspect of the whole process; a data set may contain much more than that.
• Rules valid in one framework may not be valid in another, so it is desirable to be able to extract,to learn rules regularly and easily from fresh, new data.
• Experts may not be aware of the underlying rules in existing data.
• It takes time for experts to learn rules from experience, which rules may reside peacefully in data and may have been deeply buried there.
• It is extremely difficult to articulate, maintain, update, and implement a huge number of rules.
• The path from the expert to the machine is very noisy, that is, information can be easily lost or corrupted.

One alternative to the standard approach is to build automatic learning machines which learn explicit rules from data, if enough data are available. Rules extracted from data can be subsequently used for the purpose of reasoning, explanation, understanding problems, and trying alternative techniques for solving problems, for example, fuzzy inference machines and AI rule-based techniques.

Neural networks are appropriate candidates for the task of knowledge acquisition from data because of the following:

• Neural networks can learn from past data and generalize over new data.
• Neural networks can learn to approximate a function from a table of input-output data.
• Neural networks can learn features from data.
• Neural networks can learn to distinguish groups, categories in which elements are grouped, either in a supervised or unsupervised mode.
• Neural networks can learn probabilities and statistical distribution from data.
• Rules are associations between groups ("clusters" and "patches") from the domain space and patches" from the solution space. Neural networks trained with data learn these associations. The problem is how to extract the knowledge that a neural network has accumulated during training.

There are different methods that can be applied for rules extraction from neural network architectures. They can be classified in two major groups:

(1) Destructive learning, that is, learning by pruning the neural network architecture during training.

 (2) Nondestructive learning of rules.

Learned (or articulated) initial set of rules can be used to "prewire" a neural network before consecutive training with real data for the purpose of better generalization. Two ways of achieving this task are (1) setting only selected connections in a fuzzy neural network, and (2) weights initialization of a fully connected neural network before training. The above groups of methods are presented in the following subsections.

Extracting rules from data may subsequently mean loss of information. The way knowledge is extracted restricts aspects of that knowledge and directs and biases the knowledge acquisition process. A data set may contain much more than is extracted from it, for example, many more interdependencies between attributes, many more relations, and many more inference paths. Neural networks are only one technique. Other techniques may be used alternatively or additionally, for example, genetic algorithms.