Misplaced Pages

Modular neural network

Article snapshot taken from Wikipedia with creative commons attribution-sharealike license. Give it a read and then ask your questions in the chat. We can research this topic together.

A modular neural network is an artificial neural network characterized by a series of independent neural networks moderated by some intermediary. Each independent neural network serves as a module and operates on separate inputs to accomplish some subtask of the task the network hopes to perform. The intermediary takes the outputs of each module and processes them to produce the output of the network as a whole. The intermediary only accepts the modules' outputs—it does not respond to, nor otherwise signal, the modules. As well, the modules do not interact with each other.

Biological basis

As artificial neural network research progresses, it is appropriate that artificial neural networks continue to draw on their biological inspiration and emulate the segmentation and modularization found in the brain. The brain, for example, divides the complex task of visual perception into many subtasks. Within a part of the brain, called the thalamus, lies the lateral geniculate nucleus (LGN), which is divided into layers that separately processes color and contrast: both major components of vision. After the LGN processes each component in parallel, it passes the result to another region to compile the results.

Some tasks that the brain handles, like vision, employ a hierarchy of sub-networks. However, it is not clear whether some intermediary ties these separate processes together. Rather, as the tasks grow more abstract, the modules communicate with each other, unlike the modular neural network model.

Design

Unlike a single large network that can be assigned to arbitrary tasks, each module in a modular network must be assigned a specific task and connected to other modules in specific ways by a designer. In the vision example, the brain evolved (rather than learned) to create the LGN. In some cases, the designer may choose to follow biological models. In other cases, other models may be superior. The quality of the result will be a function of the quality of the design.

Complexity

Modular neural networks reduce a single large, unwieldy neural network to smaller, potentially more manageable components. Some tasks are intractably large for a single neural network. The benefits of modular neural networks include:

Efficiency

The possible neuron (node) connections increase quadratically as nodes are added to a network. Computation time depends on the number of nodes and their connections, any increase has drastic consequences for processing time. Assigning specific subtasks to individual modules reduce the number of necessary connections.

Training

A large neural network attempting to model multiple parameters can suffer from interference as new data can alter existing connections or just serve to confuse. Each module can be trained independently and more precisely master its simpler task. This means the training algorithm and the training data can be implemented more quickly.

Robustness

Regardless of whether a large neural network is biological or artificial, it remains largely susceptible to interference at and failure in any one of its nodes. By compartmentalizing subtasks, failure and interference are much more readily diagnosed and their effects on other sub-networks are eliminated as each one is independent of the other.

Notes

  1. ^ Azam 2000.
  2. Happel & Murre 1994.
  3. Hubel & Livingstone 1990.

References

Categories: