Artificial Neural Networks (ANNs) are one of the most widely employed forms of biomorphic computation. However (unlike the biological nervous systems they draw inspiration from) the current trend is for ANNs to be structurally homogeneous. Furthermore, this structural homogeneity requires the application of complex training & learning tools that produce application specific ANNs, susceptible to pitfalls like overfitting. In this paper, an alternative approach is suggested, inspired by the role played in biology by Neural Microcircuits, the so called “fundamental processing elements” of organic nervous systems. How large neural networks can be assembled using Artificial Neural Microcircuits, intended as off-the-shelf components, is articulated; before showing the results of initial work to produce a catalogue of such Microcircuits though the use of Novelty Search.