Remember that this article is far from complete and should be considered a work in progress.
Darwin in action
Connecting neurons manually doesn't seem very fun to me and seems akin to connecting transistors with wires and a soldering iron. Let's go for evolution!
Evolution
My understanding with the intricacies of the theory of evolution isn't quite on par with Richard Dawkins, but I do understand the basic principle that anything which replicates with minor variations and is exposed to some form of selection, will converge towards the optimal variant for that particular environment.
So what we need is a replicator function and a selection scheme. The replicator function needs some way of defining the layout of neural nets by data we easily can store, alter and transfer. JSON should work fine for transfer purpose. As a selection scheme, there are various techniques I'd like to try out.
The first one will be a simple computer guided selection. We let the genes be a range from the english alpabet including punctuation, and let it try to evolve into a complete sentence: "The lazy dog jumped over the fence.". The chance of this sentence occuring randomly is only 1 out of 3554 = 2.3970*10^83 when taking into account upper- and lowercase letters plus punctuation. Note that this number is slightly larger (only a thousand times or so) than the currently estimated amount of atoms in the known universe! Evolution is not a process dictated by pure chance however, as we shall see demonstrated soon.
Replicator
The way the replicator works is it keeps a list of all the individuals, testing how well they perform in the given environment by using a fitness function. Depending on the settings, one or more of the individuals are allowed to propagate.