cat /proc/claus

Technical Rant Below. You’ve been warned.

Ok, so after many days of heavy handed procrastination, I picked up my Ant Colony Clustering program again. I had kind of dropped it after facing some difficulties in implementation, and not being able to contact the original author for clarification. Then I noticed that it has been almost 2 months since my course started, and almost 3 months since my entrance examination ended, and my research has made little if no progress. So I decided to give another shot to this simulation.

Since I’m not very sure that the current architecture will work, I decided to just make three different models, and toss them into the magical world of evolution anyway, to see if anything good comes out of it.

The problem is, I’m not sure how I should implement the evolution operators for the neural network. There are quite a few options, but I’m not sure of the consequences of each option of the convergence of the neural net, in non-classificatory environment. I mean, when they do explain the operators they use (not aways), they don’t explain why did they choose THOSE operators, and not none else. Maybe there is a paper somewhere “the ultimate guide to evolutionary neural networks operators”, but I couldn’t find it yet .

Anyway, so for implementing an evolutionary strategy for learning, you need two operators: a crossover operator, and a mutation operator. The mutation operator is quite simple, it is used, sparingly, to provide new material to the pool of solutions used in the evolutionary strategy. For the evolution of neural networks, mutation can be as simple as randomly changing the value of one connection in the network. Or maybe two. Or maybe three. That is my problem. Just how many connections should I change? Maybe a percentage of the connections? My neural networks for this program are somewhat large, with about 100 connections each. And I have no Idea on how many, or how few, of them I should flip on my mutation operator.

Also, and perhaps more importantly, you have the crossover operator. It is responsible for the search in the solution space of an evolutionary strategy training. You pick two good solutions, mix a little of each, and then get another (hopefully) good solution. The solution, in an ANN, is the vector with all the weights (I’m not messing with the ann structure just yet). So I can think of two possible crossover operators. One, where I pick some weights from one parent, and some weights from another parent, and another where I average the weights from both parents. The averaging strategy has the obvious problem that the weights will slowly erode towards 0, but instead of averaging I could use any other algebrical operator.

But the problem is, any of the above choices would be a blind one, and right now I’m not even in the cerne of my work, but really just implementing someone’s ELSE work in order to get an idea of what I could to to build on it.

Yetch… I want to go back to procrastinating…

This entry was posted on Thursday, November 24th, 2005 at 8:29 pm and is filed under Tech. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.