Evolving Spiking Neural Networks: Growth of Learning Machines

J. David Schaffer gave an excellent talk on evolving Spiking Neural Networks at the Center for Collective Dynamics of Complex Systems (CoCo) Seminar Series.

Many of today's neural network applications are based on multi-layer implementations of the perceptron. A perceptron implements a neuron model that sums weighted inputs and applies a non-linear activation function to calculate the output. Despite that this model deviates from how biological neuron networks work, the approach works and is used until today. In contrast, Spiking Neural Networks (SNNs) are a type of neural network that increase the realism in neural network simulation by introducing a time aspect into the model. Other than perceptron networks that decide upon their output at each propagation cycle, SNNs fire when a certain membrane potential is reached, which puts information into the timing of a spike.

J. David Schaffer shows how a genetic algorithm can be applied to generate an SNN for a given problem. Therefore, the chromosome representing an SNN will be mapped onto a binary string which will be evolved with mutation and recombination.

Evolving Spiking Neural Networks: Growth of Learning Machines from Complex Systems on Vimeo

Further readings on the topic:

• Evolutionary Computation at Wikipedia
• Spiking Neural Network at Wikipedia 
• J. David Schaffer's Webpage at Binghampton University
• R. Batllori, Craig B. Laramee, W. Land, J. David Schaffer. Evolving spiking neural networks for robot control. In Procedia Computer Science (6):329-334, 2011. (fulltext)
• I. Fehervari and W. Elmenreich. Evolving neural network controllers for a team of self-organizing robots. Journal of Robotics, 2010. (fulltext)
• I. Fehervari and W. Elmenreich. Evolution as a tool to design self-organizing systems. In Self-Organizing Systems, volume LNCS 8221, pages 139–144. Springer Verlag, 2014. (fulltext)
• I. Fehérvári, A. Sobe, and W. Elmenreich. Biologically sound neural networks for embedded systems using OpenCL. In Proceedings of the International Conference on NETworked sYStems (NETYS 2013). Springer Verlag, May 2013. (fulltext)
• J. David Schaffer, Darrell Whitley, Larry J Eshelman. Combinations of genetic algorithms and neural networks: A survey of the state of the art. International Workshop on Combinations of Genetic Algorithms and Neural Networks (COGANN-92), pages 1-37, 1992.

FREVO 1.2 release

We proudly announce the new release 1.2 of FREVO (FRamework for EVOlutionary design). FREVO helps to reduce the time to implement, set up and run an evolutionary algorithm to evolve an agent's behavior as a solution to a particular control problem. FREVO supports decomposing the task into problem definition, solution representation and the optimization method. The componentwise separation allows to experiment with different combinations of algorithms and neural networks for different tasks.

The following components were added to FREVO:

• HEMS – a simulation for modeling trading behavior of loads and local energy generators.
• SinglePong – a simulation of the one player pong game where several paddles can cooperate in order to achieve better performance.
• Pong – a simulation of the pong game where two teams can play against each other.

Quick start:

• download the newest version at frevo.sourceforge.net 
• unpack the ZIP file
• unless you have it already on your system, install Java 
• execute the createscrips.jar ("java -jar createscrips.jar") 
• you can now run FREVO using the script named launch_Frevo 

...or have a look at the following video explaining the basic steps to get started with FREVO:

For more information see the following sources:

• A. Sobe, I. Fehérvári, and W. Elmenreich. Frevo: A tool for evolving and evaluating self-organizing systems. InProceedings of the 1st International Workshop on Evaluation for Self-Adaptive and Self-Organizing Systems, Lyon, France, September 2012.
• A. Monacchi, S. Zhevzhyk, and W. Elmenreich. HEMS: A home energy market simulator. Computer Science - Research and Development, 2014. 
• I. Fehérvári and W. Elmenreich. Evolving neural network controllers for a team of self-organizing robots. Journal of Robotics, 2010.
• FREVO - software used for the evolutionary algorithm.
• WMOrakel - the championship between self-organized soccer teams which were evolved using FREVO.

Comparison of Metaheuristic Algorithms for Evolving a Neural Controller for an Autonomous Robot

Robots are a good way to test things. Hope our robot overlords of the future will not take this to personal…

The task

We used a simulation of a robot that is searching for a light source as a testbed to compare how well a solution can be created by evolving an artificial neural network (ANN). While ANNs are often programmed using example input-output pairs which are learned by a backpropagation algorithm (supervised learning), in our case we left the how up to the system and only required the what – the robot should be able to find the light source by operating its wheels and using its sensors – that is called learning with belated rewards or reinforcement learning. We compared different evolutionary algorithms (EA), namely simple EA, two dimensional cellular EA, and random search, according to their performance in evolving a successful algorithm for the light-searching robot. In our experiments we studied the effects of EA parameters on performance, such as population size and number of generation. The simulations have been done using the open-source tool Framework for Evolutionary Design (FREVO).

The results explain how the choice of the neural network (three-layered or fully-connected) may inf

Possible implementation in hardware

luence the quality of a final solution. The results indicate that cEA and simple EA are the most applicable for evolving a neural controller. A fully-connected ANN outperforms three-layered ANN in all conducted experiments. Based on our findings, we recommend to use cEA and fully-connected ANN for problems that require short evaluation phase. For a large number of generations and population size the efficiency of both algorithms are approximately the same. In the experiments we measured an influence of population size and number of generations on performance of metaheuristic algorithms. The dependencies on these parameters are negligible. This information is important for the conduction of experiments. To accelerate a simulation, the population size should be the same as the number of cores on the server, where these experiments will be performed.

• Sergii Zhevzhyk, Wilfried Elmenreich, Comparison of Metaheuristic Algorithms for Evolving a Neural Controller for an Autonomous Robot, Transactions on Machine Learning and Artificial Intelligence, pp. 62-76, 15, 2014.
• A. Sobe, I. Fehérvári, and W. Elmenreich. FREVO: A tool for evolving and evaluating self-organizing systems. In Proceedings of the 1st International Workshop on Evaluation for Self-Adaptive and Self-Organizing Systems, Lyon, France, September 2012.

Call for Papers 10th International Conference on Soft Computing Models in Industrial and Environmental Applications (SOCO 2015)

10th International Conference on
Soft Computing Models in Industrial
and Environmental Applications
SOCO 2015

Burgos, Spain
15th-17th June, 2015
http://soco.usal.es

In conjunction with:

8thComputational Intelligence in Security for Information Systems

CISIS 2015

and

 6th International Conference on EUropean Transnational Education

ICEUTE 2015

The 10th International Conference on Soft Computing Models in Industrial and Environmental Applications will take place in Burgos, Spain. Soft computing represents a collection or set of computational techniques in machine learning, computer science and some engineering disciplines, which investigate, simulate, and analyze very complex issues and phenomena. This conference is mainly focus on its industrial and environmental applications.

Topics of interest include, but are not limited to:
• Green Computing
• Evolutionary Computing
• Neuro Computing
• Probabilistic Computing
• Immunological Computing
• Hybrid Methods
• Causal Models
• Case-based Reasoning
• Chaos Theory Fuzzy Computing
• Intelligent Agents and Agent Theory
• Interactive Computational Models

The application fields of interest cover, but are not limited to:
• Decision Support
• Process and System Control
• System Identification and Modelling
• Optimization
• Signal or Image Processing
• Vision or Pattern Recognition
• Condition Monitoring
• Fault Diagnosis
• Systems Integration
• Internet Tools
• Human Machine Interface
• Time Series Prediction
• Robotics
• Motion Control & Power Electronics
• Biomedical Engineering
• Virtual Reality
• Reactive Distributed AI
• Telecommunications
• Consumer Electronics
• Industrial Electronics
• Manufacturing Systems
• Power and Energy
• Data Mining
• Data Visualisation
• Intelligent Information Retrieval
• Bio-inspired Systems
• Autonomous Reasoning
• Intelligent Agents

Important Dates

Paper submission deadline: 24th January, 2015
Acceptance notification: 13th February, 2015
Submission of final papers: 3rd March, 2015
Final version submission: 13th March, 2015
Conference dates: 15th-17th June, 2015

Scalability in Self-Organizing Systems

One of the properties of self-organizing systems is scalability. It means that system keeps its working capabilities even if we remove some of its components or add more of them. In our reseach, we employ different evolutionary algorithms (EAs) to create a self-organizing system. In particular, algorithms like a simple evolutionary algorithm or a two dimensional cellular EA are used  for adjusting the synaptic weights of an neural controller. The best solutions are identified based on simulations of the target application. Typically, the simulation parameters limit the applicability of the solution - there is no guarantee that an evolved solution is adaptable or scalable to situations not specified in the simulation parameters. On the other hand, there are many examples in nature where solutions could be successfully employed in other contexts. We decided to check how our soccer teams, which consist of evolved neural controllers, can scale.

For the FIFA World Cup in Brazil we organized our own tournament between evolved self-organized soccer teams. This is an exciting show - to see how simple agents having only partial information about the environment around them are reaching its goal (score a goal) as a team. Will they be able to play in the same manner if we take the contoller, trained in the simulation with 10 players per team, and increase or decrease the number of players? This question has remained open until today.

In our first scenario, we assume that we invited two soccer teams to show us a fantastic game, but due to some circumstances, only 4 players per teamshow up.
Thus our first experiment can be seen in the video below.
Despite the players being evolved in a context of 11 players on each side, reducing the number of players did not affect the ability of players to show good game.

To check the other extreme, we settled a very dangerous experiment - each team consisting of 40 players! The results were stunning (see video below). These soccer heroes could play as a team even with significantly increased number of players. Unfortunately, they could not play for a long time in this mode: Marco Materazzi headbutted Zinadine Zidane in the chest and shouted "Revenge!"; Luis Suarez bit two players in order to show his perfect teeth; Diego Maradona scored the goal by striking the ball with his hand and this time he was disqualified for this trick. We didn't care about these incidents since we got the results of our experiment:

Links:

• I. Fehervari and W. Elmenreich. Evolving neural network controllers for a team of self-organizing robots. Journal of Robotics, 2010.
• I. Fehervari and W. Elmenreich. Evolution as a tool to design self-organizing systems. In Self-Organizing Systems, volume LNCS 8221, pages 139–144. Springer Verlag, 2014.
• A. Sobe, I. Fehérvári, and W. Elmenreich. FREVO: A tool for evolving and evaluating self-organizing systems. In Proceedings of the 1st International Workshop on Evaluation for Self-Adaptive and Self-Organizing Systems, Lyon, France, September 2012.
• More simulated matches at Lakeside Labs WM Orakel http://www.robotoracle.at/

On Evolving Self-organizing Technical Systems

Moofushi Kandu fish (Image by Bruno
de Giusti under CC-BY-SA-2.5)

Individual swarm fish behave according to simple rules, which make the overall swarm an efficient entitiy for hunting and avoiding predators. Despite the simple local rules, a school of fish is a working, intelligent system. István Fehervari examined in his doctoral thesis how this behavior can be transferred to technical systems. The result is a tool making it easier to develop self-organizing systems.
Sometimes, it is possible to mimic natural self-organizing behavior for a technical system. At other times, you might not have access to such a template. "If there is no natural system, which we can copy, we have to develop it on our own," says István Fehérvári from the Institute of Networked and Embedded Systems at Alpen-Adria-Universität Klagenfurt. Fehérvári further explains: "This is very difficult because the behavior of a complex system is difficult to predict and the definition of the proper interaction behaviors is hard. Any change in the system creates an effect, often with unwanted consequences. This is why we apply an artificial evolution approach to evolve the local interaction rules,"

To make this approach feasible, the FREVO software has been developed, a tool that helps to apply the evolutionary approach in a unified way to different problem statements and settings. "FREVO provides a one-stop shop, with all the necessary steps for designing a self-organized algorithm for a given problem." says Fehérvári who devoted a major part of his thesis work to design and implement FREVO.  Now the tool is available to other researchers for further experiments and investigations. FREVO is an open-source program in Java that can be freely downloaded at frevo.sourceforge.net. 

Further readings and downloads:

• István Fehérvári, On Evolving Self-organizing Technical Systems, PhD Dissertation, Alpen-Adria-Universität Klagenfurt, 2013. (fulltext-pdf)
• A. Sobe, I. Fehérvári, and W. Elmenreich. FREVO: A tool for evolving and evaluating self-organizing systems. In Proceedings of the 1st International Workshop on Evaluation for Self-Adaptive and Self-Organizing Systems, Lyon, France, September 2012.
• Download FREVO at frevo.sourceforge.net
• 6-minute video explaining installation, configuration and operation of FREVO

Simulating the Soccer World Cup 2014

You cannot wait for the soccer World Cup to start? We proudly present a peek preview of the World Cup 2014 - played by teams created with evolutionary algorithms. Using our evolutionary tool FREVO for designing self-organizing systems we have evolved neural networks that make robots playing soccer. During the evolution phase, a fitness function combines different aspects of gameplay like zone defense, man-marking, passing, shots, and goals. By tweaking the weights for these parameters we can influence the playing style of a team while the overall gameplay is still generated automatically by the evolutionary process. Thus we can simulate playing styles of different national teams and then match them against each other.

The following video shows a simulation of Brazil versus Croatia, the opening game of the world cup. The commentary is from Toni Polster, a legendary Austrian soccer player.

While the result is credible, we have not done this to exactly predict the outcome of the games - this would spoil the whole tournament! Furthermore, our approach is not meant for prediction but a system to train a distributed agent-based system to achieve an emergent cooperative behavior in a self-organized way. Setting up this work helped us in improving our understanding how we can create and guide self-organizing systems. We have chosen the soccer simulation as a demonstration because in soccer the global goal (no pun intended) can be achieved in so many different ways , for example with a defensive, offensive, kick-and-rush, pass-intensive, etc. style. And it is nice to watch - who said good science can't be fun!

Further readings:

• I. Fehervari and W. Elmenreich. Evolving neural network controllers for a team of self-organizing robots. Journal of Robotics, 2010.
• I. Fehervari and W. Elmenreich. Evolution as a tool to design self-organizing systems. In Self-Organizing Systems, volume LNCS 8221, pages 139–144. Springer Verlag, 2014.
• A. Sobe, I. Fehérvári, and W. Elmenreich. FREVO: A tool for evolving and evaluating self-organizing systems. In Proceedings of the 1st International Workshop on Evaluation for Self-Adaptive and Self-Organizing Systems, Lyon, France, September 2012.
• More simulated matches at Lakeside Labs WM Orakel http://www.robotoracle.at/

Self-Organization for the Smart Grid

In my keynote "AI Techniques for Smart Grids" at the 2014 IEEE Innovative Smart Grid Technologies - Asia conference, I discussed the role and potential of self-organization in the smart grid. Since power systems are critical infrastructure, applying a distributed, bottom-up control structure is a delicate step. On the one hand, self-organizing systems provide very attractive properties like scalability, adaptability and robustness. On the other hand, due to non-linear interactions in the system, small changes that appear insignificant can have severe consequences. To ensure secure operation, this is not the most-wanted property. However, sometimes one gets such systems without a choice. An example is the effect of the unbundling process of power generation and system operation to create a free energy market - a decision that was taken to support distributed on-site power generation with renewable energy sources. At the UCTE grid, the largest synchronous grid in the world in terms of generation capacity, the decreased the scope of control for the Transmission System Operators (TSOs) due to the liberalization of the power market has caused the TSOs to develop a new way of understanding of the system in order to carefully and knowledgeable interact with the system. So we got a self-organization system without explicitly designing the system for being one. Another possibility where self-organization can explicitly support the smart grid is the utilization of smart meters in a way that supports the utilities while still keeping control and private information locally at the customer's side.

To enable self-organization for the smart grid, it is necessary to provide models, proofs, case studies, etc. showing that self-organizing approaches work. For finding self-organizing algorithms we provide our part: with the open-source software tool FREVO, we can evolve agent behavior for self-organizing systems using an automated process based on artificial evolution.

Links:

• Video: 6 minute introduction to FREVO
• Download FREVO (open source, platform-independent)
• A. Sobe, I. Fehérvári, and W. Elmenreich. FREVO: A tool for evolving and evaluating self-organizing systems. In Proceedings of the 1st International Workshop on Evaluation for Self-Adaptive and Self-Organizing Systems, Lyon, France, September 2012.
• A. Sobe and W. Elmenreich. Smart microgrids: Overview and outlook. In Proceedings of the ITG INFORMATIK Workshop on Smart Grids, Braunschweig, Germany, September 2012.
• IEEE Innovative Smart Grid Technologies - Asia conference website

How the body affects the mind - On the effects of robot configuration on evolved behavior

The design of robotic controllers through evolutionary methods requires making a large number of choices about the experimental setup, which are often left to the expertise or naïveté of the experimenter. Although much attention is normally given to the fitness function or the genotype-to-phenotype mapping determining the robot controller, the robot configuration is often selected with little care. Yet, an ill-defined configuration - in terms of the selected subset of the sensory-motor system, or in the pre-processing of the raw sensor data - may be decisive in determining the failure of the evolutionary process.

Different emerged patterns 
simulated with ARGoS

In our paper "On the effects of the robot configuration on evolving coordinated motion behaviors" we studied the effect of different robot configurations on the ability to evolve efficient behaviors for a swarm robotics system. In this domain, the choice of a good configuration is fundamental as even small details can lead to large differences in the group behavior. To demonstrate the importance of the robot configuration, we test different alternatives and measure the group performance on a bi-objective scale.

The results show that different configurations not only have a strong effect on performance, but they also correspond to behaviors with radically different features concerning the organization of the group.

The following video illustrates three basic behaviors that emerged: wavefront, train and flocking:


For more information, see:

I. Fehérvári, V. Trianni, and W. Elmenreich. On the effects of the robot configuration on evolving coordinated motion behaviors. In Proceedings of the IEEE Congress on Evolutionary Computation. IEEE, June 2013.

FREVO 1.0 released

We proudly announce the new release 1.0 of  FREVO (FRamework for EVOlutionary design). FREVO helps to reduce the time to implement, set up and run an evolutionary algorithm to evolve an agent's behavior as a solution to a particular control problem. FREVO is decomposing the task into problem definition, solution representation and the optimization method. These components allow to experiment with different combinations of algorithms and neural networks for different tasks.

The new version, FREVO 1.0, comes with 26 different components and an improved user interface which allows you to pause an evolution and check the intermediate results.

Quick start:

1. download the newest version at frevo.sourceforge.net 
2. unpack the ZIP file
3. unless you have it already on your system, install Java
4. execute the createscrips.jar ("java -jar  createscrips.jar")
5. you can now run FREVO using the script named launch_Frevo

...or have a look at the following video explaining the basic steps to get started with FREVO:

For more information see the following sources:

• A. Sobe, I. Fehérvári, and W. Elmenreich. Frevo: A tool for evolving and evaluating self-organizing systems. In Proceedings of the 1st International Workshop on Evaluation for Self-Adaptive and Self-Organizing Systems, Lyon, France, September 2012. 
• I. Fehérvári and W. Elmenreich. Evolving neural network controllers for a team of self-organizing robots. Journal of Robotics, 2010.
• The DEMESOS project www.demesos.tk
• Software used for the evolutionary algorithm: FREVO www.frevotool.tk

The 6-minute introduction to FREVO

Evolution is a slow process. In earth history, evolution took millions of years to achieve something. Computer simulations of evolutionary processes are much better, but still taking weeks of simulation time on a cluster to run some evolutionary algorithm.

We challenge this!

Our software FREVO (FRamework for EVOlutionary design) is aimed to reduce the time to implement, set up and run an evolutionary algorithm to evolve an agent's behavior as a solution to a particular control problem. FREVO is decomposing the task into problem definition, solution representation and the optimization method, in order to ... let's stop talking! I will show you in just 6 minutes!

As demonstrated in the video below, 6 minutes are sufficient to download the framework, install it, set up a simulation, evolve a neural network controller for an inverted pendulum problem and check the results! See for yourself!



The tool is available at http://frevo.sourceforge.com
There are also tutorials on advanced projects with FREVO. If you are doing research in engineering complex systems, this tool might be useful for your thesis ;-)

If FREVO is useful for you, please cite this paper:

A. Sobe, I. Fehérvári, and W. Elmenreich. Frevo: A tool for evolving and evaluating self-organizing systems. In Proceedings of the 1st International Workshop on Evaluation for Self-Adaptive and Self-Organizing Systems, Lyon, France, September 2012.

Evolving a cellular automaton with neural controllers

Evolution of Hungarian flag

What happens if you integrate a cellular automaton with neural network controllers? In an experiment, we extended the model of CA with a neural network that controls the cell behavior according to its internal state. The model is used to evolve an Artificial Neural Network controlling the cell behavior in a way a previously defined reference pattern emerges by interaction of the cells. Each cell is controlled by an instance of the same ANN. The ANNs have connections to neighbors and one output of each ANN determines cell color.
At the beginning of each simulation, all cells had the same state and commenced their operation at the same time - this is comparable with a number of people cooperatively drawing an image in the dark.
We used our tool FREVO for evolving the neural network in a way that it reproduces the given pattern. The best results have been achieved when evolving simple structures with large areas of a single color as they are present for example in flags. For more complex images, however, the current setup causes the evolutionary algorithm to get stuck at a suboptimal stage like depicted in the approach to learn the image of Leonardo da Vinci's Mona Lisa painting. There is, however, a large space of possibilities for variations of the model which gives rise to future work.

Complex images like this painting cannot be reproduced - Mona Lisa kinda vanished huh?

• W. Elmenreich and I. Fehérvári. Evolving self-organizing cellular automata based on neural network genotypes. In C. Bettstetter, C. Gershenson, editor, Proceedings of the Fifth International Workshop on Self-Organizing Systems, pages 16–25. Springer Verlag, 2011.
  

Evolving a self-organizing soccer team

This video shows the evolution of coordinated behavior of simulated robot soccer players. In the simulation, each soccer player is controlled by a neural network. The neural networks are evolved using an evolutionary algorithm, so generation after generation the strategy improves.
After a few hundred generations, the players of a team adopt a useful behavior. The used approach did not include a trainer telling them how to play or specifying predefined roles for the players such as being a defender, midfielder or striker. Still, during a game, different behavior of the players emerges. Thus, similar to biological systems, the entities take up different roles in a self-organizing way. Since the agents are not predefined, such systems have a high robustness against failure of come of the entities.

• I. Fehérvári and W. Elmenreich. Evolving neural network controllers for a team of self-organizing robots. Journal of Robotics, 2010.
• The DEMESOS project www.demesos.tk
• Software used for the evolutionary algorithm: FREVO www.frevotool.tk