Friday, March 6, 2020

Why it is important to share your code and make your paper accessible

In a recently published paper on "Making simulation results reproducible - Survey, guidelines, and examples based on Gradle and Docker" we asked researchers from all levels about their willingness to share the code of their simulations together with the paper. A little bit to our surprise, the answers were mostly very positive about sharing.
Still, we are currently far from a situation where every published paper is made accessible and has its code shared with the publication. Apart from the obvious cases where an industrial project might require confidentiality of some details, the most frequent reason is probably some laziness or other giving the effort to refactor your code properly for a publication a low priority. At the moment, not publishing the code is the normal habit, while providing the code is still an exception.
This needs to change for several reasons:
  • From the perspective of the researcher who reads the publication, having access to the code eases the understanding of the approach and allows building upon the work of others. The frequent argument that the code would be provided on request is mainly a lip service - first, it adds uncertainty to the reader if and when they would get the code. Second, the provider of the code might not have it prepared. Imagine the effort to dig up some code you wrote ten years ago and to make it proper so you can pass it on.
  • From the perspective of the researcher who publishes a paper, the chance to get their work read, appreciated and cited is much higher if they provide code and materials with it. Considering the time, money and effort that is put into publishing a paper, the effort of also publishing the code is well justified.
  • From a system's perspective, it is of utmost importance that we support each other. It does not make sense that brilliant minds spend time recreating implementations that have been done already. Reproducing research results is, of course, an important factor in science, but the overall ability to reproduce results and check an approach for errors increases with the possibility to have an insight into the code. 
For the same reasons, it is important to have our papers available online instead of locking them behind a paywall. Even if your university pays for the access to some literature databases there are many potential readers of your work that don't enjoy such a service, be it that their university does not provide such access or that they work from a different network at the instant they want to read your paper. My recommendation: go for open access! This could be the gold open access, where the journal provides open access to your paper at their website, however, this "gold" is usually expensive. Alternatively, several publishers offer a green open access model where you are allowed to keep a pre- or post-print version of your paper online at your private or your institution's webpage. To check if a particular publisher offers such a policy, look them up at this page about Publisher copyright policies & self-archiving.

Further reading:
Wilfried Elmenreich, Philipp Moll, Sebastian Theuermann, and Mathias Lux. Making simulation results reproducible - Survey, guidelines, and examples based on Gradle and Docker. PeerJ Computer Science, 5(e240):1–27, Dezember 2019. (doi:10.7717/peerj-cs.240)

Friday, February 7, 2020

The Wisdom of Crowds or Can many mediocre measurements produce a single good one?

In the introduction of his excellent book "The Wisdom of Crowds" James Surowiecki tells the story of the British Scientist Francis Galton who went to a country fair in 1906. There, a guessing game took place where one had to guess the weight of a bull.
800 people purchased a ticket and delivered their estimates on paper. Galton, who was curious about all kinds of things (I already told you he was a scientist, right) borrowed the tickets afterward and analyzed the results. He was expecting the average of the guesses to be far off because for each expert in the crowd (like for example a butcher) there were for sure a couple of inexperienced people. However, to his great surprise, the average guess of 1197 pounds was very close to the real result of 1198 pounds! This story, by the way, also documents the increase in weight of livestock - today a bull would be around twice as heavy!

If you are interested in the further aspects of collective human intelligence, I recommend reading the book:

James Surowiecki, The Wisdom of Crowds, Anchor, 2005.

But I was less interested in people, instead, I was wondering if this can be used for combining sensor measurements.

A couple of years ago I worked on a method for combining measurements from sensors with different accuracy. Translated to the story above this would mean if we know who are the experts and who are not, should we even bother to include the results of the latter? Actually, the answer is yes - given that the estimates have low correlation! But other than in the story above, the best way is to do a weighted average of values. The weights are derived from the error variance of the estimates, so a sensor with high error variance should get a low weight and a sensor with low error variance should get a high weight.

The resulting formula is surprisingly easy:
The paper explaining the approach in detail and showing how this can be integrated into a sensor network can be found here (link leads to freely accessible PDF):

W. Elmenreich. Fusion of continuous-valued sensor measurements using confidence-weighted averaging. Journal of Vibration and Control, 13(9-10):1303–1312, 2007. (doi:10.1177/1077546307077457)

So, just in case you have to guess the weight of a bull at a country fair, remember this approach :-)

Monday, August 7, 2017

The Spiderino Swarm Robot at Research Days 2017

In Klagenfurt from July 10-12, 2017, the Lakeside Research Days were held in collaboration between the Lakeside Labs and the Alpen-Adria-Universität Klagenfurt. Researchers presented their work and discussed open issues in self-organization and swarm intelligence in cyber physical systems.

The Research Days included also laboratory sessions with training on swarm robotics platform. Midhat Jdeed from theAlpen-Adria-Universität gave a lab session about Spiderino, how to program and implement the basic functionalities such as walking, turning and lighting two LEDs using Arduino Studio. In addition, a program has been implemented using sensors distance to make a small swarm of Spiderinos search for the light source and follow it. The sensors employed in Spiderino are CNY70s, which can measure distances and detect obstacles based on the amount of reflected light from an obstacle. This method is prone to ambient light, but it can be also used to make the robots finding a light source.

In the lab session, the Research Days participants learned about the possibilities and programming interface of Spiderino and could implement their own first program hands on. By the end of the session we had a lab of swarming spiders.

More details about Research Days 2017 can be found on the Research Days'17 webpage.

If you want to learn more about the Spiderino swarm robot, check out this paper:

M. Jdeed, S. Zhevzhyk, F. Steinkellner, and W. Elmenreich. Spiderino - a low-cost robot for swarm research and educational purposes In Proceedings of the 13th International Workshop on Intelligent Solutions in Embedded Systems (WISES'17), Hamburg, Germany, June 2017.

Tuesday, August 1, 2017

Game of Trust: a Game about Game Theory

The game of trust is a game created by Nicky Case inspired by Robert Axelrod's 1984 book, "The Evolution of Cooperation" and his 1997 sequel, "The Complexity of Cooperation", and Robert D. Putnam's 2000 book on America's declining "social capital", Bowling Alone.

The game let's you play an iterated prisoner's dilemma. Do you know the prisoners' dilemma? Not sure about the payoffs? Have a look at Nicky Case's model for an easy to remember ruleset:

You have one choice. In front of you is a machine: if you put a coin in the machine, the other player gets three coins – and vice versa. You both can either choose to COOPERATE (put in coin), or CHEAT (don't put in coin).

Experience game theory with entertaining graphics by Nicky Case

Now visit the game page of the game of trust and have fun!

Saturday, July 22, 2017

PhD position for Software Engineering of Swarm Systems

The Institute of Networked and Embedded Systems at the University of Klagenfurt, Austria, has an opening for a researcher (PhD student) on the topic of software engineering of swarm systems.

Swarm systems consist of a high number of autonomous agents, which interact with each other so that the emerging swarm performs a desired function. The topic of the PhD is to address the software design for such systems as well as investigate how unwanted swarm effects can be avoided. Other aspects involve simulation of such systems and using swarm approaches in games.

You will mainly work in an European project on Cyber-Physical Swarm Systems, which runs until end of 2019. The position is offered at the Smart Grids group at the Institute of Networked and Embedded Systems under the supervision of Professor Wilfried Elmenreich. The institute cooperates with national and international partners from industry and academia and is part of the research cluster Lakeside Labs (self-organizing networked systems).

Our team is very international and dedicated to quality research and teaching. Our working language is english. The offices and laboratories are located in the well-equipped Lakeside Science & Technology Park. Working language is English. The Institute cooperates with national and international partners in research and industry. It is part of the research cluster Lakeside Labs on self-organizing networked systems.

The candidate is expected to have a master in electrical or computer engineering, computer science, applied mathematics or equivalent. Experience in one or more of the following fields is required:

    Distributed and swarm algorithms
    Genetic algorithms and evolutionary design
    Very good programming skills in Java

The contract will be based on 40 hours per week with a gross salary of 38235 EUR per year. Please mail applications containing a letter of interest, curriculum vitae, copies of academic certificates and courses, list of publications, and contact details of two references in a single PDF file to no later than August 25, 2017. Women are especially encouraged to apply.

Wednesday, May 3, 2017

Research Days on Self-Organization and Swarm Intelligence in Cyber Physical Systems

July 10 – 12, 2017
Lakeside Labs
Klagenfurt, Austria

Systems with swarm intelligence consist of a set of agents, with local intelligence, interacting locally. Typically, approaches for swarm intelligence come from biological systems, like ant or bee colony behavior, schooling of fishes, flocking of birds, etc. As the agents follow very simple rules without a central control dictating their activities, the interactions between those agents lead to the emergence of intelligent or complex global behavior. In other words, CPS researchers need to learn how to construct and apply techniques for self- organization and swarm intelligence, their opportunities and challenges.

At the Research Days 2017, a group of international researchers will discuss how this translation between CPS, self-organization, and swarm intelligence can be achieved on a theoretical and practical basis. The Research Days are a regular event concentrating on the core competence of Lakeside Labs - Self-organizing Networked Systems. During this workshop organized by Lakeside Labs GmbH in cooperation with the University of Klagenfurt, international experts devote themselves to a special topic in self-organization. The event is organized as a workshop of several consecutive days in July. It takes place at Lakeside Labs in Klagenfurt am Wörthersee, Austria, near a beautiful lake and Alps scenery. Invited experts, local professors, and young researchers discuss and elaborate ideas in the field of Self-Organizing Systems (SOS). The main emphasis of the workshop is on soliciting discussions and creating new ideas regarding a topic related to self-organizing systems. The event greatly supports scientific exchange, networking, establishment of international collaborations, and joint research projects.

The following video gives a nice impression of the Research Days:
Learn more and registerfor the event at

Friday, February 10, 2017

School Kids Rule at Coding Contest

The results from the Advent Programming Contest 2016 made it clear: The next generation of hackers will be awesome!

The front positions are dominated by students from school, who showed great problem-solving skills and also a lot of dedication to apply these skills! Running up are excellent coders from the "other" category, thus (often professional) coders who are not studying in a school or university.

The final ranking is as follows:
RankNameCategorySolved Problems
1Marc Goritschnig school24
2ChristianGubesch school24
3DanielSemmelrock school24
4Simon school24
5jareds other22
6ck127720 other22
7Dieser1Boi other21
8gutblender other20
9nightelf other19
10MichaelGubesch other19
11maahgeh other18
12Matzoer other17
13AugustHoerandl school16
14JoelHolzbauer school16
15Kozeschnik Oskar school16

In overall, we had 175 participants from schools, 105 university students, and 115 in the "other" category.
The awarding ceremony will take place at the beginnig of the new semester as part of the semester opening game night of the IEEE Student Branch Klagenfurt. See you there