On the research front, I applied for a post-doc and didn’t get it due to lack of publications. A paper I had submitted just got rejected, yay. But on the positive side of things, my PhD thesis got published as a book. So yeah, I’m in Amazon. I never thought that would happen.

I have also been improving my Python skills through various projects like generators of Top Trumps decks and Constructed Language generators. It’s good fun.

But most of my time this past few months has been spent starting my own business. I started my own app business and this is the story of how I built my first iPhone app. This is it on the iPhone App Store, a minisite I built for it and its facebook page. I would love to hear what people think about it.

The library shown diagrammatically.

Below is a great part of my work. Tens of lines of Python code that overlap with the last year of my PhD and all the research I’ve done on my own since then. I cleaned it up a bit and provide a rudimentary “documentation” so that people will know what they can do with it. Some of this was really difficult to find (when I looked for it) and some was impossible to find so I wrote it myself. The last four procedures, smooth, _rand_sparse, sprand and sprandn are not mine so I cannot take credit for them but I put them there for completeness’ sake. Everything else is more or less “mine”. This coincides with an awesome update on WordPress: embedding of Gists. Consequently, I saw this as a great opportunity to put up my code here before I start doing nasty things to it for my next piece of research. Take it, use it, repurpose it and let me know if something isn’t working, or needs fixing or if you did something cool with it. Enjoy.

mynetlib.py:

A collection of functions for the generation of neural networks and the setup of various experiments.

mat_reorder:

Visualising the structural and functional connectivity of a neural network. This is a simple function that calculates the covariance matrix of a neural network based on its activity. It then reorders the covariance matrix to obtain a depiction of functional connectivity and based on that reordering also rearranges the connectivity matrix in order to obtain a clearer picture of its structural connectivity. Put simply, it brings closer for us to see neurons that are structurally and functionally connected. All it needs is the network’s connectivity matrix and a sample of its activity in order to calculate the covariance matrix.

ind_net:

A simple function for obtaining a spiking neural network from a binary genome and evaluating its information transmission capabilities with respect to stochasticity.

gen2phen:

A simple function for converting a binary genotype into the architecture characteristics of a spiking

neural network. It takes as input L, which is the number of layers in the network for the particular case of layered networks, gn which is the number of genes in the genome, gl, which is the length of the binary gene and gen is the current individual’s genotype. It returns the network’s characteristics, N, the number of neurons per layer, C, the connection strength for each set of connections and dn, the connectivity density for each set of connections.

eval_net:

A simple function for evaluating a spiking neural network’s information transmission capabilities with respect to noise amplitude. This function takes cf_stor as input, a vector of stored coding fractions each in response to a different value in noise level. It removes NaNs, calculates the coding fraction

at optimum noise level opt, finds the optimum noise level index, the trends of pre- and post-optimum coding fractions trajectories and their slopes. It can then use any of those values to compute a fitness score depending on some fitness function and returns it.

eval_estim:

A simple function for stimulus estimation and evaluation. The function takes x and y as input where x and y are both signals (in this particular case an input signal and the response of a neural network) and it returns cf which is the coding fraction between the input signal x and its estimate Iest.

oneur:

A simple function for simulating a single Izhikevich spiking neuron model. This function simulates a single stochastic Izhikevich neuron. It takes T, D and I as input where T is the simulation length in ms (positive integer), D is the noise amplitude (sigma of Gaussian distribution) and I is the input signal. It returns firings which is the neural response (a binary sequence).

stim_rec:

A simple function for stimulus estimation/reconstruction in a neural system using a Wiener-Kolmogorov filter. This function takes two 1-by-N arrays as input, the input signal presented to the neuron or neural net and the neural response. It also takes two integers nfft and tstep, where

nfft is the number of data points used in each block for the FFT and tstep is the sampling frequency. nfft must be even and a power of 2 is most efficient. tstep is an integer declaring the time step. It creates a WK filter that is then convolved with the neural response in order to produce an estimate of the input that produced it. It returns the input signal estimate, the zero-centered input signal and the filter.

onet:

A simple function for evaluating a spiking neural network. This is a network of Izhikevich neurons arranged with the architecture produced by netgen. It takes as input the architecture produced by netgen, the input signal produced by input_gen, the length of the simulation T and the noise level D. It returns a matrix of 0 and 1 where 1 signifies a spike and 0 the lack of neural activity.

input_gen:

A simple input generation function. This function generates an analogue signal for the neural net. It uses a Gaussian distribution for the values and then smooths them using a moving window method. The input Ia is the input amplitude T is the length of the simulation in ms, K is the number of neurons

in the population, ind are the indices of the neurons the input will be presented to, smin is the length of the smoothing window and norm is the option of whether to bring the entire input signal above zero

or not. It outputs a matrix with the signal values for all rows with index ind and zeros for every other neuron.

Netgen:

A generic network connectivity architecture. This is a simple function that generates a variety of network connectivities and consequently architectures anywhere from a three-layer feedforward network to a neural pool of dynamics. It supports recurrent connections, lateral and self-connections, feedforward connections (obviously), any degree of sparsity (0 to 100% connectivity) and both excitatory and inhibitory connections. This function takes three arguments: N, C and D. N is a

1 by n list of integers larger than 1, where n is an integer from 1 to 3 and signifies the number of layers

in the architecture of the network. Each value in N is the number of neurons in each layer. C and D are L by L lists where L is the number of layers in the architecture. C stores the synaptic strength of each set of connections and D stores the sparsity of each set of connections. This function builds

subsets of connections with individual strengths and sparsities which connect any one layer to another. The subsets are then concatenated into a big matrix which describes the connectivity and the overall architecture of the network.

smooth:

Smooth the data using a window with requested size. This method is based on the convolution of a scaled window with the signal. The signal is prepared by introducing reflected copies of the signal (with the window size) in both ends so that transient parts are minimized in the beginning and end part of the output signal.

rand_sparse:

Used to generate an M-by-N sparse matrix of any chosen density.

sprand:

Used to build a sparse uniformly distributed random matrix.

sprandn:

Used to build a sparse normally distributed random matrix.

Why would I want to do that? Neurons of course. It’s always about neurons. I want to test how well Spiking Neural Networks (SNNs) will fare against some physical problems. I will find out what the best SNNs are for controlling systems with varying degrees of freedom using Genetic Algorithms (GAs). And then I will dissect them. I will study their topology, their connectivity properties, their information processing capabilities, everything and anything that can clearly demonstrate why some SNNs would do better than others as control systems.

This is going to be a long project. The first step was to find a sort of “physics engine” for my simulated “real-world problems”. Now that is done, I will revamp my neuroevolution Python code to work for this set of experiments and hopefully work better. After that I should be able to start evolving SNNs that will solve physical problems and then I will have to start collecting tools for analysing them. As always, I will show you everything I’m doing as I slowly work on this project. Stay with me!

I imagine this happens in all forms of science and artistry. You think you are good at something, and maybe you are, but then you look at your work after years of improvement and feel a little embarrassed. It wasn’t that my old MATLAB code was not good, I just thought I could do a lot better now. So I decided to rebuild my toy control problems in order to use them in my new set of experiments with spiking neural networks.

The first and simplest of these problems is the simulation of a pendulum (a mass hanging from a string in a world with gravity and friction) more commonly known as an inverted pendulum. My first implementation was a direct translation from my MATLAB code into Python. It is a set of two relatively simple differential equations that are solved using fourth order Runge-Kutta methods. Normally, differential equations are solved by the very famous Euler method, but Runge-Kutta methods are much more accurate. And I like accuracy. Click here for the first version of my simulated pendulum. Also, here is an example of the results:

A lovely, simple pendulum.

Everything looks fine and works as it should. Unfortunately, the code is not elegant or reusable. This is not important just to me. This is important for everyone. Imagine having to hammer a nail: a rock would work fine, but a hammer is a much more elegant, not to mention far more long-lived solution. So I need to build me a hammer. Enter pendulum v 2.0. This is a much more elegant attempt at simulating the pendulum. And here is an example of the results:

An equally pretty but less accurate pendulum.

Unfortunately, I can only get the differential equation solver that comes with Python’s Scipy and uses the Euler method to work. The one that uses the 4th order Runge-Kutta method, I haven’t cracked yet mainly because of horrendous documentation. The really annoying thing is that this is a gorgeous piece of code because if I can get it to work, I can then theoretically feed it any set of differential equations (ie. simulation of a physical problem) and it will solve it (ie. simulate it). I won’t have to change anything other than the set of equations for each new problem. This will make the code infinitely reusable for me and everyone else.

So let me return to bashing my head against Python code and pendulums and differential equation solvers and if by any chance any of you know anything about this and can help me, I will name the first spiking neural network that can keep the pendulum upright after you (I will also be very grateful). It also goes without saying that I will share the code and results.

My project went really well I think considering I have never done this before, I am a new scientist and I don’t have much of an online presence. However, I do feel a little sad. I wanted to do better and in order to do that I have to figure out what, if anything, went wrong. Inevitably, I turned to the data collected during the second round of #SciFund (I also analysed the data collected from the first round but that was just for reference, it bears no significance here). I decided to perform a simple analysis, something that I can then tell everyone else how to do, and see what is important for a successful #SciFund project.

The data analysed was the financial Goal of project, which I like to think is linked to how ambitious the project is. The amount Raised, Percent Funded and whether the project was Fully Funded or Not which are indicators of how successful the project was. The number of Contributors, Contributions and Mean Donation are simple statistics about how the project was funded and could be said to be linked to its popularity. And finally, the number of Tweets, Facebook Likes and Video Views which I like to think are indicators of exposure.

The best thing about the analysis I carried out is that it was dead easy, anyone should be able to do it. I took the data, put it in a lovely LibreOffice spreadsheet and fed it to Wolfram Alpha. Uploading files is a pro feature which means you have to register but fortunately there is a free trial period of two weeks. I won’t bore you with all of the analysis, I’ll just give some highlights and my own, possibly unsafe conclusions along with a small hypothesis.

First, I’ll throw some pretty pictures in your face. Here is one with the distributions of the data:

And here is another one with everything plotted against everything (I like to call it shotgun-science):

In both figures above, the Yes and No groups are the projects that were fully funded and not fully funded respectively.

The really interesting bit comes next though. Apart from being extremely easy to use, Wolfram Alpha is also super helpful. You just give it the data, ask it which variable to focus on (which is the dependent variable), set the confidence, it runs a regression analysis and then very helpfully tells you in english, not in math, what the analysis tells us. Here is an example:

This shows the effect of all variables on the amount Raised. Some are intuitive and some are not. The most important thing to notice though is that ambition, popularity and exposure of the project do not seem to have a statistically significant effect on the amount of funds raised. The same can be said about the less absolute Percent Funded variable:

Again, nothing seems to affect the result of the crowdfunding campaign of a project. And finally something that resolutely reinforces that conclusion:

The regression analysis has spoken.

What does this mean for a researcher that wants to crowdfund a science project? Well, so far it is obvious that having an ambitious project is not an obstacle and that popularity and exposure are not surefire ways of successfully funding a scientific project. This is also backed by the data of the first round of #SciFund. So what is important when crowdfunding science? My hypothesis is that right now, at this stage of the crowdfunding culture and particularly when talking about science, the most important factor is having a supporting community. This could translate into Twitter followers, Facebook friends, number of blog/site readers per month and a host of other things. Also, this notion seems to be backed by the analysis carried out on the data collected after the last round of #SciFund.

Maybe this will change as the crowdfunding phenomenon grows, but for now, it seems that the best preparation one can make is to nurture a community around their work that will support them when they decide to crowdfund their research.