attempted to train a Spiking neural net from scratch again!
this time it relies on repeated mutation-and-selection cycles to refine the connection weights. results at the bottom of post.
Link to code in comment
50 neurons arranged in a 5×10 grid. Each neuron fires brief electrical “spikes,” and only the last two columns represent digits 0–9 (one neuron per digit).
When we want the network to add two numbers, we feed the first number’s spikes into column 0 and the second number’s spikes into column 1.
Those pulses ripple from left to right through the grid, and whichever output neuron spikes the most (and fastest) is the sum the network “believes” is correct.
To make it learn, we keep a population of ten such networks. Each generation, every network tries fifteen random addition problems.
Their fitness score rewards accuracy and confidence, but penalizes slow or wildly wrong answers. We keep the top three, mutate their connection weights randomly (no crossover- on a second thought, it should have crossover), and use those to refresh the new generation.
Over generations, networks that spike in ways that resemble actual addition are kept; the rest get replaced. Meanwhile, the frontend plays back each winner’s spike sequence so you can literally watch information flow from inputs to outputs in that neon cyberpunk interface.
RESULTS: I still couldn't get spiking networks to learn properly and well(accuracy mostly remains at chance). But this was an interesting experiment. it also has a detailed readme and "how it works" page if you want to learn more. The UI looks pretty cool tho :)
you see the accuracy jump in increments of ~6.7% cuz we are testing 15 examples per trial
inspired by Gibbs phenomenon in Fourier series approximations
Open source- link below
Fourier series can approximate the curve for any function, using only sine and cosine waves, but with discontinuous functions with sharp vertical lines it requires infinite many terms
when we don't use infinite many terms we get these cool looking oscillations at the corners
fascinating stuff
----
if you like this consider becoming a patron and access to 500+ projects, exclusive videos and weekly meetings