bekolay.org

A lot of times, I will try to explain the NEF to people, and it is not a simple thing to do. But while I was in the bathroom I came up with what I believe is a useful analogy, so I wanted to write it down before I forget about it.

Our research is based on the Neural Engineering Framework (NEF), created by Chris Eliasmith (my supervisor) and Charles Anderson. The principles of the NEF allow us to encode a signal using a group of neurons, do some transformation on it (say, double the signal) and then decode the result in another group of neurons.

So, maybe the signal is just a constant value of 0.4. We can encode that in a population of neurons, and then find a set of connection weights such that we can decode the value 0.8 in the next population of neurons.

Well wait, you may think. Are you saying that neurons in the brain represent numbers, and that the complexity of our behaviour is just some mathematical transformations of those numbers?

Yes and no.

Consider the physical world. Look at a metre stick. Obviously, you might tell me, a metre stick is 100 cm long. If you glued two metre-sticks together, end-to-end, you would end up with a piece of wood that's 200 cm long. You've doubled the length of that stick, and you can easily define that transformation: 100 to 200, you multiplied the first number by two. You doubled the length. Simple!

But why do you consider that metre stick to be 100 cm long? In another part of the world, it's 39.4 inches long. On another planet, that uses base 16 because they have 8 fingers per hand, it's 2C kerplorks long.

The way that we understand the length of the metre-stick is just a convenient way of understanding the world around us. We all (in the countries that use the metric system) agree on how long a centimetre is, so we can all agree that the metre-stick is 100 cm long.

But this is really just for convenience, because whether you say it's 100 cm or 39.4 inches, the transformation is the same -- you take the number and double it. Even if you use an entirely different system of math, that has no concept of doubling, there will be a way to take one number and get twice that number as a result. Or rather, there will be way of describing a length of material, and a method for describing a length of material that is twice the original length.

We think of doubling because it's the easiest way for us to understand it.

The same is true for the way that the NEF represents and transforms signals in a neural simulation. We have a group of neurons that fire at specific times. Those spikes induce current into another group of neurons, which spikes at specific times. We, to make it understandable, have defined a method to represent those firing patterns as numbers, and using that convenient way of looking at neurons, derived a method of finding connection weight matrices that will give us twice the value in the second group of neurons.

So, we have a profile of firing patterns for the first group that represents a number, and a way of eliciting a firing pattern for the second group that represents double that number.

It's really no different than gluing two metre-sticks together.

From Dang-Vu et al. 2010

A Wired article referencing a short article in the recently published volume of Current Biology makes the claim that "spontaneous brain rhythms" predict how heavy or light a sleeper you are.

Quick summary: if you take an EEG recording of a sleeping person, they will have little bursts of 11-15 hz oscillations called "sleep spindles" (pictured above). There is evidence that the number of sleep spindles positively correlates with learning potential -- that is, the more sleep spindles that are produced when you're sleeping, the higher your IQ. This new research adds that there is also a positive correlation between sleep spindle production and resistance to sleep disturbances -- that is, the more sleep spindles that are produced when you're sleeping, the harder you are to wake up.

It's an interesting article, and the research is sound, but there are some disconnects between the way I look at brains and how these researchers (and perhaps many biologists) do.

First, these studies are trying to decipher brain functions using EEG. EEG is a measurement of electrical activity on the scalp -- it gives you an idea of the average electrical activity of a large group of neurons (on the order of thousands or millions). This doesn't give us enough granularity to investigate function, just like looking at the NASDAQ doesn't tell you very much about a company's or a group of companies' financials. You can find thousands of phenomena that a particular pattern of EEG activity is correlated with, but no amount will tell you how the brain performs a certain function.

Second, the title of the original article calls sleep spindles "spontaneous brain activity". Perhaps this is just a case of different jargon, but in the way I see the brain, there is no such thing as spontaneous activity. There is some base firing rate for each neuron, but once you connect them together the activity of each neuron is just a function of the inputs it receives from other neurons. Even if we take the definition of spontaneous as something that happens without an identifiable stimulus, everyone is constantly being bombarded with stimuli, whether it be visual or tactile and so on. When you're asleep, you're responding to the accumulation of stimuli throughout the day, picking through which are important, replaying them, and consolidating those memories in cortex. Classifying brain activity as spontaneous or not spontaneous tells us absolutely no information, unless you think the brain is acausal or magic.

Even in interpreting the results, the investigators think differently than I. In the Wired article, one of the researchers writes, "During sleep, our neurons are busy doing very complicated processing, including, this study shows, generating sleep spindles to protect us from being awoken from noises in the environment."

Wouldn't an evolutionary biologist suggest the exact opposite? Evolutionarily, it's more advantageous to wake up when you are being attacked, or are otherwise in peril -- sleeping through it is a death sentence!

It should be noted that this notion is not present in the original article, perhaps because of the peer review process. Well, like Wired, bekolay.org goes through no peer review, so I'm going to make some claims of my own.

During sleep, our day's experiences -- temporarily held in the hippocampus -- get selectively replayed and consolidated into cortex. The better you are at this process, the better you are at efficiently forming memories. The better you are at forming memories, the more activity will occur during sleep -- producing more "sleep spindles," an artifact of the memory consolidation process. Since your brain is attending to memory consolidation, it takes a higher threshold of disturbance to disrupt sleep -- just like someone concentrating while conscious can ignore distractions (to a point).

The evolutionary significance of this suggests some kind of plateau to the overall intelligence of a group of animals living in a dangerous environment. The animals with the best memory will the last to wake up when the group is attacked.

While that's a bit of a bummer, it kind of makes me optimistic. Now that we live in a decidedly safe environment, being a heavy sleeper is all positive -- meaning modern humans' intelligence shouldn't plateau when we become a world of incredibly heavy sleeping super geniuses.

I'm not sure if it's because I'm back in Winnipeg and consequently thinking about the past, but I've been reading some old things I wrote and one entry in particular caught my attention. I guess it's something that's been on my mind passively for a while, so it's interesting to read how I thought about the subject two years ago.

Originally posted as a Facebook note, edited for boooring:

It's been a few weeks since my last update -- I've been busy okay? But it's reading week so I have enough time to blatantly self-promote.

I got a gig writing guides for the How-To Geek. I'll be writing a fair bit about Linux, which I realize has already excluded many of you from caring, but I'll do Windows articles as well.

So far I've had four articles published, so please do visit them. They're great, I swear.

• How To Hide Kernel Updates in Ubuntu
• Getting Started with SMPlayer on Windows (to Play Movies Better)
• How To Fix Sound Issues in Ubuntu 9.10
• Fix Windows Update Errors by Letting ActiveX Traffic Through

Another project of mine has been putting up all of the old Manitoban articles I wrote; really, I'm fine with just linking to the Manitoban website, but the articles from the 2008-2009 publishing year aren't online, so I must preserve them, lest someone wish to read "The 'meatatarian' saga: douche vs. bully" and not be able to.

They're going up gradually; you can see them as they're uploaded here.

Finally, another tasty morsel that I added to this here website recently is the transcript of an interview I did with a master's of social work student for her thesis on student exchange experiences. Basically, it was an excuse for me to wax nostalgic about my year in Tokyo, and I think I produced some decent sound bites. The full transcript is here, but I'll end this post with a few choice passages.

In a previous post I voiced my concern that scientific researchers do not have to provide the source code that produces their results. Now, Darrel Ince, Professor of Computing at The Open University, has written an article for the Guardian also calling for scientists to release their source code. Slashdot covered the story this morning.

Ince argues this as part of the fallout of "climategate," the incident in which it was shown that climate researchers had been fudging scientific data to produce desired results. It is a compelling example, but Ince goes on to say that "the situation is by no means bad across academia." He cites the example of the four colour theorem from graph theory, which was proved partly by a computer program. The program was published and highly scrutinized, and eventually found to be correct.

However, not all results are that nice in mathematics and computer science. In his PhD thesis, Terry Stewart notes one result that, to this day, has not been recreated.

SHRDLU

SHRDLU was a natural language understanding system developed from 1968-1970 by Terry Winograd at MIT. This 40-year old system produces some very interesting interaction, and was one of the first great results in artificial intelligence research. The following are two snippets from a demo found at Winograd's website:

That is crazy impressive. The question is: why haven't we seen systems like it in common use today? A page maintained by Semaphore Corp. that provides source code for an implementation of SHRDLU gives some insight:

40 years of natural language research and infinitely more powerful computers are still not able to reproduce SHRDLU's 1970 results. How is this possible?

One of Winograd's students, Dave McDonald, notes, "In the rush to get [SHRDLU] ready for his thesis defense [Winograd] made some direct patches to the Lisp assembly code and never back propagated them to his Lisp source... We kept around the very program image that [Winograd] constructed and used it whenever we could."

Back in those days, most computers ran different operating systems, so as changes were made to the operating system of the computer Winograd and his students were using, the program became less robust (a phenomenon known as software rot).

So, even though the source code exists in some form, the original results he produced cannot be replicated because they don't include Winograd's direct patches, and because the platform that the code was created for longer exists.

The lesson

What should be taken from this example is that mathematics and computer science do still suffer the same problem of irreproducible results. Ever-changing platforms are less of a problem now -- a Matlab program produced on one computer should run the same on any other computer -- but source code is rarely shared as part of the publication process.

I think it should be. But for that to happen we need a central repository to store code and meaningfully link it to the papers that use it.

This repository should have the same amount of peer review and, therefore, authority that scientific journals have now. Maybe that can happen by existing journals adding the ability to link code to a paper (and enforce that any code used to generate results is included), or maybe a new organization has to rise up to the challenge (I would love to see code.arxiv.org).

Already I can hear the outcry of scientists claiming that their code is "sloppy" and "not ready to be released," but those concerns are simply irrelevant: all that matters is that the code produces the output cited in the paper given the input cited in the paper. That's it. If another researcher finds your result interesting, then let it be up to them to wade through your code -- it's probably still way better than trying to reproduce your result based on the prose that describes your algorithm.