[<<][math][>>][..]Mon May 11 09:54:21 CEST 2009

This is a rehash of the previous post. For bit vectors b of dimension n, the function bits : 2^n -> [0..n] introduces a partition[1] of 2^n. The equivalence classes have an order relation, determined by the bits function. The central idea in using this partition to represent numbers in terms of bit vectors is the distribution of bits(v) when v is drawn from a uniform distribution. It is a binomial distribution[2]. Such a distribution naturally encodes "agreement" (most bits zero or one) as an exceptionally ordered state, and "disagreement" as the natural disordered state. This non-linearity is quite a good match to the non-linear behaviour of neural networks when modeled as an inner product followed by a limiting nonlinearity, without ever using any explicit computations. Simply concatenating a number of such vectors gives a resulting vector that represents anything on the scale of no->disagrement->yes. The size of an individual bit vectors dermines its weight in the vote: v_out = [ v_1 | ... | v_n ] Let's call a single bit vector v \in {0,1}^n a "vote". Each vote has a weight n which is its dimensionality. Now, to make this managable, the only real "computation" that has to happen is to reduce the weigth of votes so they can be combined with other votes to form new ones. The essential observation is that the distribution of a vote with weight n is qualitatively similar to the distribution of a vote with weigth n-1. In other words, simply discarding one element of the bit vector on average has no qualitative effect. Let's call this "weight loss". The same reasoning goes for randomly flipping a bit. On average this has little effect, giving rise to good noise immunity. This gives some form of computation (it can do anything feedforward neural networks do) in a statistical way. The next step is to make weight loss programmable. A vote should be able to inhibit another vote. The problem here is that it is a physical operation. Inhibition means to cut wires. Is it possible to do a non-phsyical inhibition? Yes. The and/or gate to introduce assymetry between yes and no. One vote could be combined with another vote (gate it). So, what then is a nerve cell? It's where multiple wires come together and a random part of them is discarded. Forgetting is the essence of computation. ;) So, given a (huge) set of binary nodes, there are only two operations: - reduction: taking a subset of signals to create a node - introduction of assymetry (inhibition through AND|OR) = gating [1] http://en.wikipedia.org/wiki/Partition_of_a_set [2] http://en.wikipedia.org/wiki/Binomial_distribution

[Reply][About]

[<<][math][>>][..]