Shriphani ‘PSP’ Palakodety

This post is slightly psychology influenced. Well, in the Psychology class I have been taking this semester (Introduction To Cognitive Learning), I was taught about Neural Networks and a certain Hebb’s Rule which allows a network to remember a previous configuration and use this configuration to interpret new information. We use the following model to represent a cell:

1. A cell (neuron) has only two activation states: On or Off.

• On: Cell has high firing rate. (High number of action potentials fired in unit time).
• Off: Cell has low firing rate.

2. Cell weights are reciprocal. Weights symbolize the size of the synapse between two cells. Greater size = greater strength.

3. If cells fire simultaneously, they develop positive weights. The active cells (activation = on) inhibit the inactive cells (develop negative weights.

4. The network organizes itself based on sensory input. And new sensory info is interpreted by previously developed weights.

5. The network updates itself one at a time.

6. The activation of a cell can be determined by the following formula:

1 if ∑w_ija_j > 0
0 otherwise

The weights lend a sort of memory to the network since weights influence the interpretation of subsequent sensory input.

Based on these rules, I wrote something that simulated a network that implemented the Hebb’s rule.

First, we need to describe a neuron. Our neurons represent an uppercase letter of the english alphabet. We also need a way to represent the weights with each of the other neurons.

So a basic class to describe a neuron:

Now, the network has a collection of neurons (26 in our case which signify each letter of the alphabet). Hence, the init method for the neuron class:

Now that we have a network that is populated, we need a method that receives input and figures out the changes in the weights between cells (or the strength of the connection between cells). This method is then fed by another method that figures out the overall change in the weights and computer which cells are active:

Then we run this network with:

Before I run this, let me mention that this thing only accepts uppercase letters and breaks if a letter is repeated in a word. This is because neurons rarely have their own axon connected to their own dendrite (neurons are not connected to themselves).

So, a sample run:

tark-b-183:scripts shriphani\> python hebb.py
Sensory Interface: HELP
{'A': -4, 'C': -4, 'B': -4, 'E': 3, 'D': -4, 'G': -4, 'F': -4, 'I': -4, 'H': 3, 'K': -4, 'J': -4, 'M': -4, 'L': 3, 'O': -4, 'N': -4, 'Q': -4, 'P': 3, 'S': -4, 'R': -4, 'U': -4, 'T': -4, 'W': -4, 'V': -4, 'Y': -4, 'X': -4, 'Z': -4}
The possible interpretations of this input might be anagrams of the subsets of: 

['E', 'H', 'L', 'P']
Sensory Interface: LOSER
{'A': -9, 'C': -9, 'B': -9, 'E': 7, 'D': -9, 'G': -9, 'F': -9, 'I': -9, 'H': -2, 'K': -9, 'J': -9, 'M': -9, 'L': 7, 'O': 0, 'N': -9, 'Q': -9, 'P': -2, 'S': 0, 'R': 0, 'U': -9, 'T': -9, 'W': -9, 'V': -9, 'Y': -9, 'X': -9, 'Z': -9}
The possible interpretations of this input might be anagrams of the subsets of: 

['E', 'L']
Sensory Interface: HOMELY
{'A': -12, 'C': -12, 'B': -12, 'E': 9, 'D': -12, 'G': -12, 'F': -12, 'I': -12, 'H': 4, 'K': -12, 'J': -12, 'M': -1, 'L': 9, 'O': 4, 'N': -12, 'Q': -12, 'P': -6, 'S': -6, 'R': -6, 'U': -12, 'T': -12, 'W': -12, 'V': -12, 'Y': -1, 'X': -12, 'Z': -12}
The possible interpretations of this input might be anagrams of the subsets of: 

['E', 'H', 'L', 'O']
Sensory Interface: LOSER
{'A': -17, 'C': -17, 'B': -17, 'E': 13, 'D': -17, 'G': -17, 'F': -17, 'I': -17, 'H': -5, 'K': -17, 'J': -17, 'M': -9, 'L': 13, 'O': 8, 'N': -17, 'Q': -17, 'P': -11, 'S': 1, 'R': 1, 'U': -17, 'T': -17, 'W': -17, 'V': -17, 'Y': -9, 'X': -17, 'Z': -17}
The possible interpretations of this input might be anagrams of the subsets of: 

['E', 'L', 'O', 'R', 'S']

So we observe that in the first case, four neurons were activated. The word “LOSER” led to H and P being deactivated because the synapse weakened. The next input “HOMELY” causes H and O to activate because O began developing positive weights with other cells and when LOSER was submitted as input and developed a strong enough connection when it was seen in another sensory input. When LOSER is flashed again, it leads to H being forgotten and causes R and S to activate.

Now, a particular case which (I think) indicates the use of this “learning” technique is when we are playing scrabble. Assume that I have words such as NEAR in my memory and when I see N and E on a board, I should be able to complete these words. Watch:

Sensory Interface: NEAR
{'A': 3, 'C': -4, 'B': -4, 'E': 3, 'D': -4, 'G': -4, 'F': -4, 'I': -4, 'H': -4, 'K': -4, 'J': -4, 'M': -4, 'L': -4, 'O': -4, 'N': 3, 'Q': -4, 'P': -4, 'S': -4, 'R': 3, 'U': -4, 'T': -4, 'W': -4, 'V': -4, 'Y': -4, 'X': -4, 'Z': -4}
The possible interpretations of this input might be anagrams of the subsets of: 

['A', 'E', 'N', 'R']
Sensory Interface: NE
{'A': 1, 'C': -6, 'B': -6, 'E': 4, 'D': -6, 'G': -6, 'F': -6, 'I': -6, 'H': -6, 'K': -6, 'J': -6, 'M': -6, 'L': -6, 'O': -6, 'N': 4, 'Q': -6, 'P': -6, 'S': -6, 'R': 1, 'U': -6, 'T': -6, 'W': -6, 'V': -6, 'Y': -6, 'X': -6, 'Z': -6}
The possible interpretations of this input might be anagrams of the subsets of: 

['A', 'E', 'N', 'R']
Sensory Interface:

Well, this semester, I have already been taught some insanely great stuff. I have a bunch of exams coming up. Great scores will be the perfect way to make an awesome semester even better.

The script can be downloaded here.
The material here was used from Professor Greg Francis’ notes on Neural Learning.