Path Selection and Planning

Contents:

1. Requirements for advancement to the next tasks
2. Multiple place field resolutions
3. Path formation
4. Active output
5. Sparse encoding and recoding
6. Minicolumns
   1. Model Options 1: Sub- or suprathreshold connectivity from population a to population x
   2. Encoding Protocol Step 3
   3. Encoding Protocol Step 4
   4. Encoding Protocol Step 5
   5. Encoding Protocol Steps 6 to 8
   6. Encoding Protocol Step 9: Output and activity
   7. Encoding Protocol Summary
   8. Retrieval Protocol Step 1
   9. Retrieval Protocol Step 2
   10. Retrieval Protocol Step 3
   11. Retrieval Protocol Step 4
   12. Retrieval Protocol Step 5

• Minicolumns: The Thorpe Task
• Minicolumns: The Schultz Task

Requirements for advancement to the next tasks

• Catacomb models of Mike Hasselmo's proposed ``Minicolumns'' for rule learning in the neocortex.

• Path learning that includes information beyond nearest neighbour connectivity between place cells. This can involve:
  1. Adjustment of the STM model to enable connections between more place cells in an episode.
  2. Improvement of the place cell model:
     1. Non-geometric layout; place cells assigned according to significant features.
        
     2. Overlapping place fields at multiple resolutions, forming multi-cell autoassociative items.
        

• A forward-only path retrieval method for the detection of paths to a goal.
  • This can use the neocortex to select a route, based on the results of rapid retrieval that detects paths that reach the goal.
  (Alternatively, a goal ``context'' may be linked to a path during learning, as described previously.)

A. Multiple place field resolutions

1. This approach does not require adjustment of the STM buffer to handle more than 2 items during heteroassociative learning.

2. It does require adjustment of maximum conductances on recurrent connections, so that recall of the next item in a path depends on the activity of all elements of the previous item.

3. This can be implemented using n (n>1) feature discretizer components in Catacomb, each of which discretizes at a different resolution. Their outputs can be considered different strands of feature information that arrive at EC layers, just as smell or sound can supply additional feature information.
   

B. Path formation

CA1: Place fields are crisp. Each ``item'' is sparsely encoded.

For the planning phase, we need a rapid forward recall that can test which paths lead to a goal. Here we define:

• path = multiple sparsely encoded ``items'' in CA3, episodes.
• rapid = within one retrieval phase, i.e. an episode may need to consist of place fields linked at different resolutions. For example:
  

C. Active output

D. Sparse encoding and recoding

CA3 would encode sparsely, based on episodes at various resolutions (``any features''), items that are encountered and can be decoded:

The above ties into the use of the neocortex for rule learning and planning stages. During the planning phase, possible paths are tested. During the decision (or execution) phase, a winner is selected and retrieval is elicited accordingly.

Minicolumns

During construction I will initially identify subthreshold and suprathreshold connection by setting suprathreshold connections to 30 nS and subthreshold connections to 0.1 nS.

For maximal clarity during initial testing, I've made every population spike and vector output available through relays in the minicolumn networks. Eventually these should be replaced with just a few relays that receive spikes and vectors from concatenators. There could be one set of relays for all interneuron populations, one for x-y, one for r-s, one for p-q and one for a.

I made X and Y into distinct populations so that the weights can begin with different initial values without having to add unused synapse populations.

The diffuse input to X from A should probably last longer than the momentary input from Y output of other minicolumns. I therefore adjust the timing characteristics of the receiving synapse populations in X accordingly. I also insure that X will spike only when receiving a combination of diffuse input from A and sharp input from external Ys. nX activation in response to mY activation by mA during initial encoding (when WB is initialized as ones) is now successfully gated by diffuse input from nA in minicolumn-2-captured-20021011.ccm.

In order to avoid relying on an LTD learning rule, I am setting initial Wb weights to 50 percent of their maximum value. That value in the absence of input from na is then made to be subthreshold on nx and suprathreshold in the presence of input from na to nx. Synaptic transmission modulation can be used to alter those conditions in different phases. Learning in Wb works according to these specifications in minicolumn-2-captured-20021011.ccm. Prior to training, my alone could not spike nx. After training it can.

I'm temporarily using two vector objects to control the excitatory versus inhibitory transmission strength from a to y in accordance with encoding or retrieval dynamics being worked on.

As far as encoding is concerned, I can now test the learning of nmWf between ns and mr due to activation of na followed by ma. Using an event sequencer to activate mx as a result of a theorized following minicolum's activation can then cause ms to activate and train mmWif.

The synaptic weights on the synapse population transmitting activity of a and of x to s is updated so that each is subthreshold individually, but suprathreshold together. There also has to be a delay (currently implemented by a simple delay on the spike projection) from a to s, so that a and x input can coincide. Perhaps it is better to create a separate synapse population for a input, which can then be of diffuse longer duration.

At this point the training of Wf from 50 percent diffuse to focused patterns at 100 percent is almost working as it should. The s population is gated by x and the r population is gated by y so that only specific elements spike and a sensible pattern of connectivity is learned in Wf.

1 - achieve holding of r and s in p and q
- retrieval dynamics:
  - forward spread: planning and reality

3  - during retrieval, goals provide backwards information (into x),
    sensory input provides information in a, retrieve path
    - can test with temporary "fake" weights that signify encoded path
      goal input enters x

2  - do the max function in mr, take input from ns subthreshold and spike
    only if my is matching (no effect from mp necessary then). Otherwise,
    spike if mp and my are matching. Once a spike is successful, inhibit
    other elements. To test this, use fake Wf weights
    planning: if p inactive -> activate r according to sWf+y match by elements
    and inhibit rest; if p pactive -> activate r according to sWf+y+p match
    by elements and inhibit rest. p must then help, since it also produces
    inhibition. Both sWf and y must be subthreshold on their own, but
    suprathreshold together. An element p must cancel out the p group
    inhibition effect. y also has endstopping - so, if a is active then
    y will not be active and hence activity cannot propagate forward through
    r.
    (i) If (ma) then mr=[0 0 ... 0].
    (ii) If (!ma) and (!mp) and (nsWf*my) then mr=Comp(nsWf*my).
    (* = dot product; Comp() = competition that allows only one element to be
    active)
    (iii) If (!ma) and (mp) and (nsWf*my*mp) then mr=Comp(nsWf*my*mp).

4 - import the theta modulation circuit and set up proper encoding/retrieval
    phases

To test and tune retrieval, I temporarily disconnect transmission from ns to mr, set the mr receiving synapse to a subthreshold explicit value (after deactivating the LTP function), and provide simulated ``nsWf'' spikes to mr from a SpikeSequencer. In order to provide spikes to my from a SpikeSequencer I temporarily set the initial weight of the synapse population that would ordinarily receive input from mx to the maximum value.

When both my and nxsWf inputs are active at about the same time (a few ms earlier on my) and on the same element then the corresponding element in mr spikes as desired. When the activities are out of synchronization (i.e. the elements are not active together) then mr appropriately does not spike.

The holding cells p and q properly repeat activity now that they receive rhythmic input in synchrony with the encoding-retrieval modulation of synaptic transmission on y. If retrieval in p and q is a bit early, the phase can be shifted easily by adding a delay to the projections from the spike forwarder to the theta synapses.

Status of the r function:
Rule (i) untested.
Rule (ii) works for coinciding nsWf and y activity, lack of y untested, competition untested (but some inhibition clearly follows). (Activation of r is followed by holding in p.)
Rule (iii) untested.

I'm setting conductances on RS excitatory and inhibitory synapses receiving input from p and q to 2.0 nS and 2.5 nS respectively, which make their effect subthreshold with a peak influence of about 5 mV. In addition to the slightly lower amplitude, the excitatory synapses were also slowed down compared to the inhibitory synapses to make up for the greater potential difference. This is necessary for rule (iii) at r. It remains to be seen if different synaptic conductances are needed at s. When p is holding an active element it now properly adds inhibition to elements of r that map to inactive elements of p, while having no effect on the element that maps to the active element in p.

Status of the r function (minicolumn-retrieval-holding-phases.20021018.ccm):
Rule (i) properly deactivates y and any subsequent activity in r. This rules appears to work properly.
Rule (ii) works for coinciding nsWf and y activity, r is not activated in the absence of y (y gates r), competition appears to work now that fall times for excitatory and inhibitory input from p to r have been increased. (Activation of r is followed by holding in p.) This rule appears to work properly.
Rule (iii) in the presence of p, sWf without y properly did not active r, with both sWf and y elements active that differ from the element held in p the element in r was properly not activated, with matching p, sWf and y elements (although later than the non-matching set) it properly does spike. This rule appears to work properly.

For further retrieval testing, I'm temporarily setting the input synapse on mx to its maximum conductance of 3 nS (from its normal initial value of 1.5 nS) and deactivating the LTP rule.

Retrieval rules involving a, x, s and q:
(iv) If (!mrWif) and (!mq) and (ma) and (mx) then ms=Comp(mx).
(v) If (mrWif*mx) and (!mq) then ms=Comp(mrWif*mx).
(vi) If (mrWif*mx*mq) then ms=Comp(mrWif*mx*mq).

I've set the delay on the spike projection from ma to ms to 1 ms, instead of the 20 ms delay used in encoding, so that there are no issues with that. The delay may not be necessary during encoding now that the effect of ma on ms has been prolonged with synaptic timing similar to that used for synaptic effects of p and q on r and s respectively. Encoding must be retested! I am not changing any parameters in ms, since the same neuron template is used for mr, which is already tuned. I move the connection from ma to ms to the synapse that also receives input from mq, since that synapse has a prolonged subthreshold effect. I must also test the effect of retrieval in the x-s-q group on the y-r-p group, since activation of ma currently causes excitation of mx and subsequently of my, despite the end-stopping inhibition on my. Perhaps that inhibition must have a longer lasting effect. I decreased the amplitude of the conductance from ma to mx to 1.5 nS so that it is indeed subthreshold. Similarly, I am reducing the conductance from ma onto ms (that is experimental, since mr requires a certain level of conductance there so that the balance of excitation and inhibition from mp to mr has its desired effect).

Rule (iv) appears to work properly, as demonstrated between t=800 ms and t=1000 ms in minicolumn-retrieval-holding-phases.20021020.ccm.

Currently, there is no inhibitory population that balances excitation with inhibition from mq to ms. This inhibitory population may need to be added. I should also test to see if there is a need for the inhibitory population that implements competition between elements of mr and between elements of ms. That competition may already be implemented via the holding populations mp and mq and their respective inhibitory populations. The additional inhibitory population for the inhibitory effect of all elements in mq on ms is added in the model minicolumn-rxp-syq-matching-competition.20021020.ccm. Rule (vi) now appears to work properly. How is mq to be deactivated so that Rule (v) can be tested (and by extension, how are mp and mq deactivated so that new activity can be held)?

Rules (i) to (vi) are tested in the model minicolumn-retrieval-clean.20021021.ccm in the order: (ii), (ii), (iii), (i), (iv), (iv), (vi), (v). STM deactivation occurs at t=1160 ms.

Retrieval rules in r:

Retrieval rules in s:

With the local rules (i) to (vi) in effect, I now test associative and forward convergence in the two minicolumns n and m in the model minicolumn-retrieval-convergence.20021022.ccm. Retrieval will be tested before encoding, so excitation at mx is imposed through inputs temporarily set to the maximum trainable conductance. Similarly, the connection from x to y is also assumed to be at maximum conductance for the appropriate mapping of elements. On the forward path, we assume that Wf between n and m has learned a maximal conductance mapping.

When m is assumed to represent a location adjacent to the current location, and n is assumed to represent the current location, and m is on a known path to a goal, then the current location activity of na should combine with associatively propagated activity in nx due to activity caused in my by mx. That should activate ns and consequently mr.

As mx(0) is activated at t=180 ms, the activity spreads to all my, since all connections in mWib are temporarily set to maximum conductance (this is akin to a case in which all elements of my connect to valid associated minicolumns). Catcomb currently lacks the ability to discriminate spike sources when crossing network boundaries. For that reason I am temporarily adding individual spike relays for my(i) element output. Element my(1) is connected to minicolumn n through nx(2). Activity mx(0) at t=180 ms [a] leads to activity my(1) at about t=190 ms [b]. That leads to activity nx(2) at about t=191 ms [c], which in turn activated all ny due to the temporary maximum conductance setting on all nWib connections.

When current location is activated at n through na at t=170 ms [d], then activity nx(2) alone manages to activate ns(2) [e], in accordance with rule (iv). Activity at na should also cause end-stopping at ny. To make that work I increased the characteristic fall timing of the inhibitory synapse from a to y to 20 ms and decreased the maximum conductance on Wib connections to 10 nS, providing longer and stronger end-stopping inhibition. End-stopping at ny is now achieved [f]. The activity of ns(2) is held in working memory in nq(2) [g]. Again, individual spike relays are added to temporarily offset Catacomb's inability to propagate spikes from specific ns sources through the network boundary. The activity of ns(2) now propagates forward along the connections in nmWf (temporarily assumed trained to maximal conductances) to mr(1) [h], which activates at about t=202 ms in response to the successful gating of that input by the activity of my(1), in accordance with rule (ii). The activity of mr(1) is subseqeuntly held in mp(1) [i].

Interlude: Major revisions in Catacomb require that I bring the minicolumn model up to date before proceeding with further functional construction. The first step is to get the model to run in the new version of Catacomb, the second to improve the model's minicolumn connection capabilities, using the new spike relays and the connection router.

There are several different ways to go about changing the size of the minicolumn networks. In versions up to minicolumn-retrieval-convergence-layers.20030312.ccm (a Catacomb 2.093 script), a single minicolumn contained dimensions of x,y,r,s,p and q that determine the local connectivity, i.e. the number of other minicolumns that a minicolumn can connect to directly. The advantage of this approach is that networks of minicolumns and their connectivity are clearly visible. The drawback is that manual placement and linkage of each individual minicolumn becomes prohibitively labour intensive as the number of minicolumns increases.

Another approach is to assign a dimension greater than one to the captured minicolumn, and to add recurrent connection routers to that. This is attempted in minicolumn-retrieval-convergence-layers-by-object-dimension.20030313.ccm, but does not work at this time due to limitations of Catacomb.

Finally, it is possible to relinquish the notion of visibly individual minicolunms alltogether, and instead to modify the dimensions of a,x,y,r,s,p and q to match maximal requirements. For N minicolumns, the dimension of a is N, the dimension of each of the layers x,y,r,s,p and q is N*M, where M is the number of minicolumns each minicolumn can connect to according to the desired connectivity ratio. In the simple test case, M=N. Connection routers within the minicolumn structure are then used to make the appropriate connections within minicolumns that allocate subsets of the a,x,y,r,s,p and q cell layers and between the minicolumns thus defined.

Seeking to equal the encoding and retrieval achievement of minicolumn-mike-environments-20030326.ccm: Mike's network manages to encode and retrieve a sequence of 7 minicolumn activations. Each episodic connection is learned in a single cycle.

The model in minicolumn-retrieval-convergence-layers-within-object.20030425.ccm implements an environment and layered minicolumns within a captured object.

In the current implementation, populations that support connections between minicolumns (x,y,r and s) have only a single synaptic connection to a cell in another minicolumn. If this restriction is removed, it may be possible to directly implement conditional relationships (e.g. "AND" and "OR" combinations based on subthreshold and suprathreshold contributions), as well as multiple resolutions for place cell connectivity.

In order to enable learning from occasional signals arriving at arbitrary times, such as while exploring a new environment, a STM buffer is added to population a (minicolumn-retrieval-convergence-layers-within-object.20030502.ccm). We are assuming that the minicolumn structures are located in prefrontal cortex, in which the presence of working memory (WM) is not controversial. As such, the minicolumn networks can also supply input to entorhinal cortex (EC) and our hippocampal models.

Functional Status: The a-STM buffer works most of the time. Whenever the buffer is full and a new afferent input appears, the entire previous sequence is lost. It appears that the time intervals between sequence items and between the last sequence item and an afferent input are not tuned in the manner prescribed by the STM buffer implementation in newtmaze models.

I therefore create a separate STM buffer object ``a-STM-buffer'' so that the parameters can be set correctly without affecting populations p and q. A working layered minicolumn model with working memory (WM) for population a is consequently available in minicolumn-layers-aWM.20030506.ccm.

The working memory enables minicolumn networks to deal with relationships between any set of events occurring at arbitrary times.

In order to align the STM repetition in WM with encoding activation of the y population by the a population, a number of modifications are made. Firstly, the transmission modulation of connections from a to y is taken from the transmission modulation signal, rather than its inverse so that a broader interval of transmission is achieved. Secondly, the transmission modulation of end-stopping is moved from the synapses at the y population to the synapses at the interneuronal population receiving input from the a population. By doing so, end-stopping inhibition is removed entirely instead of merely reduced during the encoding phase. The conductances of the transmission modulated synapses at the y and interneuron populations must now be tuned so that a phase broad enough to transmit all encoding signals in a sequence causes spiking during encoding and the remainder of a cycle enables inhibitory end-stopping spiking. Encoding should pass spikes during an interval of about 50 ms. If retrieval is also enabled during about 50 ms that leaves a good 25 ms for separation between the two modes. In accordance with thse guidelines, the y-population synapses receiving excitatory input from the a-population were set to a conductance 4.8 nS. The same conductance is set for the interneuronal synapses propagating input from the a population.

Model Options 1: Sub- or suprathreshold connectivity from population a to population x.

Option	Notes
Subthreshold connections from population a to population x during encoding. [This is the chosen option in current models (20030507).]	These demand non-zero initial connection strengths, so that action potentials can be achieved in the x population and LTP can be established. It is possible for this initialization to take place during development or a special prior learning phase aimed at generating functional and recruitable minicolumns. The presence of non-zero connection strength prior to learning does not lead to functional complications as long as each cell in x receives input from a y population cell in only one other minicolumn, since there can be no additive effects. This option simplifies directed learning with LTP on the y to x connections, since x can only fire after a spike is received from y. No special attention needs to be given to the shape of the a to x synaptic response. It need merely persist long enough to support spiking when a spike is received from y. This option can be used to attain bidirectional learning with a realistic LTP response function. If both a to x and y to x connections are initially subthreshold and their synaptic response functions are both persistent, spiking in x can be achieved for both sequence directions.
Suprathreshold connections from population a to population x during encoding.	This option removes the need for non-zero initialization of connection strengths from population y to population x. This option can guarantee unidirectional learning if that is desired, even with a realistic LTP response function, since spike times can be determined entirely by the local minicolumn, thereby retaining the spiking order. If subthreshold a to x responses are needed during retrieval, transmission modulation can achieve it. This option makes encoding Wb independent of propagation. In other words, transmission modulation can deactivate the y-to-x pathway during encoding if propagation along paths with previously established LTP interferes with proper encoding.

We expect the buffered interval between spikes in a sequence to be between 20 and 30 ms (about 25 ms). At present, the synaptic response in x due to a takes about 7 ms to rise to -55 mV and remains depolarized above that level for slightly more than 30 ms (synaptic parameters: G_0=1.5 nS, rT=2 ms, fT=20 ms). If the response due to y rises more quickly, this subthreshold response due to a should persist sufficiently to achieve spiking when a response to y appears.

The y to x synaptic responses are above -55 mV in the absence of any other contributions for only about 6 ms (synaptic parameters: G_0=1.5 nS, rT=2 ms, fT=4 ms). Sequence order should be preserved by combining the persistent response to input from the a population with this rapid input from the y population.

Figure 9: Cells in population x spike as combinations of persistent subthreshold responses to input from population a and rapid subthreshold responses to input from population y exceed membrane thresholds. Specific y-to-x connections between minicolumns can thereby establish LTP on synapses in Wb in the reverse order of activation at minicolumn population a. The first spike is at x(3) in minicolumn 2, encoding the reverse of the sequence of buffered activity in minicolumns 2 and 3. The second spike is caused by unintended spiking of population y in minicolumn 1 due to propagation along the untuned connection from x to y within minicolumns. This is an artifact of the step-wise process of tuning the minicolumn model for correct encoding operations. (Responses depicted were produced with minicolumn-layers-aWM.20030507.ccm.)

Note: While testing encoding spikes in the x population and learning in Wb, I've temporarily disabled the x-to-y connections within minicolumns (by setting G_0 from 10 nS to 0 nS and learning from LTPWb to none), since propagation through that untuned connection was interfering with the desired activity (see the unintended second spike in Figure 9 above).

For the present set of experiments it is useful if strong LTP is established after about 2 repetitions of a sequence. That learning rate can avoid some effects of spurious spiking, while achieving reliable encoding within a duration of the experiment that is acceptable in terms of simulation time. The maximum conductance is set to G_max=3.2 nS. Since the current version of Catacomb (2.098) does not allow me to directly inspect synaptic efficacies, I ensure that the proper learning rate is implemented by first setting it so that 3 repetitions would be needed. Since the sequence 2-3 repeats only twice, subsequent activity at minicolumn 3 should not lead to activity at population x in minicolumn 2. I then increase the learning rate so that the LTP achieved with 2 repetitions is just enough to achieve subsequent activity at population x in minicolumn 2 when there is activity in minicolumn 3. This was achieved with an LTPWb profile with height parameter y=120.

Figure 10: Two repetitions of the minicolumn activation sequence 2-3 lead to sufficient LTP on the y-to-x connection from minicolumn 3 to minicolumn 2 so that ensuing activity in minicolumn 3 causes spiking at cell x(3) in minicolumn 2. The broad responses precipitating spiking between t=1400 ms and t=1600 ms show the a-to-x contributions during encoding. Subsequent spikes are achieved with only the more rapid y-to-x contributions. (Responses depicted were produced with minicolumn-layers-aWM.20030507.ccm.)

Next steps:

Individual y cells must activate in a meaningful way in response to the activation of x cells, i.e. x cells should not activate all y cells, as would be the case if a is suprathreshold on all y during encoding of Wib. This is essential for the role of the y population in the gating of the r population. During encoding, y cells may receive subthreshold input from the a population and from the r population. The r population provides information about preceding active minicolumns in a sequence. Make the a-to-y excitatory connection subthreshold. Add a subthreshold connection from r to y, so that the combined effect of a and r inputs on y is similar to that in the encoding protocol applied to the x population. If a sequence contains activity at minicolumns n-1, n and n+1, then encoding Wib requires x activity (via encoded Wb connections) due to n+1 (or due to n and n+1 when encoding is still taking place at the x population) and y activity due to n-1 and n. The activity of n is represented by the subthreshold input from the a population. Activity of n-1 at population r would have subsided by the time the working memory for the a population repeats n and n+1 activities. A buffer for the most recent activity in the r population is needed. It may be possible to use population p as that buffer during encoding. Use buffering in p or add a specific r buffer for encoding. Insure that timing is such that specific x spiking causes y cells with specific subthreshold depolarization to spike so that LTP is established in Wib. Note: A similar set of options exists here as in the encoding of x, since we could use a protocol with suprathreshold connections from the r-buffer to y during encoding. In order for the r population to provide sensible subthreshold inputs, LTP must be established at Wf synapses. During encoding, population a could provide suprathreshold input to population s and subthreshold input to population r. (Subthreshold input at r is not optional, since very specific r population activity may be necessary at that time for subsequent use in the encoding of Wib.) The encoding protocol for s activity may need to be modified if it interferes with requirements for the encoding of Wif. A stepwise approach is likely to most rapidly produce a successful model of the minicolumn, so I will deal with such potential modification needs once the steps above have been made and verified. Also, since this is a reimplementation within a behavioural task, I will construct encoding without heed to effects on retrieval at this time. Since the utilization of a loop via s (with suprathreshold input from a) and r to specific y activities changes the activity of y during Wb encoding from that used above, the proper establishment of LTP in Wb (as shown in Figure 10) must be retested.

The y activity that is needed both for Wb and Wib encoding must now involve specific excitation from r. That loop begins with activity in the s population, since that is where suprathreshold activity is currently assumed. Consequently, I begin by defining a separate Transient-S cell type and set conductances from x (originally G_0=2.6 nS) and r (originally G_0=2.5 nS) populations to G_0=0 nS so that the reimplementation can be done in concrete steps. Since the interneuronal contribution to population s, critical for the many retrieval rules, may also interfere during encoding, its conductance is also set to zero (from G_0=4 nS). Unlike the Transient-RS cells, the a-to-s synapses are now set for rapid suprathreshold activity (parameters: G_0=3 nS, rT=2 ms, fT=4 ms, duration=20 ms). During encoding, activity in all cells of the s population follows activity in the a population (minicolumn-layers-aWM.20030508.ccm).

Encoding Protocol:

1. a_new ---(sup)---> y
2. a_old -----------> x => Wb
   y ----------/
3. x ---(sup)---> s (not gating, not as in retrieval)
4. s ---------------> r (initial Wf = 0.5) => Wf
   y ----------/
   (all, diffuse)
5. r2_old * s => Wif
6. r2_old ---(sup)---> y
7. r -----------> r2_new
8. y * x => Wib (left column)
9. [o_random * s => Wo (self-organization)] or s(state) = o

Having established the encoding protocol, it is clear that specific y is not needed during the initial learning of Wb and Wf. I therefore revert back to the model version minicolumn-layers-aWM.20030507.ccm before continuing with the following steps.

Encoding protocol steps 1 and 2 have already been completed above.

E.P. step 3:

• I created a separate population "Transient-S", so that it can be modified without changing parameters for r.
• I made the x to s connection suprathreshold. The connection strength can be modulated during retrieval if it should then be subthreshold.
• I temporarily deactivated inhibitory input to s from interneuron populations by setting the corresponding synaptic conductance to 0 nS (from the previous value 4 nS). This allows me to concentrate on one construction issue at a time.
• I temporarily deactivate input to s from a and from q by setting the corresponding synaptic conductance to 0 nS (from the previous value 1.5 nS). If these connections are used during retrieval, and if they would create interfering activity during encoding, they can be modulated to near zero conductance during encoding.

E.P. step 4:

- set r parameters to 0,1.5 (from 2.5),3.2 (from 2.5),LTPWb (from none),2.0,
  4.0,20 (from 15)
- set p->r to 0 (from 1.5)
- set int.n input to r to 0 (from 4.0)
- set y->r to 0 (from 2.6) temporarily (testing subthreshold s->r input
= s11==s2-3 (counting backwards due to catacomb oddity) causes subthreshold
  activity in r18==r3-2, s20==s3-4 causes subthreshold activity in r27==r4-3.
  This is the correct encoding according to the Wb connection table, achieving
  part of encoding step 4.
- QUESTION: Does step 4 really need y input? The s->r input is already very
  specific due to the setup of the Wb connection table. Perhaps suprathreshold
  activity from s to r would suffice. Perhaps dependence on y is more useful
  when Wb connections are not as constrained.
- set y->r to 2.5 (originally 2.6)
- set y->r timing to 2,4,20 (from 1,2,10)
= spiking in r is achieved by the combination of s and y, but how can I test
  LTP on Wf as specified in step 4? Is there a part of the sequence where
  y is no longer spiking, but input to r still arrives from s? Probably not.
- QUESTION: How do I test Wf? I could take a part of the experiment and
  stimulate s directly.
- QUESTION: What is r2_old initialized as before the first time an r->r2_new
  assignment is done?
- QUESTION: If r2_old * s is used for Hebbian update of Wif, how does that
  plausibly involve the same synapses as from r to s?
- QUESTION: Update of r2_old from r is done when other update steps using
  r2_old are completed. Those may involve multiple cycles, so what is the
  mechanism to update r2? One possibility is to use a 2-item buffer, as for
  a, and to use transmission modulation or lateral inhibition so that only
  the oldest item in the buffer affects encoding updates where r2_old is
  required.
  NOTE: If a buffer already holds both r2 and r, then a separate r population
  may be unnecessary. This would also solve the issue of using the same
  synapses, as the choice to use r or r2 becomes one of transmission
  modulation intervals, not different synapses.
  The concept of "cycles" or steps seems different here than in Matlab.
- implemented r2 (actually r and r2) as a 2 item buffer
- QUESTION: As in previous models, to avoid flushing the buffer with multiple
  presentations of the same item, it may be necessary to filter input to
  detect when a "new" item appears, yet how would that deal with repetition of
  the same item? Perhaps this is not a problem when items are always sequences
  of STATE-ACTION, since a repetition STATE-STATE or ACTION-ACTION would not
  occur.
- In previous models, the filter was a simple after-hyperpolarization, relying
  on the features of the task, that a virtual rat would not take longer than a
  certain time to traverse a place field and that a place field would not be
  revisited before a certain time had passed. In the current, more flexible
  model, it is preferable that the presence of an item in the 2nd position of
  the buffer (i.e. r) should inhibit afferent input of that same item.
  It may also be possible that an AHP is sufficient, if it is long enough to
  allow the induction of LTP for the items in the buffer. A subsequent
  buffering of two instances of the same item should not lead to difficulties,
  since self-association is not enabled in the network connectivity.
  Advantage of the AHP approach: It is very simple, possibly more plausible
  as a result.
  Advantage of the identity-input-suppressing approach: It is information
  dependent, not time dependent. It will also work for rapidly changing
  input without skipping items, eventhough multiple passes may then be needed
  to achieve strong LTP.
- I'll first try it with the AHP approach. I create a filter that reacts to a
  single input and follows that with an extended AHP (r-buffer-input-filter).
- As far as BUFFER TIMING is concerned, I currently work with the assumption
  that new buffer input should arrive out of phase with buffer retrieval, and
  buffer retrieval should be in phase with the encoding operations taking
  place. If this needs to be changed, modify: (a) r-buffer-input-filter
  synchronization (theta) input delay (currently 0), (b) buffer synchronization
  (theta) timing for membrane potentials (currently 0 delay), and (c) buffer
  transmission modulation delays (currently 112 ms).
- A little timing check on the route from place input to the a-STM-buffer:
  place-input-synchronizer oscillation phase offset 35.9, minicolumn
  oscillatory input phase offset 35.9, FIFO inhibition population
  synchronization input delay 30 ms, a-STM-buffer oscillatory input delay 0,
  a-STM-buffer transmission modulation delays 112 ms, encoding phase a-to-y
  input transmission modulation delay 75 ms (this can always be adjusted to
  place the encoding phase into synchrony with the activation of a by the
  a-STM-buffer).
  In the newtmaze model: theta phase offset 359, place-input filter theta
  delay 115 ms, place-input filter transmission modulation delay 115 ms,
  FIFO inhibition population theta delay 30 ms, STM buffer theta delay 112 ms,
  STM buffer transmission modulation delays 112 ms.
  Clearly, timing in the minicolumn network needs to be fixed!
- Fixing timing:
  - I'm switching to a network-wide oscillator phase of 359 degrees.
  - I'm adding a 115 ms delay to the synchronization input of the
    place-input-synchronization population, as well as a 115 ms delayed
    transmission modulation.
  - I'm adding a 112 ms delay to the oscillatory input to the a-STM-buffer.
  - I'm changing the encoding phase transmission modulation delay to 95 ms, and
    the retrieval phase transmission modulation delay to 33 ms, and the a-to-y
    conductance to 5 nS (from 4.8 nS), so that the desired encoding activity
    is achieved (E.P. step 1).
  = E.P. steps 1, 2, 3 and 4 appear to be in good working order after fixing
    the timing.
- Returning to the BUFFER TIMING issue above, the filter must now be tuned to
  present new items from r at the afferent input phase of the r-STM-buffer,
  while the r-STM-buffer should reactivate items at the minicolumn encoding
  phase, i.e. at the phase at which old and new items are appearing in a, y,
  x, s and r. First, I'll have to make the synaptic response of r input to
  the filter slower, so that a single input can cause it to fire when the
  synchronization input appears. I can then tune the synchronization timing
  as necessary for afferent input to the r-STM-buffer. I then first set up
  the r-STM-buffer so that its retrieval phase coincides with the minicolumn
  encoding phase, before setting the afferent input timing.
- (1) make sure the r-STM-buffer works, i.e. buffers properly. (2) have the
  r-STM-buffer retrieve during the minicolumn encoding phase. (3) provide
  afferent input to the r-STM-buffer at its afferent input phase.
  - (1) is achieved, (2) the buffer retrieves about 10 ms earlier than the
    regular r activity
  - Since the delays on the filter would become 125 ms after adding 10 ms, no
    delays are included there, since the delay matches the oscillation period.
  - (2) and (3) are also achieved.
- An r2 is now available for E.P. step 5, and its separation from r can be
  increased if necessary by increasing the interval produced by recurrent
  inhibition.

E.P. step 5:

• In order not to have interference from r during the encoding with Wif with r2 and s, the second buffered spike (r) in the r-STM-buffer should not be propagated to the synapses to be updated. This is done by adding a filter population after the output from the r-STM-buffer, where the receiving synapses are transmission modulated to enable spiking during encoding only in the first STM item phase.

  This phase dependent transmission modulation approach works as shown in Figure 11, but becomes less reliable if there is more variation in the item spike times. Biological plausibility may also decrease as more specific STM items need to be extracted, since that involves superpositions of tuned transmission modulation functions in a cascade of filters.

  
  Figure 11: The output of the STM buffer of r population activity in (a) is filtered by phase dependent transmission modulation on the synapses of a filter population. Spikes propagated in (b) correspond to the memory r2. Some spurious spikes are propagated where new afferent input arrives in the r STM buffer. (Responses depicted were produced with minicolumn-layers-aWM.20030604b.ccm.)

  As an alternative for the extraction of items in the buffer, a cascade of filters can be used, in which recurrent lateral inhibition suppresses items following the item to be extracted. In order to extract the second item in a second level filter, feed-forward inhibition from the first level filter in the cascade can suppress first item spiking, and so forth through the cascade. It is conceivable that a learning procedure can produce such a cascade from a sufficiently large pool of neurons with random initial connectivity to and from interneurons. Transmission modulation is applied here only as a means to suppress spiking from the r-STM-buffer that is due to the arrival of new afferent input. This approach is implemented in minicolumn-layers-aWM.20030604c.ccm.

  In order to achieve spiking from selective filter output for r during retrieval and for r2 during encoding through the same synapses onto population s, the construct in Figure 12 can be used.

  
  Figure 12: The same synapses with the target population s are used during encoding and retrieval. The buffer sustains both r2 and r. Selective filters extract items r2 and r. Transmission modulation on the synapses from these filters propagate r during retrieval and r2 during encoding. The r' population spikes accordingly, causing the proper update of synapses with population s during encoding and using those same synapses during retrieval.

  (The construct in Figure 12 has not yet been added, as the current focus is on the encoding protocol. Once this construct is needed, the population r' must be added between the filter(s) and population s.)

• The suprathreshold activity from x, established in E.P. step 3 drives s. That activity, together with the presynaptic activity r2 lead to Wif. The initial connection strength from population r (actually r') is therefore set low (0.1 nS). Even during retrieval, activity in s should be gated by x. The maximum connectivity strength is therefore kept at 2.5 nS.
• Timing: In order to bring the timing of r2 activity at the filter output (or population r' in Figure 12) within the necessary interval (20 ms for the LTPWb modification rule used at the r to s synapses) and before spiking activity in population s, it is possible to delay the buffer retrieval phase with a later synchronizing oscillation phase. The r2 activity (and r during retrieval) can then arrive at the desired phase on the next cycle of activity in the minicolumn. Those cycles are maintained by the a-STM-buffer during encoding. Note that this timing issue, the need to present r2 to s, is another good reason for an oscillatory sustained activity design of the minicolumn. This stage of the encoding protocol is implemented in minicolumn-layers-aWM.20030605.ccm.

  Learning does appear to take place on the r to s connection in Figure 13, although a good test of the strength that is established remains to be done in the absence of a working SynapseRecorder object in Catacomb.

  
  Figure 13: Connection strength between r and s grows during encoding. The increasing size of the EPSP caused by subthreshold input from r prior to the spikes elicited by input from x is evidence that LTP is being established. The timing of r2 input and the learning rate of the LTP function during encoding may yet need to be adjusted if the connection strength that is achieved is not enough for spiking gated by x during retrieval.

  In minicolumn-layers-aWM.20030605b.ccm, the output of population s and the filtered output of the r-STM-buffer are combined so that the spike intervals can be measured precisely and the spiking order can be verified (Fig.14). If the order is correct and the intervals are smaller than 20 ms then the level of LTP that is established depends on the number of cycles with the same spike pairs and the chosen learning rate.

  
  Figure 14: Retrieval spike timing in the r-STM-buffer and population s spike timing can elicit the desired LTP. The filtered spikes from the r-STM-buffer (lower offset) arrive before the spikes in population s (upper offset). The spike interval is usually less than 20 ms, although sometimes at about 20 ms.

E.P. steps 6 to 8:

• Instead of the diffuse y activity caused by population a in encoding protocol step 1, we must now provide the specific y activity required in E.P. step 6. A single population y can provide these conflicting patterns of activity only with strong and very precise inhibitory and excitatory inputs that are transmission modulated for specific intervals of the encoding protocol. Additionally, the arrival of such inputs and the use of y output must then be precisely tuned for those specific intervals. A more robust alternative proposal for the encoding design is presented in Figure 15.

  
  Figure 15: Encoding tasks for the establishment of Wb and Wib are delegated to separate populations that act together during retrieval. A population y-diffuse is driven by input from population a during the encoding of Wb. A population y-specific is driven by input from the filtered r2 output of the r-STM-buffer during encoding of Wib. At that time, transmission from y-specific to y-diffuse is suppressed. During retrieval phases, suprathreshold input from y-specific drives y-diffuse, so that they act as one population y for the retrieval of memories encoded on synapses in Wb and Wib.

  This design is implemented in minicolumn-layers-aWM.20030609.ccm. The r2 output properly drives y-specific. And during encoding, the spiking of y-diffuse appears to be properly independent of the spiking in y-specific. Spike timing in population x and y-specific are within the interval needed to elicit LTP. Proper Wib encoding will be verified during retrieval.

• The replacement of r2 with r in E.P. step 7 is already included as part of the r-STM-buffer FIFO queue mechanism.

E.P. step 9: Output and activity

• Some redesign is necessary. While a component is available that can direct the virtual rat in terms of spike commands for up, down, left and right, running on a trajectory does not produce a perception of which direction was taken to reach the next place field. The trajectory is therefore replaced by a sequence of spikes that direct the virtual rat through the environment during exploration (minicolumn-layers-aWM.200306010.ccm).
• In order to be able to control the virtual rat in all 4 directions (currently set with the n_directions=2 parameter setting in the DiscreteStepper component) a new device is created: vrat4dircontroller. This device takes spike event commands with the following meanings:
  

  index	command
  0	stop
  1	go right
  2	go up
  3	go left
  4	go down

  
  When a command is received, the corresponding events for the DiscreteStepper are sustained at a repetition rate of 50 Hz. The device implemented in minicolumn-layers-aWM.20030611b.ccm is usable only with 4 directions, due to limitations in Catacomb. I attempted to create a more flexible device that could expand its number of outputs, by capturing a single vector switch and threshold component row and using a population of such a device. It appears that the vector switch does not know that it can accept only 2 spike indices, and a connection table therefore does not properly route spikes. A very useful addition to Catacomb would be a component that relays spikes, accepts only a specific number of spike indices and produces output with spikes renumbered beginning with index 0.
• The perceptions of position and of motor action must now be combined into one sequence of perceptions for encoding. A way to achieve this in a regular manner is proposed in Figure 16.

  
  Figure 16: (...figure caption pending...) This design is implemented in minicolumn-layers-aWM.20030612.ccm.

  Concern: The data provided by the state-action pairs may be insufficient for successful navigation. For instance, if the state is not detailed enough, a change of direction part-way through a place field during exploration may not be correctly repeated during navigation. The sequence would remember the place, the first direction, then the same place and the second direction. There is then no information about the distance travelled within the place field in either direction. Depending on the hetero-associative connectivity that is available during encoding, successive pairs with the same state may also pose difficulties for path learning.

  
  Figure 17: The Catacomb implementation of the design in Figure 16.

  
  Figure 18: State-action pairs produced by perception during the initial exploration of the T-maze. The first pair is place 1 (index 4) and direction up (index 1). The second pair is place 2 (index 5) and direction up (index 1). The third pair is shown to include place 3 (index 6).

  Note: This is only one of many possible ways to encode the combination of position and motor action. The two types of information could also be joined auto-associatively as one memory pattern for each combination of position and motor action. Or sequences could be created with different amounts of each type of input.

  The additional processing stages in the input circuitry do not appear to cause a significant delay, as the state-action perceptions arrive at the afferent input phase of the a-STM-buffer and are correctly buffered.

• The NxN outputs of the s population need to be transformed into useful output for the direction of motion. The s population output of minicolumns associated with actions can be used to drive motion when spikes are gated by backwards spread through the x population. Successful gating by backwards spready indicates action that leads to a place along a path to the goal. If the s population output of place minicolumns is used instead, then it may be possible to take specific s spikes as specific direction controls. A threshold detection population can connect any successful s spikes at a minicolumn represention action to the corresponding directional control.

  An output population with 4 cells representing the 4 directions of motion is added with connections from the s population so that the s outputs of the 4 minicolumns representing actions cause the output cells to spike. The spike indices from these output cells are increased by one to conform with those expected by the vrat4dircontroller device. Input to the output cell population is transmission modulated. Transmission is suppressed during the exploration part of the experiment so that the virtual rat is driven only by exploratory events. At the onset of the navigation part of the experiment a vector switch switches the transmission modulation to an oscillatory signal that allows only retrieval spikes from the s population in the minicolumns to propagate to the output population.

The "place-input-synchronizer" is desensitized a bit, so that a synchronous spike will not be caused by the first spike from a new place cell at the same time as a synchronous spike due to cumulative spikes from the previous place cell.

E.P. Summary:

The switch to state-action pair input to the minicolumns, different place field sizes and other changes of the data set affect the encoded patterns in Wb, Wf, Wib and Wif. It is also important that none of the encoding protocol breaks while adding the retrieval protocol. For these reasons, a summary of results with the model minicolumn-layers-aWM.20030617.ccm if presented here for the encoding protocol.

Figure 19: Place (state) perception input during exploration. As the virtual rat explores the T-maze it traverses four place fields.

Figure 20: Direction (action) perception input during exploration. For a little over 3 seconds, the virtual rat explores in an upward direction (index 1) through the stem.of the T-maze. It then explores to the left (index 2) through the left arm of the T-maze.

Figure 21: State-action pairs produced as input to the minicolumns during exploration. The change detection network detects changes in the perceived place and direction of motion. Each time when a change occurs, both the new state (place) and action (direction) are transmitted to the minicolumns. The network that produces these state-action pairs also imposes an intervening interval that allows buffering in the minicolumns to establish significant LTP between the items in the sequence. The state-action pairs shown here encode the following: (1) bottom of stem direction up, (2) middle of stem direction up, (3) top of stem direction up, (4) top of stem direction left, (5) left arm direction left.

Figure 22: Activity in the minicolumns a population during exploration. The activity of a population cells that are receptive to action input (lower signal plots) is offset from that of cells receptive to state input (upper signal plots) for clarity. The desired sequence order is achieved with the a-STM-buffer. (1) Red state is followed by green direction. (2) As a new state-action pair arrives, green direction is followed by green state. (3) Then the action of the new state-action pair is buffered, green state is followed by green direction. (4) Green direction is followed by blue state. (5) Blue state is followed by green direction. (6) The virtual rat changes direction within the place field represented by blue state spikes as it enters the left arm of the T-maze, green direction is followed by blue state. (7) The new direction enters the buffer, blue state is followed by blue direction. (8) Blue direction is followed by brown state. (9) Brown state is followed by blue direction.

Figure 23: During encoding the activity in the y population is a diffuse copy of the activity in the a population.

Correction of initial Wb learning: There should be a connection from ``middle of stem'', y[41], to ``up'', x[13]. It looks like the lack of input spikes does not allow sufficient update, a problem that could be removed by strengthening the update, by insuring that fewer gaps occur in the spike trains, or by slowing down the virtual rat's movements. Part of the reason appears to be a problem with the STM buffer. FIFO replacement fails after the first two items and leaves the first in, while retiring the second as the third comes in. A close look reveals that the onset of FIFO inhibition is just too late to catch the first spike by about 1 ms. I therefore shift the FIFO phase 2 ms earlier. Since the same problem may be occurring in the r-STM-buffer, I adjust the FIFO delay there to 38 ms (from 40 ms, with a buffer phase of 122 ms instead of the 112 ms in the a-STM-buffer). These modifications are stored in minicolumn-layers-aWM.20030619.ccm. Now, why does a not always propagate to y? I think I should suppress afferent a-STM-buffer spikes so that they do not appear in a, since that phase of a activity should be reserved for a sustained representation of current location that can be obtained from the a-STM-buffer via a filter such as the one used with the r-STM-buffer. It appears that the transmission modulation stopped propagation from a to y when spikes were slightly earlier. I've modified the conductance from 5.0 nS to 5.2 nS, which is hopefully also correct for input from the r-STM-buffer filter output to y-specific. The issue of large gaps is thereby solved. It seems that the first contributions of a and y at x do not reliably cause spiking. Contributions from a are modified to have a slower rise time (12 ms) so that the peak of the rise coincides with the appearance of y input at the smallest interval. The fall time is set to 20 ms and the conductance is lowered to 1 nS. Any side-effects would show up in the graphs. Spiking in x is now properly elicited. If two spikes are not enough for sufficient update, either the learning rate can be increased or the interval between the state and action spikes can be increased to allow another spike to occur. Plasticity modulation may be necessary to prevent undesired updates during the retrieval phase. The speed of the virtual rat and other experiment timing are set back to the original with a run-time of 5000 ms in minicolumn-layers-aWM.20030619b.ccm.

Figure 24: Combinations of activity in the a population and the y population cause spiking in the x population so that connections in Wb are encoded. As pairs of minicolumn activity propagate through the a-STM-buffer in first-in-first-out (FIFO) manner, new a activity that diffusely activates y together with old a activity that contributes to membrane potentials in x cause spiking in x and corresponding updates of Wb. When new a becomes old a in the buffer, it continues to provide diffuse activation of y. Strengthened connections in Wb can then cause cells in the x population to spike without the contribution from a. (1) After the first 500 ms, new a for the ``up'' direction depolarizes x[8-15] and y[8-15], while old a for the ``bottom of stem'' location depolarizes x[32-39] and y[32-39]. Updates of Wb should strengthen the connection from the ``up'' minicolumn to the ``bottom of stem'' minicolumn, i.e. y[12] to x[33]. As expected, x[33] first spikes due to the combination of input from a and y. Around t=1200 ms input to x[33] from a ceases (see Fig.22), yet spiking of x[33] continues due to input from y (see Fig.23). The connection was correctly strengthened. (2) As ``middle of stem'' data enters the a-STM-buffer the connection to ``up'', y[41] to x[13], is strengthened as x[13] spikes twice. (3) The action ``up'' is then activated again, so that the connection from y[13] to x[41] should be strengthened. Wb is correctly updated, as evidenced by the continued spiking of x[41] after activity in the a population for ``middle of stem'' subsides after t=2500 ms. Activity for the ``up'' direction also causes renewed spiking of x[33] along the first strengthened connection. During this encoding, from about t=1800 ms, buffered old a represents ``middle of stem'' and new a represents ``up''. Thus the activity of y[41] at the old a phase propagates through the connection strengthened previously, causing x[13] to spike. (4) State ``top of stem'' appears, connections to ``up'' (y[49] to x[14]) are strengthened, x[14] spikes. Buffered old a also causes spiking along known paths from ``up'' to ``bottom of stem'' (x[33]) and ``middle of stem'' (x[41]). (5) After t=3000 ms, direction ``up'' completes the new state-action pair, so that x[49] spikes as the connection from y[14] is strengthened. Buffered ``top of stem'' and ``up'' activities also propagate through known paths to x[14], x[33] and x[41]. (6) The ``top of stem'' state appears again with ``up'' in the buffer as old a, causing a repeat of updates and activity as in step (4), as well as spiking in x[49] due to the recently strengthened connection from the minicolumn that represents the ``up'' direction. (7) Action ``left'' appears, so that the connection from y[22] to x[50] is strengthened as x[50] spikes. Buffered ``top of stem'' activity propagates and causes x[14] to spike. (8) The state ``left arm'' appears and the connection to ``left'' direction (y[58] to x[23]) is strengthened as x[23] spikes. Direction ``left'' propagates to ``top of stem'' and causes x[50] to spike. (9) Direction ``left'' completes the state-action pair. The connection to ``left arm'' (y[23] to x[58]) is strengthened as x[58] spikes. Previously strengthened connection propagate activity to ``top of stem'' (x[50]) and ``left'' (x[23]).

Note: The need for multi-cycle intervals between item replacements in the a-STM-buffer and other STM buffers is removed if the buffers can store more than 2 items.

Note: If propagation due to diffuse y activity caused by buffered old a as in step (3) of Fig.24 is undesired it can be suppressed, since it is not needed for encoding. This can be done by using phase-filtered y output implemented in the same manner as the phase-filtered output of the r-STM-buffer.

Figure 25: The synaptic connections between minicolumns representing places and directions in the T-maze, in the order in which they are updated in Wb through the steps in Fig.24 (red arrows and numbers). The route of associative spread from a goal in the left arm of the T-maze is shown in accordance with strong synaptic connections established in Wb (blue dashed arrows).

Figure 26: During encoding, suprathreshold connections from the x population drive spiking in the s population.

Figure 27: Synaptic connectivity in Wf is established by causing spiking in the r population through a combination of inputs from y and s populations. (1) Activity at ``bottom of stem'' cell s[33] and in the y population of the ``up'' minicolumn elicit spiking in r[12]. A connection from ``bottom of stem'' to ``up'' is established in Wf. (2) Activity at ``up'' cell s[13] and in the y population of the ``middle of stem'' minicolumn elicit spiking in r[41]. A connection from ``up'' to ``middle of stem'' is established in Wf. (3) Similarly, activation of r[13] established a connection from ``middle of stem'' to ``up'', activation of r[49] establishes a connection from ``up'' to ``top of stem'', and activation of r[14] establishes a connection from ``top of stem'' to ``up''. This process continues until encoding in Wf is complete.

Diffuse y activity caused by the a population propagates via Wb to the x and subsequently the s population and the combination of s and y activity then establishes Wf connection strengths with the r population. Similarly, diffuse y activity caused by the buffered old a population activity leads to updates in Wf. These are identical to the updates elicited when old a was new in the a-STM-buffer. The effect is therefore a prolongation of the learning period for associations encoded in Wf.

Figure 28: The input filter to the r-STM-buffer is phase-locked and uses a large AHP. Together this means that only new r can cause spiking in the filter that is propagated as afferent input to the r-STM-buffer, not old r spikes. It also means that a specific itme can only spike again after a long duration, which is a simple (and not very flexible) way to counter interference problems during learning.

Figure 29: The r-STM-buffer maintains and ordered queue of the new r activities.

Figure 30: The previous item activity r2, buffered in the r-STM-buffer, is obtained through a phase-locked filter with lateral inhibition. At the beginning of exploration the r2 output is not void but rather presents r, since only a single item is in the r-STM-buffer. If this needs to be avoided for proper encoding, an initial r2 item may be inserted into the buffer.

Figure 31: The connectivity of Wif involves dense local connectivity.

Comparing activities in Figures 26 and 30 and taking the connectivity in Figure 31 into account, the connections that are updated can be listed in the following table.

interval containing updates	minicolumn	r to	s	updates
t=[1000,1900] ms	up	r[12] from bottom to up	s[13] from up to middle	2 intervals less than 20 ms
t=[1900,2750] ms	middle	r[41] from up to middle	s[41] from middle to up	3+ intervals less than 15 ms
t=[2750,3100] ms	up	r[13] from middle to up	s[14] from up to top	2 intervals less than 20 ms
t=[3100,4000] ms	top	r[49] from up to top	s[49] from top to up	3+ intervals less than 20 ms
Etc. with left and arm.

This appears to be correct and the updates should suffice if two or more Hebbian occurrances elicit enough LTP.

Note: It would be more direct and secure to inspect the connection strengths directly, once Catacomb makes this possible.

Figure 32: During encoding, the separated population y-specific is driven by r2 as propagated from the r2 output filter of the r-STM-buffer (see Fig.30).

Local connectivity in Wib is the same as shown in Figure 31. Comparing x spiking in Figure 24 with y-specific spiking in Figure 32 and taking the Wib connectivity into account, the connections that are updated can be listed in the following table.

interval containing updates	minicolumn	x to	y	updates
t=[1000,1700] ms	up	x[13] from middle to up	y[12] from up to bottom	1 interval less than 20 ms
t=[1700,2800] ms	middle	x[41] from up to middle	y[41] from middle to up	3+ intervals less than 20 ms
t=[2800,3100] ms	up	x[14] from top to up	y[13] from up to middle	2 intervals less than 5 ms
t=[3100,4000] ms	top	x[49] from up to top	y[49] from top to up	3+ intervals less than 20 ms
Etc. with left and arm.

The encoding appears to be correct, although the number of updates available may be too few to elicit strong LTP with the current learning rate.

R.P. step 1:

Before proceeding through these retrieval protocol steps, I will list the retrieval rules that will be implemented at this time. We are not yet taking into account populations p and q. Transmission to and from those populations is temporarily set to zero conductance.

a status	retrieval operations
a active	end-stopping in y no spikes in r s = competition between cells with contributions from x
a inactive	r = competition between cells with contributions from nWf and y s = competition between cells with contributions from rWif and x

For the above rules, we want the activity of the a population during retrieval to reflect the virtual rat's current location. This means that we do not want any confusion with old a activity, and the current location activity should appear on every cycle. Note that the a-STM-buffer shifts the a activity representing location (state) in the a-STM-buffer in order to process a direction (action). The most straightforward way to achieve these requirements is to obtain the state signal once it has been synchronized, but before it is interleaved with the action signal (see Fig.16). This synchronized state signal can be delayed as necessary so that it can be presented when appropriate for the retrieval phase. This is implemented in minicolumn-layers-aWM.20030625.ccm.

At this point, the retrieval state activity "a" can cause activation of the same population a that is used during encoding, or it can be sent to some other populations directly. The direct approach may avoid some conflicts in which transmission modulation has to be applied and complicates the circuitry. Nevertheless, I begin by routing through the a population, simply because that was the initial design strategy. This may be revised shortly (see R.P.Summary).

End-stopping in y is implemented in minicolumn-layers-aWM.20030625b.ccm, although it can only be tested once backward retrieval is completely incorporated.

R.P. step 2:

The dependence at r on the absence of activity in a can be implemented either through active inhibition via the interneuron population that achieves end-stopping in y. Or the reliance on gating by y can be used without additional direct inhibition at r. If spiking in r during retrieval depends on a combination of nWf and y, then the absence of spikes in y due to end-stopping also implies that there will be no spiking in r. During encoding, untrained nWf connections could cause spiking in the presence of input from y. Trained nWf connections became even stronger. In order to assure that y fulfills a gating function during retrieval, the strength of nWf connections must be decreased by transmission modulation.

This is done in minicolumn-layers-aWM.20030625b.ccm without any direct inhibition of r. Transmission modulation is constrained so that maximal transmission amplitudes are achieved during encoding and about half of that amplitude is achieved during retrieval. This can only be tested when the virtual rat is placed back at the beginning of a learned path. Competition must also be implemented and be active only during retrieval - although perhaps it may also be active during encoding. Plasticity modulation may also be necessary.

R.P. step 3:

In order that x input to s can take on a gating role during retrieval, transmission modulation is applied. The transmission modulation enables strong conductance during encoding and conductance up to half of the maximum conductance amplitude during retrieval.

The maximum conductance that is achieved through learning in Wif is set to be subthreshold so that x can fulfill its gating function during retrieval when a minicolumn does not receive ``current location'' state activity.

The subthreshold input that is needed if ``current location'' state activity is available at a minicolumn cen be provided direction from place state input or via transmission modulated retrieval activity in the a population. While constructing this basic retrieval architecture, the input from q to s has been (temporarily) removed. This was a 1-1 network link to the same synapses on s that receive the a/state signal.

I am currently opting for the direct state signal to s connection, since this guarantees there is no input during encoding that could offset the probability that a cell in s will spike when x or rWif inputs arrive. For this reason, I also remove the network link that was previously in place between a and s. That input had zero conductance so that it did not interfere with encoding and used the UseWa connection table. Spike timing or synaptic response durations may need some adjustment during testing. Additionally, competition must be implemented as in R.P. step 2. The retrieval architecture in R.P. step 3 is implemented in minicolumn-layers-aWM.20030626.ccm.

Having implemented basic components of the tabulated retrieval rules, it is clear that there is no functional reason why state signal retrieval contributions would need to be run through the a population. That signal affects activity in y and in s. Combining this signal with encoding activity in the a population raises other questions, such as the possible effect during retrieval on spiking in x if subthreshold yWb input is encountered. So why is encoding activity in the a-STM-buffer routed through the a population instead of using direct trsnsmission modulated inputs to target populations y and x? The a population acts as a filter that improves the reliability of operations by assuring that low transmission modulation levels on its inputs (e.g. suppression of the transmission of a-STM-buffer afferent input spikes) result in zero contribution.

R.P. step 4:

We must now complete the circuitry between y-specific and y-diffuse in Fig.15, as well as the circuitry that insures that r uses the same synapses as r2 during retrieval (Fig.12). We can then provide goal input at x and perform tests to assure that retrieval is achieved without disturbances during encoding.

The first circuitry is completed by setting the transmission modulated conductance from y-specific to y-diffuse to 5.2 nS, the same as the transmission modulated suprathreshold input conductance from the a population to y-diffuse (minicolumn-layers-aWM.20030626b.ccm). During retrieval, propagation from y-specific to y-diffuse causes the to populations to act as one y population that interacts with the learned synaptic strengths in both Wb and Wib.

Input from r to s can be obtained during retrieval either through a connection from the r population to the r' population in Fig.12 or through a connection from an r output filter from the r-STM-buffer to the r' population in Fig.12. The functional difference is that r output from the r-STM-buffer will be maintained for a while, even without r population activity elicited by inputs. That would circumvent the rule imposed by end-stopping in y, that r should be inactive for the ``current location'' minicolumn and therefore should not contribute rWif input to s that could interfere with the competition between cells in s receiving state input and input from the x population. The circuitry in Figure 12 is therefore modified as shown in Figure 33.

Figure 33: Signals from r and r2, active during retrieval and encoding phases respectively, are combined at r' so that they use the same plastic synaptic connections to the s population. This is a modified version of Figure 12, since activity in r is propagated if available, instead of using r activity that is sustained in the r-STM-buffer.

The first spike from r2 is not properly propagated. After temporarily removing transmission modulation it is clear that this is caused by the transmission modulation, possibly due to its phase. The first r2 spike does indeed fall right into the trough of the transmission modulation. Whether this needs to be aleviated, for instance by shifting the transmission modulation slightly, remains to be determined.

R.P. step 5:

Here we perform tests of retrieval. Necessary modifications will ensure that retrieval can function without disturbing encoding. In order to do the tests, the experimental task is now extended so that the virtual rat receives goal activity and corresponding retrieval activity is elicited.

Note: As goal location should be discovered in a realistic fashion, I temporarily move the goal into the range of exploration in minicolumn-layers-aWM.20030627.ccm. Population sizes can therefore remain at their current size during the design and construction of the model.

This version of the model also switches to the diverse populations option, so that the population a that filters a-STM-buffer output no longer receives state input during retrieval. The state input is delivered directly to the interneuron population that achieves end-stopping in y, just as it was already delivered directly to the s population.

The settings of the goal location and the range at which it is detected cause a few changes in the place input. The total number of places visited during the exploration at this stage is now 5. I wish to constrain the number of minicolumns to 8 during the design and construction phase, and to minimize the change in the input signals so that maintenance of proper encoding can be confirmed by comparison with the E.P. Summary. For this reason, I carefully adjust the detection range and location of the goal and restrict the "nout" parameter of the connection router on the input pathway to 8. The goal-state range can be decreased later for greater precision.

It is not necessary in prefrontal cortex that the goal-state is obtained from place cell information. A separate perception that a goal is present, that a reward is obtained can produce state input. This is a more direct approach to the incorporation of various forms of information, eventhough different place fields are probably also formed near a goal.

Figure 34: Circuitry that provides goal-state input from an arbitrary perception source. Instead of using the vector produced by proximity to the goal to create a different place field, spikes caused by proximity to the goal are joined directly with place cell spikes to form state information. A connection table in a router that is appended there can be used to select sources of input that the model can deal with. (Due to problems with the router implementation in Catacomb 2.104 a connection table is used in the SpikeProjection from the joining relay instead.)

The circuitry in Figure 34 (implemented in minicolumn-layers-aWM.20030627b.ccm) can produce multiple simultaneous state spikes, e.g. goal spikes + environment place cell spikes, or goal spikes + place cell spikes for place field in vicinity of goal (this occurs when the vector switch allows goal vector input to generate a new place field in the vicinity of the goal). If the selection router is not used to avoid multiple concurrent state spikes then multiple paths are generated during encoding in the minicolumns. E.g. an action minicolumn will be bound through Wb, Wf, Wif and Wib with goal state as well as with a place state. During retrieval, activation of either the goal y population or the corresponding place y population can then recall the same path. Note: It is possible that this does not occur if STM in the a-STM-buffer is preceded by associative binding, e.g. through high-level encoding such as that which is hypothesized to take place in dentate gyrus.

Diffuse activation of the goal y population can initiate associative spread during retrieval. Since end-stopping and activity in the a population have been uncoupled, the obvious way to achieve this is through spiking in the goal a population. Propagation of afferent spikes from the a-STM-buffer to the a population is already supressed and encoding spikes from the a-STM-buffer can arrive at the a population only during the encoding phase, while no retrieval spikes yet target the a population. It is therefore safe to remove the transmission modulation from the a population to the y-diffuse population synapses. This is done in minicolumn-layers-aWM.20030630.ccm. Removal of this transmission modulation does add y population spikes, since transmission modulation previously suppressed the propagation of spikes that recently entered the a-STM-buffer.

Note: If this causes any problems, such as learning of the reverse direction in Wb, then the transmission modulation should be returned and goal activity sent directly to the y population.

In further development of this model, it is possible to create a representation in the minicolumns for "goals found" or "goals reached". Those could have associations with all goals in an environment, so that retrieval leads the virtual rat to seek out the nearest goal. For now, the discovery of a goal will lead to maintenance of that discovery in a STM buffer. The STM buffer or other recall mechanism will be added once the proper operation of the minicolumns is ensured. As a temporary solution, goal spikes are hardwired to appear once exploration is complete.

In this version, I also corrected an omitted routing table in the spike projection from state input to the s population. That connection now uses the UseWa table, so that the 8 inputs are sent as diffuse input to the 64 cells in the s population.

The virtual rat is first guided against the wall, due to spiking in s[21] ("from left to middle") at about t=5342 ms. Why does that cell spike? There is a corresponding spike in x[21] at about t=5339 ms. There is spiking via Wib at about t=5330 ms in cell r'[23] ("arm to left"). And there is a spike at about t=5330 ms in a[5] ("middle"). For some as yet incomprehensible reason, the feature discretizer now decides to begin its discretization with spikes with index 1 instead of 0, so I'm adjusting the stateinputselection router accordingly. After this awkward adjustment, the state spikes in the a population are once again correct, one for the goal location during retrieval and one for the current location, "bottom of stem", during encoding. The spikes that cause a left motion there through s[20] ("from left to bottom"0 are now x[20] and a[4], while r'[23] still spikes.

It is possible that moving the virtual rat from the goal back into the bottom of the stem creates an association from goal to bottom of stem. But that should not influence retrieval at the bottom of the stem, since the route was from goal to bottom, not bottom to goal. Current location "bottom" should generate activity that is end-stopping in y[32-39], causes no retrieval spiking in r[32-39] and supports spiking in s[32-39] diffusely. The actual cell in s[32-39] that spikes depends on x, therefore on Wb. The only strengthened synapse that is applicable should be the one to x[33], so that s[33] spikes and leads to motion in the "up" direction.

Goal activity does cause the goal y population to spike diffusely, but that should not propagate to the "bottom" minicolumn, since there should be no corresponding route through Wb.

After some more corrections due to a broken binary signal generator, the cell that spikes in population s during the retrieval phase is now s[39] ("from bottom to arm") at t=5260, 5385, ... ms. There should be no significant connection through s[39]! The virtual rat moves only left from the bottom location after a single direction signal at about t=5350 ms. Note that this direction signal is received before the retrieval activity in the s population! Just after t=5340 ms, there was a spike in s[20] ("from left to bottom"), i.e. apparently due to the state spike ("bottom of stem") in the encoding phase. There should be no significant connection through s[20]! The undesired connection is established due to the repetition of the action-state pair "left" and "bottom" in the a-STM-buffer. That can be avoided if the a-STM-buffer is flushed when the rat is picked up and placed back at the starting location.

• Navigation cannot be allowed to be driven by encoding spikes. This is accomplished in minicolumn-layers-aWM.20030701.ccm.
• The virtual rat cannot be given the impression that going left from the goal arm leads to the bottom of the stem. There must either be a minicolumn representation for being "picked up", or being picked up must cause a different state of mind in the virtual rat, so that oscillations sustaining STM buffers in the minicolumns subside and the buffers are flushed by hyperpolarization. I will here assume the latter. Although a more detailed implementation may be along the lines of changes in the oscillations and a hyperpolarization, a simple way to achieve the same result here is through a reset spike relay that sends a high frequency spike train to the FIFO inhibitory inputs of STM buffers. That is already implemented for the p and q populations in the minicolumns.

Note: The transmission modulation on the output filter can easily be replaced by an oscillatory timing dependence, the need for direction input to the filter to be concurrent with oscillatory input.

Since the driving circuitry requires an explicit "stop" command so that the virtual rat will not continue moving left, I also connect the spike train that clears the STM buffers (with spike index 0) to the input of the driving circuitry. This may need to be changed if we need a distinction between a change in the state of mind and a momentary pause. If the signals are provided separately, a momentary pause does not lead to drastic changes in the minicolumn activity. After the pause, a new state-action pair would be sent to the minicolumns, since a change was detected, and the state and action would shift into the FIFO STM buffers so that they can form relationships with the activities that were maintained in those buffers over the duration of the pause.

The only actvity in the s population after t=5000 ms is now spiking in s[39], starting at t=5260 ms. Just before that, there is activity in a["goal"] as well as x[39]. There is no activity through Wif until after t=5270 ms, at r'Wif[60] ("bottom to goal"). That activity should only be caused by contributions from Wf and y, but the corresponding Wf synapse should not have been updated during exploration! This may be because current location and goal location spikes occur within a short interval once as current location re-enters the cleared a-STM-buffer during the encoding phase. It may be avoidable by not propagating goal activity through the a population, but instead chosing the alternative routing directly to the y population.

In minicolumn-layers-aWM.20030728.ccm the problem is investigated using the new ObservationRecorder and ObservationRelay objects in Catacomb 2.106. The Wb weights show up as a single vector of 64 weights, which makes sense as the Wb connection table creates only single connections from one source to one destination. Eacb synapse in Wb that is strengthened implies a connection from the source that was determined in the connection matrix to the destination indicated by the index of the synapse. For example, a strengthened synapse Wb[13] means that the single connection from y[41] to x[13] in Wb is potentiated.

Wb synapse strengthened	Wb connection strengthened	Wb y and x cell index meanings	Wb connection meaning
13	y[41] to x[13]	middle[up] to up[middle]	y[middle] to x[up]
14	y[49] to x[14]	top[up] to up[top]	y[top] to x[up]
23	y[58] to x[23]	arm[left] to left[arm]	y[arm] to x[left]
33	y[12] to x[33]	up[bottom] to bottom[up]	y[up] to x[bottom]
41	y[13] to x[41]	up[middle] to middle[up]	y[up] to x[middle]
49	y[14] to x[49]	up[top] to top[up]	y[up] to x[top]
50	y[22] to x[50]	left[top] to top[left]	y[left] to x[top]
58	y[23] to x[58]	left[arm] to arm[left]	y[left] to x[arm]

These are the same 8 synaptic connections as those in Figure 25. Weights in Wb were correctly learned during encoding. But, during retrieval another connection is strengthened, namely Wb[39], that is the connection y[arm] to x[bottom]. This means that there is activity during retrieval that maintains those two positions in a buffer long enough to establish significant LTP. This begins around t=5260 ms and continues at least until t=5670 ms.

Wf synapse strengthened	Wf connection strengthened	Wf s and r cell index meanings	Wf connection meaning
12	s[33] to r[12]	bottom[up] to up[bottom]	s[bottom] to r[up]
13	s[41] to r[13]	middle[up] to up[middle]	s[middle] to r[up]
14	s[49] to r[14]	top[up] to up[top]	s[top] to r[up]
22	s[50] to r[22]	top[left] to left[top]	s[top] to r[left]
23	s[58] to r[23]	arm[left] to left[arm]	s[arm] to r[left]
41	s[13] to r[41]	up[middle] to middle[up]	s[up] to r[middle]
49	s[14] to r[49]	up[top] to top[up]	s[up] to r[top]
58	s[23] to r[58]	left[arm] to arm[left]	s[left] to r[arm]

The 8 connections that are strengthened provide the correct Wf encoding. Note that synapse Wf[23] is a little weaker than the others. During retrieval, after t=5260 ms, the connection Wb[60] is also strengthened. That is a connection s[bottom] to r[arm]. These updates continue at least until t=5890 ms.

The connection table for Wib enables 8 blocks of 8 by 8 connections internal to each minicolumn. That is a total of 512 synapses. The ObservationRecorder displays only 64 by 2 synapses, a total of 128 synapses. This bug in the current version of Catacomb interferes with a detailed analysis.

Figure 35: Activity of x[39] (x[bottom,arm]) that causes the undesired encoding in synapse Wb[39] during the retrieval task.

Figure 36: Activity of y[60] (y[arm,bottom]) that causes the undesired encoding in synapse Wb[39] during the retrieval task.

A possible explanation is that the retrieved goal location and current location lead to encoding. Once that is encoded, the bottom location can retrieve the arm location, which in turn retrieves the left direction. That may cause the virtual rat to move left into the wall of the stem of the T-maze.

Figure 37: Activity of a during the retrieval task.

Figure 38: Phases of activity during the retrieval task.

In Fig.37 it appears there is only one time when the appearance of the goal activity is close enough to the reactivation of location actvity propagated from the STM buffer during encoding to elicit LTP. On subsequent cycles, eventhough the location activity is shifted into its proper encoding phase, goal activity retrieves the location (y[arm] to x[bottom]). Tests in which the strength of the contribution from a to x was varied determined that this is largely caused by the diffuse a to x contribution. If the duration of the a to x response is reduced from 80 ms to 60 ms then all but the first undesired update of Wb[39] are removed.

A reliable way to avoid undesired updates is the use transmission and plasticity modulation on the synapse populations that experience LTP. For now, it is preferable that we avoid such modulation. The chosen solution is therefore a combination of three modifications: (1) The retrieval phase is shifted to occur 9 ms later, thereby increasing the separation between new buffered encoding spikes and goal retrieval spikes. (2) The duration of the synaptic response to diffuse contributions from population a to population x is decreased from 80 ms to 60 ms. (3) The first undesired update is removed by suppressing the goal retrieval spike when a new buffered encoding spike is recalled. The single update would not be problematic in itself, since the resulting LTP is insufficient to propagate spikes from y to x, but the undesired updates could accumulate if the virtual rat navigates the T-maze several times. These modifications are implemented in minicolumn-layers-aWM.20030806.ccm.

All updates of Wb[39] were successfully removed. Now a new undesired set of updates at Wb[20] commences at t=5390 ms. This is a connection to the x population of the "left" minicolumn from the y population of the "bottom" minicolumn (y[34] to x[20]). Functionally, this makes sense, since left and bottom are both in the encoding buffer during that time. The question is, why is "left" motion active in the buffer?

Figure 39: While "stop" commands insure that no direction spike is supplied on the inputs to the minicolumns (only the first half of the state-action pair is delivered, "bottom" location), this graph shows that the STM buffer of the a population is not flushed properly.

The synapse population in the STM buffer that receives the buffer clearing signal is the same one as that used to provide inhibition for the FIFO queing operation. Its inhibition strength and duration was specified to achieve suppression of a first item until a second buffered item spikes. The inhibition requirements for a complete buffer flush are greater.

There are three ways to implement the buffer clear:

1. Picking up the virtual rat causes a specific GABAergic response in the buffers.
2. Picking up the virtual rat causes a theta reset in the buffers.
3. Picking up the virtual rat causes a theta reset in the whole system.

There doesn't appear to be any propagation through Wib. Despite this, there appeared to be updates of the connection from x[23] to y[22] on at least two occasions. The second occasion may even have been the result of some propagation during training. A working ObservationRecorder or a means of specifically testing that connection would be helpful.

Further testing reveals Catacomb bugs with regards to SpikeProjection objects with connection tables and delays used in SpikeProjection objects. The first task undertaken to avoid these problems prior to a fix in Catacomb is to replace all SpikeProjection objects using delays with SpikeProjection objects that do not use delays or with a construct of JoinCloneRelay-SpikeDelay-JoinDivideRelay objects. While doing so, the delay between the x and y-specific populations is removed and a delay of 1 ms added to the connection from the r-STM-buffer to the y-specific population. This is done to test if better encoding can be achieved by assuring that more Hebbian pre- and post-synaptic spikes appear at Wib synapses. This is done in minicolumn-layers-aWM.20030807b.ccm.

Initial results are promising. Encoding of Wb and Wf appears to remain correct after these small timing modifications and some spike propagation is now taking place through Wib connections. The Wib connection from x_middle[from_up] to y_middle[to_up] appears to be correctly strengthened. (x[41] to y[41] connection in the Wib encoding table). The desired Wib connection from x_up[from_middle] to y_up[to_bottom] (x[13] to y[12] in the Wib encoding table) is not updated. This is probably due to the relative lack of Hebbian spiking, as indicated in the Wib encoding table. The absence of updates on this connection contributes to the virtual rat's inability to move up along the path through the T-maze.

The full list of connections in Wib that are strengthened is: weak ym(u) from xm(u), strong yu(m) from xu(t), strong yt(u) from xt(u), weak yu(t) from xu(t), strong yl(t) from xl(a), weak ya(l) from xa(l). Connections such as ya(l) from xa(l) are correctly strengthened in Wib if Wb contains strengthened connections in both directions between minicolumns representing arm and left. Weak connections and no strengthening on the connection between bottom and up make it impossible for retrieval from goal input to succeed in moving the virtual rat up the stem of the T-maze.

During retrieval, the following spikes are recorded in a Wb-Wib path retrieval sequence: x[23], y[22], x[50], y[49], x[14], y[13]. This is translated as: x_left[from_arm], y_left[to_top], x_top[from_left], y_top[to_up], x_up[from_top], y_up[to_middle]. A connection to the bottom location is missing.

We desire update of the Wib synapse from x[13] to y-specific[12], so that x[13] must spike less than 20 ms before y-specific[12]. During encoding, y-specific is driven by the filtered r2 output of the r STM buffer. Depending on the learning rate, multiple instances of such Hebbian spiking are necessary.

A close examination of the candidate spikes between t=1400 ms and t=1900 ms leads to the following questions:

1. Why does x[13] spike twice, then experience an interval of inactivity, followed by more spikes? The "middle" state is first a new a-STM item, then shifts to the old a-STM phase (i.e. earlier). That is the cause of late x spiking, followed by a gap and then earlier x spiking. Solution: Spiking in x can precede spiking in y-specific if the driving spikes from the r2 filtered r-STM-buffer output are delayed by at least 11 ms. This delay can be implemented as a transmission delay or incorporated in the phase of oscillation of the r-STM-buffer.
2. Why does y[12] stop spiking after the third instance of a spike at x[13]? Activity shifts out of r2 as r_up[from_bottom] is replaced with r_middle[from_up]. The r-STM-buffer is shifting state down as an action arrives. Solution: An increase in the number of cycles between state input and action input of a new state-action pair would delay the shift, so that more Hebbian updates are possible for the connection from x[13] to y-specific[12].

The virtual rat does not move up along the retrieved path, since retrieval is unable to complete within a retrieval phase. The interval between retrieval of one x population item to the next is 15 ms. One approach is to increase the rapidity of individual synaptic and membrane responses. Another is to increase the duration of a retrieval phase by switching to a lower oscillation frequency. The delta frequency range between 0.5 Hz and 4 Hz is a candidate, as it has been associated with sensory detection and decision making (Basar-Eroglu et al., Int J Psychophysiol 1992; Fellous and Sejnowski, Hippocampus 2000).

An alternative solution is to switch to a framework with periods of encoding and periods of retrieval that are not locked into specific phases of a fixed brain rhythm. During encoding, input can drive the minicolumns with bursts. During retrieval, cycles of retrieval are restarted whenever a retrieval is completed and a decision is made. Switching to encoding can occur whenever new input awaits. Switching to retrieval can take place whenever encoding is completed.

Wif updates found:

• Wif[13]: s[13] = s_up[to_middle], r_up[from_bottom] = r[12]
• Wif[14]: s[14] = s_up[to_top], r_up[from_middle] = r[13]
• Wif[23]: s[23] = s_left[to_arm], r_left[from_top] = r[22]
• Wif[41]: s[41] = s_middle[to_up], r_middle[from_up] = r[41]
• Wif[49]: s[49] = s_top[to_uo], r_top[from_up] = r[49]

At the onset of navigation, the retrieval input for "bottom" state is expected to depolarize cells s[33-39]. Gating by the x population should allow s[33] to spike, which should propagate to r[12]. Instead, only activity at s[14] is observed. As it turns out, retrieval input was not propagated correctly due to a delay parameter in the incomplete SpikeProjection object. The SpikeProjection is now preceded by a construct that implements the delay. Consequently, s[33] receives the anticipated depolarization. Since path retrieval from the goal does is not yet able to complete during a retrieval phase, the gating spike from the x population does not arrive and the virtual rat does not move.

The undesired spike in s[14] is caused by the retrieval of x[14]. This should not happen, since transmission modulation should make the input from the x population subthreshold during retrieval. Currently, this is solved by shifting the transmission modulation 40 ms later and increasing the strength of the connection from the x population to the s population from 4.8 nS to 6.0 nS so that the x population still properly drives the s population during encoding. These modifcations are implemented in minicolumn-layers-aWM.20030815.ccm.

An alternative implementation can avoids the use of transmission modulation and insure that the same level of subthreshold depolarization is used throughout the retrieval phase, while the same level of suprathreshold depolarization is applied throughout the encoding phase. In this implementation the connection from the x population to the s population is split into two pathways. One pathway, the retrieval pathway, goes through a gating population that receives subthreshold depolarization from a rhythmic oscillation during the retrieval phase. The other pathway, the encoding pathway, goes through a gating population that receives subthreshold depolarization from a rhythmic oscillation during the encoding phase. Connections from the x population to the gates is subthreshold on both pathways. The connection from the gate on the retrieval pathway to the s population is subthreshold, while the connection from the gate on the encoding pathway to the s population is suprathreshold. This solution has the added benefit that depolarization that controls the gates can be provided by sources other than a rhythmic oscillation if a different approach to switches between encoding and retrieval is chosen in accordance with the possibility mentioned above.

The same test should be done for gated spiking in the r population. And placing the virtual rat in the "middle" place field should be tested.

There are two promising long-term approaches to dealing with the restricted retrieval length:

1. Associations between items in a sequence can span further than the nearest neighbour in that sequence. Such associations in the Wb-Wib paths provide short-cuts for retrieval.
2. Switching between encoding and retrieval modes can be based on control mechanisms other than switching that is phase-locked with an oscillation. This possibility is presented above.

A quick solution that can be applied with minimal modifications is a buffering mechanisms for the activity in the Wb-Wib path. The last activity can be used as a cue in the same manner as goal input. The result is a continuation of retrieval in the next retrieval phase. Where retrieval elicited by goal input is independent of retrieval elicited by the proposed retrieval buffer, multiple retrieval sequences can occur simultaneously. The ability to sustain multiple concurrent retrieval waves means that a long retrieval path can be completed while the goal remains the same and retrieval can focus on a new goal immediately. Means of implementation are:

1. A one-item buffer that replaces previous items immediately and reactivates when triggered by a (rhythmic) signal.
   This is a bit complex for this limited task, since items are not supplied to the buffer at a single phase in an oscillatory cycle.
2. A buffer population with slow fall-time subthreshold input responses. The most recent input is strongest and reactivates first when a (rhythmic) signal appears.
   This has the advantages that (1) the number of items retrieved is not problematic, since recency gives priority, and (2) modifications of the timing are not necessarily problematic. It can also work in a framework that does not switch between encoding and retrieval rhythmically, but that feature is less importance, since such a framework may have no need for a buffer that spans retrieval cycles.
3. Assume that the last item of an incomplete retrieval appears at a known phase. Filter activity to detect that item. Transfer it with the necessary delay.
   The specification of the last item is reliable and does not depend on the separation between multiple non-linear decay functions. This implementation is well-geared toward a rigidly oscillatory framework.

From the analysis, I conclude that the third x activation during retrieval is indicative of an incomplete path retrieval. The y_diff population connected to that x via Wib cannot be activated specifically during the remainder of the retrieval phase, while the third x activation would not have occurred if path retrieval was completed. This is true if end-stopping was correctly implemented at the y_spec population of the minicolumn that represents current location.

The activity of the x population during retrieval can be transferred to a buffer population with a subthreshold response with slow decay. The third x activity will be the strongest residual response when a trigger input arrives to reactivate the x activity at the onset of the retrieval phase in the next rhythmic cycle. Lateral inhibition can prevent other cells in the buffer from spiking. In order to take only x activity during retrieval, a filter population precedes the buffer, so that spiking in accordance with input from the x population is conditional upon rhythmic membrane depolarization.

Figure 40: 1. add buffer pop with trans mod, test spiking only for retrieval x 2. reduce transmission to subthreshold level, check that order is apparent in amplitude 3. add synapse for trigger, with rapid timing, add rhythmic input with delay 4. set delay for spiking on onset of next retrieval phase 5. add lateral inhibition, test that it all works 6. add connection back to the x population, test that retrieval continues 7. perhaps move the buffer into the minicolumn object

The buffer output could be connected to the same synapses that are trained to connect the y population to the x population. That is not as easily accomplished in accordance with neurophysiology, so a separate set of synapses receiving the 1-1 connections is added to the x population. If necessary, the retrieval buffer output can inhibit goal input. Without such inhibition, the new retrieval in the x population toggles between the sequence 13, 23, 33&41, 50, 14 and the sequence 14, 23, 33&41, 50, 13. The x[23] and x[50] retrievals are caused by the goal input. Retrieval elicited by the retrieval buffer toggles, since the last reactivation becomes the first so that another cell reactivates last in the subsequent sequence. The new retrieval in the x population, beginning with x[14] is translated as follows: x_up[from_top], x_bottom[from_up] & x_middle[from_up], x_up[from_middle]. Mediated by activity in the y population, these retrievals correspond well with the possible paths of retrieval through Wb, as shown in Fig.25. This is achieved with minicolumn-layers-aWM.20030827.ccm.

The output of the s population that is used to drive the virtual rat during the navigation part of the task now toggles between s[14] and s[13] (s_up[to_top] and s_up[to_middle]) between t=5000 ms and t=6000 ms. From comparisons of the spike timing in the s population with that in the x population, it appears that spiking in the s population corresponds to the first spike in a retrieval sequence in the x population. There are several questions:

1. Should s population activity at the "up" minicolumn be used during navigation, or should s population activity with the index "to_up" be used? The minicolumn representing the "up" direction is an associative neighbour of the minicolumn representing the "bottom" location, but is there anything limiting the retrieval through Wif and Wf to immediate neighbours of the current location?
2. Is the activity in the s population based on current location activity that is gated by the x population activity? From the relative timing of the activity in the x population and in the s population, that appears to be true.
3. Why is the first spike in the x population during retrieval being used? This may well result simply from competitive spiking and a match with the first x activity at neighbours of current location.
4. When there is activity in the s population of the minicolumn representing the "up" direction, why does the virtual rat not move up?

While the virtual rat can now move up, it does not make the left turn. No left directives are received. There is no activity in populations r and s related to the top location and left direction during navigation. Inspection shows that this may be related to the drop-out of current location input for the retrieval phase after about t=8000 ms. That time corresponds with a point in the simulation where the virtual rat entered another place field above the "top" place field. That place field was identified as the same place field assigned to the left "arm". This leads to two questions: (1) Why does Catacomb assign the same place field at two different borders of the "top" place field? (2) Why is the "arm" place field created by the FeatureDiscretizer still being generated when goal location was reimplemented as a sensation of proximity to reward.

In minicolumn-layers-aWM.20030828.ccm all remainder of the circuitry involved in 3 dimensional place field generation using proximity information is removed. In two dimensions, the FeatureDiscretizer still produces the place field assignment questioned in the preceding paragraph. While this is a strange way to generate place fields, the problem should not occur if retrieval leads to "left" directives once "top" is current location.

The current specifications for retrieval imply that forward retrieval through paths involving Wf and Wif should retrieve complete paths, since s population cells can activate in the absence of current location a population activity. Such full forward retrieval can combine with backward retrieval to lead to direction output that is not useful for navigation from the virtual rat's current location. If full retrieval is needed for other tasks, output for the navigation task must gate activity with current location a population activity.

When gated by current location, direction output can be obtained either from an associative neighbour minicolumn r population or from the current location minicolumn s population. In the first possibility, activity is gated both by the current location x population and the neighbouring y population, while the connection router table needed to extract direction output is straightforward. In the second possibility, s population activity is gated by x population activity, while the connection router table needs to route specifically those s population outputs that lead to minicolumns representing directions. The second possibility, using s population output, is a better choice here, since it is more easily gated by the corresponding a population activity.

Diffuse activation of the y_diff population for minicolumns that are active in in the a-STM-buffer during encoding appears to interfere with the specific activation of y_diff cells during the retrieval phase. This interference also appears to propagate back to x and y_spec populations. Since directions can be associated with various parts of a path, retrieval through x and y populations must be able to pass through diffusely activated y_diff populations during the retrieval phase without interference. The cells must be able to spike as needed in the separate phases.

1. Build the s copy gated population and specific connection table. Since the output population should not spike when the s population spikes for complete path retrieval, the output population needs only fixed strength synapses from the x population and the y population.
2. Fix the AHP so that y_diff, y_spec and x populations can spike properly in encoding and retrieval phases.

The output needs the right connection table and actual spikes in the output population. These are now achieved.

The virtual rat must be pulled to the left before running into trouble with the FeatureDiscretizer issues. For this it may be possible to increase the learning rate. Also, the activity in population x and y must pass through.

I am now testing with modified values of the input from x and from a to the output population. The tranmission modulated input from x to s and the input from a to s is received with the following characteristic parameters of synapses from x and a:

x	a
6	2.5
1	2
2	4
10	15

And the x conductance is always greater than 1/2 the specified amplitude due to transmission modulation. The corresponding connections to the output population are:

x	a
3	2.5
1	2
2	4
10	15

Bold type indicates parameter values that may need to be adjusted. First I attempt only an adjustment of the x conductance to 3.5 nS, to reflect the fact that the corresponding connection to the s population is usually modulated to a value greater than 3 nS.

I then changed the a connection parameters to: 2,4,8,30.
OR SHOULD THIS BE TESTED WITH POPULATION s INSTEAD?