Neuronal Circuitry: Oscillatory STM

Figure 1: Simulations that include a model of the short-term memory buffer are an integral part of the feedback loop between theory and experiment.

1. First investigation: Development of the Catacomb implementation of persistent spiking STM
2. Second investigation: Describing and Evaluating the model for publication [KOE:STMFIFO]
   For additional information, see the comments displayed with SHOWCOMMENTStrue in the STM-FIFO paper.
   • For information about canonical reference simulations, see TL#200602110555.1.

Development of the Catacomb implementation of persistent spiking STM

• make a list of things that need to be done to make an STM buffer, as implemented in the model (prior to sAHP or competitive suppression)
• current components and parameters used for STM buffer
  • use the tables above to obtain the parameters
• improvements
  • 3-4 items buffer
  • oscillations occur
  • necessary connections unknown during rapid acquisition
  • must capture input immediately
  • robust, perhaps not to severe disruptions, but certainly to a large degree of internal uncertainty in the functionality of components and some percentage variations in the environmental conditions

See derivations of dynamics and corresponding parameters from functional requirements in the DenseLTM Hypothesis 2 Experiment 1 log.

Methods of Achieving Item Expiry in Reliable FIFO Order

1. buffer loads up
2. items compete, produce inhibition
3. inhibition mounts until at N items, the items are barely still activating
4. next afferent item (N+1) adds inhibition that is too much for items to handle
5. as a consequence, item 1 does not fire - that brings the activation over the recent period detected back to N, so the remaining items (including the new one) manage to activate

• mutual inhibition (can be interneuron population, or more specific, e.g. population may spike if N activities in prior recall phase + 1 activity in the subsequent afferent input phase)
• specific span of detection
• inhibition should cause item to turn off, not just to delay and merge with the next item - that might require very precise tuning of the ADP curve
• interneuronal network sensitive to N+1 overlapping responses can cause spike that is transmission modulated to impact the first item most; the impact can be so severe that recovery time brings the cell past its useful ADP range

• MUST TARGET ITEM 1 WHEN ITEM N+1 APPEARS IN ORDER TO HAVE FIFO
• MUST SHUT OFF RATHER THAN SIMPLY DELAY
  • repetition was due to ADP+theta, hence the use of at least one of these must be affected to avoid reactivation
  • if you don't want items to merge, the process has to be relatively sudden (it may be possible to have items merge over very few cycles, if LTP is set not to accumulate too quickly - it may also be useful to counteract such merger LTP by LTD in proper sequences, so that coherent spiking in an interval of 35 ms causes LTP, an interval between 35 and 50 ms causes some LTD, and greater intervals cause neither)
  • normally, SHUT-OFF = loss of ADP+theta momentum (missing of interval where that can cause spiking)
• NEW ITEM MUST TAKE UP THE N-TH POSITION IN THE SEQUENCE
  • this implies that other items must shift forwards within one theta cycle, as the first item is no longer reactivated, which in turn implies that the ADP curve is such that the items would activate sooner than they do, if they were not held back by inhibition during gamma intervals

• ADP was timed so that delaying an item would cause it to no longer reactivate
  • this requires a very specific ADP curve
• New item presentation was used to cause the delay
• The delay somehow only occurred if the item presented was number N+1

• To keep items distinct:
  • presentation distinct enough to delay by time taken to cause gamma inhib.
  • ADP *not flat* at reactivation point, so that distinctness remains
    • V(t) *may not* have reached balance of leaky integrator! (conductance curve may be flat)
  • *strong* gamma can *chew* into integration of V, really delay curve of V(t) [TEST]
• To shift forward if space available:
  • all items should want to spike earlier if not delayed by gamma
• Normally:
  • ADP matches theta, peak of ADP is where repetition of first item occurs
  • gamma creates larger gap than would otherwise exist, but only so far that rising theta compensates, as does still rising ADP, hence other items are recalled
• Replacement:
  • gamma caused by afferent pushes 1st item beyond its ADP peak, theta cannot compensate *quickly* enough, *next item spikes* and its gamma compounds the delay for 1st item - which then never manages to spike
  • (second item must not undergo this treatment, hence gamma cannot affect it as badly)
• Why first item and not other affected so much? Because 1st doesn't spike until the PEAK of its ADP, others have more room, becuase theta is greater. (If theta rises faster than ADP, can also do this with 1st spiking after its ADP peak.)
  • This does require precise tuning and a *steep* theta (could be inverted normal theta, e.g. using excitatory instead of inhibitory influences or vice-versa if the normal one is produced differently, or generated by other parameter settings)!
  • shifted, but why not in trouble with gamma of next if less than N in buffer?

• An interneuron (population) can receive recurrent input from the buffer as well as feed-forward input from the afferent sources. Recurrent input from N active buffer cells can bring the interneuron (population) close to spiking activity. An additional afferent input (N+1) can cause interneuron (population) spiking. The inhibitory signal can reach the buffer a time delta-T after the arrival of the afferent input, just in time to affect the first item in the buffer and to delay it, thereby suppressing its activation as described above.
• This is NOT *cell* targeted, but *time* targeted, which may be more plausible.
• While sensitivity to N+1 responses with timing differences may be achievable in an interneuron population with internal propagation of activity, it may be less plausible (especially if a simple interneuronal representation is used). In that case, the N+1 responses can be synchronized by prepending the interneuron (population) with a synchronization circuit.

• A grid of neurons (M*N for M cells in the STM buffer and an STM capacity of N items) can remember which cell is involved in the representation of an item at a particular slot in the ordered STM buffer.
• Each time afferent input is received in the grid row that corresponds to an STM-buffer cell receiving the afferent input, spiking of the currently active cell in that row can combine with the afferent input to cause the next cell in the row to spike. The spiking is maintained through internal dynamics, such as ADP.
• Spiking of the N-th cell in a row can combine with the N+1th afferent activity to cause the spiking of an interneuron that inhibits the corresponding cell in the STM-buffer so that it shuts off.
• This implements an STM limit through structural constraints instead of physiological constraints on the dynamics. There are some plausibility concerns with this implementation.

• maintenace of items could be made dependent on interplay between the first and a second buffer - this is similar to the use of a context population in some ways (SHUT-OFF = loss of maintenance)
• what if afferent input could affect transmission modulation somewhere that could have a differential effect on the first versus other items in STM?
• expiry of items could be linked to a need for active support from memory of the autoassociative items; this is similar to Marc's hypothesized sequential memory system with links to context memory; an item could be allowed to activate in STM only when it is receiving enough afferent input from a context population; the context could shift as new items appear

Plausible Implementations of Item Expiry I: Targeted Time-Specific Inhibition

The first task is to design a synchronization circuit that is sensitive to N+1 sequential depolarizing responses. The synchronization circuit should be built according to the precision timing methods outlined for synchronization circuitry, and should be tuned to respond to the presence of two items plus the appearance of an afferent input.

Construction stages:

1. Construct a type 1 synchronization circuit with the added feature of amplitude thresholding. This circuit can be created in one of two ways. The first involves synchronization stages preceding the thresholding. The second involves the inclusion of amplitude dependence within a single synchronization stage.

   

   The amplitude threshold gains an additional degree of separation for the case where afferent input leads to N+1, compared to minor increases in the combined response amplitude due to shifts within the STM buffer, by depending not on N+1, but N+2 to reach threshold. N+2 is achieved by adding a response caused through a separate synapse from the afferent input to the spike response of the STM buffer to the afferent input.

   The desired behaviour is obtained with the second option by setting synaptic parameters as follows:

   Synchronization signal	G=0.6	$t_{rise}=0.1$ ms	$t_{fall}=20$ ms	duration=100 ms
   STM recall	G=0.25	$t_{rise}=80$ ms	$t_{fall}=60$ ms	duration=140 ms
   Afferent input	G=0.4	$t_{rise}=15$ ms	$t_{fall}=30$ ms	duration=60 ms

   The afferent response is designed to be more rapid than the responses to STM recall, which is given a slow rise time. This assures that the neuronal circuit will react more strongly to an afferent input following to recalled items than to an afferent input that is followed by one recalled item and the subsequent recall of the item elicited by the afferent input. A stronger response at the time of afferent input than at the time of its recall is further mandated by a difference in the synaptic conductances (G=0.25 versus G=0.4).

   The arrival of the synchronizing signal can be timed such that a spike propagates to inhibitory synapses of the STM buffer at the phase needed to suppress the item that would otherwise reactivate after afferent input was received. The synchronization signal derived from septal theta input is therefore given a delay of 35 ms. Satisfactory suppression is then a matter of tuning the inhibitory response to the dynamics of the STM buffer circuitry. This state of the implementation is stored as tmaze-STM.time-specific-inhibition.combination-option2.20020423.ccm.gz.

   Insuring that a clear distinction is made between the case where afferent input is preceded by the recall of two items and the case where afferent input is followed by the recall of one older item and the item it elicited is even more straightforward with the first option. When that option is implemented, synchronization precedes combination, so that recall followed by afferent input combines to attain threshold, while afferent input followed by recall does not.

2. The STM buffer dynamics must be reconstructed to respond properly to inhibition that targets the theta phase at which the first item is recalled. The theta modulation in the STM buffer must be steeper in its high phase. The ADP should have a shorter period, so that the time taken to rise to maximum corresponds closely with a theta period. Recalled items should shift to earlier phases in the absence of gamma inhibition.

   The fall-time constant of the theta synapses is increased from 14 ms to 35 ms and their duration parameter set to span the entire theta period of 125 ms. The time-constant of the STM-buffer principal cells is reduced from 10 ms to 9 ms. The resulting membrane potential modulation does not approach a plateau, but the maximum depolarization due to theta membrane modulation is slightly lower at about -63 mV.

3. Tuning of inhibitory response.
4. Optional testing for other numbers of items.

Perhaps the theta shape has to be adjusted again - perhaps a bit flatter, since the theta rise may have to be less steep than the ADP fall. The conductances of ADP and theta should be similar if they are to be comparable in their drive - of course, by making the ADP conductance stronger and letting it drop off low enough, it may be easier to deal with steeper theta. Neither ADP nor theta should be capable of causing spiking on their own - well, that isn't quite necessary, as it is possible to allow ADP to cause spiking when theta is around its mid-value, since strong suppression at the peak of ADP can then still cause drop-out.

1. THETA alone does not cause spiking. [DONE]

2. ADP+HIGH THETA causes repetition of new input in same theta period near end of STM cycle - HIGH THETA must be high enough to combine with less than MAX ADP to exceed threshold. [DONE]

3. MAX ADP + MID THETA at desired ONSET time of recall phase causes recall of first item - MAX ADP must be just high enough to ensure that threshold is reached and ADP TIMING must correlate with THETA PERIOD.

   This is not so easy. The current implementation of the ADP assumes that the fall timing is passive, not an active hyperpolarizing pull. So, the limit is an alpha function that necessarily has a longer fall time than rise time. The ADP drop-off can be made more dramatic, by specifying a duration of 125 ms or less, so that the capacitance then reduces the depolarization quickly. There is clearly some room for future improvement of the ADP implementation. Using this modification of the ADP function, it is also possible to lengthen the rise time further, if that proves to be necessary in order to maintain distinctness of items.

   Constraints:

   • MAX THETA + LESSER ADP = threshold.
   • MAX ADP + MID THETA = threshold.
   Candidate ADP: -45, 20, 125, 0, 130
   Candidate THETA: -90, 20, 100, 0.1, 20, 125
   These reach a MAX ADP + MID THETA limit at about 73 ms into a theta period.

   If we wish to make that an earlier phase to make more room for recall, we can decrease the hyperpolarization caused by theta, or shorten the AHP, or strengthen the ADP.

   [DONE, except that the recall phase my be broadened if necessary]

4. GAMMA inhibition must have RAPID ONSET and be STRONG but SHORT to avoid merging of items.

   Original: -70, 20, 100, 1, 5, 20
   Candidate: -90, 40, 100, 0.01, 2, 10

   If gamma needs to be even shorter and sharper, the duration can be reduced to something like 3 ms, so that the capacitance immediately starts bringing the membrane potential back. The gamma inhibition was given a very strong conductance, since it is the full responsibility of that inhibition to keep the items apart - at the distance imposed by the fall time, duration and capacitance parameters.

   Note: When the gamma period is shortened, it is important to adjust the AHP within the interneuronal population, so that another gamma period can begin when the next spike occurs in the STM buffer! For this reason, I created the interneuronal population "STM-short-gamma" with AHP parameters: -90, 100, 10^-4, 4, 10.

   For the sake of completeness, I am investigating the behaviour of the new STM buffer when holding 3 items. While a third item is now properly introduced to the buffer, it merges with the second item upon recall in a subsequent theta cycle. The theta modulation appears to be pushing the responses together, despite different phases of the ADP responses. One way to deal with this is to increase the relative strength of the ADP by increasing the ratio of ADP to theta conductance. While doing so, the behaviour previously achieved should be carefully maintained. Since high theta is in effect a return to resting potential, a simple way to try this is to reduce the theta conductance, which should not affect the modulated membrane potential at high theta.

   Changing theta conductance from G=20 to G=10 seems to retain behaviour with one item in the buffer. The onset of the recall phase was shifted slightly, to about 58 ms into a theta period.

   Candidate THETA: -90, 10, 100, 0.1, 20, 125.

   Since the theta influence is now less, it is important that the ADPs not be in a flat portion together. To avoid that, the AHP is increased.

   Candidate AHP: -90, 25, 10^-4, 30, 125.

   Interplay: THETA combines with the MAX ADP to designate the onset of the STM recall phase.
   ADP and AHP are stronger than THETA, to insure that items are not merged by becoming too strongly adjusted to the common theta response.
   A short but strong AHP insures that the ADP does not flatten too rapidly as it approaches its maximum, thereby helping to maintain separation between items.
   The ADP drops off rapidly, beginning 5 ms after its maximum (achieved by the membrane capacitance), so that suppression of the first item at the moment of MAX ADP can cause the item to be retired from the buffer.

   [DONE, perhaps adjust further to enable 4 items to be held in STM]

   Three items in the STM buffer: tmaze-STM.time-specific-inhibition.20020424.ccm.gz. (Note: Three items enter the buffer consecutively, only two are maintained in the buffer at any given time.)

5. THETA must promote shifting to first position in absence of GAMMA inhibition.

6. TIME-SPECIFIC INHIBITION must be STRONG and LONG enough to suppress the part of the first item's membrane response that could exceed threshold, but SHORT enough not to block second item spiking.

   The delay of the theta modulation in the time-specific inhibition circuitry is increased to from 25 ms to 35 ms to match the new onset of the STM recall phase.

   Candidate Inhibition: -90, 40, 100, 1, 3, 15.

   [DONE, appears to work quite well]

   This version is stored as: tmaze-STM.time-specific-inhibition.20020424-2.ccm.gz. (This runs with Catacomb 2.034.)

7. AHP must have RAPID ONSET and be STRONG enough to suppress multiple spiking, but must not interfere with the interaction of dynamics.

   Candidate AHP: -90, 10, 10^-4, 30, 125.
   New Candidate AHP: -90, 25, 10^-4, 30, 125. (After update in D.)

   In order to make it all work with the signals from the feature discretizer, the place input synchronization circuit needs to be replaced with a more precise version (at type 1 synchronization circuit), to insure that afferent input arrives early enough for it to spike in the recall phase of the theta period. This new place input population is created as "Place-Input-type1".

   place-input-type1:
   cell: 22,1,10,1,-60,-60
   AHP: -90,10,4,1000,1200
   theta: 0,2,100,0.1,20,100
   input: 0,1,100,1,15,30

   We want the first spike to occur between t=106 ms and t=108 ms. For this, a delay of 105 ms is introduced into the theta synchronization signal (and the corresponding transmission modulation).

   GREEN wasn't properly retired when BLUE appeared - the inhibition should last a bit longer.

   Inhibition: -90, 40, 100, 1, 5, 15.

   The blue one is really the problem now. All others seem to move forward in the phase, but the blue one stubbornly stays near the last gamma cycle available in the theta period.

   ADP: G=25 instead of 20.
   AHP: G=30 instead of 25.
   Correspondingly, move theta phase for inhibitor earlier by 15 ms (from 35 ms to 25 ms).

   Some moving together occurs - can try to keep ADP steeper and/or increase gamma (which makes sense, since ADP conductance was increased).
   gamma: -90, G=50 instead of 40, 50, 100, 0.01, 3 instead of 2, 12 instead of 10.

   Perhaps changing gamma wasn't really necessary, since removing the sAHP got rid of remaining merging. Instead of removing the sAHP, it can be set to participate must more gradually, such as with sAHP: G=0.01.

The following graph demonstrates the output of the older STM buffer circuitry with improvised item retirement through cell specific counting.

The output of the more plausible STM buffer circuitry with time-specific inhibition is shown in the graph below.

Importing the new STM buffer implementation with its more plausible item retirement circuitry back into the newtmaze.ccm model requires some careful testing and minor adjustments. These adjustments can be necessary, since the exact phase at which items in STM are repeated is slightly different than that in the previous STM buffer implementation. The propagation of the STM buffer activity must be adapted accordingly or the theta phase in the new STM buffer must be tuned.

The onset of STM recall in the new STM buffer implementation is a bit earlier. It is necessary for the second item in the buffer to fall within the strong phase of plasticity modulation in ECIII and CA3. The arrival of spikes from the STM buffer must therefore be delayed by about 10 ms. To avoid implausibly long delays on the synaptic pathways from the EC STM buffer to ECIII and CA3, the theta cycles in the STM buffer, its time-specific inhibition circuit and its input synchronization population must be shifted.

For testing purposes, I am temporarily simply adding a 10 ms delay to the output projections from the new STM buffer, so that the propagated spikes arrive at the expected times for transmission and plasticity modulation in ECIII and CA3. Using the delay improvisation, the new STM buffer circuitry appears to work properly with the newtmaze.ccm model.

Now the necessary theta phases can be adjusted as described above.

Component	Current	Delayed by 10 ms
EC-STM	theta mod: 102 in trans mod: 102 gamma trans mod: 102	theta mod: 112 in trans mod: 112 gamma trans mod: 112
STM-inhib	theta mod: 20	theta mod: 30
Place-input	theta mod: 105 in trans mod: 105	theta mod: 115 in trans mod: 115

Problem:
The new timing causes the pairing of the fourth and fifth place cell activities to appear one theta cycle later (the fifth place cell probably just missed its chance in the previous cycle due to the 10 ms delay). That makes the presence of that pair very short (two cycles). That appears to be insufficient to acquire enough LTP so that the virtual rat can retrieve the next location.

I can either try to adjust the timing, so that an additional theta cycle is obtained. Or I can try to make the interval of traversal of the fifth place field longer. Or I can decrease the size of the update window of the LTP function, while increasing the update ratio.
I will first attempt the adjustment of the LTP function. I am decreasing the width from 35 ms to 25 ms, and increasing the height from 200 to 400.
This did not solve the problem.

In order to experiment with solutions, I am letting the virtual rat run more slowly on the forward traversal of the left arm of the T-maze. The onward speed on all other segments of the trajectory was 0.15 mm/ms. On the modified segment, the onward speed was slowed to 0.08 mm/ms. This allowed the problematic pairing to be repeated over 5 theta cycles. The virtual rat was subsequently able to navigate to the goal when driven by the network. This clearly demonstrates the significance of the number of recalls achieved for a particular episode in STM during exploratory traversal of the environment.

The above is not a problem that is specific to the new STM implementation, rather it demonstrates a larger issue. The issue is a competition between the advantages of high-resolution place mapping and sufficient repetition of episodes in STM to achieve memories based on LTP from a single presentation (traversal) of the environment. To easily achieve both, the virtual rat would have to explore very slowly, which would in essence nullify the advantage of learning from a single traversal of a novel environment. Two modifications of the model may be helpful here:

1. The use of multi-cell item representations for a high-resolution mapping.
2. Repetition of more than two items in STM (which can allow items to remain in STM longer).

Describing and Evaluating the model for publication

For additional notes about this description and evaluation process, see notes taken in the Task Log for DIL#20040312094513.1.

Once a working extracted version of the most recent STM implementation is ready, its Catacomb representation can be used for a schematic representation of the model and event sequencer input can be used to produce sample output.