While much is known about the limiting effect of neural noise on the fidelity of sensory coding representation, knowledge about the impact of noise in short-term memory and integrator networks has remained more elusive. (Integrator networks are networks of nodes – in this case neurons in a biological network – often recurrently connected, whose time dynamics settle to stable stationary, cyclic, or chaotic patterns, that can integrate or store memories of external inputs.)
Recently, however, scientists at The Hebrew University of Jerusalem, Harvard University and University of Texas, Austin used statistical and dynamical approaches to investigate how neural noise interacts with neural and network parameters to limit memory. They derived a series of unanticipated results – including the implications that short-term memory may be co-localized with sensory representation – by establishing a fundamental limit on the network’s ability to maintain a persistent neural state.
Assistant Professors Ila R. Fiete and Yoram Burak described the various challenges they encountered. “The dynamics of spiking neural networks are in general highly nonlinear and involve a very large number of degrees of freedom,” Fiete tells Phys.org, addressing their analysis of how stored memory in continuous attractor networks will stochastically degrade over time. She adds that most work on such networks is focused on deterministic dynamics. “A priori,” she continues, “it wasn’t obvious that one could evaluate precisely how noise affects the state of the system,” pointing out that investigations into noise affecting a memory state in such networks was previously done for very simple systems with linear neurons (those with no nonlinearity in the neural response), with noise being externally injected and having simple statistical properties.
“By contrast,” Burak explains, “we wanted to understand the role of noise that originates within the network – that is, noise intrinsic to single neurons or synapses, as opposed to simple external noise.” Intrinsic neuronal noise has a more complicated form, and its properties vary in each neuron, based on the neuron’s firing rate at that moment in time. “Unlike external noise, which is assumed to directly affect the memory state, internal noise must be passed through the nonlinear dynamics of the system, to derive its effects on the memory state. We wanted to obtain a general theory, without making particular assumptions about network connectivity and neural nonlinearity.”
Their work began with an intuitive idea they had before doing calculations about continuous attractor networks, says Fiete. “The network’s limited ability to read its own past state from its spikes, so to speak, must limit its ability to maintain that past state into the future. This must limit the accuracy of persistent activity. The biggest challenge here was to translate this intuitive idea into a rigorous formal statement about a concrete model of spiking neurons. The formal statement is given by a combination of a statistical limit with a dynamical property, in the form of an information-diffusion inequality.”
Among the study’s unexpected consequences, Fiete continues, was that despite the long persistence time of short-term memory networks, it does not pay to accumulate spikes for much longer than the short time-constant of individual neurons, to read out the contents of the network. “This result was born out of our attempt to understand the consequences of the gradual loss of accuracy in storing a variable in a memory network due to diffusive dynamics. Our initial intuition was that in a network with persistent memory, the longer one observes the spikes generated by the network, the better one should be able to infer its state to arbitrary precision. However, while one is collecting spikes to improve statistical precision in estimating the network’s state, the state itself drifts due to diffusion. As a result, the state of the network cannot be inferred to arbitrary precision.” What surprised the researchers was that there was actually no benefit to collecting spikes for any appreciable length of time beyond a very short time scale – that is, the intrinsic time constant of single neurons. Via Of noise and neurons: Sensory coding, representation and short-term memory.
