Researcher: Mehrdad Jazayeri

How expectation influences perception

by Anne Trafton |

For decades, research has shown that our perception of the world is influenced by our expectations. These expectations, also called “prior beliefs,” help us make sense of what we are perceiving in the present, based on similar past experiences. Consider, for instance, how a shadow on a patient’s X-ray image, easily missed by a less experienced intern, jumps out at a seasoned physician. The physician’s prior experience helps her arrive at the most probable interpretation of a weak signal.

The process of combining prior knowledge with uncertain evidence is known as Bayesian integration and is believed to widely impact our perceptions, thoughts, and actions. Now, MIT neuroscientists have discovered distinctive brain signals that encode these prior beliefs. They have also found how the brain uses these signals to make judicious decisions in the face of uncertainty.

“How these beliefs come to influence brain activity and bias our perceptions was the question we wanted to answer,” says Mehrdad Jazayeri, the Robert A. Swanson Career Development Professor of Life Sciences, a member of MIT’s McGovern Institute for Brain Research, and the senior author of the study.

The researchers trained animals to perform a timing task in which they had to reproduce different time intervals. Performing this task is challenging because our sense of time is imperfect and can go too fast or too slow. However, when intervals are consistently within a fixed range, the best strategy is to bias responses toward the middle of the range. This is exactly what animals did. Moreover, recording from neurons in the frontal cortex revealed a simple mechanism for Bayesian integration: Prior experience warped the representation of time in the brain so that patterns of neural activity associated with different intervals were biased toward those that were within the expected range.

MIT postdoc Hansem Sohn, former postdoc Devika Narain, and graduate student Nicolas Meirhaeghe are the lead authors of the study, which appears in the July 15 issue of Neuron.

Ready, set, go

Statisticians have known for centuries that Bayesian integration is the optimal strategy for handling uncertain information. When we are uncertain about something, we automatically rely on our prior experiences to optimize behavior.

“If you can’t quite tell what something is, but from your prior experience you have some expectation of what it ought to be, then you will use that information to guide your judgment,” Jazayeri says. “We do this all the time.”

In this new study, Jazayeri and his team wanted to understand how the brain encodes prior beliefs, and put those beliefs to use in the control of behavior. To that end, the researchers trained animals to reproduce a time interval, using a task called “ready-set-go.” In this task, animals measure the time between two flashes of light (“ready” and “set”) and then generate a “go” signal by making a delayed response after the same amount of time has elapsed.

They trained the animals to perform this task in two contexts. In the “Short” scenario, intervals varied between 480 and 800 milliseconds, and in the “Long” context, intervals were between 800 and 1,200 milliseconds. At the beginning of the task, the animals were given the information about the context (via a visual cue), and therefore knew to expect intervals from either the shorter or longer range.

Jazayeri had previously shown that humans performing this task tend to bias their responses toward the middle of the range. Here, they found that animals do the same. For example, if animals believed the interval would be short, and were given an interval of 800 milliseconds, the interval they produced was a little shorter than 800 milliseconds. Conversely, if they believed it would be longer, and were given the same 800-millisecond interval, they produced an interval a bit longer than 800 milliseconds.

“Trials that were identical in almost every possible way, except the animal’s belief led to different behaviors,” Jazayeri says. “That was compelling experimental evidence that the animal is relying on its own belief.”

Once they had established that the animals relied on their prior beliefs, the researchers set out to find how the brain encodes prior beliefs to guide behavior. They recorded activity from about 1,400 neurons in a region of the frontal cortex, which they have previously shown is involved in timing.

During the “ready-set” epoch, the activity profile of each neuron evolved in its own way, and about 60 percent of the neurons had different activity patterns depending on the context (Short versus Long). To make sense of these signals, the researchers analyzed the evolution of neural activity across the entire population over time, and found that prior beliefs bias behavioral responses by warping the neural representation of time toward the middle of the expected range.

“We have never seen such a concrete example of how the brain uses prior experience to modify the neural dynamics by which it generates sequences of neural activities, to correct for its own imprecision. This is the unique strength of this paper: bringing together perception, neural dynamics, and Bayesian computation into a coherent framework, supported by both theory and measurements of behavior and neural activities,” says Mate Lengyel, a professor of computational neuroscience at Cambridge University, who was not involved in the study.

Embedded knowledge

Researchers believe that prior experiences change the strength of connections between neurons. The strength of these connections, also known as synapses, determines how neurons act upon one another and constrains the patterns of activity that a network of interconnected neurons can generate. The finding that prior experiences warp the patterns of neural activity provides a window onto how experience alters synaptic connections. “The brain seems to embed prior experiences into synaptic connections so that patterns of brain activity are appropriately biased,” Jazayeri says.

As an independent test of these ideas, the researchers developed a computer model consisting of a network of neurons that could perform the same ready-set-go task. Using techniques borrowed from machine learning, they were able to modify the synaptic connections and create a model that behaved like the animals.

These models are extremely valuable as they provide a substrate for the detailed analysis of the underlying mechanisms, a procedure that is known as “reverse-engineering.” Remarkably, reverse-engineering the model revealed that it solved the task the same way the monkeys’ brain did. The model also had a warped representation of time according to prior experience.

The researchers used the computer model to further dissect the underlying mechanisms using perturbation experiments that are currently impossible to do in the brain. Using this approach, they were able to show that unwarping the neural representations removes the bias in the behavior. This important finding validated the critical role of warping in Bayesian integration of prior knowledge.

The researchers now plan to study how the brain builds up and slowly fine-tunes the synaptic connections that encode prior beliefs as an animal is learning to perform the timing task.

The research was funded by the Center for Sensorimotor Neural Engineering, the Netherlands Scientific Organization, the Marie Sklodowska Curie Reintegration Grant, the National Institutes of Health, the Sloan Foundation, the Klingenstein Foundation, the Simons Foundation, the McKnight Foundation, and the McGovern Institute.

How we make complex decisions

by Anne Trafton |

When making a complex decision, we often break the problem down into a series of smaller decisions. For example, when deciding how to treat a patient, a doctor may go through a hierarchy of steps — choosing a diagnostic test, interpreting the results, and then prescribing a medication.

Making hierarchical decisions is straightforward when the sequence of choices leads to the desired outcome. But when the result is unfavorable, it can be tough to decipher what went wrong. For example, if a patient doesn’t improve after treatment, there are many possible reasons why: Maybe the diagnostic test is accurate only 75 percent of the time, or perhaps the medication only works for 50 percent of the patients. To decide what do to next, the doctor must take these probabilities into account.

In a new study, MIT neuroscientists explored how the brain reasons about probable causes of failure after a hierarchy of decisions. They discovered that the brain performs two computations using a distributed network of areas in the frontal cortex. First, the brain computes confidence over the outcome of each decision to figure out the most likely cause of a failure, and second, when it is not easy to discern the cause, the brain makes additional attempts to gain more confidence.

“Creating a hierarchy in one’s mind and navigating that hierarchy while reasoning about outcomes is one of the exciting frontiers of cognitive neuroscience,” says Mehrdad Jazayeri, the Robert A. Swanson Career Development Professor of Life Sciences, a member of MIT’s McGovern Institute for Brain Research, and the senior author of the study.

MIT graduate student Morteza Sarafyzad is the lead author of the paper, which appears in Science on May 16.

Hierarchical reasoning

Previous studies of decision-making in animal models have focused on relatively simple tasks. One line of research has focused on how the brain makes rapid decisions by evaluating momentary evidence. For example, a large body of work has characterized the neural substrates and mechanisms that allow animals to categorize unreliable stimuli on a trial-by-trial basis. Other research has focused on how the brain chooses among multiple options by relying on previous outcomes across multiple trials.

“These have been very fruitful lines of work,” Jazayeri says. “However, they really are the tip of the iceberg of what humans do when they make decisions. As soon as you put yourself in any real decision-making situation, be it choosing a partner, choosing a car, deciding whether to take this drug or not, these become really complicated decisions. Oftentimes there are many factors that influence the decision, and those factors can operate at different timescales.”

The MIT team devised a behavioral task that allowed them to study how the brain processes information at multiple timescales to make decisions. The basic design was that animals would make one of two eye movements depending on whether the time interval between two flashes of light was shorter or longer than 850 milliseconds.

A twist required the animals to solve the task through hierarchical reasoning: The rule that determined which of the two eye movements had to be made switched covertly after 10 to 28 trials. Therefore, to receive reward, the animals had to choose the correct rule, and then make the correct eye movement depending on the rule and interval. However, because the animals were not instructed about the rule switches, they could not straightforwardly determine whether an error was caused because they chose the wrong rule or because they misjudged the interval.

The researchers used this experimental design to probe the computational principles and neural mechanisms that support hierarchical reasoning. Theory and behavioral experiments in humans suggest that reasoning about the potential causes of errors depends in large part on the brain’s ability to measure the degree of confidence in each step of the process. “One of the things that is thought to be critical for hierarchical reasoning is to have some level of confidence about how likely it is that different nodes [of a hierarchy] could have led to the negative outcome,” Jazayeri says.

The researchers were able to study the effect of confidence by adjusting the difficulty of the task. In some trials, the interval between the two flashes was much shorter or longer than 850 milliseconds. These trials were relatively easy and afforded a high degree of confidence. In other trials, the animals were less confident in their judgments because the interval was closer to the boundary and difficult to discriminate.

As they had hypothesized, the researchers found that the animals’ behavior was influenced by their confidence in their performance. When the interval was easy to judge, the animals were much quicker to switch to the other rule when they found out they were wrong. When the interval was harder to judge, the animals were less confident in their performance and applied the same rule a few more times before switching.

“They know that they’re not confident, and they know that if they’re not confident, it’s not necessarily the case that the rule has changed. They know they might have made a mistake [in their interval judgment],” Jazayeri says.

Decision-making circuit

By recording neural activity in the frontal cortex just after each trial was finished, the researchers were able to identify two regions that are key to hierarchical decision-making. They found that both of these regions, known as the anterior cingulate cortex (ACC) and dorsomedial frontal cortex (DMFC), became active after the animals were informed about an incorrect response. When the researchers analyzed the neural activity in relation to the animals’ behavior, it became clear that neurons in both areas signaled the animals’ belief about a possible rule switch. Notably, the activity related to animals’ belief was “louder” when animals made a mistake after an easy trial, and after consecutive mistakes.

The researchers also found that while these areas showed similar patterns of activity, it was activity in the ACC in particular that predicted when the animal would switch rules, suggesting that ACC plays a central role in switching decision strategies. Indeed, the researchers found that direct manipulation of neural activity in ACC was sufficient to interfere with the animals’ rational behavior.

“There exists a distributed circuit in the frontal cortex involving these two areas, and they seem to be hierarchically organized, just like the task would demand,” Jazayeri says.

Daeyeol Lee, a professor of neuroscience, psychology, and psychiatry at Yale School of Medicine, says the study overcomes what has been a major obstacle in studying this kind of decision-making, namely, a lack of animal models to study the dynamics of brain activity at single-neuron resolution.

“Sarafyazd and Jazayeri have developed an elegant decision-making task that required animals to evaluate multiple types of evidence, and identified how the two separate regions in the medial frontal cortex are critically involved in handling different sources of errors in decision making,” says Lee, who was not involved in the research. “This study is a tour de force in both rigor and creativity, and peels off another layer of mystery about the prefrontal cortex.”

Mehrdad Jazayeri

The long-term objective of the Jazayeri lab is to develop a mathematical framework for understanding the link between the brain and the mind. To tackle this problem, the lab records and perturbs brain signals in animal models while they perform mental computations. They use normative theories, computational models, and artificial neural networks to understand the building blocks of the mind in terms of the mechanisms and algorithms implemented by the brain.

The lab currently focuses on the following mental computations: (1) anticipation and planning, (2) integration and inference, (3) hierarchical and counterfactual reasoning, and (4) mental navigation.

Meeting of the minds

Anne Trafton |

In the summer of 2006, before their teenage years began, Mahdi Ramadan and Alexi Choueiri were spirited from their homes amid political unrest in Lebanon. Evacuated on short notice by the U.S. Marines, they were among 2,000 refugees transported to the U.S. on the aircraft carrier USS Nashville.

The two never met in their homeland, nor on the transatlantic journey, and after arriving in the U.S. they went their separate ways. Ramadan and his family moved to Seattle, Washington. Choueiri’s family settled in Chandler, Arizona, where they already had some extended family.

Yet their paths converged 11 years later as graduate students in MIT’s Department of Brain and Cognitive Sciences (BCS). One day last fall, on a walk across campus, Ramadan and Choueiri slowly unraveled their connection. With increasing excitement, they narrowed it down by year, by month, and eventually, by boat, to discover just how closely their lives had once come to one another.

Lebanon, the only Middle Eastern country without a desert, enjoys a lush, Mediterranean climate. Amid this natural beauty, though, the country struggles under the weight of deep political and cultural divides that sometimes erupt into conflict.

Despite different Lebanese cultural backgrounds — Ramadan’s family is Muslim and Choueiri’s Christian — they have had remarkably similar experiences as refugees from Lebanon. Both credit those experiences with motivating their interest in neuroscience. Questions about human behavior — How do people form beliefs about the world? Can those beliefs really change? — led them to graduate work at MIT.

In pursuit of knowledge

When they first immigrated to the U.S., school symbolized survival for Ramadan and Choueiri. Not only was education a mode of improving their lives and supporting their families, it was a search for objectivity in their recently upended worlds.

As the family’s primary English speaker, Ramadan became a bulwark for his family in their new country, especially in medical matters; his little sister, Ghida, has cerebral palsy. Though his family has limited financial resources, he emphasizes that both he and his sister have been constantly supported by their parents in pursuit of their educations.

In fact, Ramadan feels motivated by Ghida’s determination to complete her degree in occupational therapy: “That to me is really inspirational, her resilience in the face of her disability and in the face of assumptions that people make about capability. She’s really sassy, she’s really witty, she’s really funny, she’s really intelligent, and she doesn’t see her disability as a disability. She actually thinks it’s an advantage — it actually motivated her to pursue [her education] even more.”

Ramadan hopes his own educational journey, from a low-income evacuee to a neuroscience PhD, can show others like him that success is possible.

Choueiri also relied on academics to adapt to his new world in Arizona. Even in Lebanon, he remembers taking solace from a chaotic world in his education, and once in the U.S., he dove headfirst into his studies.

Choueiri’s hometown in Arizona sometimes felt homogenous, so coming to MIT has been a staggering — and welcome — experience. “The diversity here is phenomenal: meeting people from different cultures, upbringings, countries,” he says. “I love making friends from all over and learning their stories. Being a neuroscientist, I like to know how they were brought up and how their ideas were formed. … It’s like Disneyland for me. I feel like I’m coming to Disneyland every day and high-fiving Mickey Mouse.”

At home at MIT

Ramadan and Choueiri revel in the freedom of thought they have found in their academic home here. They say they feel taken seriously as students and, more importantly, as thinkers. The BCS department values interdisciplinary thought, and cultivates extracurricular student activities like philosophy discussion groups, the development of neuroscience podcasts, and independent, student-led lectures on myriad neuroscience-adjacent topics.

Both students were drawn to neuroscience not only by their experiences as Lebanese-Americans, but by trying to make sense of what happened to them at a young age.

Ramadan became interested in neuroplasticity through self-observation. “You know that feeling of childhood you have where everything is magical and you’re not really aware of things around you? I feel like when I immigrated to the U.S., that feeling went away and I had to become extra-aware of everything because I had to adapt so quickly. So, something that intrigued me about neuroscience is how the brain is able to adapt so quickly and how different experiences can shape and rewire your brain.”

Now in his second year, Ramadan plans to pursue his interest in neuroplasticity in Professor Mehrdad Jazayeri’s lab at the McGovern Institute by investigating how learning changes the brain’s underlying neural circuits; understanding the physical mechanism of plasticity has application to both disease states and artificial intelligence.

Choueiri, a third-year student in the program, is a member of Professor Ed Boyden’s lab at the McGovern Institute. While his interest in neuroscience was similarly driven by his experience as an evacuee, his approach is outward-looking, focused on making sense of people’s choices. Ultimately, the brain controls human ability to perceive, learn, and choose through physiological changes; Choueiri wants to understand not just the human brain, but also the human condition — and to use that understanding to alleviate pain and suffering.

“Growing up in Lebanon, with different religions and war … I became fundamentally interested in human behavior, irrationality, and conflict, and how can we resolve those things … and maybe there’s an objective way to really make sense of where these differences are coming from,” he says. In the Synthetic Neurobiology Group, Choueiri’s research involves developing neurotechnologies to map the molecular interactions of the brain, to reveal the fundamental mechanisms of brain function and repair dysfunction.

Shared identities

As evacuees, Ramadan and Choueiri left their country without notice and without saying goodbye. However, in other ways, their experience was not unlike an immigrant experience. This sometimes makes identifying as a refugee in the current political climate complex, as refugees from Syria and other war-ravaged regions struggle to make a home in the U.S. Still, both believe that sharing their personal experience may help others in difficult positions to see that they do belong in the U.S., and at MIT.

Despite their American identity, Ramadan and Choueiri also share a palpable love for Lebanese culture. They extol the diversity of Lebanese cuisine, which is served mezze-style, making meals an experience full of variety, grilled food, and yogurt dishes. The Lebanese diaspora is another source of great pride for them. Though the population of Lebanon is less than 5 million, as many as 14 million live abroad.

It’s all the more remarkable, then, that Ramadan and Choueiri intersected at MIT, some 6,000 miles from their homeland. The bond they have forged since, through their common heritage, experiences, and interests, is deeply meaningful to both of them.

“I was so happy to find another student who has this story because it allows me to reflect back on those experiences and how they changed me,” says Ramadan. “It’s like a mirror image. … Was it a coincidence, or were our lives so similar that they led to this point?”

This story was written by Bridget E. Begg at MIT’s Office of Graduate Education.

Study reveals how the brain overcomes its own limitations

Anne Trafton |

Imagine trying to write your name so that it can be read in a mirror. Your brain has all of the visual information you need, and you’re a pro at writing your own name. Still, this task is very difficult for most people. That’s because it requires the brain to perform a mental transformation that it’s not familiar with: using what it sees in the mirror to accurately guide your hand to write backward.

MIT neuroscientists have now discovered how the brain tries to compensate for its poor performance in tasks that require this kind of complicated transformation. As it also does in other types of situations where it has little confidence in its own judgments, the brain attempts to overcome its difficulties by relying on previous experiences.

“If you’re doing something that requires a harder mental transformation, and therefore creates more uncertainty and more variability, you rely on your prior beliefs and bias yourself toward what you know how to do well, in order to compensate for that variability,” says Mehrdad Jazayeri, the Robert A. Swanson Career Development Professor of Life Sciences, a member of MIT’s McGovern Institute for Brain Research, and the senior author of the study.

This strategy actually improves overall performance, the researchers report in their study, which appears in the Oct. 24 issue of the journal Nature Communications. Evan Remington, a McGovern Institute postdoc, is the paper’s lead author, and technical assistant Tiffany Parks is also an author on the paper.

Noisy computations

Neuroscientists have known for many decades that the brain does not faithfully reproduce exactly what the eyes see or what the ears hear. Instead, there is a great deal of “noise” — random fluctuations of electrical activity in the brain, which can come from uncertainty or ambiguity about what we are seeing or hearing. This uncertainty also comes into play in social interactions, as we try to interpret the motivations of other people, or when recalling memories of past events.

Previous research has revealed many strategies that help the brain to compensate for this uncertainty. Using a framework known as Bayesian integration, the brain combines multiple, potentially conflicting pieces of information and values them according to their reliability. For example, if given information by two sources, we’ll rely more on the one that we believe to be more credible.

In other cases, such as making movements when we’re uncertain exactly how to proceed, the brain will rely on an average of its past experiences. For example, when reaching for a light switch in a dark, unfamiliar room, we’ll move our hand toward a certain height and close to the doorframe, where past experience suggests a light switch might be located.

All of these strategies have been previously shown to work together to increase bias toward a particular outcome, which makes our overall performance better because it reduces variability, Jazayeri says.

Noise can also occur in the mental conversion of sensory information into a motor plan. In many cases, this is a straightforward task in which noise plays a minimal role — for example, reaching for a mug that you can see on your desk. However, for other tasks, such as the mirror-writing exercise, this conversion is much more complicated.

“Your performance will be variable, and it’s not because you don’t know where your hand is, and it’s not because you don’t know where the image is,” Jazayeri says. “It involves an entirely different form of uncertainty, which has to do with processing information. The act of performing mental transformations of information clearly induces variability.”

That type of mental conversion is what the researchers set out to explore in the new study. To do that, they asked subjects to perform three different tasks. For each one, they compared subjects’ performance in a version of the task where mapping sensory information to motor commands was easy, and a version where an extra mental transformation was required.

In one example, the researchers first asked participants to draw a line the same length as a line they were shown, which was always between 5 and 10 centimeters. In the more difficult version, they were asked to draw a line 1.5 times longer than the original line.

The results from this set of experiments, as well as the other two tasks, showed that in the version that required difficult mental transformations, people altered their performance using the same strategies that they use to overcome noise in sensory perception and other realms. For example, in the line-drawing task, in which the participants had to draw lines ranging from 7.5 to 15 centimeters, depending on the length of the original line, they tended to draw lines that were closer to the average length of all the lines they had previously drawn. This made their responses overall less variable and also more accurate.

“This regression to the mean is a very common strategy for making performance better when there is uncertainty,” Jazayeri says.

Noise reduction

The new findings led the researchers to hypothesize that when people get very good at a task that requires complex computation, the noise will become smaller and less detrimental to overall performance. That is, people will trust their computations more and stop relying on averages.

“As it gets easier, our prediction is the bias will go away, because that computation is no longer a noisy computation,” Jazayeri says. “You believe in the computation; you know the computation is working well.”

The researchers now plan to further study whether people’s biases decrease as they learn to perform a complicated task better. In the experiments they performed for the Nature Communications study, they found some preliminary evidence that trained musicians performed better in a task that involved producing time intervals of a specific duration.

The research was funded by the Alfred P. Sloan Foundation, the Esther A. and Joseph Klingenstein Fund, the Simons Foundation, the McKnight Endowment Fund for Neuroscience, and the McGovern Institute.

How the brain performs flexible computations

Anne Trafton |

Humans can perform a vast array of mental operations and adjust their behavioral responses based on external instructions and internal beliefs. For example, to tap your feet to a musical beat, your brain has to process the incoming sound and also use your internal knowledge of how the song goes.

MIT neuroscientists have now identified a strategy that the brain uses to rapidly select and flexibly perform different mental operations. To make this discovery, they applied a mathematical framework known as dynamical systems analysis to understand the logic that governs the evolution of neural activity across large populations of neurons.

“The brain can combine internal and external cues to perform novel computations on the fly,” says Mehrdad Jazayeri, the Robert A. Swanson Career Development Professor of Life Sciences, a member of MIT’s McGovern Institute for Brain Research, and the senior author of the study. “What makes this remarkable is that we can make adjustments to our behavior at a much faster time scale than the brain’s hardware can change. As it turns out, the same hardware can assume many different states, and the brain uses instructions and beliefs to select between those states.”

Previous work from Jazayeri’s group has found that the brain can control when it will initiate a movement by altering the speed at which patterns of neural activity evolve over time. Here, they found that the brain controls this speed flexibly based on two factors: external sensory inputs and adjustment of internal states, which correspond to knowledge about the rules of the task being performed.

Evan Remington, a McGovern Institute postdoc, is the lead author of the paper, which appears in the June 6 edition of Neuron. Other authors are former postdoc Devika Narain and MIT graduate student Eghbal Hosseini.

Ready, set, go

Neuroscientists believe that “cognitive flexibility,” or the ability to rapidly adapt to new information, resides in the brain’s higher cortical areas, but little is known about how the brain achieves this kind of flexibility.

To understand the new findings, it is useful to think of how switches and dials can be used to change the output of an electrical circuit. For example, in an amplifier, a switch may select the sound source by controlling the input to the circuit, and a dial may adjust the volume by controlling internal parameters such as a variable resistance. The MIT team theorized that the brain similarly transforms instructions and beliefs to inputs and internal states that control the behavior of neural circuits.

To test this, the researchers recorded neural activity in the frontal cortex of animals trained to perform a flexible timing task called “ready, set, go.” In this task, the animal sees two visual flashes — “ready” and “set” — that are separated by an interval anywhere between 0.5 and 1 second, and initiates a movement — “go” — some time after “set.” The animal has to initiate the movement such that the “set-go” interval is either the same as or 1.5 times the “ready-set” interval. The instruction for whether to use a multiplier of 1 or 1.5 is provided in each trial.

Neural signals recorded during the “set-go” interval clearly carried information about both the multiplier and the measured length of the “ready-set” interval, but the nature of these representations seemed bewilderingly complex. To decode the logic behind these representations, the researchers used the dynamical systems analysis framework. This analysis is used in the study of a wide range of physical systems, from simple electrical circuits to space shuttles.

The application of this approach to neural data in the “ready, set, go” task enabled Jazayeri and his colleagues to discover how the brain adjusts the inputs to and initial conditions of frontal cortex to control movement times flexibly. A switch-like operation sets the input associated with the correct multiplier, and a dial-like operation adjusts the state of neurons based on the “ready-set” interval. These two complementary control strategies allow the same hardware to produce different behaviors.

David Sussillo, a research scientist at Google Brain and an adjunct professor at Stanford University, says a key to the study was the research team’s development of new mathematical tools to analyze huge amounts of data from neuron recordings, allowing the researchers to uncover how a large population of neurons can work together to perform mental operations related to timing and rhythm.

“They have very rigorously brought the dynamical systems approach to the problem of timing,” says Sussillo, who was not involved in the research.

“A bridge between behavior and neurobiology”

Many unanswered questions remain about how the brain achieves this flexibility, the researchers say. They are now trying to find out what part of the brain sends information about the multiplier to the frontal cortex, and they also hope to study what happens in these neurons as they first learn tasks that require them to respond flexibly.

“We haven’t connected all the dots from behavioral flexibility to neurobiological details. But what we have done is to establish an algorithmic understanding based on the mathematics of dynamical systems that serves as a bridge between behavior and neurobiology,” Jazayeri says.

The researchers also hope to explore whether this type of model could help to explain behavior of other parts of the brain that have to perform computations flexibly.

The research was funded by the National Institutes of Health, the Sloan Foundation, the Klingenstein Foundation, the Simons Foundation, the McKnight Foundation, the Center for Sensorimotor Neural Engineering, and the McGovern Institute.

Study reveals how the brain tracks objects in motion

Anne Trafton |

Catching a bouncing ball or hitting a ball with a racket requires estimating when the ball will arrive. Neuroscientists have long thought that the brain does this by calculating the speed of the moving object. However, a new study from MIT shows that the brain’s approach is more complex.

The new findings suggest that in addition to tracking speed, the brain incorporates information about the rhythmic patterns of an object’s movement: for example, how long it takes a ball to complete one bounce. In their new study, the researchers found that people make much more accurate estimates when they have access to information about both the speed of a moving object and the timing of its rhythmic patterns.

“People get really good at this when they have both types of information available,” says Mehrdad Jazayeri, the Robert A. Swanson Career Development Professor of Life Sciences and a member of MIT’s McGovern Institute for Brain Research. “It’s like having input from multiple senses. The statistical knowledge that we have about the world we’re interacting with is richer when we use multiple senses.”

Jazayeri is the senior author of the study, which appears in the Proceedings of the National Academy of Sciences the week of March 5. The paper’s lead author is MIT graduate student Chia-Jung Chang.

Objects in motion

Much of the information we process about objects moving around us comes from visual tracking of the objects. Our brains can use information about an object’s speed and the distance it has to cover to calculate when it will reach a certain point. Jazayeri, who studies how the brain keeps time, was intrigued by the fact that much of the movement we see also has a rhythmic element, such as the bouncing of a ball.

“It occurred to us to ask, how can it be that the brain doesn’t use this information? It would seem very strange if all this richness of additional temporal structure is not part of the way we evaluate where things are around us and how things are going to happen,” Jazayeri says.

There are many other sensory processing tasks for which the brain uses multiple sources of input. For example, to interpret language, we use both the sound we hear and the movement of the speaker’s lips, if we can see them. When we touch an object, we estimate its size based on both what we see and what we feel with our fingers.

In the case of perceiving object motion, teasing out the role of rhythmic timing, as opposed to speed, can be difficult. “I can ask someone to do a task, but then how do I know if they’re using speed or they’re using time, if both of them are always available?” Jazayeri says.

To overcome that, the researchers devised a task in which they could control how much timing information was available. They measured performance in human volunteers as they performed the task.

During the task, the study participants watched a ball as it moved in a straight line. After traveling some distance, the ball went behind an obstacle, so the participants could no longer see it. They were asked to press a button at the time when they expected the ball to reappear.

Performance varied greatly depending on how much of the ball’s path was visible before it went behind the obstacle. If the participants saw the ball travel a very short distance before disappearing, they did not do well. As the distance before disappearance became longer, they were better able to calculate the ball’s speed, so their performance improved but eventually plateaued.

After that plateau, there was a significant jump in performance when the distance before disappearance grew until it was exactly the same as the width of the obstacle. In that case, when the path seen before disappearance was equal to the path the ball traveled behind the obstacle, the participants improved dramatically, because they knew that the time spent behind the obstacle would be the same as the time it took to reach the obstacle.

When the distance traveled to reach the obstacle became longer than the width of the obstacle, performance dropped again.

“It’s so important to have this extra information available, and when we have it, we use it,” Jazayeri says. “Temporal structure is so important that when you lose it, even at the expense of getting better visual information, people’s performance gets worse.”

Integrating information

The researchers also tested several computer models of how the brain performs this task, and found that the only model that could accurately replicate their experimental results was one in which the brain measures speed and timing in two different areas and then combines them.

Previous studies suggest that the brain performs timing estimates in premotor areas of the cortex, which plays a role in planning movement; speed, which usually requires visual input, is calculated in visual cortex. These inputs are likely combined in parts of the brain responsible for spatial attention and tracking objects in space, which occurs in the parietal cortex, Jazayeri says.

In future studies, Jazayeri hopes to measure brain activity in animals trained to perform the same task that human subjects did in this study. This could shed further light on where this processing takes place and could also reveal what happens in the brain when it makes incorrect estimates.

The research was funded by the McGovern Institute for Brain Research.

How the brain keeps time

Anne Trafton |

Timing is critical for playing a musical instrument, swinging a baseball bat, and many other activities. Neuroscientists have come up with several models of how the brain achieves its exquisite control over timing, the most prominent being that there is a centralized clock, or pacemaker, somewhere in the brain that keeps time for the entire brain.

However, a new study from MIT researchers provides evidence for an alternative timekeeping system that relies on the neurons responsible for producing a specific action. Depending on the time interval required, these neurons compress or stretch out the steps they take to generate the behavior at a specific time.

“What we found is that it’s a very active process. The brain is not passively waiting for a clock to reach a particular point,” says Mehrdad Jazayeri, the Robert A. Swanson Career Development Professor of Life Sciences, a member of MIT’s McGovern Institute for Brain Research, and the senior author of the study.

MIT postdoc Jing Wang and former postdoc Devika Narain are the lead authors of the paper, which appears in the Dec. 4 issue of Nature Neuroscience. Graduate student Eghbal Hosseini is also an author of the paper.

Flexible control

One of the earliest models of timing control, known as the clock accumulator model, suggested that the brain has an internal clock or pacemaker that keeps time for the rest of the brain. A later variation of this model suggested that instead of using a central pacemaker, the brain measures time by tracking the synchronization between different brain wave frequencies.

Although these clock models are intuitively appealing, Jazayeri says, “they don’t match well with what the brain does.”

No one has found evidence for a centralized clock, and Jazayeri and others wondered if parts of the brain that control behaviors that require precise timing might perform the timing function themselves. “People now question why would the brain want to spend the time and energy to generate a clock when it’s not always needed. For certain behaviors you need to do timing, so perhaps the parts of the brain that subserve these functions can also do timing,” he says.

To explore this possibility, the researchers recorded neuron activity from three brain regions in animals as they performed a task at two different time intervals — 850 milliseconds or 1,500 milliseconds.

The researchers found a complicated pattern of neural activity during these intervals. Some neurons fired faster, some fired slower, and some that had been oscillating began to oscillate faster or slower. However, the researchers’ key discovery was that no matter the neurons’ response, the rate at which they adjusted their activity depended on the time interval required.

At any point in time, a collection of neurons is in a particular “neural state,” which changes over time as each individual neuron alters its activity in a different way. To execute a particular behavior, the entire system must reach a defined end state. The researchers found that the neurons always traveled the same trajectory from their initial state to this end state, no matter the interval. The only thing that changed was the rate at which the neurons traveled this trajectory.

When the interval required was longer, this trajectory was “stretched,” meaning the neurons took more time to evolve to the final state. When the interval was shorter, the trajectory was compressed.

“What we found is that the brain doesn’t change the trajectory when the interval changes, it just changes the speed with which it goes from the initial internal state to the final state,” Jazayeri says.

Dean Buonomano, a professor of behavioral neuroscience at the University of California at Los Angeles, says that the study “provides beautiful evidence that timing is a distributed process in the brain — that is, there is no single master clock.”

“This work also supports the notion that the brain does not tell time using a clock-like mechanism, but rather relies on the dynamics inherent to neural circuits, and that as these dynamics increase and decrease in speed, animals move more quickly or slowly,” adds Buonomano, who was not involved in the research.

Neural networks

The researchers focused their study on a brain loop that connects three regions: the dorsomedial frontal cortex, the caudate, and the thalamus. They found this distinctive neural pattern in the dorsomedial frontal cortex, which is involved in many cognitive processes, and the caudate, which is involved in motor control, inhibition, and some types of learning. However, in the thalamus, which relays motor and sensory signals, they found a different pattern: Instead of altering the speed of their trajectory, many of the neurons simply increased or decreased their firing rate, depending on the interval required.

Jazayeri says this finding is consistent with the possibility that the thalamus is instructing the cortex on how to adjust its activity to generate a certain interval.

The researchers also created a computer model to help them further understand this phenomenon. They began with a model of hundreds of neurons connected together in random ways, and then trained it to perform the same interval-producing task they had used to train animals, offering no guidance on how the model should perform the task.

They found that these neural networks ended up using the same strategy that they observed in the animal brain data. A key discovery was that this strategy only works if some of the neurons have nonlinear activity — that is, the strength of their output doesn’t constantly increase as their input increases. Instead, as they receive more input, their output increases at a slower rate.

Jazayeri now hopes to explore further how the brain generates the neural patterns seen during varying time intervals, and also how our expectations influence our ability to produce different intervals.

The research was funded by the Rubicon Grant from the Netherlands Scientific Organization, the National Institutes of Health, the Sloan Foundation, the Klingenstein Foundation, the Simons Foundation, the Center for Sensorimotor Neural Engineering, and the McGovern Institute.

A sense of timing

by Charles Jennings |

The ability to measure time and to control the timing of actions is critical for almost every aspect of behavior. Yet the mechanisms by which our brains process time are still largely mysterious.

We experience time on many different scales—from milliseconds to years— but of particular interest is the middle range, the scale of seconds over which we perceive time directly, and over which many of our actions and thoughts unfold.

“We speak of a sense of time, yet unlike our other senses there is no sensory organ for time,” says McGovern Investigator Mehrdad Jazayeri. “It seems to come entirely from within. So if we understand time, we should be getting close to understanding mental processes.”

Singing in the brain

Emily Mackevicius comes to work in the early morning because that’s when her birds are most likely to sing. A graduate student in the lab of McGovern Investigator Michale Fee, she is studying zebra finches, songbirds that learn to sing by copying their fathers. Bird song involves a complex and precisely timed set of movements, and Mackevicius, who plays the cello in her spare time, likens it to musical performance. “With every phrase, you have to learn a sequence of finger movements and bowing movements, and put it all together with exact timing. The birds are doing something very similar with their vocal muscles.”

A typical zebra finch song lasts about one second, and consists of several syllables, produced at a rate similar to the syllables in human speech. Each song syllable involves a precisely timed sequence of muscle commands, and understanding how the bird’s brain generates this sequence is a central goal for Fee’s lab. Birds learn it naturally without any need for training, making it an ideal model for understanding the complex action sequences that represent the fundamental “building blocks” of behavior.

Some years ago Fee and colleagues made a surprising discovery that has shaped their thinking ever since. Within a part of the bird brain called HVC, they found neurons that fire a single short burst of pulses at exactly the same point on every repetition of the song. Each burst lasts about a hundredth of a second, and different neurons fire at different times within the song. With about 20,000 neurons in HVC, it was easy to imagine that there would be specific neurons active at every point in the song, meaning that each time point could be represented by the activity of a handful of individual neurons.

Proving this was not easy—“we had to wait about ten years for the technology to catch up,” says Fee—but they finally succeeded last year, when students Tatsuo Okubo and Galen Lynch analyzed recordings from hundreds of individual HVC neurons, and found that they do indeed fire in a fixed sequence, covering the entire song period.

“We think it’s like a row of falling dominoes,” says Fee. “The neurons are connected to each other so that when one fires it triggers the next one in the chain.” It’s an appealing model, because it’s easy to see how a chain of activity could control complex action sequences, simply by connecting individual time-stamp neurons to downstream motor neurons. With the correct connections, each movement is triggered at the right time in the sequence. Fee believes these motor connections are learned through trial and error—like babies babbling as they learn to speak—and a separate project in his lab aims to understand how this learning occurs.

But the domino metaphor also begs another question: who sets up the dominoes in the first place? Mackevicius and Okubo, along with summer student Hannah Payne, set out to answer this question, asking how HVC becomes wired to produce these precisely timed chain reactions.

Mackevicius, who studied math as an undergraduate before turning to neuroscience, developed computer simulations of the HVC neuronal network, and Okubo ran experiments to test the predictions, recording from young birds at different stages in the learning process. “We found that setting up a chain is surprisingly easy,” says Mackevicius. “If we start with a randomly connected network, and some realistic assumptions about the “plasticity rules” by which synapses change with repeated use, we found that these chains emerge spontaneously. All you need is to give them a push—like knocking over the first domino.”

Their results also suggested how a young bird learns to produce different syllables, as it progresses from repetitive babbling to a more adult-like song. “At first, there’s just one big burst of neural activity, but as the song becomes more complex, the activity gradually spreads out in time and splits into different sequences, each controlling a different syllable. It’s as if you started with lots of dominos all clumped together, and then gradually they become sorted into different rows.”

Does something similar happen in the human brain? “It seems very likely,” says Fee. “Many of our movements are precisely timed—think about speaking a sentence or performing a musical instrument or delivering a tennis serve. Even our thoughts often happen in sequences. Things happen faster in birds than mammals, but we suspect the underlying mechanisms will be very similar.”

Speed control

One floor above the Fee lab, Mehrdad Jazayeri is also studying how time controls actions, using humans and monkeys rather than birds. Like Fee, Jazayeri comes from an engineering background, and his goal is to understand, with an engineer’s level of detail, how we perceive time and use it flexibly to control our actions.

To begin to answer this question, Jazayeri trained monkeys to remember time intervals of a few seconds or less, and to reproduce them by pressing a button or making an eye movement at the correct time after a visual cue appears on a screen. He then recorded brain activity as the monkeys perform this task, to find out how the brain measures elapsed time. “There were two prominent ideas in the field,” he explains. “One idea was that there is an internal clock, and that the brain can somehow count the accumulating ticks. Another class of models had proposed that there are multiple oscillators that come in and out of phase at different times.”

When they examined the recordings, however, the results did not fit either model. Despite searching across multiple brain areas, Jazayeri and his colleagues found no sign of ticking or oscillations. Instead, their recordings revealed complex patterns of activity, distributed across populations of neurons; moreover, as the monkey produced longer or shorter intervals, these activity patterns were stretched or compressed in time, to fit the overall duration of each interval. In other words, says Jazayeri, the brain circuits were able to adjust the speed with which neural signals evolve over time. He compares it to a group of musicians performing a complex piece of music. “Each player has their own part, which they can play faster or slower depending on the overall tempo of the music.”

Ready-set-go

Jazayeri is also using time as a window onto a broader question—how our perceptions and decisions are shaped by past experience. “It’s one of the great questions in neuroscience, but it’s not easy to study. One of the great advantages of studying timing is that it’s easy to measure precisely, so we can frame our questions in precise mathematical ways.”

The starting point for this work was a deceptively simple task, which Jazayeri calls “Ready-Set-Go.” In this task, the subject is given the first two beats of a regular rhythm (“Ready, Set”) and must then generate the third beat (“Go”) at the correct time. To perform this task, the brain must measure the duration between Ready and Set and then immediately reproduce it.

Humans can do this fairly accurately, but not perfectly—their response times are imprecise, presumably because there is some “noise” in the neural signals that convey timing information within the brain. In the face of this uncertainty, the optimal strategy (known mathematically as Bayesian Inference) is to bias the time estimates based on prior expectations, and this is exactly what happened in Jazayeri’s experiments. If the intervals in previous trials were shorter, then people tend to under-estimate the next interval, whereas if the previous intervals were longer, they will over-estimate. In other words, people use their memory to improve their time estimates.

Monkeys can also learn this task and show similar biases, providing an opportunity to study how the brain establishes and stores these prior expectations, and how these expectations influence subsequent behavior. Again, Jazayeri and colleagues recorded from large numbers of neurons during the task. These patterns are complex and not easily described in words, but in mathematical terms, the activity forms a geometric structure known as a manifold. “Think of it as a curved surface, analogous to a cylinder,” he says. “In the past, people could not see it because they could only record from one or a few neurons at a time. We have to measure activity across large numbers of neurons simultaneously if we want to understand the workings of the system.”

Computing time

To interpret their data, Jazayeri and his team often turn to computer models based on artificial neural networks. “These models are a powerful tool in our work because we can fully reverse-engineer them and gain insight into the underlying mechanisms,” he explains. His lab has now succeeded in training a recurrent neural network that can perform the Ready-Set-Go task, and they have found that the model develops a manifold similar to the real brain data. This has led to the intriguing conjecture that memory of past experiences can be embedded in the structure of the manifold.

Jazayeri concludes: “We haven’t connected all the dots, but I suspect that many questions about brain and behavior will find their answers in the geometry and dynamics of neural activity.” Jazayeri’s long-term ambition is to develop predictive models of brain function. As an analogy, he says, think of a pendulum. “If we know its current state—its position and speed—we can predict with complete confidence what it will do next, and how it will respond to a perturbation. We don’t have anything like that for the brain—nobody has been able to do that, not even the simplest brain functions. But that’s where we’d eventually like to be.”

A clock within the brain?

It is not yet clear how the mechanisms studied by Fee and Jazayeri are related. “We talk together often, but we are still guessing how the pieces fit together,” says Fee. But one thing they both agree on is the lack of evidence for any central clock within the brain. “Most people have this intuitive feeling that time is a unitary thing, and that there must be some central clock inside our head, coordinating everything like the conductor of the orchestra or the clock inside your computer,” says Jazayeri. “Even many experts in the field believe this, but we don’t think it’s right.” Rather, his work and Fee’s both point to the existence of separate circuits for different time-related behaviors, such as singing. If there is no clock, how do the different systems work together to create our apparently seamless perception of time? “It’s still a big mystery,” says Jazayeri. “Questions like that are what make neuroscience so interesting.”

 

How the brain keeps time

Anne Trafton |

Keeping track of time is critical for many tasks, such as playing the piano, swinging a tennis racket, or holding a conversation. Neuroscientists at MIT and Columbia University have now figured out how neurons in one part of the brain measure time intervals and accurately reproduce them.

The researchers found the lateral intraparietal cortex (LIP), which plays a role in sensorimotor function, represents elapsed time, as animals measure and then reproduce a time interval. They also demonstrated how the firing patterns of population of neurons in the LIP could coordinate sensory and motor aspects of timing.

LIP is likely just one node in a circuit that measures time, says Mehrdad Jazayeri, the lead author of a paper describing the work in the Oct. 8 issue of Current Biology.

“I would not conclude that the parietal cortex is the timer,” says Jazayeri, an assistant professor of brain and cognitive sciences at MIT and a member of the McGovern Institute for Brain Research. “What we are doing is discovering computational principles that explain how neurons’ firing rates evolve with time, and how that relates to the animals’ behavior in single trials. We can explain mathematically what’s going on.”

The paper’s senior author is Michael Shadlen, a professor of neuroscience and member of the Mortimer B. Zuckerman Mind Brain Behavior Institute at Columbia University.

As time goes by

Jazayeri, who joined the MIT faculty in 2013, began studying timing in the brain several years ago while a postdoc at the University of Washington. He began by testing humans’ ability to measure and reproduce time using a task called “ready, set, go.” In this experiment, the subject measures the time between two flashes (“ready” and “set”) and then presses a button (“go”) at the appropriate time — that is, after the same amount of time that separated the “ready” and “set.”

From these studies, he discovered that people do not simply measure an interval and then reproduce it. Rather, after measuring an interval they combine that measurement, which is imprecise, with their prior knowledge of what the interval could have been. This prior knowledge, which builds up as they repeat the task many times, allows people to reproduce the interval more accurately.

“When people reproduce time, they don’t seem to use a timer,” Jazayeri says. “It’s an active act of probabilistic inference that goes on.”

To find out what happens in the brain during this process, Jazayeri recorded neuronal activity in the LIP of monkeys trained to perform the same task. In these recordings, he found distinctive patterns in the measurement phase (the interval between “ready” and “set”), and the production phase (the interval between “set” and “go”).

During the measurement phase, neuron activity increases, but not linearly. Instead, the slope of activity begins as a steep curve that gradually flattens out as time goes by, until the “set” signal is given. This is key because the slope at the end of the measurement interval predicts the slope of activity in the production phase.

When the interval is short, the slope during the second phase is steep. This allows the activity to increase quickly so that the animal can produce a short interval. When the interval is longer, the slope is gentler and it takes longer to reach the time of response.

“As time goes by during the measurement, the animal knows that the interval that it has to produce is longer and therefore requires a shallower slope,” Jazayeri says.

Using this data, the researchers could correctly predict, based on the slope at the end of the measurement phase, when the animal would produce the “go” signal.

“Previous research has shown that some neurons exhibit a ramping up of their firing rate that culminates with the onset of a timed motor response. This research is exciting because it provides the first hint as to what may control the slope of this ‘neural ramping,’ specifically that the slope of the ramp may be determined by the firing rate at the beginning of the timed interval,” says Dean Buonomano, a professor of behavioral neuroscience at the University of California at Los Angeles who was not involved in the research.

“A highly distributed problem”

All cognitive and motor functions rely on time to some extent. While LIP represents time during interval reproduction, Jazayeri believes that tracking time occurs throughout brain circuits that connect subcortical structures such as the thalamus, basal ganglia, and cerebellum to the cortex.

“Timing is going to be a highly distributed problem for the brain. There’s not going to be one place in the brain that does timing,” he says.

His lab is now pursuing several questions raised by this study. In one follow-up, the researchers are investigating how animals’ behavior and brain activity change based on their expectations for how long the first interval will last.

In another experiment, they are training animals to reproduce an interval that they get to measure twice. Preliminary results suggest that during the second interval, the animals refine the measurement they took during the first interval, allowing them to perform better than when they make just one measurement.