Research Area: Cognitive Neuroscience

​perception, memory, language, social cognition, attention, empathy, brain disorders, aging, cognitive flexibility, developing brain

Computational model mimics humans’ ability to predict emotions

Anne Trafton |

When interacting with another person, you likely spend part of your time trying to anticipate how they will feel about what you’re saying or doing. This task requires a cognitive skill called theory of mind, which helps us to infer other people’s beliefs, desires, intentions, and emotions.

MIT neuroscientists have now designed a computational model that can predict other people’s emotions — including joy, gratitude, confusion, regret, and embarrassment — approximating human observers’ social intelligence. The model was designed to predict the emotions of people involved in a situation based on the prisoner’s dilemma, a classic game theory scenario in which two people must decide whether to cooperate with their partner or betray them.

To build the model, the researchers incorporated several factors that have been hypothesized to influence people’s emotional reactions, including that person’s desires, their expectations in a particular situation, and whether anyone was watching their actions.

“These are very common, basic intuitions, and what we said is, we can take that very basic grammar and make a model that will learn to predict emotions from those features,” says Rebecca Saxe, the John W. Jarve Professor of Brain and Cognitive Sciences, a member of MIT’s McGovern Institute for Brain Research, and the senior author of the study.

Sean Dae Houlihan PhD ’22, a postdoc at the Neukom Institute for Computational Science at Dartmouth College, is the lead author of the paper, which appears today in Philosophical Transactions A. Other authors include Max Kleiman-Weiner PhD ’18, a postdoc at MIT and Harvard University; Luke Hewitt PhD ’22, a visiting scholar at Stanford University; and Joshua Tenenbaum, a professor of computational cognitive science at MIT and a member of the Center for Brains, Minds, and Machines and MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL).

Predicting emotions

While a great deal of research has gone into training computer models to infer someone’s emotional state based on their facial expression, that is not the most important aspect of human emotional intelligence, Saxe says. Much more important is the ability to predict someone’s emotional response to events before they occur.

“The most important thing about what it is to understand other people’s emotions is to anticipate what other people will feel before the thing has happened,” she says. “If all of our emotional intelligence was reactive, that would be a catastrophe.”

To try to model how human observers make these predictions, the researchers used scenarios taken from a British game show called “Golden Balls.” On the show, contestants are paired up with a pot of $100,000 at stake. After negotiating with their partner, each contestant decides, secretly, whether to split the pool or try to steal it. If both decide to split, they each receive $50,000. If one splits and one steals, the stealer gets the entire pot. If both try to steal, no one gets anything.

Depending on the outcome, contestants may experience a range of emotions — joy and relief if both contestants split, surprise and fury if one’s opponent steals the pot, and perhaps guilt mingled with excitement if one successfully steals.

To create a computational model that can predict these emotions, the researchers designed three separate modules. The first module is trained to infer a person’s preferences and beliefs based on their action, through a process called inverse planning.

“This is an idea that says if you see just a little bit of somebody’s behavior, you can probabilistically infer things about what they wanted and expected in that situation,” Saxe says.

Using this approach, the first module can predict contestants’ motivations based on their actions in the game. For example, if someone decides to split in an attempt to share the pot, it can be inferred that they also expected the other person to split. If someone decides to steal, they may have expected the other person to steal, and didn’t want to be cheated. Or, they may have expected the other person to split and decided to try to take advantage of them.

The model can also integrate knowledge about specific players, such as the contestant’s occupation, to help it infer the players’ most likely motivation.

The second module compares the outcome of the game with what each player wanted and expected to happen. Then, a third module predicts what emotions the contestants may be feeling, based on the outcome and what was known about their expectations. This third module was trained to predict emotions based on predictions from human observers about how contestants would feel after a particular outcome. The authors emphasize that this is a model of human social intelligence, designed to mimic how observers causally reason about each other’s emotions, not a model of how people actually feel.

“From the data, the model learns that what it means, for example, to feel a lot of joy in this situation, is to get what you wanted, to do it by being fair, and to do it without taking advantage,” Saxe says.

Core intuitions

Once the three modules were up and running, the researchers used them on a new dataset from the game show to determine how the models’ emotion predictions compared with the predictions made by human observers. This model performed much better at that task than any previous model of emotion prediction.

The model’s success stems from its incorporation of key factors that the human brain also uses when predicting how someone else will react to a given situation, Saxe says. Those include computations of how a person will evaluate and emotionally react to a situation, based on their desires and expectations, which relate to not only material gain but also how they are viewed by others.

“Our model has those core intuitions, that the mental states underlying emotion are about what you wanted, what you expected, what happened, and who saw. And what people want is not just stuff. They don’t just want money; they want to be fair, but also not to be the sucker, not to be cheated,” she says.

“The researchers have helped build a deeper understanding of how emotions contribute to determining our actions; and then, by flipping their model around, they explain how we can use people’s actions to infer their underlying emotions. This line of work helps us see emotions not just as ‘feelings’ but as playing a crucial, and subtle, role in human social behavior,” says Nick Chater, a professor of behavioral science at the University of Warwick, who was not involved in the study.

In future work, the researchers hope to adapt the model so that it can perform more general predictions based on situations other than the game-show scenario used in this study. They are also working on creating models that can predict what happened in the game based solely on the expression on the faces of the contestants after the results were announced.

The research was funded by the McGovern Institute; the Paul E. and Lilah Newton Brain Science Award; the Center for Brains, Minds, and Machines; the MIT-IBM Watson AI Lab; and the Multidisciplinary University Research Initiative.

Real-time feedback helps adolescents with depression quiet the mind

by Jennifer Michalowski |

Real-time feedback about brain activity can help adolescents with depression or anxiety quiet their minds, according to a new study from MIT scientists. The researchers, led by McGovern research affiliate Susan Whitfield-Gabrieli, have used functional magnetic resonance imaging (fMRI) to show patients what’s happening in their brain as they practice mindfulness inside the scanner and to encourage them to focus on the present. They report in the journal Molecular Psychiatry that doing so settles down neural networks that are associated with symptoms of depression.

McGovern research affiliate Susan Whitfield-Gabrieli in the Martinos Imaging Center.

“We know this mindfulness meditation is really good for kids and teens, and we think this real-time fMRI neurofeedback is really a way to engage them and provide a visual representation of how they’re doing,” says Whitfield-Gabrieli. “And once we train people how to do mindfulness meditation, they can do it on their own at any time, wherever they are.”

The approach could be a valuable tool to alleviate or prevent depression in young people, which has been on the rise in recent years and escalated alarmingly during the Covid-19 pandemic. “This has gone from bad to catastrophic, in my perspective,” Whitfield-Gabrieli says. “We have to think out of the box and come up some really innovative ways to help.”

Default mode network

Mindfulness meditation, in which practitioners focus their awareness on the present moment, can modulate activity within the brain’s default mode network, which is so named because it is most active when a person is not focused on any particular task. Two hubs within the default mode network, the medial prefrontal cortex and the posterior cingulate cortex, are of particular interest to Whitfield-Gabrieli and her colleagues, due to a potential role in the symptoms of depression and anxiety.

“These two core hubs are very engaged when we’re thinking about the past or the future and we’re not really engaged in the present moment,” she explains. “If we’re in a healthy state of mind, we may be reminiscing about the past or planning for the future. But if we’re depressed, that reminiscing may turn into rumination or obsessively rehashing the past. If we’re particularly anxious, we may be obsessively worrying about the future.”

Whitfield-Gabrieli explains that these key hubs are often hyperconnected in people with anxiety and depression. The more tightly correlated the activity of the two regions are, the worse a person’s symptoms are likely to be. Mindfulness, she says, can help interrupt that hyperconnectivity.

“Mindfulness really helps to focus on the now, which just precludes all of this mind wandering and repetitive negative thinking,” she explains. In fact, she and her colleagues have found that mindfulness practice can reduce stress and improve attention in children. But she acknowledges that it can be difficult to engage young people and help them focus on the practice.

Tuning the mind

To help people visualize the benefits of their mindfulness practice, the researchers developed a game that can be played while an MRI scanner tracks a person’s brain activity. On a screen inside the scanner, the participant sees a ball and two circles. The circle at the top of the screen represents a desirable state in which the activity of the brain’s default mode network has been reduced, and the activity of a network the brain uses to focus on attention-demanding tasks—the frontal parietal network—has increased. An initial fMRI scan identifies these networks in each individual’s brain, creating a customized mental map on which the game is based.

“They’re training their brain to tune their mind. And they love it.” – Susan Whitfield-Gabrieli

As the person practices mindfulness meditation, which they learn prior to entering the scanner, the default mode network in the brain quiets while the frontal parietal mode activates. When the scanner detects this change, the ball moves and eventually enters its target. With an initial success, the target shrinks, encouraging even more focus. When the participant’s mind wanders from their task, the default mode network activation increases (relative to the frontal parietal network) and the ball moves down towards the second circle, which represents an undesirable state. “Basically, they’re just moving this ball with their brain,” Whitfield-Gabrieli says. “They’re training their brain to tune their mind. And they love it.”

Nine individuals between the ages of 17 and 19 with a history of major depression or anxiety disorders tried this new approach to mindfulness training, and for each of them, Whitfield-Gabrieli’s team saw a reduction in connectivity within the default mode network. Now they are working to determine whether an electroencephalogram, in which brain activity is measured with noninvasive electrodes, can be used to provide similar neurofeedback during mindfulness training—an approach that could be more accessible for broad clinical use.

Whitfield-Gabrieli notes that hyperconnectivity in the default mode network is also associated with psychosis, and she and her team have found that mindfulness meditation with real-time fMRI feedback can help reduce symptoms in adults with schizophrenia. Future studies are planned to investigate how the method impacts teens’ ability to establish a mindfulness practice and its potential effects on depression symptoms.

2023 MacVicar Faculty Fellows named

Anne Trafton |

The Office of the Vice Chancellor and the Registrar’s Office have announced this year’s Margaret MacVicar Faculty Fellows: professor of brain and cognitive sciences John Gabrieli, associate professor of literature Marah Gubar, professor of biology Adam C. Martin, and associate professor of architecture Lawrence “Larry” Sass.

For more than 30 years, the MacVicar Faculty Fellows Program has recognized exemplary and sustained contributions to undergraduate education at MIT. The program is named in honor of Margaret MacVicar, the first dean for undergraduate education and founder of the Undergraduate Research Opportunities Program (UROP). New fellows are chosen every year through a competitive nomination process that includes submission of letters of support from colleagues, students, and alumni; review by an advisory committee led by the vice chancellor; and a final selection by the provost. Fellows are appointed to a 10-year term and receive $10,000 per year of discretionary funds.

Gabrieli, Gubar, Martin, and Sass join an elite group of more than 130 scholars from across the Institute who are committed to curricular innovation, excellence in teaching, and supporting students both in and out of the classroom.

John Gabrieli

“When I learned of this wonderful honor, I felt gratitude — for how MIT values teaching and learning, how my faculty colleagues bring such passion to their teaching, and how the students have such great curiosity for learning,” says new MacVicar Fellow John Gabrieli.

Gabrieli PhD ’87 received a bachelor’s degree in English from Yale University and his PhD in behavioral neuroscience from MIT. He is the Grover M. Hermann Professor in the Department of Brain and Cognitive sciences. Gabrieli is also an investigator in the McGovern Institute for Brain Research and the founding director of the MIT Integrated Learning Initiative (MITili). He holds appointments in the Department of Psychiatry at Massachusetts General Hospital and the Harvard Graduate School of Education, and studies the organization of memory, thought, and emotion in the human brain.

He joined Course 9 as a professor in 2005 and since then, he has taught over 3,000 undergraduates through the department’s introductory course, 9.00 (Introduction to Psychological Science). Gabrieli was recognized with departmental awards for excellence in teaching in 2009, 2012, and 2015. Highly sought after by undergraduate researchers, the Gabrieli Laboratory (GabLab) hosts five to 10 UROPs each year.

A unique element of Gabrieli’s classes is his passionate, hands-on teaching style and his use of interactive demonstrations, such as optical illusions and personality tests, to help students grasp some of the most fundamental topics in psychology.

His former teaching assistant Daniel Montgomery ’22 writes, “I was impressed by his enthusiasm and ability to keep students engaged throughout the lectures … John clearly has a desire to help students become excited about the material he’s teaching.”

Senior Elizabeth Carbonell agrees: “The excitement professor Gabrieli brought to lectures by starting with music every time made the classroom an enjoyable atmosphere conducive to learning … he always found a way to make every lecture relatable to the students, teaching psychological concepts that would shine a light on our own human emotions.”

Lecturer and 9.00 course coordinator Laura Frawley says, “John constantly innovates … He uses research-based learning techniques in his class, including blended learning, active learning, and retrieval practice.” His findings on blended learning resulted in two MITx offerings including 9.00x (Learning and Memory), which utilizes a nontraditional approach to assignments and exams to improve how students retrieve and remember information.

In addition, he is known for being a devoted teacher who believes in caring for the student as a whole. Through MITili’s Mental Wellness Initiative, Gabrieli, along with a compassionate team of faculty and staff, are working to better understand how mental health conditions impact learning.

Associate department head and associate professor of brain and cognitive sciences Josh McDermott calls him “an exceptional educator who has left his mark on generations of MIT undergraduate students with his captivating, innovative, and thoughtful approach to teaching.”

Mariana Gomez de Campo ’20 concurs: “There are certain professors that make their mark on students’ lives; professor Gabrieli permanently altered the course of mine.”

Laura Schulz, MacVicar Fellow and associate department head of brain and cognitive sciences, remarks, “His approach is visionary … John’s manner with students is unfailingly gracious … he hastens to remind them that they are as good as it gets, the smartest and brightest of their generation … it is the kind of warm, welcoming, inclusive approach to teaching that subtly but effectively reminds students that they belong here at MIT … It is little wonder that they love him.”

Marah Gubar

Marah Gubar joined MIT as an associate professor of literature in 2014. She received her BA in English literature from the University of Michigan at Ann Arbor and a PhD from Princeton University. Gubar taught in the English department at the University of Pittsburgh and served as director of the Children’s Literature Program. She received MIT’s James A. and Ruth Levitan Teaching Award in 2019 and the Teaching with Digital Technology Award in 2020.

Gubar’s research focuses on children’s literature, history of children’s theater, performance, and 19th- and 20th-century representations of childhood. Her research and pedagogies underscore the importance of integrated learning.

Colleagues at MIT note her efficacy in introducing new concepts and new subjects into the literature curriculum during her tenure as curricular chair. Gubar set the stage for wide-ranging curricular improvements, resulting in a host of literature subjects on interrelated topics within and across disciplines.

Gubar teaches several classes, including 21L.452 (Literature and Philosophy) and 21L.500 (How We Got to Hamilton). Her lectures provide uniquely enriching learning experiences in which her students are encouraged to dive into literary texts; craft thoughtful, persuasive arguments; and engage in lively intellectual debate.

Gubar encourages others to bring fresh ideas and think outside the box. For example, her seminar on “Hamilton” challenges students to recontextualize the hip-hop musical in several intellectual traditions. Professor Eric Klopfer, head of the Comparative Media Studies Program/Writing and interim head of literature, calls Gubar “a thoughtful, caring instructor, and course designer … She thinks critically about whose story is being told and by whom.”

MacVicar Fellow and professor of literature Stephen Tapscott praises her experimentation, abstract thinking, and storytelling: “Professor Gubar’s ability to frame intellectual questions in terms of problems, developments, and performance is an important dimension of the genius of her teaching.”

“Marah is hands-down the most enthusiastic, effective, and engaged professor I had the pleasure of learning from at MIT,” writes one student. “She’s one of the few instructors I’ve had who never feels the need to reassert her place in the didactic hierarchy, but approaches her students as intellectual equals.”

Tapscott continues, “She welcomes participation in ways that enrich the conversation, open new modes of communication, and empower students as autonomous literary critics. In professor Gubar’s classroom we learn by doing … and that progress also includes ‘doing’ textual analysis, cultural history, and abstract literary theory.”

Gubar is also a committed mentor and student testimonials highlight her supportive approach. One of her former students remarked that Gubar “has a strong drive to be inclusive, and truly cares about ‘getting it right’ … her passion for literature and teaching, together with her drive for inclusivity, her ability to take accountability, and her compassion and empathy for her students, make [her] a truly remarkable teacher.”

On receiving this award Marah Gubar writes, “The best word I can think of to describe how I reacted to hearing that I had received this very overwhelming honor is ‘plotzing.’ The Yiddish verb ‘to plotz’ literally means to crack, burst, or collapse, so that captures how undone I was. I started to cry, because it suddenly struck me how much joy my father, Edward Gubar, would have taken in this amazing news. He was a teacher, too, and he died during the first phase of this terrible pandemic that we’re still struggling to get through.”

Adam C. Martin

Adam C. Martin is a professor and undergraduate officer in the Department of Biology. He studies the molecular mechanisms that underlie tissue form and function. His research interests include gastrulation, embryotic development, cytoskeletal dynamics, and the coordination of cellular behavior. Martin received his PhD from the University of California at Berkeley and his BS in biology (genetics) from Cornell University. Martin joined the Course 7 faculty in 2011.

“I am overwhelmed with gratitude knowing that this has come from our students. The fact that they spent time to contribute to a nomination is incredibly meaningful to me,” says Martin. “I want to also thank all of my faculty colleagues with whom I have taught, appreciate, and learned immensely from over the past 12 years. I am a better teacher because of them and inspired by their dedication.”

He is committed to undergraduate education, teaching several key department offerings including 7.06 (Cell Biology), 7.016 (Introductory Biology), 7.002 (Fundamentals of Experimental Molecular Biology), and 7.102 (Introduction to Molecular Biology Techniques).

Martin’s style combines academic and scientific expertise with creative elements like props and demonstrations. His “energy and passion for the material” is obvious, writes Iain Cheeseman, associate department head and the Herman and Margaret Sokol Professor of Biology. “In addition to creating engaging lectures, Adam went beyond the standard classroom requirements to develop videos and animations (in collaboration with the Biology MITx team) to illustrate core cell biological approaches and concepts.”

What sets Martin apart is his connection with students, his positive spirit, and his welcoming demeanor. Apolonia Gardner ’22 reflects on the way he helped her outside of class through his running group, which connects younger students with seniors in his lab. “Professor Martin was literally committed to ‘going the extra mile’ by inviting his students to join him on runs around the Charles River on Friday afternoons,” she says.

Amy Keating, department head and Jay A. Stein professor of biology, and professor of biological engineering, goes on to praise Martin’s ability to attract students to Course 7 and guide them through their educational experience in his role as the director of undergraduate studies. “He hosts social events, presides at our undergraduate research symposium and the department’s undergraduate graduation and awards banquet, and works with the Biology Undergraduate Student Association,” she says.

As undergraduate officer, Martin is involved in both advising and curriculum building. He mentors UROP students, serves as a first-year advisor, and is a current member of MIT’s Committee on the Undergraduate Program (CUP).

Martin also brings a commitment to diversity, equity, and inclusion (DEI) as evidenced by his creation of a DEI journal club in his lab so that students have a dedicated space to discuss issues and challenges. Course 7 DEI officer Hallie Dowling-Huppert writes that Martin “thinks deeply about how DEI efforts are created to ensure that department members receive the maximum benefit. Adam considers all perspectives when making decisions, and is extremely empathetic and caring towards his students.”

“He makes our world so much better,” Keating observes. “Adam is a gem.”

Lawrence “Larry” Sass

Larry Sass SM ’94, PhD ’00 is an associate professor in the Department of Architecture. He earned his PhD and SM in architecture at MIT, and has a BArch from Pratt Institute in New York City. Sass joined the faculty in the Department of Architecture in 2002. His work focuses on the delivery of affordable housing for low-income families. He was included in an exhibit titled “Home Delivery: Fabricating the Modern Dwelling” at the Museum of Modern Art in New York City.

Sass’s teaching blends computation with design. His two signature courses, 4.500 (Design Computation: Art, Objects and Space) and 4.501 (Tiny Fab: Advancements in Rapid Design and Fabrication of Small Homes), reflect his specialization in digitally fabricating buildings and furniture from machines.

Professor and head of architecture Nicholas de Monchaux writes, “his classes provide crucial instruction and practice with 3D modeling and computer-generated rendering and animation …  [He] links digital design to fabrication, in a process that invites students to define desirable design attributes of an object, develop a digital model, prototype it, and construct it at full scale.”

More generally, Sass’ approach is to help students build confidence in their own design process through hands-on projects. MIT Class of 1942 Professor John Ochsendorf, MacVicar Fellow, and founding director of the Morningside Academy for Design with appointments in the departments of architecture and civil and environmental engineering, confirms, “Larry’s teaching is a perfect embodiment of the ‘mens et manus’ spirit … [he] requires his students to go back and forth from mind and hand throughout each design project.”

Students say that his classes are a journey of self-discovery, allowing them to learn more about themselves and their own abilities. Senior Natasha Hirt notes, “What I learned from Larry was not something one can glean from a textbook, but a new way of seeing space … he tectonically shifted my perspective on buildings. He also shifted my perspective on myself. I’m a better designer for his teachings, and perhaps more importantly, I better understand how I design.”

Senior Izzi Waitz echoes this sentiment: “Larry emphasizes the importance of intentionally thinking through your designs and being confident in your choices … he challenges, questions, and prompts you so that you learn to defend and support yourself on your own.”

As a UROP coordinator, Sass assures students that the “sky is the limit” and all ideas are welcome. Postgraduate teaching fellow and research associate Myles Sampson says, “During the last year of my SM program, I assisted Larry in conducting a year-long UROP project … He structured the learning experience in a way that allowed the students to freely flex their design muscles: no idea was too outrageous.”

Sass is equally devoted to his students outside the classroom. In his role as head of house at MacGregor House, he lives in community with more than 300 undergraduates each year, providing academic guidance, creating residential programs and recreational activities, and ensuring that student wellness and mental health is a No. 1 priority.

Professor of architecture and MacVicar Fellow Les Norford says, “In two significant ways, Larry has been ahead of his time: combining digital representation and design with making and being alert to the well-being of his students.”

“In his kindness, he honors the memory of Margaret MacVicar, as well as the spirit of MIT itself,” Hirt concludes. “He is a designer, a craftsman, and an innovator. He is an inspiration and a compass.”

On receiving this award, Sass is full of excitement: “I love teaching and being part of the MIT community. I am grateful for the opportunity to be part of the MacVicar family of fellows.”

Studies of unusual brains reveal critical insights into brain organization, function

by Thea Singer |

EG (a pseudonym) is an accomplished woman in her early 60s: she is a college graduate and has an advanced professional degree. She has a stellar vocabulary—in the 98th percentile, according to tests—and has mastered a foreign language (Russian) to the point that she sometimes dreams in it.

She also has, likely since birth, been missing her left temporal lobe, a part of the brain known to be critical for language.

In 2016, EG contacted McGovern Institute Investigator Evelina Fedorenko, who studies the computations and brain regions that underlie language processing, to see if her team might be interested in including her in their research.

“EG didn’t know about her missing temporal lobe until age 25, when she had a brain scan for an unrelated reason,” says Fedorenko, the Frederick A. (1971) and Carole J. Middleton Career Development Associate Professor of Neuroscience at MIT. “As with many cases of early brain damage, she had no linguistic or cognitive deficits, but brains like hers are invaluable for understanding how cognitive functions reorganize in the tissue that remains.”

“I told her we definitely wanted to study her brain.” – Ev Fedorenko

Previous studies have shown that language processing relies on an interconnected network of frontal and temporal regions in the left hemisphere of the brain. EG’s unique brain presented an opportunity for Fedorenko’s team to explore how language develops in the absence of the temporal part of these core language regions.

Greta Tuckute, a graduate student in the Fedorenko lab, is the first author of the Neuropsychologia study. Photo: Caitlin Cunningham

Their results appeared recently in the journal Neuropsychologia. They found, for the first time, that temporal language regions appear to be critical for the emergence of frontal language regions in the same hemisphere — meaning, without a left temporal lobe, EG’s intact frontal lobe did not develop a capacity for language.

They also reveal much more: EG’s language system resides happily in her right hemisphere. “Our findings provide both visual and statistical proof of the brain’s remarkable plasticity, its ability to reorganize, in the face of extensive early damage,” says Greta Tuckute, a graduate student in the Fedorenko lab and first author of the paper.

In an introduction to the study, EG herself puts the social implications of the findings starkly. “Please do not call my brain abnormal, that creeps me out,” she . “My brain is atypical. If not for accidentally finding these differences, no one would pick me out of a crowd as likely to have these, or any other differences that make me unique.”

How we process language

The frontal and temporal lobes are part of the cerebrum, the largest part of the brain. The cerebrum controls many functions, including the five senses, language, working memory, personality, movement, learning, and reasoning. It is divided into two hemispheres, the left and the right, by a deep longitudinal fissure. The two hemispheres communicate via a thick bundle of nerve fibers called the corpus callosum. Each hemisphere comprises four main lobes—frontal, parietal, temporal, and occipital. Core parts of the language network reside in the frontal and temporal lobes.

Core parts of the language network (shown in teal) reside in the left frontal and temporal lobes. Image: Ev Fedorenko

In most individuals, the language system develops in both the right and left hemispheres, with the left side dominant from an early age. The frontal lobe develops slower than the temporal lobe. Together, the interconnected frontal and temporal language areas enable us to understand and produce words, phrases, and sentences.

How, then, did EG, with no left temporal lobe, come to speak, comprehend, and remember verbal information (even a foreign language!) with such proficiency?

Simply put, the right hemisphere took over: “EG has a completely well-functioning neurotypical-like language system in her right hemisphere,” says Tuckute. “It is incredible that a person can use a single hemisphere—and the right hemisphere at that, which in most people is not the dominant hemisphere where language is processed—and be perfectly fine.”

Journey into EG’s brain

In the study, the researchers conducted two scans of EG’s brain using functional magnetic resonance imaging (fMRI), one in 2016 and one in 2019, and had her complete a range of behaviorial tests. fMRI measures the level of blood oxygenation across the brain and can be used to make inferences about where neural activity is taking place. The researchers also scanned the brains of 151 “neurotypical” people. The large number of participants, combined with robust task paradigms and rigorous statistical analyses made it possible to draw conclusions from a single case such as EG.

Magnetic resonance image of EG’s brain showing missing left temporal lobe. Image: Fedorenko Lab

Fedorenko is a staunch advocate of the single case study approach—common in medicine but not currently in neuroscience. “Unusual brains—and unusual individuals more broadly—can provide critical insights into brain organization and function that we simply cannot gain by looking at more typical brains.” Studying individual brains with fMRI, however, requires paradigms that work robustly at the single-brain level. This is not true of most paradigms used in the field, which require averaging many brains together to obtain an effect. Developing individual-level fMRI paradigms for language research has been the focus of Fedorenko’s early work, although the main reason for doing so had nothing to do with studying atypical brains: individual-level analyses are simply better—they are more sensitive and their results are more interpretable and meaningful.

“Looking at high-quality data in an individual participant versus looking at a group-level map is akin to using a high-precision microscope versus looking with a naked myopic eye, when all you see is a blur,” she wrote in an article published in Current Opinion in Behaviorial Sciences in 2021. Having developed and validated such paradigms, though, is now allowing Fedorenko and her group to probe interesting brains.

While in the scanner, each participant performed a task that Fedorenko began developing more than a decade ago. They were presented with a series of words that form real, meaningful sentences, and with a series of “nonwords”—strings of letters that are pronounceable but without meaning. In typical brains, language areas respond more strongly when participants read sentences compared to when they read nonword sequences.

Similarly, in response to the real sentences, the language regions in EG’s right frontal and temporal lobes lit up—they were bursting with activity—while the left frontal lobe regions remained silent. In the neurotypical participants, the language regions in both the left and right frontal and temporal lobes lit up, with the left areas outshining the right.

fMRI showing EG’s language activation on the brain surface. The right frontal lobe shows robust activations, while the left frontal lobe does not have any language responsive areas. Image: Fedorenko lab

“EG showed a very strong response in the right temporal and frontal regions that process language,” says Tuckute. “And if you look at the controls, whose language dominant hemisphere is in the left, EG’s response in her right hemisphere was similar—or even higher—compared to theirs, just on the opposite side.”

Leaving no stone unturned, the researchers next asked whether the lack of language responses in EG’s left frontal lobe might be due to a general lack of response to cognitive tasks rather than just to language. So they conducted a non-language, working-memory task: they had EG and the neurotypical participants perform arithmetic addition problems while in the scanner. In typical brains, this task elicits responses in frontal and parietal areas in both hemisphers.

Not only did regions of EG’s right frontal lobe light up in response to the task, those in her left frontal lobe did, too. “Both EG’s language-dominant (right) hemisphere, and her non-language-dominant (left) hemisphere showed robust responses to this working-memory task ,” says Tuckute. “So, yes, there’s definitely cognitive processing going on there. This selective lack of language responses in EG’s left frontal lobe led us to conclude that, for language, you need the temporal language region to ‘wire up’ the frontal language region.”

Next steps

In science, the answer to one question opens the door to untold more. “In EG, language took over a large chunk of the right frontal and temporal lobes,” says Fedorenko. “So what happens to the functions that in neurotypical individuals generally live in the right hemisphere?”

Many of those, she says, are social functions. The team has already tested EG on social tasks and is currently exploring how those social functions cohabit with the language ones in her right hemisphere. How can they all fit? Do some of the social functions have to migrate to other parts of the brain? They are also working with EG’s family: they have now scanned EG’s three siblings (one of whom is missing most of her right temporal lobe; the other two are neurotypical) and her father (also neurotypical).

The “Interesting Brains Project” website details current projects, findings, and ways to participate.

The project has now grown to include many other individuals with interesting brains, who contacted Fedorenko after some of this work was covered by news outlets. A website for this project can be found here. The project promises to provide unique insights into how our plastic brains reorganize and adapt to various circumstances.

 

Season’s Greetings from the McGovern Institute

by McGovern Institute |

This year’s holiday video (shown above) was inspired by Ev Fedorenko’s July 2022 Nature Neuroscience paper, which found similar patterns of brain activation and language selectivity across speakers of 45 different languages.

Universal language network

Ev Fedorenko uses the widely translated book “Alice in Wonderland” to test brain responses to different languages. Photo: Caitlin Cunningham

Over several decades, neuroscientists have created a well-defined map of the brain’s “language network,” or the regions of the brain that are specialized for processing language. Found primarily in the left hemisphere, this network includes regions within Broca’s area, as well as in other parts of the frontal and temporal lobes. Although roughly 7,000 languages are currently spoken and signed across the globe, the vast majority of those mapping studies have been done in English speakers as they listened to or read English texts.

To truly understand the cognitive and neural mechanisms that allow us to learn and process such diverse languages, Fedorenko and her team scanned the brains of speakers of 45 different languages while they listened to Alice in Wonderland in their native language. The results show that the speakers’ language networks appear to be essentially the same as those of native English speakers — which suggests that the location and key properties of the language network appear to be universal.

The many languages of McGovern

English may be the primary language used by McGovern researchers, but more than 35 other languages are spoken by scientists and engineers at the McGovern Institute. Our holiday video features 30 of these researchers saying Happy New Year in their native (or learned) language. Below is the complete list of languages included in our video. Expand each accordion to learn more about the speaker of that particular language and the meaning behind their new year’s greeting.

Brains on conlangs

by Jennifer Michalowski |

For a few days in November, the McGovern Institute hummed with invented languages. Strangers greeted one another in Esperanto; trivia games were played in High Valyrian; Klingon and Na’vi were heard inside MRI scanners. Creators and users of these constructed languages (conlangs) had gathered at MIT in the name of neuroscience. McGovern Institute investigator Evelina Fedorenko and her team wanted to know what happened in their brains when they heard and understood these “foreign” tongues.

The constructed languages spoken by attendees had all been created for specific purposes. Most, like the Na’vi language spoken in the movie Avatar, had given identity and voice to the inhabitants of fictional worlds, while Esperanto was created to reduce barriers to international communication. But despite their distinct origins, a familiar pattern of activity emerged when researchers scanned speakers’ brains. The brain, they found, processes constructed languages with the same network of areas it uses for languages that evolved naturally over millions of years.

The meaning of language

“There’s all these things that people call language,” Fedorenko says. “Music is a kind of language and math is a kind of language.” But the brain processes these metaphorical languages differently than it does the languages humans use to communicate broadly about the world. To neuroscientists like Fedorenko, they can’t legitimately be considered languages at all. In contrast, she says, “these constructed languages seem really quite like natural languages.”

The “Brains on Conlangs” event that Fedorenko’s team hosted was part of its ongoing effort to understand the way language is generated and understood by the brain. Her lab and others have identified specific brain regions involved in linguistic processing, but it’s not yet clear how universal the language network is. Most studies of language cognition have focused on languages widely spoken in well-resourced parts of the world—primarily English, German, and Dutch. There are thousands of languages—spoken or signed—that have not been included.

Brain activation in a Klingon speaker while listening to English (left) and Klingon (right). Image: Saima Malik Moraleda

Fedorenko and her team are deliberately taking a broader approach. “If we’re making claims about language as a whole, it’s kind of weird to make it based on a handful of languages,” she says. “So we’re trying to create tools and collect some data on as many languages as possible.”

So far, they have found that the language networks used by native speakers of dozens of different languages do share key architectural similarities. And by including a more diverse set of languages in their research, Fedorenko and her team can begin to explore how the brain makes sense of linguistic features that are not part of English or other well studied languages. The Brains on Conlangs event was a chance to expand their studies even further.

Connecting conlangs

Nearly 50 speakers of Esperanto, Klingon, High Valyrian, Dothraki, and Na’vi attended Brains on Conlangs, drawn by the opportunity to connect with other speakers, hear from language creators, and contribute to the science. Graduate student Saima Malik-Moraleda and postbac research assistant Maya Taliaferro, along with other members of both the Fedorenko lab and brain and cognitive sciences professor Ted Gibson’s lab, and with help from Steve Shannon, Operations Manager of the Martinos Imaging Center, worked tirelessly to collect data from all participants. Two MRI scanners ran nearly continuously as speakers listened to passages in their chosen languages and researchers captured images of the brain’s response. To enable the research team to find the language-specific network in each person’s brain, participants also performed other tasks inside the scanner, including a memory task and listening to muffled audio in which the constructed languages were spoken, but unintelligible. They performed language tasks in English, as well.

To understand how the brain processes constructed languages (conlangs), McGovern Investigator Ev Fedorenko (center) gathered with conlang creators/speakers Marc Okrand (Klingon), Paul Frommer (Na’vi), Damian Blasi, Jessie Sams (méníshè), David Peterson (High Valyrian and Dothraki) and Aroka Okrent at the McGovern Institute for the “Brains on Colangs” event in November 2022. Photo: Elise Malvicini

Prior to the study, Fedorenko says, she had suspected constructed languages would activate the brain’s natural language-processing network, but she couldn’t be sure. Another possibility was that languages like Klingon and Esperanto would be handled instead by a problem-solving network known to be used when people work with some other so-called “languages,” like mathematics or computer programming. But once the data was in, the answer was clear. The five constructed languages included in the study all activated the brain’s language network.

That makes sense, Fedorenko says, because like natural languages, constructed languages enable people to communicate by associating words or signs with objects and ideas. Any language is essentially a way of mapping forms to meanings, she says. “You can construe it as a set of memories of how a particular sequence of sounds corresponds to some meaning. You’re learning meanings of words and constructions, and how to put them together to get more complex meanings. And it seems like the brain’s language system is very well suited for that set of computations.”

Machine learning can predict bipolar disorder in children and teens

by Jennifer Michalowski |

Bipolar disorder often begins in childhood or adolescence, triggering dramatic mood shifts and intense emotions that cause problems at home and school. But the condition is often overlooked or misdiagnosed until patients are older. New research suggests that machine learning, a type of artificial intelligence, could help by identifying children who are at risk of bipolar disorder so doctors are better prepared to recognize the condition if it develops.

On October 13, 2022, researchers led by McGovern Institute investigator John Gabrieli and collaborators at Massachusetts General Hospital reported in the Journal of Psychiatric Research that when presented with clinical data on nearly 500 children and teenagers, a machine learning model was able to identify about 75 percent of those who were later diagnosed with bipolar disorder. The approach performs better than any other method of predicting bipolar disorder, and could be used to develop a simple risk calculator for health care providers.

Gabrieli says such a tool would be particularly valuable because bipolar disorder is less common in children than conditions like major depression, with which it shares symptoms, and attention-deficit/ hyperactivity disorder (ADHD), with which it often co-occurs. “Humans are not well tuned to watch out for rare events,” he says. “If you have a decent measure, it’s so much easier for a machine to identify than humans. And in this particular case, [the machine learning prediction] was surprisingly robust.”

Detecting bipolar disorder

Mai Uchida, Director of Massachusetts General Hospital’s Child Depression Program, says that nearly two percent of youth worldwide are estimated to have bipolar disorder, but diagnosing pediatric bipolar disorder can be challenging. A certain amount of emotional turmoil is to be expected in children and teenagers, and even when moods become seriously disruptive, children with bipolar disorder are often initially diagnosed with major depression or ADHD. That’s a problem, because the medications used to treat those conditions often worsen the symptoms of bipolar disorder. Tailoring treatment to a diagnosis of bipolar disorder, in contrast, can lead to significant improvements for patients and their families. “When we can give them a little bit of ease and give them a little bit of control over themselves, it really goes a long way,” Uchida says.

In fact, a poor response to antidepressants or ADHD medications can help point a psychiatrist toward a diagnosis of bipolar disorder. So too can a child’s family history, in addition to their own behavior and psychiatric history. But, Uchida says, “it’s kind of up to the individual clinician to pick up on these things.”

Uchida and Gabrieli wondered whether machine learning, which can find patterns in large, complex datasets, could focus in on the most relevant features to identify individuals with bipolar disorder. To find out, they turned to data from a study that began in the 1990s. The study, headed by Joseph Biederman, Chief of the Clinical and Research Programs in Pediatric Psychopharmacology and Adult ADHD at Massachusetts General Hospital, had collected extensive psychiatric assessments of hundreds of children with and without ADHD, then followed those individuals for ten years.

To explore whether machine learning could find predictors of bipolar disorder within that data, Gabrieli, Uchida, and colleagues focused on 492 children and teenagers without ADHD, who were recruited to the study as controls. Over the ten years of the study, 45 of those individuals developed bipolar disorder.

Within the data collected at the study’s outset, the machine learning model was able to find patterns that associated with a later diagnosis of bipolar disorder. A few behavioral measures turned out to be particularly relevant to the model’s predictions: children and teens with combined problems with attention, aggression, and anxiety were most likely to later be diagnosed with bipolar disorder. These indicators were all picked up by a standard assessment tool called the Child Behavior Checklist.

Uchida and Gabrieli say the machine learning model could be integrated into the medical record system to help pediatricians and child psychiatrists catch early warning signs of bipolar disorder. “The information that’s collected could alert a clinician to the possibility of a bipolar disorder developing,” Uchida says. “Then at least they’re aware of the risk, and they may be able to maybe pick up on some of the deterioration when it’s happening and think about either referring them or treating it themselves.”

Not every reader’s struggle is the same

Anne Trafton |

Many children struggle to learn to read, and studies have shown that students from a lower socioeconomic status (SES) background are more likely to have difficulty than those from a higher SES background.

MIT neuroscientists have now discovered that the types of difficulties that lower-SES students have with reading, and the underlying brain signatures, are, on average, different from those of higher-SES students who struggle with reading.

In a new study, which included brain scans of more than 150 children as they performed tasks related to reading, researchers found that when students from higher SES backgrounds struggled with reading, it could usually be explained by differences in their ability to piece sounds together into words, a skill known as phonological processing.

However, when students from lower SES backgrounds struggled, it was best explained by differences in their ability to rapidly name words or letters, a task associated with orthographic processing, or visual interpretation of words and letters. This pattern was further confirmed by brain activation during phonological and orthographic processing.

These differences suggest that different types of interventions may needed for different groups of children, the researchers say. The study also highlights the importance of including a wide range of SES levels in studies of reading or other types of academic learning.

“Within the neuroscience realm, we tend to rely on convenience samples of participants, so a lot of our understanding of the neuroscience components of reading in general, and reading disabilities in particular, tends to be based on higher-SES families,” says Rachel Romeo, a former graduate student in the Harvard-MIT Program in Health Sciences and Technology and the lead author of the study. “If we only look at these nonrepresentative samples, we can come away with a relatively biased view of how the brain works.”

Romeo is now an assistant professor in the Department of Human Development and Quantitative Methodology at the University of Maryland. John Gabrieli, the Grover Hermann Professor of Health Sciences and Technology and a professor of brain and cognitive sciences at MIT, is the senior author of the paper, which appears today in the journal Developmental Cognitive Neuroscience.

Components of reading

For many years, researchers have known that children’s scores on standardized assessments of reading are correlated with socioeconomic factors such as school spending per student or the number of children at the school who qualify for free or reduced-price lunches.

Studies of children who struggle with reading, mostly done in higher-SES environments, have shown that the aspect of reading they struggle with most is phonological awareness: the understanding of how sounds combine to make a word, and how sounds can be split up and swapped in or out to make new words.

“That’s a key component of reading, and difficulty with phonological processing is often one of the hallmarks of dyslexia or other reading disorders,” Romeo says.

In the new study, the MIT team wanted to explore how SES might affect phonological processing as well as another key aspect of reading, orthographic processing. This relates more to the visual components of reading, including the ability to identify letters and read words.

To do the study, the researchers recruited first and second grade students from the Boston area, making an effort to include a range of SES levels. For the purposes of this study, SES was assessed by parents’ total years of formal education, which is commonly used as a measure of the family’s SES.

“We went into this not necessarily with any hypothesis about how SES might relate to the two types of processing, but just trying to understand whether SES might be impacting one or the other more, or if it affects both types the same,” Romeo says.

The researchers first gave each child a series of standardized tests designed to measure either phonological processing or orthographic processing. Then, they performed fMRI scans of each child while they carried out additional phonological or orthographic tasks.

The initial series of tests allowed the researchers to determine each child’s abilities for both types of processing, and the brain scans allowed them to measure brain activity in parts of the brain linked with each type of processing.

The results showed that at the higher end of the SES spectrum, differences in phonological processing ability accounted for most of the differences between good readers and struggling readers. This is consistent with the findings of previous studies of reading difficulty. In those children, the researchers also found greater differences in activity in the parts of the brain responsible for phonological processing.

However, the outcomes were different when the researchers analyzed the lower end of the SES spectrum. There, the researchers found that variance in orthographic processing ability accounted for most of the differences between good readers and struggling readers. MRI scans of these children revealed greater differences in brain activity in parts of the brain that are involved in orthographic processing.

Optimizing interventions

There are many possible reasons why a lower SES background might lead to difficulties in orthographic processing, the researchers say. It might be less exposure to books at home, or limited access to libraries and other resources that promote literacy. For children from this background who struggle with reading, different types of interventions might benefit them more than the ones typically used for children who have difficulty with phonological processing.

In a 2017 study, Gabrieli, Romeo, and others found that a summer reading intervention that focused on helping students develop the sensory and cognitive processing necessary for reading was more beneficial for students from lower-SES backgrounds than children from higher-SES backgrounds. Those findings also support the idea that tailored interventions may be necessary for individual students, they say.

“There are two major reasons we understand that cause children to struggle as they learn to read in these early grades. One of them is learning differences, most prominently dyslexia, and the other one is socioeconomic disadvantage,” Gabrieli says. “In my mind, schools have to help all these kinds of kids become the best readers they can, so recognizing the source or sources of reading difficulty ought to inform practices and policies that are sensitive to these differences and optimize supportive interventions.”

Gabrieli and Romeo are now working with researchers at the Harvard University Graduate School of Education to evaluate language and reading interventions that could better prepare preschool children from lower SES backgrounds to learn to read. In her new lab at the University of Maryland, Romeo also plans to further delve into how different aspects of low SES contribute to different areas of language and literacy development.

“No matter why a child is struggling with reading, they need the education and the attention to support them. Studies that try to tease out the underlying factors can help us in tailoring educational interventions to what a child needs,” she says.

The research was funded by the Ellison Medical Foundation, the Halis Family Foundation, and the National Institutes of Health.

Understanding reality through algorithms

by Leah Campbell | School of Science |

Although Fernanda De La Torre still has several years left in her graduate studies, she’s already dreaming big when it comes to what the future has in store for her.

“I dream of opening up a school one day where I could bring this world of understanding of cognition and perception into places that would never have contact with this,” she says.

It’s that kind of ambitious thinking that’s gotten De La Torre, a doctoral student in MIT’s Department of Brain and Cognitive Sciences, to this point. A recent recipient of the prestigious Paul and Daisy Soros Fellowship for New Americans, De La Torre has found at MIT a supportive, creative research environment that’s allowed her to delve into the cutting-edge science of artificial intelligence. But she’s still driven by an innate curiosity about human imagination and a desire to bring that knowledge to the communities in which she grew up.

An unconventional path to neuroscience

De La Torre’s first exposure to neuroscience wasn’t in the classroom, but in her daily life. As a child, she watched her younger sister struggle with epilepsy. At 12, she crossed into the United States from Mexico illegally to reunite with her mother, exposing her to a whole new language and culture. Once in the States, she had to grapple with her mother’s shifting personality in the midst of an abusive relationship. “All of these different things I was seeing around me drove me to want to better understand how psychology works,” De La Torre says, “to understand how the mind works, and how it is that we can all be in the same environment and feel very different things.”

But finding an outlet for that intellectual curiosity was challenging. As an undocumented immigrant, her access to financial aid was limited. Her high school was also underfunded and lacked elective options. Mentors along the way, though, encouraged the aspiring scientist, and through a program at her school, she was able to take community college courses to fulfill basic educational requirements.

It took an inspiring amount of dedication to her education, but De La Torre made it to Kansas State University for her undergraduate studies, where she majored in computer science and math. At Kansas State, she was able to get her first real taste of research. “I was just fascinated by the questions they were asking and this entire space I hadn’t encountered,” says De La Torre of her experience working in a visual cognition lab and discovering the field of computational neuroscience.

Although Kansas State didn’t have a dedicated neuroscience program, her research experience in cognition led her to a machine learning lab led by William Hsu, a computer science professor. There, De La Torre became enamored by the possibilities of using computation to model the human brain. Hsu’s support also convinced her that a scientific career was a possibility. “He always made me feel like I was capable of tackling big questions,” she says fondly.

With the confidence imparted in her at Kansas State, De La Torre came to MIT in 2019 as a post-baccalaureate student in the lab of Tomaso Poggio, the Eugene McDermott Professor of Brain and Cognitive Sciences and an investigator at the McGovern Institute for Brain Research. With Poggio, also the director of the Center for Brains, Minds and Machines, De La Torre began working on deep-learning theory, an area of machine learning focused on how artificial neural networks modeled on the brain can learn to recognize patterns and learn.

“It’s a very interesting question because we’re starting to use them everywhere,” says De La Torre of neural networks, listing off examples from self-driving cars to medicine. “But, at the same time, we don’t fully understand how these networks can go from knowing nothing and just being a bunch of numbers to outputting things that make sense.”

Her experience as a post-bac was De La Torre’s first real opportunity to apply the technical computer skills she developed as an undergraduate to neuroscience. It was also the first time she could fully focus on research. “That was the first time that I had access to health insurance and a stable salary. That was, in itself, sort of life-changing,” she says. “But on the research side, it was very intimidating at first. I was anxious, and I wasn’t sure that I belonged here.”

Fortunately, De La Torre says she was able to overcome those insecurities, both through a growing unabashed enthusiasm for the field and through the support of Poggio and her other colleagues in MIT’s Department of Brain and Cognitive Sciences. When the opportunity came to apply to the department’s PhD program, she jumped on it. “It was just knowing these kinds of mentors are here and that they cared about their students,” says De La Torre of her decision to stay on at MIT for graduate studies. “That was really meaningful.”

Expanding notions of reality and imagination

In her two years so far in the graduate program, De La Torre’s work has expanded the understanding of neural networks and their applications to the study of the human brain. Working with Guangyu Robert Yang, an associate investigator at the McGovern Institute and an assistant professor in the departments of Brain and Cognitive Sciences and Electrical Engineering and Computer Sciences, she’s engaged in what she describes as more philosophical questions about how one develops a sense of self as an independent being. She’s interested in how that self-consciousness develops and why it might be useful.

De La Torre’s primary advisor, though, is Professor Josh McDermott, who leads the Laboratory for Computational Audition. With McDermott, De La Torre is attempting to understand how the brain integrates vision and sound. While combining sensory inputs may seem like a basic process, there are many unanswered questions about how our brains combine multiple signals into a coherent impression, or percept, of the world. Many of the questions are raised by audiovisual illusions in which what we hear changes what we see. For example, if one sees a video of two discs passing each other, but the clip contains the sound of a collision, the brain will perceive that the discs are bouncing off, rather than passing through each other. Given an ambiguous image, that simple auditory cue is all it takes to create a different perception of reality.

There’s something interesting happening where our brains are receiving two signals telling us different things and, yet, we have to combine them somehow to make sense of the world.

De La Torre is using behavioral experiments to probe how the human brain makes sense of multisensory cues to construct a particular perception. To do so, she’s created various scenes of objects interacting in 3D space over different sounds, asking research participants to describe characteristics of the scene. For example, in one experiment, she combines visuals of a block moving across a surface at different speeds with various scraping sounds, asking participants to estimate how rough the surface is. Eventually she hopes to take the experiment into virtual reality, where participants will physically push blocks in response to how rough they perceive the surface to be, rather than just reporting on what they experience.

Once she’s collected data, she’ll move into the modeling phase of the research, evaluating whether multisensory neural networks perceive illusions the way humans do. “What we want to do is model exactly what’s happening,” says De La Torre. “How is it that we’re receiving these two signals, integrating them and, at the same time, using all of our prior knowledge and inferences of physics to really make sense of the world?”

Although her two strands of research with Yang and McDermott may seem distinct, she sees clear connections between the two. Both projects are about grasping what artificial neural networks are capable of and what they tell us about the brain. At a more fundamental level, she says that how the brain perceives the world from different sensory cues might be part of what gives people a sense of self. Sensory perception is about constructing a cohesive, unitary sense of the world from multiple sources of sensory data. Similarly, she argues, “the sense of self is really a combination of actions, plans, goals, emotions, all of these different things that are components of their own, but somehow create a unitary being.”

It’s a fitting sentiment for De La Torre, who has been working to make sense of and integrate different aspects of her own life. Working in the Computational Audition lab, for example, she’s started experimenting with combining electronic music with folk music from her native Mexico, connecting her “two worlds,” as she says. Having the space to undertake those kinds of intellectual explorations, and colleagues who encourage it, is one of De La Torre’s favorite parts of MIT.

“Beyond professors, there’s also a lot of students whose way of thinking just amazes me,” she says. “I see a lot of goodness and excitement for science and a little bit of — it’s not nerdiness, but a love for very niche things — and I just kind of love that.”

Modeling the social mind

by Rachel Yang |

Typically, it would take two graduate students to do the research that Setayesh Radkani is doing.

Driven by an insatiable curiosity about the human mind, she is working on two PhD thesis projects in two different cognitive neuroscience labs at MIT. For one, she is studying punishment as a social tool to influence others. For the other, she is uncovering the neural processes underlying social learning — that is, learning from others. By piecing together these two research programs, Radkani is hoping to gain a better understanding of the mechanisms underpinning social influence in the mind and brain.

Radkani lived in Iran for most of her life, growing up alongside her younger brother in Tehran. The two spent a lot of time together and have long been each other’s best friends. Her father is a civil engineer, and her mother is a midwife. Her parents always encouraged her to explore new things and follow her own path, even if it wasn’t quite what they imagined for her. And her uncle helped cultivate her sense of curiosity, teaching her to “always ask why” as a way to understand how the world works.

Growing up, Radkani most loved learning about human psychology and using math to model the world around her. But she thought it was impossible to combine her two interests. Prioritizing math, she pursued a bachelor’s degree in electrical engineering at the Sharif University of Technology in Iran.

Then, late in her undergraduate studies, Radkani took a psychology course and discovered the field of cognitive neuroscience, in which scientists mathematically model the human mind and brain. She also spent a summer working in a computational neuroscience lab at the Swiss Federal Institute of Technology in Lausanne. Seeing a way to combine her interests, she decided to pivot and pursue the subject in graduate school.

An experience leading a project in her engineering ethics course during her final year of undergrad further helped her discover some of the questions that would eventually form the basis of her PhD. The project investigated why some students cheat and how to change this.

“Through this project I learned how complicated it is to understand the reasons that people engage in immoral behavior, and even more complicated than that is how to devise policies and react in these situations in order to change people’s attitudes,” Radkani says. “It was this experience that made me realize that I’m interested in studying the human social and moral mind.”

She began looking into social cognitive neuroscience research and stumbled upon a relevant TED talk by Rebecca Saxe, the John W. Jarve Professor in Brain and Cognitive Sciences at MIT, who would eventually become one of Radkani’s research advisors. Radkani knew immediately that she wanted to work with Saxe. But she needed to first get into the BCS PhD program at MIT, a challenging obstacle given her minimal background in the field.

After two application cycles and a year’s worth of graduate courses in cognitive neuroscience, Radkani was accepted into the program. But to come to MIT, she had to leave her family behind. Coming from Iran, Radkani has a single-entry visa, making it difficult for her to travel outside the U.S. She hasn’t been able to visit her family since starting her PhD and won’t be able to until at least after she graduates. Her visa also limits her research contributions, restricting her from attending conferences outside the U.S. “That is definitely a huge burden on my education and on my mental health,” she says.

Still, Radkani is grateful to be at MIT, indulging her curiosity in the human social mind. And she’s thankful for her supportive family, who she calls over FaceTime every day.

Modeling how people think about punishment

In Saxe’s lab, Radkani is researching how people approach and react to punishment, through behavioral studies and neuroimaging. By synthesizing these findings, she’s developing a computational model of the mind that characterizes how people make decisions in situations involving punishment, such as when a parent disciplines a child, when someone punishes their romantic partner, or when the criminal justice system sentences a defendant. With this model, Radkani says she hopes to better understand “when and why punishment works in changing behavior and influencing beliefs about right and wrong, and why sometimes it fails.”

Punishment isn’t a new research topic in cognitive neuroscience, Radkani says, but in previous studies, scientists have often only focused on people’s behavior in punitive situations and haven’t considered the thought processes that underlie those behaviors. Characterizing these thought processes, though, is key to understanding whether punishment in a situation can be effective in changing people’s attitudes.

People bring their prior beliefs into a punitive situation. Apart from moral beliefs about the appropriateness of different behaviors, “you have beliefs about the characteristics of the people involved, and you have theories about their intentions and motivations,” Radkani says. “All those come together to determine what you do or how you are influenced by punishment,” given the circumstances. Punishers decide a suitable punishment based on their interpretation of the situation, in light of their beliefs. Targets of punishment then decide whether they’ll change their attitude as a result of the punishment, depending on their own beliefs. Even outside observers make decisions, choosing whether to keep or change their moral beliefs based on what they see.

To capture these decision-making processes, Radkani is developing a computational model of the mind for punitive situations. The model mathematically represents people’s beliefs and how they interact with certain features of the situation to shape their decisions. The model then predicts a punisher’s decisions, and how punishment will influence the target and observers. Through this model, Radkani will provide a foundational understanding of how people think in various punitive situations.

Researching the neural mechanisms of social learning

In parallel, working in the lab of Professor Mehrdad Jazayeri, Radkani is studying social learning, uncovering its underlying neural processes. Through social learning, people learn from other people’s experiences and decisions, and incorporate this socially acquired knowledge into their own decisions or beliefs.

Humans are extraordinary in their social learning abilities, however our primary form of learning, shared by all other animals, is learning from self-experience. To investigate how learning from others is similar to or different from learning from our own experiences, Radkani has designed a two-player video game that involves both types of learning. During the game, she and her collaborators in Jazayeri’s lab record neural activity in the brain. By analyzing these neural measurements, they plan to uncover the computations carried out by neural circuits during social learning, and compare those to learning from self-experience.

Radkani first became curious about this comparison as a way to understand why people sometimes draw contrasting conclusions from very similar situations. “For example, if I get Covid from going to a restaurant, I’ll blame the restaurant and say it was not clean,” Radkani says. “But if I hear the same thing happen to my friend, I’ll say it’s because they were not careful.” Radkani wanted to know the root causes of this mismatch in how other people’s experiences affect our beliefs and judgements differently from our own similar experiences, particularly because it can lead to “errors that color the way that we judge other people,” she says.

By combining her two research projects, Radkani hopes to better understand how social influence works, particularly in moral situations. From there, she has a slew of research questions that she’s eager to investigate, including: How do people choose who to trust? And which types of people tend to be the most influential? As Radkani’s research grows, so does her curiosity.