[Women in Science] When your brain searches, Koel Das watches

The electrical engineer-turned-cognitive neuroscientist talks about how the brain ‘sees’ and ‘finds’.

Humans have always been extremely good at looking and finding – whether it is while spotting a ripe berry on a tree while escaping from a wild animal, or while eyeing an empty seat in a crowded bus. So much that according to cognitive neuroscientist Koel Das who works at Indian Institute of Science Education and Research (IISER) Kolkata, even with all the advances in artificial intelligence, we are yet to devise a program that can search the way we do. “It’s not clear what brain mechanism underlies the efficiency with which we find things,” said Koel, introducing her research – “exactly how do we search? How do we strategise?”

Decision-making based on visual cues might be a way to understand how our brain searches. Koel has been exploring this since her postdoctoral days in the 2000s. She recently concluded an experiment focused on how contextual information influences our visual searches. “Say, I was looking for my lab keys on this table. I will probably look for something golden and shiny. I manage to find it pretty fast, considering how cluttered my table usually is,” she said, adding, “Today, I cleaned it up for you.”

She continued: “We seem to be using templates from our memory to perform a sort of template matching. This is where contextual information helps. I already have an understanding of what I’m looking for, what it’s shape and size is, and so on,” Koel said.

Of cues and contexts

Her lab aims to pinpoint the mechanism used by the brain to allow us to successfully conduct a visual search. One of her recent experiments explored how humans use ‘scene context’ while searching for objects or cues. “We showed subjects some photographs of real-life scenes. They were given cues or targets to look for in the photograph – for example, a sink, a chimney or a rug… ”

However, the experiment is designed such that out of 800 images shown to subjects, only 400 actually had the target (i.e. the rug/chimney, sink) present. The target-absent images did not have the items themselves, but the ‘context’ of the items was included. “If they are looking for a boat, even if there is no boat, there will be water; if they are looking for a chimney, there will be a roof.” The task for the subjects was to search the image and indicate if the target was present.

Koel is especially interested in how the subjects reacted to the target-absent images. These are more useful to examine the effect of contextual information in our search for something. For this, she connects the subjects to an electroencephalography (EEG) set-up as they searched the images. EEG is a method that detects electrical activity in the brain, which is considered to be synonymous with the brain’s activity.

After their analyses, they saw that the subject was able to predict the location of the target; even when the target was absent, the subjects looked at a specific area where they expected to find it (for example, they looked at the floor, even if there was no rug). This was something Koel observed in her PhD research too, and it’s significant because it confirms the role of contextual information in a human’s visual search. As pointed out in this Science Daily report of Koel’s postdoctoral research:

“So, if you’re looking for a computer mouse on a cluttered desk, a machine would be looking for things shaped like a mouse. It might find it, but it might see other objects of similar shape, and classify that as a mouse,” said Eckstein, co-author of the study and Koel’s advisor. Computer vision systems might also not associate their target with specific locations or other objects. So, to a machine, the floor is just as likely a place for a mouse as a desk.”

This makes it clear how far computers have to go to be as efficient at searching as the human brain is.

How to read a brain

To find out which area in the brain, which neural circuit, is key for this kind of visual search, another technology is required – functional magnetic resonance imaging (fMRI). While, EEG is good enough to track what happens in the brain at specific moments in time, functional MRI is much better suited for information about spatial locations of brain activity, Koel explained.

During her postdoctoral research, fMRI scans helped Koel’s team identify three areas that seemed key to the brain activity in question. One is an area called the lateral occipital complex (LOC), which could predict the location of where the observers expected to find targets in target-absent images. Then came an area called intraparietal sulcus (IPS), which was linked to cue-based visual activity. Finally came the frontal region, the executive areas of the brain, which is where the actual decision-making takes place.

Koel’s lab is currently expanding on this research; primarily using EEG. For her EEG experiments, Koel’s subjects are all IISER students.

The findings from these experiments match the already established knowledge of these regions in the brain, but Koel said her research is trying something different. “A lot of research is done on the visual cortex, which is a huge chunk of our brain. But previous ones focused on synthetic cues (i.e. patterns and other artificial pictures, not images of real scenes) because those were easier to do. These do not actually imitate what we do in real life.” Her recent findings were presented last year at the European Conference on Visual Perception.

“In a way, I’m an anomaly”

Koel works at the intersection of two departments at IISER Kolkata – Mathematics and Statistics and Biological Sciences. An electrical engineer by training, she is well aware of the boundaries her research requires her to cross. “In a way, I am an anomaly, but that doesn’t hamper me because the IISERs were created to develop this kind of interdisciplinary research. Currently, I think that some very interesting science is taking place at these boundaries!”

For example, Koel’s lab is also focusing on how collective decision-making is done – a phenomenon referred to by The New Yorker journalist James Surowiecki as ‘the wisdom of crowds’ – from the perspective of brain circuits. In addition, she is working with collaborators who provide her with fMRI data of persons affected with autism, schizophrenia and other behavioural conditions. “We want to try and predict from their brain signals if they are healthy or not. And we are getting some interesting results, especially with schizophrenic data. It looks like we can predict who is high-risk for these conditions.”

Koel suspects that it is her training in engineering that draws her beyond basic research to application-oriented research like this. Growing up in Calcutta, she followed in her electrical engineer father’s footsteps. She studied engineering in a government college at Guwahati before leaving to the US to do her masters. So far, a pretty predictable path for an engineer. So when exactly did she diverge into neuroscientific research?

“When I became a teaching assistant for a newly joined professor at University of California Irvine, I got interested in his work on brain-computer interface. It deals with the technology used by Stephen Hawking to communicate. I found it fascinating and joined him as his first PhD student.”

A different kind of culture shock

Her choice to go into research was met with disapproval from her father. “He has a transformer company and being the only child, it was kind of a given that I would join him. I think he is still not convinced… He is very sceptical about what I do,” she laughed. “My father is still kind of in denial but my mother is very supportive.”

Despite this, Koel stresses that she was brought up very progressively. There was never any resistance to her pursuing higher studies and this was something she took for granted till she saw how it was for less privileged peers. “I didn’t have culture shock in the US, but I experienced it only when I talked to my friends from different parts of India about the struggle they faced in getting educated and because of their gender.”

After her PhD, Koel went to University of California Santa Barbara, to pursue postdoctoral studies in pure psychology and brain science. Throughout her decade-long academic journey in California, there was no doubt in her mind that she would eventually return to India. This made it imperative for her to gather as many skills as possible. “During my postdoc, I learnt to do EEGs and fMRIs by myself and developed the tools for running an experimental lab. I did not want to search for and depend on a collaborator for data. I wanted to be self-sufficient in India.”

When she finally returned in 2010, Koel first joined her physicist husband Supratim Sengupta in New Delhi. Supratim at the time had a job at Jawaharlal Nehru University. She looked for positions in and around the city with limited success.

Support in the midst of challenges

For a few months, she worked at South Asian University at New Delhi, but was not satisfied there. “My research was very interdisciplinary and at that time there was no infrastructure [at SAU] for this kind of research,” she recalled. That’s why IISER-K came as a breath of fresh air. Situated about 40 kilometres from the city of Kolkata, the new institute was not only welcoming of Koel’s boundary-blurring research but also her husband’s. “Usually, you will see the wives sacrificing [their careers], following the husband, but for me, it was the other way,” she smiled. “My husband was quite settled at JNU but I said ‘Let’s go!’”

Though she had a Ramalingaswami Fellowship at hand, starting out at her new workplace came with some challenges. Firstly, Koel was in the middle of a difficult pregnancy. Soon as she joined IISER-K in 2011, she went on a maternity leave. She returned in the fall of 2012.

Setting up lab was no walk in the park, either. “I came from a theory background and had no connections in India since I had left after my Bachelor’s,” she said, admitting to having hugely relied on advice from biologist colleagues. Now, her lab is up and running.

After a long wait, Koel has finally been assigned on-campus housing. “Our six-year-old son, who has been parceled off [to family who lives close to the city] while we move, is eagerly waiting to stay on campus since he can play around with all his friends (our colleagues’ kids) and walk to his campus school,” she said in an email update.