Multi-modal and high-throughput behavioral phenotyping

12 minute read

Phenotyping refers to the process of observing and measuring the physical and biological traits of an organism, also known as its phenotype. In the context of animal behavior studies, phenotyping might involve assessing animals’ behavior, motor functions, cognitive abilities, and other brain-related traits. The goal is often to understand how these traits might be influenced by various factors such as genetic modifications, environmental changes, or neurological diseases.

Experiments where, e.g., many cameras or a broad range of different sensors are used to track as much of the animal’s behavior as possible are typically referred to as multi-modal behavioral phenotyping or tracking studies. These approaches allow to observe and quantify a large number of behavioral parameters simultaneously and with a high degree of precision (Ziv et al. (2013)ꜛ, Anderson& Perona (2014)ꜛ, Gomez-Marin et al. (2014)ꜛ, Hong et al. (2015)ꜛ, Berman et al. (2014)ꜛ).

Multi-modal behavior tracking enables the recording of complex behavioral phenotypes. Image generated with DALL-E (sourceꜛ)

These methods often employ machine vision algorithms and other automated systems to detect and track the movement and behaviors of the animals being studied. Some setups can record a wide array of behaviors, from simple locomotor activity to complex social interactions, in diverse species ranging from insects to rodents and more. They provide a holistic view of animal behavior, making it possible to capture subtle changes that may be missed in more conventional assays.

Furthermore, such multi-modal tracking setups are often paired with other neuroscientific methods, like calcium imaging or optogenetics, allowing researchers to correlate specific neuronal activity with particular behaviors, thus providing a more in-depth understanding of the neural underpinnings of behavior.

These high-throughput and automated tracking approaches are becoming increasingly important in neuroscience and other fields, especially in the context of studying neurodegenerative diseases or the effects of drugs, where subtle behavioral changes may have significant implications.

What are high-throughput behavioral experiments?

High-throughput behavioral experiments refer to a type of experimental setup where a large number of animals can be observed and their behavior tracked simultaneously. These setups typically employ automated tracking systems and machine learning algorithms to detect and quantify a wide array of behavioral parameters with a high degree of precision and minimal human intervention (Hong et al. (2015)ꜛ, Anderson& Perona (2014)ꜛ, Berman et al. (2014)ꜛ, Gomez-Marin et al. (2014)ꜛ).

High-throughput behavioral phenotyping allows for the systematic analysis of large cohorts of animals in socially behaving groups. Image generated with DALL-E (sourceꜛ)

These high-throughput experiments serve multiple purposes. Firstly, they allow for the observation of more naturalistic social behaviors, as conditions mimic the wild more closely than in isolated experiments. Secondly, they significantly increase the amount of data that can be collected at a given time, thereby enhancing the statistical power and reliability of the findings.

High-throughput behavioral experiments have been used in a variety of contexts, from studying the effects of genetic modifications to understanding the impact of environmental changes or drug treatments on animal behavior. They have been particularly valuable in the field of neuroscience, where they have been also paired with techniques like optogenetics and imaging to correlate specific patterns of neural activity with particular behaviors.

Examples

In the following, some example of multi-modal and high-throughput behavioral experiments are presented. Note, that this list is not exhaustive, and that there are many other examples of such experiments in the literature. In principle, any combination advanced tracking and imaging techniques can be used to study animal behavior in a multi-modal and high-throughput fashion.

Before we begin, let’s shortly recap what we understand by common behavioral experiments for mice (in contrast to multi-modal and high-throughput behavioral experiments). The following overview is classified according the cognitive or sensory system the according experiment addresses, following Buccafusco (2008)ꜛ. However, the classification is not strict, as many experiments address multiple systems at once:

Common behavioral experiments for mice.

Two-photon calcium imaging of moving mice

Two-photon calcium imaging is a powerful technique used to capture neural activity in awake and moving animals, including mice on a linear treadmill. It employs the principles of two-photon excitation microscopy and the use of calcium indicators to visualize neuronal activity, as the intracellular calcium concentration is tightly linked with neuronal firing (Dombeck et al. (2010)ꜛ, Rickgauer et al. (2014)ꜛ, Somarowthu et al. (2021)ꜛ, Voigts & Harnett (2018)ꜛ).

Sketch depicting a two-photon calcium imaging experiment of a mouse on a linear treadmill. Several other sensors such as cameras and speedometers can be used to record additional behavioral parameters.

When a mouse is placed on a linear treadmill, it can freely move (i.e., run or walk) while the activity of neurons in a specific brain area can be recorded. The advantages of this setup include maintaining physiological relevance by allowing the animal to move and behave more naturally and the ability to observe brain activity during particular behaviors.

These imaging studies can be further enriched by combining them with simultaneous sensor recordings of various behaviors. Cameras and other sensors can be employed to track and record additional parameters such as pupil dilation, whisker movements, paw movements, and nose movements. Additionally, speedometers can measure the animal’s movement speed on the treadmill. By correlating these behavioral data with the neuronal activity captured by two-photon calcium imaging, one can gain deep insights into the neural substrates of behaviors.

Recording of the calcium activity in mice brain. YouTube video source (Columbia University channel)ꜛ

Tracking of a mouse hand using a high-speed camera and DeepLabCutꜛ. YouTube video source (DeepLabCut channel)ꜛ

Miniscopes

Miniature microscopes, or “miniscopes”ꜛ (check also open-source Miniscopeꜛ for reference), are portable, lightweight microscopy devices that allow the imaging of neural activity in freely moving animals. These devices can be attached to the head of small animals, like mice, and used to record the activity of hundreds to thousands of neurons in specific regions of the brain in real time. As a result, one can correlate the neural activity with the animal’s behavior, offering unprecedented insights into how different patterns of neural activity drive different behaviors (Ghosh et al. (2012)ꜛ, Ziv et al. (2013)ꜛ).

Sketch depicting a multi-modal behavioral experiment with a miniscope. The miniscope experiment, i.e., the recording of neural activity, is combined with a common behavioral experiment, e.g., assessing social behavior in an open field arena, and multiple cameras to record the mice’ behavior.

In the context of multi-modal behavior phenotyping, miniscopes can provide crucial neurophysiological data that can be matched with the rich behavioral data obtained from multi-modal behavioral tracking systems. By combining miniscopes with common behavioral experiments, we can gain a comprehensive picture of how specific patterns of neural activity give rise to diverse behaviors. For instance, using a miniscope, one could record the neural activity in the mouse’s hippocampus while the mouse is exploring its environment, e.g., in a Y-maze. Since miniscopes enable the free movement of the animal, one can also record the neural activity while the mouse is performing more complex behaviors or interacting with other mice.

Example of in-vivo imaging of freely moving mice with a miniscope in hippocamus CA1 region. YouTube video source (Miniscope Org channel)ꜛ

Miniscope recording of mice on long track. YouTube video source (Miniscope Org channel)ꜛ

Automated home-cage monitoring systems

Automated behavioral assessment systems are used to track and record the behavior of animals in a high-throughput manner. These systems can be used to study a wide range of behaviors, including social interactions, locomotion, and feeding. They are typically composed of a camera placed above the animals’ cage and a computer vision algorithm that can detect and track the animals. Specific computer vision algorithm can be trained to detect and track specific body parts, such as the animals’ nose, ears, and tail. The algorithm can also be trained to detect and track multiple animals simultaneously. The data obtained from these systems can be used to quantify the animals’ behavior and to study the effects of genetic, neuropathological, pharmacological, and environmental factors on behavior.

Sketch depicting a high-throughput and multi-modal behavior experiment in an automated home-cage monitoring system. Several animals can be monitored simultaneously in their home cage. The system can be equipped with multiple cameras and sensors to record the animals’ behavior. Since such system usually require a minimum of human intervention and, therefore, reduce the human bias on the experiment, they can be used to monitor the animals for long periods of time (e.g., weeks or months).

One such system is, e.g., the IntelliCageꜛ. The IntelliCage is a fully automated system used for behavioral experiments with mice. It’s designed to allow long-term, high-throughput, and minimal-stress monitoring of behaviors in a socially and environmentally enriched setting. The system comprises of a large cage that can house multiple mice together and is equipped with four operant conditioning corners. Each corner has a set of doors that grant access to water bottles, and these doors can be independently controlled to design a variety of behavioral tasks, including learning and memory tasks Kiryk et al. (2020)ꜛ.

The IntelliCage allows to collect continuous and highly detailed data on individual mouse behavior, despite them living in a social group. It uses transponders (RFID chips) implanted in the animals to identify each individual mouse when it visits the conditioning corners. The system can track and record a variety of behaviors, including nose pokes, licks, corner visits, and more. It’s been used to study a range of behaviors and neurological disorders in mice.

Movie demonstrating the functionality of the IntelliCage. YouTube video source (TSE Systems channel)ꜛ

Wireless optogenetic systems

Optogenetical stimulation usually requires a physical connection between the light source and the animal. This can be problematic for long-term studies, as the physical connection can interfere with the animal’s natural behavior. Wireless optogenetic systems allow for minimally invasive control of specific neurons within freely moving animals, opening up new possibilities for long-term studies of neural circuits in naturalistic settings.

Sketch depicting a high-throughput and multi-modal behavior experiment in a home-cage combined with optogenetical stimulation. Several animals can be monitored simultaneously by multiple sensors under controlled optogenetic stimulation.

Neuroluxꜛ is such a system, developed to provide wireless, battery-free optogenetic stimulation that can be used in freely behaving animals. It consists of implantable optoelectronic devices that are powered and controlled via induction, eliminating the need for any physical connections. This allows for minimally invasive control of specific neurons within freely moving animals, opening up new possibilities for long-term studies of neural circuits in naturalistic settings. The system can be combined with many of the common behavior experiments shown in the overview above (Ouyang et al. (2023)ꜛ, Luyao et al. (2018)ꜛ, Park et al. (2015)ꜛ).

One of the key benefits of such systems is that they facilitates experiments that were difficult or impossible with tethered setups, such as social interactions, complex motor behaviors, or studies in larger, more complex environments. This makes these systems a valuable tool to study the neural circuits underlying cognition and behavior in conditions that closely mimic the natural environment.

Wireless optogenetic stimulation of a freely moving mice in the Neurolux system. YouTube video source (NeuroLux Optogenetics channel)ꜛ

What is multi-modal behavior phenotyping