publications
Journal publications and preprints, listed in reversed chronological order.
2025
- E3-Pose: equivariant symmetry-aware rigid pose estimation in fetal brain MRIRamya Muthukrishnan, Benjamin Billot, Borjan Gagoski, and 3 more authors2025
Rigid pose estimation relative to a canon- ical coordinate frame is paramount in medical imaging. It provides consistent anatomical alignment across time, modalities, and subjects, enabling tasks such as regis- tration and motion tracking. Here we present E3-Pose, a method that estimates a canonical 3D pose from volumetric brain MRI. Importantly, E3-Pose uses a E(3)-equivariant convolutional neural network that exploits intrinsic spa- tial symmetries in rigid pose estimation. In turn, this formulation enables us to introduce a novel 9D rotation parametrization, which uses regular and pseudovectors to tackle ambiguities in contralateral brain symmetry. We show that E3-Pose outperforms state-of-the-art methods on motion quantification in fetal brain MRI, and that its symmetry-driven architectural constraints enable it to gen- eralize to challenging and out-of-distribution samples, such as younger fetuses with underdeveloped anatomy. Finally, we demonstrate utility for the clinical application of auto- navigated slice prescription, where extensive simulations show that E3-Pose significantly improves brain coverage and slice readability. Our code and model are available at https://github.com/ramyamut/E3-Pose.
- arXiVSpatial regularisation for improved accuracy and interpretability in keypoint-based registrationBenjamin Billot, Ramya Muthukrishnan, Esra Abaci-Turk, and 4 more authors2025
Unsupervised registration strategies bypass requirements in ground truth transforms or segmentations by optimising similarity metrics between fixed and moved volumes. Among these methods, a recent subclass of approaches based on unsupervised keypoint detection stand out as very promising for interpretability. Specifically, these methods train a network to predict feature maps for fixed and moving images, from which explainable centres of mass are computed to obtain point clouds, that are then aligned in closed-form. However, the features returned by the network often yield spatially diffuse patterns that are hard to interpret, thus undermining the purpose of keypoint-based registration. Here, we propose a three-fold loss to regularise the spatial distribution of the features. First, we use the KL divergence to model features as point spread functions that we interpret as probabilistic keypoints. Then, we sharpen the spatial distributions of these features to increase the precision of the detected landmarks. Finally, we introduce a new repulsive loss across keypoints to encourage spatial diversity. Overall, our loss considerably improves the interpretability of the features, which now correspond to precise and anatomically meaningful landmarks. We demonstrate our three-fold loss in foetal rigid motion tracking and brain MRI affine registration tasks, where it not only outperforms state-of-the-art unsupervised strategies, but also bridges the gap with state-of-the-art supervised methods. Our code is available at https://github.com/BenBillot/spatial regularisation.
- arXiVLPAC: Learnable perception-action-communication loops with applications to coverage controlSaurav Agarwal, Ramya Muthukrishnan, Walker Gosrich, and 2 more authors2025
Coverage control is the problem of navigating a robot swarm to collaboratively monitor features or a phenomenon of interest not known a priori. The problem is challenging in decentralized settings with robots that have limited communication and sensing capabilities. We propose a learnable Perception-Action-Communication (LPAC) architecture for the problem, wherein a convolution neural network (CNN) processes localized perception; a graph neural network (GNN) facilitates robot communications; finally, a shallow multi-layer perceptron (MLP) computes robot actions. The GNN enables collaboration in the robot swarm by computing what information to communicate with nearby robots and how to incorporate received information. Evaluations show that the LPAC models—trained using imitation learning—outperform standard decentralized and centralized coverage control algorithms. The learned policy generalizes to environments different from the training dataset, transfers to larger environments with more robots, and is robust to noisy position estimates. The results indicate the suitability of LPAC architectures for decentralized navigation in robot swarms to achieve collaborative behavior.
2023
- AAAIInvRT: Solving Radar Inverse Problems with TransformersRamya Muthukrishnan, Justin Goodwin, Adam Kern, and 2 more authors2023
A wide variety of applications rely on the characterization of objects from radar observations. Existing approaches in this space primarily focus on inferring object type. In this work, we focus on inferring the input parameters of a physics-based radar simulator from its output, a temporal sequence of radar observations. Sequential radar observations inherently have a complex spatio-temporal structure that is difficult to capture with many standard vision-based deep learning architectures. We model such complex phenomena as a sequence to sequence prediction problem and use a transformer architecture, taking advantage of its ability to capture contextual temporal dependencies. We demonstrate that our method, Inverse Radar Transformer (InvRT), outperforms baseline approaches in predicting object properties, for both high and low observability settings. Furthermore, its errors are highly correlated with the level of object observability, highlighting its potential to learn the geometric limitations of radar sensing.
- NeurRepsSymmetric Models for Radar Response ModelingColin Kohler, Nathan Vaska, Ramya Muthukrishnan, and 5 more authors2023
Many radar applications require complex radar signature models that incorporate characteristics of an object’s shape and dynamics as well as sensing effects. Even though high fidelity, first-principles radar simulators are available, they tend to be resource intensive and do not easily support the requirements of agile and large-scale AI development and evaluation frameworks. Deep learning represents an attractive alternative to these numerical methods, but can have large data requirements and limited generalization ability. In this work, we present the Radar Equivariant Model (REM), the first SO(3)-equivariant model for predicting radar responses from object meshes. By constraining our model to the symmetries inherent to radar sensing, REM is able to achieve a high-level reconstruction of signals generated by a first-principles radar model and shows improved performance and sample efficiency.
2022
- NeuroImage ClinDeep learning-based automated segmentation of resection cavities on postsurgical epilepsy MRIT. Campbell Arnold, Ramya Muthukrishnan, Akash R. Pattnaik, and 9 more authorsNeuroImage: Clinical, 2022
Accurate segmentation of surgical resection sites is critical for clinical assessments and neuroimaging research applications, including resection extent determination, predictive modeling of surgery outcome, and masking image processing near resection sites. In this study, an automated resection cavity segmentation algorithm is developed for analyzing postoperative MRI of epilepsy patients and deployed in an easy-to-use graphical user interface (GUI) that estimates remnant brain volumes, including postsurgical hippocampal remnant tissue. This retrospective study included postoperative T1-weighted MRI from 62 temporal lobe epilepsy (TLE) patients who underwent resective surgery. The resection site was manually segmented and reviewed by a neuroradiologist (JMS). A majority vote ensemble algorithm was used to segment surgical resections, using 3 U-Net convolutional neural networks trained on axial, coronal, and sagittal slices, respectively. The algorithm was trained using 5-fold cross validation, with data partitioned into training (N = 27) testing (N = 9), and validation (N = 9) sets, and evaluated on a separate held-out test set (N = 17). Algorithm performance was assessed using Dice-Sørensen coefficient (DSC), Hausdorff distance, and volume estimates. Additionally, we deploy a fully-automated, GUI-based pipeline that compares resection segmentations with preoperative imaging and reports estimates of resected brain structures. The cross-validation and held-out test median DSCs were 0.84 ± 0.08 and 0.74 ± 0.22 (median ± interquartile range) respectively, which approach inter-rater reliability between radiologists (0.84–0.86) as reported in the literature. Median 95% Hausdorff distances were 3.6 mm and 4.0 mm respectively, indicating high segmentation boundary confidence. Automated and manual resection volume estimates were highly correlated for both cross-validation (r = 0.94, p < 0.0001) and held-out test subjects (r = 0.87, p < 0.0001). Automated and manual segmentations overlapped in all 62 subjects, indicating a low false negative rate. In control subjects (N = 40), the classifier segmented no voxels (N = 33), <50 voxels (N = 5), or a small volumes<0.5 cm3 (N = 2), indicating a low false positive rate that can be controlled via thresholding. There was strong agreement between postoperative hippocampal remnant volumes determined using automated and manual resection segmentations (r = 0.90, p < 0.0001, mean absolute error = 6.3 %), indicating that automated resection segmentations can permit quantification of postoperative brain volumes after epilepsy surgery. Applications include quantification of postoperative remnant brain volumes, correction of deformable registration, and localization of removed brain regions for network modeling.
- JAMIAExtracting seizure frequency from epilepsy clinic notes: a machine reading approach to natural language processingKevin Xie, Ryan S Gallagher, Erin C Conrad, and 19 more authorsJournal of the American Medical Informatics Association, Feb 2022
Seizure frequency and seizure freedom are among the most important outcome measures for patients with epilepsy. In this study, we aimed to automatically extract this clinical information from unstructured text in clinical notes. If successful, this could improve clinical decision-making in epilepsy patients and allow for rapid, large-scale retrospective research.We developed a finetuning pipeline for pretrained neural models to classify patients as being seizure-free and to extract text containing their seizure frequency and date of last seizure from clinical notes. We annotated 1000 notes for use as training and testing data and determined how well 3 pretrained neural models, BERT, RoBERTa, and Bio_ClinicalBERT, could identify and extract the desired information after finetuning.The finetuned models (BERTFT, Bio_ClinicalBERTFT, and RoBERTaFT) achieved near-human performance when classifying patients as seizure free, with BERTFT and Bio_ClinicalBERTFT achieving accuracy scores over 80\%. All 3 models also achieved human performance when extracting seizure frequency and date of last seizure, with overall F1 scores over 0.80. The best combination of models was Bio_ClinicalBERTFT for classification, and RoBERTaFT for text extraction. Most of the gains in performance due to finetuning required roughly 70 annotated notes.Our novel machine reading approach to extracting important clinical outcomes performed at or near human performance on several tasks. This approach opens new possibilities to support clinical practice and conduct large-scale retrospective clinical research. Future studies can use our finetuning pipeline with minimal training annotations to answer new clinical questions.