Research Activities

Actively contributing to advancing Large Language Models and AI Systems at the Frontier Language AI Research (FLAIR) Lab, focusing on efficiency, safety, and multimodal capabilities.

Ongoing Research Projects

Investigating Cache Merging Strategies for Efficient Multimodal LLMs

Current multimodal Large Language Models (e.g., Vision-Language Models) face significant computational challenges when processing long context inputs, impacting inference speed and cost. This project focuses on developing and evaluating novel cache merging techniques specifically tailored for multimodal architectures.

  • Problem Addressed: High latency and computational cost associated with long sequences in multimodal attention mechanisms.
  • Methodology: Exploring strategies like selective key-value (KV) caching, cross-modal attention optimization, and adaptive merging algorithms to reduce the size of the attention cache without significant performance degradation.
  • Goal: To significantly improve the inference efficiency (latency, memory footprint) of multimodal LLMs on tasks requiring long-context understanding, enabling broader applicability.
  • Status: Actively developing methodologies and setting up experimental benchmarks.

Keywords: Multimodal LLMs, Vision-Language Models, Long Context, Inference Optimization, Efficiency, KV Cache, Attention Mechanisms.

Advancing Targeted Unlearning in Large Language Models

Building upon my work for SemEval-2025 Task 4, this research continues to explore robust and efficient methods for machine unlearning or selective knowledge removal in LLMs. The focus is on refining techniques for isolating and modifying specific information (e.g., sensitive data, outdated facts) while preserving the model's general capabilities and mitigating unintended consequences.

  • Problem Addressed: Safely and precisely removing unwanted knowledge from pre-trained LLMs without costly retraining or significant degradation of useful abilities.
  • Methodology: Extending causal-informed optimization techniques, investigating the scalability of layer-specific interventions (like those applied to OLMo models), and evaluating impacts on model safety, fairness, and robustness beyond standard benchmarks like MMLU.
  • Goal: To develop practical and scalable unlearning algorithms that enhance LLM trustworthiness and maintainability for real-world deployment.
  • Status: Exploring extensions and further analysis based on SemEval findings.

Keywords: Machine Unlearning, LLM Safety, Responsible AI, Privacy, Model Editing, Causal Inference, Constrained Optimization, Trustworthy NLP.

Presentations & Academic Engagement

Sharing research findings and engaging with the academic community through presentations:

  • LLM Unlearning Mechanisms & SemEval Contribution 📄
    Venue: FLAIR Lab, TAMU
    Focus: Provided a comprehensive overview of Machine Unlearning concepts, algorithms (including Causal Mediation and Knowledge Editing), and presented mechanistic insights derived from our SemEval-2025 research.
  • Critique of DPO and Knowledge Editing for LLM Detoxification 📄
    Venue: NLP Research Group, TAMU
    Focus: Analyzed limitations of Direct Preference Optimization (DPO) for detoxification, highlighting how it may manipulate attention rather than remove toxicity, and presented Knowledge Editing as a potentially more robust alternative based on recent literature.
  • Adversarial Example Generation via Syntactically Controlled Paraphrasing 📄
    Venue: Trustworthy NLP Seminar, TAMU
    Focus: Examined the Syntactically Controlled Paraphrase Network (SCPN) approach for generating fluent, semantically similar adversarial examples while controlling specific syntactic structures.
  • Exploring the Visual Shortcomings of Multimodal LLMs ("Eyes Wide Shut?") 📄
    Venue: Trustworthy NLP Seminar, TAMU
    Focus: Investigated systematic visual weaknesses in current Multimodal LLMs linked to CLIP limitations ("CLIP-blind pairs"), introduced the MMVP benchmark, and discussed the Mixture of Features (MoF) technique for improving visual grounding.
  • Frugal Streaming Algorithms for Quantile Estimation 📄
    Venue: Data Streaming and Algorithms Course, TAMU
    Focus: Explained the Frugal 1U and Frugal 2U algorithms as memory-efficient methods for estimating quantiles within data stream processing constraints.

Broader Research Interests

Beyond my active projects, I am keenly interested in the intersection of efficient AI systems, cross-modal reasoning capabilities in foundation models, and exploring their potential applications in complex scientific domains.