Jamuna S Murthy
Assistant Professor · Ramaiah Institute of Technology
I am an Assistant Professor in the Department of Computer Science and Engineering at Ramaiah Institute of Technology, Bengaluru, India, affiliated with the Center for Imaging Technologies. My research centers on vision–language models and scalable AI systems — models that ground language in images, video, and audio, and the efficient architectures and infrastructure that let them run reliably in the real world, from edge devices to cloud services. My latest focus is the theory of controllable image generation: treating controllability as a property of structured latent geometry rather than inference-time conditioning (Controlla), and measuring what generated images lose in meaning (KathaTrace), both as a research collaborator with Prof. Rajiv Ramnath's CETI research group at The Ohio State University. I received my M.Tech. from Ramaiah Institute of Technology, where my thesis won the Best Master's Thesis Award (2017), and in 2024 I received the STEM Young Researcher Award.
News
- Sep 2026 New Multimodal Humor Detection in Videos via Domain-Specific Representations and Deep Hybrid Fusion Networks — up to 95.8% accuracy on IEMOCAP, CMU-MOSI, and CMU-MOSEI — accepted in Soft Computing (Springer Nature).
- Jul 2026 CAMMEL-ViT — context-aware memory with lightweight vision transformers for 2D and 3D medical image segmentation — accepted and presented at MIUA 2026, University College Dublin (video).
- Jul 2026 KathaTrace — diagnosing semantic trajectory collapse in generated visual narratives, with the KathaBench-25K benchmark — published on arXiv and under submission to a CORE A* venue. New work with the CETI research group, The Ohio State University.
- May 2026 Controlla — learning controllability via graph-constrained latent geometry, with the AffectHuman-43K benchmark — published on arXiv and under submission to a CORE A* venue. New work with the CETI research group, The Ohio State University.
- 2026 Serving as a reviewer for AAAI 2027, AISTATS 2027, and ACM Multimedia 2027.
- 2025 Serving as a reviewer for NeurIPS 2026, AAAI 2026, AISTATS 2026, ACM Multimedia 2026, and ACM Multimedia Asia 2026.
- Nov 2025 Completed the Samsung PRISM worklet NextGen AI Datastore for multimodal video retrieval as faculty mentor (certificate).
- Aug 2025 FewShot-SPT — few-shot spatiotemporal perception transformer for unseen behavioral anomalies, deployed on edge hardware — published at IEEE AVSS 2025 (CORE A). Work in collaboration with The Ohio State University and Ming Chi University of Technology.
- Apr 2025 Joined Verity Labs (California) as an AI Researcher on the AI-ACC-Stack project.
- 2025 Deep Hybrid Fusion published in Multimedia Tools and Applications.
- Nov 2024 Received the Young Researcher Award from the STEM Research Society.
- Jun 2024 Faculty mentor for the Samsung PRISM program on expression correction in photos.
Research Interests
Two threads — vision–language models and scalable AI systems — with controllable image generation as my current theoretical focus.
Controllable Image Generation: Theory Latest
Controllability as a property of latent geometry — factorized identity and attribute spaces aligned with graph priors through optimal transport, so edits follow consistent trajectories while identity is preserved — and principled measures of whether generated images still carry the intended meaning.
Vision–Language & Multimodal Understanding
Grounding language in visual and acoustic signals — cross-modal attention and hybrid fusion of text, image, and audio representations for understanding subtle, human-centric content such as humor and sarcasm in video.
Efficient Vision Transformers for Real-World Perception
Lightweight, memory-efficient transformers that keep global context affordable — for medical image segmentation, few-shot video anomaly detection on edge hardware, real-time driver assistance, and surgical video analytics.
CAMMEL-ViT · FewShot-SPT · ObjectDetect · Surgeon's Eye (patent)
Scalable AI Systems
Infrastructure that takes models to production — vision–language retrieval over large video collections, RAG and tool-calling agent stacks, and low-latency streaming analytics on distributed systems.
Video Retrieval Datastore · AI-ACC-Stack · TwitSenti · Twitter Trend Analysis
Selected Publications
- Under Submission
KathaTrace: Diagnosing Semantic Trajectory Collapse in Generated Visual Narratives - Under Submission
Controlla: Learning Controllability via Graph-Constrained Latent Geometry - Journal 2026
Multimodal Humor Detection in Videos via Domain-Specific Representations and Deep Hybrid Fusion Networks - MIUA 2026
CAMMEL-ViT: Context-Aware Memory with Lightweight Vision Transformers for Medical Image Segmentation - AVSS 2025
Towards Secure Video Surveillance: A Few-Shot Spatiotemporal Perception Transformer for Unseen Behavioral Anomalies - Journal 2025
Multimedia Video Analytics using Deep Hybrid Fusion Algorithm - Journal 2024
A Smart Video Analytical Framework for Sarcasm Detection using Adaptive Fusion and SarcasNet-99 - Journal 2022
ObjectDetect: A Real-Time Object Detection Framework for Advanced Driver Assistant Systems Using YOLOv5
Show more publications
Full list on the publications page and on Google Scholar.
Experience
- Assistant Professor — Ramaiah Institute of Technology, Department of CSE
- Lab affiliation: Center for Imaging Technologies.
- Research collaborator with Prof. Rajiv Ramnath's CETI research group, The Ohio State University, on controllable image generation and generated visual narratives.
- Teaches Artificial Intelligence and Machine Learning, Object Oriented Programming, Cloud Computing and Big Data, and Introduction to Deep Learning; faculty sponsor of the RIT ACM Student Chapter.
- AI Researcher — Verity Labs Inc. (AI-ACC-Stack)
- Built a modular AI backend stack covering retrieval-augmented generation, LLM integration, document ingestion, tool-calling agents, structured querying, and authentication.
- Deployed AI services on cloud platforms (AWS, Azure, GCP) with cross-functional teams, using PyTorch, TensorFlow, and Scikit-learn.
- Faculty Mentor — Samsung PRISM (Samsung R&D)
- Expression Correction in Photos: GAN-based affective computing models for emotion-aware image enhancement, including blink and expression refinement. Certificate
- NextGen AI Datastore for Video Retrieval (May – Nov 2025, completed): a scalable vector database for multimodal video retrieval with vision–language embeddings, focused on semantic search, indexing, and real-time retrieval. Certificate
- Assistant Professor — BNM Institute of Technology, Department of ISE
- Head of the Google Cloud Center of Excellence.
- Assistant Professor — PES University, Department of CSE
- Lab affiliation: Centre for Data Science & Applied Machine Learning (CDSAML).
- Research Assistant — Indian Institute of Science, Department of CDS
- Intern — Amazon
Funding & Patents
- FitQua: Smart Water Bottle for AI-Based Health Tracking — DST-funded project
- Surgeon's Eye: Intelligent Video Analytical Framework for Robotic and Laparoscopic Surgery — Patent
Academic Service
Reviewing
| Venue | Reviewing |
|---|---|
| NeurIPS | 2026 |
| AAAI | 2025, 2026, 2027 |
| AISTATS | 2026, 2027 |
| ICCV | 2025 |
| ECCV | 2026 |
| ACM Multimedia | 2025, 2026, 2027 |
| ACM Multimedia Asia | 2025, 2026 |
| Springer Nature Journals | Gold Reviewer, 2020 – presentScientific Reports · Journal of Big Data · Signal, Image and Video Processing · Discover Informatics · Language Resources and Evaluation · Discover Artificial Intelligence · International Journal of Machine Learning and Cybernetics · Journal of Ambient Intelligence and Humanized Computing · Discover Education · International Journal of Computational Intelligence Systems · The Journal of Supercomputing |
Student Mentoring
- Smart India Hackathon 2023 — Winning team, as faculty mentor
- Unisys Innovation Program — Faculty mentor
- ICPC — Mentor and coach
- Samsung PRISM — Faculty mentor
Leadership & Memberships
- Faculty Sponsor, RIT ACM Student Chapter
- Member, Association for Computing Machinery (ACM)
- Lifetime Member, Indian Society for Technical Education (ISTE)
Awards & Honors
- 2024 Young Researcher Award, STEM Research Society.
- 2023 Smart India Hackathon — Winner (as faculty mentor of the winning team), Ministry of Education, Government of India. Certificate
- 2017 Best Master's Thesis Award, Ramaiah Institute of Technology. For Distributed Framework for Real-Time Twitter Sentiment Analysis and Visualization (TwitSenti).
- 2016–17 TEQIP-funded Graduate Teaching Assistantship, Ramaiah Institute of Technology.
Education
- M.Tech. in Computer Software Engineering — Ramaiah Institute of Technology, Bengaluru
- B.E. in Computer Science and Engineering — Dr. Ambedkar Institute of Technology, Bengaluru
