Sponsor Talk
Ravi Mahatme
Senior Director, Product Management, Photonic Fabric Business Unit, Marvell
Title: A Shared Memory Architecture for Token-Efficient AI Infrastructure
Abstract:
As AI inference scales, infrastructure performance is increasingly constrained by the ability to move data efficiently between compute and memory rather than by compute alone. Traditional server architectures, built around fixed memory-to-compute ratios, limit memory capacity, infrastructure utilization, and overall AI efficiency.
This presentation introduces a shared memory architecture that enables memory to scale independently of compute while maintaining the low latency and high bandwidth required for modern AI workloads. It examines the architectural principles behind a new shared memory tier, the role of optical connectivity in extending memory across servers and racks, and the system-level considerations required to deliver near-local memory performance at pod scale.
Attendees will gain insight into how this architectural approach improves infrastructure utilization, reduces unnecessary data movement and recomputation, and enables more token-efficient AI systems. The session also explores how advances in optical interconnect technology are making shared memory practical for next-generation AI infrastructure.
This presentation introduces a shared memory architecture that enables memory to scale independently of compute while maintaining the low latency and high bandwidth required for modern AI workloads. It examines the architectural principles behind a new shared memory tier, the role of optical connectivity in extending memory across servers and racks, and the system-level considerations required to deliver near-local memory performance at pod scale.
Attendees will gain insight into how this architectural approach improves infrastructure utilization, reduces unnecessary data movement and recomputation, and enables more token-efficient AI systems. The session also explores how advances in optical interconnect technology are making shared memory practical for next-generation AI infrastructure.
Biography:
Ravi Mahatme is a product and systems leader focused on building next-generation AI infrastructure at the intersection of silicon, photonics, systems, and software.
Ravi joined Marvell through its acquisition of Celestial AI and is currently Senior Director of Product. He leads product strategy for rack-scale AI memory and interconnect platforms based on Photonic Fabric™ optical technology. His responsibilities span product architecture, hardware and software definition, hyperscaler engagement, ecosystem partnerships, and bringing new infrastructure platforms to market.
Prior to Celestial AI, Ravi worked on distributed training and profiling capabilities for large-scale GPU workloads at AWS SageMaker. At Arm, he led product management for Ethos NPUs and spent more than a decade across engineering, business development, and product leadership roles, helping bring processor, accelerator, and platform technologies from concept to deployment.
His interests include memory-centric computing, rack-scale architectures, AI inference efficiency, hardware-software co-design, advanced interconnects, and the economics of scaling AI infrastructure.
Ravi holds an M.S. in Computer Engineering from North Carolina State University and a B.E. in Electronics Engineering from the University of Mumbai.
Ravi joined Marvell through its acquisition of Celestial AI and is currently Senior Director of Product. He leads product strategy for rack-scale AI memory and interconnect platforms based on Photonic Fabric™ optical technology. His responsibilities span product architecture, hardware and software definition, hyperscaler engagement, ecosystem partnerships, and bringing new infrastructure platforms to market.
Prior to Celestial AI, Ravi worked on distributed training and profiling capabilities for large-scale GPU workloads at AWS SageMaker. At Arm, he led product management for Ethos NPUs and spent more than a decade across engineering, business development, and product leadership roles, helping bring processor, accelerator, and platform technologies from concept to deployment.
His interests include memory-centric computing, rack-scale architectures, AI inference efficiency, hardware-software co-design, advanced interconnects, and the economics of scaling AI infrastructure.
Ravi holds an M.S. in Computer Engineering from North Carolina State University and a B.E. in Electronics Engineering from the University of Mumbai.