2026-d1-0915-keynote-1-lessons-from-networking-metas-gigawatt-scale-ai-fleet

conf26// should work

@Ryan Scherbarth (nvidia) is there a typo in the zoom prompt

@Matthew Fricke / @Subhadeep Bhattacharya I'm not sure where this is configured off the top of my head

just followed the link and type the username/password. it does not work

@Weikai Sun Just to double check, does the prompt match the note in the #C0154F3UDTQ channel?

https://hot-interconnects.slack.com/archives/C0154F3UDTQ/p1786739573517959

Fixed the lobby announcement.

thanks. working now. missed the "//" in password in earlier announcement

Thanks a lot @Omar Baldonado (Meta)! Ethernet connection for RDMA, plus some in-tray specific local links ... like PCIe or UCIe?

That's a pretty good summary - first you have to see for that ASIC what type of server/rack system it is sitting in (and what traffic requirements the ASIC will have in/out) - basically what scale-up is it going to sit in (or if it sits in one at all, vs. being part of say a regular front-end fabric). Then you can see what requirements need to go in/out of the ASIC and that informs the building-level fabric requirements (whether a backend scale-out fabric or a front-end fabric).

"For memory" - do you mean like what is RDMA traffic vs non-RDMA traffic? If so, then yes, we have telemetry from all the switches that is monitoring the traffic (protocol/endpoints/...)

The number of historical 802.x standards should we've continually been putting pressure on Ethernet over the decades 🙂 But seriously, not all of Ethernet is appropriate for scale-up/low-latency/high-bandwidth efficiency, and I wouldn't put an Ethernet switch in the middle of a compute tray for die-to-die communication if I didn't have to. We push for Ethernet because it provides a well-known, mature ecosystem of products, vendors, and researchers, even as we all push it to do new things.

Meta had a paper at SIGCOMM a few years ago (2023?) that gave examples of how we plan and do "drains" of fabrics. A lot of that is applied to our scale-out network. Something we also push for is warm-boot and ISSU-style capabilities - you can see within the OCP Switch Abstraction Interface (SAI) project of how we ask for this, because it lets us do these upgrades with less impact.

Trying 🙂 One of the takeaways I shared is that scale-up is heavily part of the AI system and that varies heavily. ESUN is a multi-vendor initiative within OCP, including a number of those AI vendors. Small note: I believe SUE has moved to be more focused on transport layer (SUE-T), even as it leverages Ethernet underneath.

Great question - internally, we've continually automate/standardize this as much as possible, with higher workload-level benchmarking/test suites as well as component-level benchmarks/test suites. Also, note there is a tradeoff between time to bring up capacity and time to optimize it. So in certain situations, we may take X% of roofline performance and have it working sooner, and then spend time optimizing it in parallel (rather than delaying the capacity, waiting for 100% roofline). This is because often there is so much to learn operationally when inserting a new component, that I want the experience with it as soon as possible.

That makes sense, Omar, getting safe capacity online early can be worth more than waiting for the last few percent. I am wondering then what defines the "safe to run real jobs" gate, and do issues found during bring-up normally become regression tests for future rollouts?

We have a few different levels of qualification gates defined, so that folks that want/get "early access" know that we're going to be working some things out still . And those issues indeed get folded in for future verification test suites for future capacity,

I love routing (back in early 2000's, I worked for a company named RouteScience!). In scale-up, the routing problem is pretty easy 🙂 and in scale-across, there's definitely a similar routing problem. We use BGP actively even in our scale-out fabrics, but where we've had to extend the routing control plane some is to make sure failure handling is fast and propagated well. There's also a lot of work in load balancing (ECMP) to further ensure failures given all the parallel links we have.

@Omar Baldonado (Meta) , does Meta uses MRC?

I'd encourage you to come to our networking @scale event next week to see what we're doing in this transport area! atscaleconference.com/events/networking-2026

It's based on some older generation infrastructure from 2023/2024, but we shared some of AI infrastructure reliability learnings in our lengthy Llama3 paper (arxiv.org/pdf/2407.21783). Section 3.3 covers some of the failure rates/components.

Without going into the specifics of this framework, I can say that it relates to some of the lessons I presented in the talk. For example, the bubble elimination in a collective by overlapping comms and compute is an example where cross-layer/whole system knowledge is required. Topology-awareness is critical for jobs/collectives, especially when dealing with scale-across networks where there is much more heterogeneity in the network (as opposed to scale-up and scale-out). There are multiple ways to bring this sort of awareness into the researchers' development workflow.

I'm not totally aware of this - I know how the switch/network ASIC vendors are responding, but we have our own in-house SD-WAN controller system.

Meta will be talking more about some of our thoughts on protocols and NICs in our Networking @ Scale conference next week. atscaleconference.com/events/networking-2026 - encourage you all to attend! Regarding FPGA-based NICs, yes we have looked at them, but those come with their pros/cons. Often in these technologies, we'll evaluate and use some at a certain scale, even as we push for more scalable/performant solutions in parallel.

Thank you

As you probably saw and refer to, we've been very active in evaluating CPO in general as a technology (initially in scale-out - where we announced some reliability results at ECOC last year). We also announced earlier the OCI MSA, which is related. Optics in scale-up has a clear value prop assuming you are moving to a certain size scale-up (which we want).

I think heterogeneity in the fleet (at different layers like switches, NICs, etc) is different from GPU/XPU interoperability. I think that is a really hard problem and not just because of network but the whole software stack, numerics accuracy & compatibility, etc. In a way, the network protocols are an easier part of the problem.

Does Meta uses disaggregated inference?

It really depends on how interoperable the previous generation of hardware is. upgrading to the next-gen switch or NIC can keep leverage a lot of previous infrastructure, but later AI-system-level upgrades aren't compatible or cost effective with the existing building (e.g., we do liquid cooling in buildings without facility-level cooling, but we have to tradeoff cost/benefit/feasibility of such upgrades)

I see that makes sense Omar thank you so much for the reply.

A lot of plans are focused on training - indeed, our Llama4 pretraining was done using the scale-across connecting buildings of our 129K cluster and we continue to do that. However, "inference" has a lot of different parts now between GPUs, CPUs, and storage. Hard right now to say how broad that will go.