2026-d3-0900-tutorial-track-1-gpu-communication-libraries-for-accelerating-hpc

For the session of "GPU Communication Libraries for Accelerating HPC and AI Applications" at Hot Interconnects 2026 please familiarize yourself with document below. It explains first steps required for the tutorial. In order to get the most of our tutorial time, please log in to the machine before the course. We want to focus on actual content, and not debugging remote connections. If any issue arises please do not hesitate to contact me.

great opportunity for hands on! @channel

Anyone able to tell us the magic constant?

I don't believe I have any institutional access, is there any way to log in with a personal email?

Yes, please follow USAGE.pdf above.

The Jupyter link seems to be not responding

Michal will help us take a look after he is done speaking, I am sure.

You have some time until exercises start

@here After setting up JuDoor it will take 15 mins for the changes to reflect.

Yes, jupyter.jsc.fz-juelich.de/workshops/hoti26 seems to be timing out

To help you out with access via SSH: apps.fz-juelich.de/jsc/hps/jupiter/access.html#…

Please try to do exercise 0 in the background.

One question, why in broadcast, root is not excluded for receiving?

So, device sends data to itself?

I was able to connect via SSH! Trying exercise 0

By the end of today you will be able to write fused GEMM + Allreduce using NCCL device APIs. 😃 please let us know if you are facing any issues on the labs

Should I be using the files directly in /e/project1/training2633/hoti-tutorial or copy them to my home dir? I'm guessing copy

@skimberk1 you can use jsc-material-sync that would take care of the copy

I am still getting permission denied on public key. Probably it hasn't gone through yet. Is the material available in any other places than Jupiter systems?

Yeah I already set that phrase

Github repo would be awesome! thanks

github.com/NVIDIA/hoti-gpu-comms-tutorial Tutorial material

Not sure if there was a specific system you needed to add SSH keys to, I added to them all

Can I find api docs somewhere?

if it wasn't quiet, in what form does the notification arrive at the peer and how is it handled?

is there any kind of fencing that guarantees completion before moving ahead

We can hear you loud and clear!

Getting the following error running the solved nccl put:

[kimberk1@jpbl-s03-01 02-host-put-symmetric-operands]$ make run_SOLVED
salloc --reservation hoti26 --partition booster --gres=gpu:4  --time 0:10:00 srun --cpu-bind=sockets --unbuffered -n 4 ./host_put_symmetric_operands_SOLVED
salloc: Granted job allocation 1445768
salloc: Waiting for resource configuration
salloc: Nodes jpbo-002-16 are ready for job
This exercise requires exactly two MPI ranks.
srun: error: jpbo-002-16: tasks 0-3: Exited with exit code 1
salloc: Relinquishing job allocation 1445768
make: *** [Makefile:34: run_SOLVED] Error 1

Got the solution for this exercise to run! Haha I'm not quite quick enough to actually think it through/code it up myself

PE 0 observed 0
PE 1 observed 42

For the original question...I feel like it wouldn't be defined on pe1? I forget the exact function calls, but I don't think we were doing any waits on pe1 (but I'd have to look at the slide again)

If it makes it easier jupyter.jsc.fz-juelich.de/workshops/hoti26 is back online

I was able to run the 5b solution (after setting export NP=2). Looked at the problem code too, I think i'd be able to figure it out with a bit more time

I guess no Makefile for this one, I'll try following the README.md instructions with setting NVSHMEM_HOME=/usr/local/nvshmem

can you pls post the open source link to the more PPA optimized GEMM?

Looks like the py Makefile is failing bc NCCL4PY_SOURCE isn't defined

Link to NCCL-EP paper: arxiv.org/abs/2603.13606

Will you be publishing slides?

how can I an account to practice the tutorial

I was late for the tutorial

@deyuzhou see the pinned messages

@Sayan Ghosh Can I sign up later today to tomorrow or has to be today during the session?

That was great, thanks! To what extent are you planning on supporting these exercises/tutorials for the future? Is this going to be an ongoing tutorial or one-and-done?

@m.rajski please see below and comment. Thanks!

Thanks for joining this tutorial session. Our next tutorial "Principles and Practice of Scalable and Distributed AI Training and Inference" in track 1 will start at 1:30pm PT: #C0BQ9G452NQ

I’m currently looking for an opportunity where I can gain hands-on experience and become involved in practical lab/research work. I would be grateful if you could review my CV and suggest where I should apply or which opportunities may be suitable for my background. If there is any opportunity to work with you or your team, either online or physically, I would be very interested. Your guidance would mean a lot to me.