Triton-distributed-ascend documentation

Triton-distributed-ascend is a distributed computing framework extended from the Triton language and optimized for Ascend AI processors. It enables efficient distributed computing on Ascend NPUs, with communication primitives and computation-communication overlap optimizations aimed at large-scale distributed training and inference.

The project builds on the multi-level abstraction capabilities of MLIR to map distributed operators written in Triton onto Ascend AI processors, and exposes fine-grained interfaces for controlling on-chip memory and pipeline synchronization.

Getting Started

Work through these in order:

• Installation — hardware and software prerequisites, CANN setup, and Python dependencies.

• Build from Source — building Triton-distributed-ascend, AscendNPU-IR, and shmem from source.

• Quick Start — running your first distributed kernel and verifying the installation.

See Release Note for the versioning policy, branch lifecycle, and compatibility matrices.

Key Features

• Distributed communication primitives — AllReduce, AllGather, ReduceScatter, All2All.

• Computation-communication overlap — overlapped execution for improved throughput.

• AscendNPU IR-based compiler optimizations — optimizations that use Ascend hardware features directly.

• Python-friendly interfaces — concise Python APIs that integrate into existing deep learning frameworks.

Architecture

• Distributed Architecture Design — Distributed architecture design, terminology, compilation pipeline, and communication engine details.

API Reference

The API documentation covers five main areas:

• Triton-distributed Semantics — Design philosophy, semantic model, and programming patterns for tile-centric computation-communication overlap.

• triton_dist.language API — Distributed operations for Triton kernels (wait, consume_token, rank, num_ranks, symm_at, notify, extern_call).

• SHMEM Device API — Device-side ACLSHMEM APIs for use within kernels (RMA operations, signaling, barriers, and synchronization).

• SHMEM Host API — Host-side ACLSHMEM APIs for initialization, memory management, and host-initiated communication.

• Swizzle Helper Functions — Swizzle helper functions for GEMM and communication tile scheduling.

• Autotuning API — Distributed autotune API for multi-device kernel tuning.

Developer Guide

• Kernel Debugging — Extracting IR, printing compiler passes, and debugging synchronization issues with bishengir-compile flags.

• Environment Variables and Compiler Options — Environment variables and compiler options for debugging and optimization.

• Multi-Card Communication-Computation Fusion Operator Precision Verification — Multi-card fusion operator precision verification.

• Single Operator Development — Single operator development with distributed kernels.

• Performance Optimization Guide — Performance optimization techniques and best practices.

• Operator Performance Testing and Tuning: autotune Feature Usage Guide — Operator performance testing and autotune feature usage.

• Unified Ascend operator tests — Kernel performance profiling guide.

Tutorials

• AllGather-GEMM (Allgather Gemm) — AllGather+GEMM fusion optimization tutorial.

• Autotuning Demo: UDMA Reverse All2All Optimization — Autotune example and optimization practices.

• Reverse All2All (EP MoE Distributed Kernel) — Reverse All2All for MoE distributed kernels.

FAQ

• FAQ Documentation — Frequently asked questions and troubleshooting.