NCP-AIN NVIDIA-Certified Professional AI Networking Questions and Answers

A rail-optimized topology is designed to enhance GPU-to-GPU communication by connecting each GPU's Network Interface Card (NIC) to a dedicated rail switch. This configuration ensures predictable traffic patterns and minimizes network interference between data flows, which is crucial for the performance of large-scale AI workloads, such as training large language models. By reducing contention and latency, this topology supports efficient and scalable AI training environments.​

Reference Extracts from NVIDIA Documentation:

"Rail-optimized network topology helps maximize all-reduce performance while minimizing network interference between flows."​

"A Rail Optimized Stripe Architecture provides efficient data transfer between GPUs, especially during computationally intensive tasks such as AI Large Language Models (LLM) training workloads, where seamless data transfer is necessary to complete the tasks within a reasonable timeframe."​

The NVIDIA Network Operator is designed to simplify and automate the deployment and management of networking components in Kubernetes clusters, particularly those utilizing NVIDIA Spectrum-X switches. It manages the installation and configuration of necessary drivers, plugins, and other networking resources to enable features like RDMA and GPUDirect RDMA, which are essential for high-performance AI workloads.​

By leveraging Kubernetes Custom Resource Definitions (CRDs) and the Operator Framework, the Network Operator ensures that networking components are consistently and correctly configured across the cluster, reducing manual intervention and potential configuration errors.​

The NVIDIA UFM Cyber-AI Platform is specifically designed to enhance security and operational efficiency in InfiniBand data centers. It leverages AI-powered analytics to detect security threats, operational anomalies, and predict potential network failures. By analyzing real-time telemetry data, it identifies abnormal behaviors and performance degradation, enabling proactive maintenance and threat mitigation.​

This platform integrates with existing UFM Enterprise and Telemetry services to provide a comprehensive view of the network's health and security posture. It utilizes machine learning algorithms to establish baselines for normal operations and detect deviations that may indicate security breaches or hardware issues.​

Accessing the BlueField DPU Operating System (OS) is possible through rshim , either over PCIe or USB, and via SSH through the OOB interface when in DPU mode.

From the NVIDIA BlueField Software Documentation :

"You can access the BlueField OS through the rshim interface. The rshim module enables host-to-DPU communication either via PCIe (default) or USB."

B. rshim over PCIe : Default when BlueField is installed in a host.

D. rshim over USB : Useful for provisioning or systems without PCIe drivers.

Incorrect Options:

A (NIC mode) : BlueField acts as a transparent NIC; OS access is not available to the host.

C (Redfish) : Redfish is for out-of-band management, not direct OS-level access.

In Cumulus Linux, microbursts are short-lived, high-volume traffic bursts that often go undetected by coarse-grained monitoring like SNMP.

The two tools specifically used for this purpose are:

What Just Happened (WJH)

"WJH provides real-time packet drop visibility and classifies drops by reason (e.g., congestion, ACLs, etc.), enabling microburst detection."

ASIC monitoring at millisecond granularity

"Deep telemetry is enabled via the switch ASIC, which provides sub-second counters that capture microburst patterns otherwise missed by SNMP."

Incorrect Options:

A and C provide low-frequency sampling, insufficient for microbursts which last milliseconds.

The NVIDIA Spectrum-X Reference Architecture (RA) 1.0.1 is designed for Ethernet AI cloud deployments and includes the SN5600 Spectrum-4 switches and BlueField-3 SuperNICs. This architecture supports adaptive routing and DOCA programmable congestion control (PCC) for lossless RoCE traffic, optimizing performance for AI workloads.

The SN5600 switch offers 64 ports of 800GbE in a dense 2U form factor, providing high throughput and low latency essential for AI applications.

In Cumulus Linux, configuring a bond interface with Link Aggregation Control Protocol (LACP) involves setting the bond mode to 'lacp'. The correct command to achieve this is:​

nv set interface bond1 bond mode lacp​

This command sets the bonding mode of 'bond1' to LACP, enabling dynamic link aggregation for increased bandwidth and redundancy.​

Reference Extracts from NVIDIA Documentation:

"To reset the link aggregation mode for bond1 to the default value of 802.3ad, run the nv set interface bond1 bond mode lacp command."​

The RDMA Shared Device Plugin is a critical component in the NicClusterPolicy custom resource for enabling Remote Direct Memory Access (RDMA) capabilities in Kubernetes clusters. RDMA allows for high-throughput, low-latency networking, which is essential for efficient GPU-to-GPU communication across nodes in AI workloads. By deploying the RDMA Shared Device Plugin, the cluster can leverage RDMA-enabled network interfaces, facilitating direct memory access between GPUs without involving the CPU, thus optimizing performance.​

Reference Extracts from NVIDIA Documentation:

"RDMA Shared Device Plugin: Deploy RDMA Shared device plugin. This plugin enables RDMA capabilities in the Kubernetes cluster, allowing high-speed GPU-to-GPU communication across nodes."​

"The RDMA Shared Device Plugin is responsible for advertising RDMA-capable network interfaces to Kubernetes, enabling pods to utilize RDMA for high-performance networking."​

The MINHOP routing protocol, while efficient in finding minimal paths, does not inherently prevent credit loops. This can lead to deadlocks in the network. In contrast, routing protocols like UPDOWN and FAT TREE are designed to avoid such loops, ensuring more reliable network operation.​

The ibnetdiscover utility is used to perform InfiniBand subnet discovery and outputs a human-readable topology file. GUIDs, node types, and port numbers are displayed, as well as port LIDs and node descriptions. All nodes and links are displayed, providing a full topology. This utility can also be used to list the current connected nodes. The output is printed to the standard output unless a topology file is specified.​

InfiniBand is a high-performance, low-latency interconnect technology used in AI and HPC data centers, particularly for east-west traffic between compute nodes in large-scale fabrics. Ensuring seamless data flow requires tools to troubleshoot and monitor the network, including the ability to trace and discover network paths between nodes. The question asks for the specific tool used to trace and discover paths in an InfiniBand fabric, which is a key task in InfiniBand troubleshooting.

According to NVIDIA’s official InfiniBand documentation, the ibnetdiscover tool is designed to discover and map the topology of an InfiniBand fabric, including the paths between nodes. It scans the fabric, queries the subnet manager, and generates a topology map that details the connections between switches, Host Channel Adapters (HCAs), and other devices. This tool is essential for verifying connectivity, identifying routing paths, and troubleshooting issues like misconfigured routes or link failures in large-scale InfiniBand fabrics.

Exact Extract from NVIDIA Documentation:

“The ibnetdiscover tool is used to discover the InfiniBand fabric topology and generate a map of the network. It queries the subnet manager to retrieve information about all nodes, switches, and links in the fabric, providing a detailed view of the paths between nodes. This tool is critical for troubleshooting connectivity issues and ensuring proper routing in InfiniBand networks.”

— NVIDIA InfiniBand Networking Guide

This extract confirms that ibnetdiscover is the correct tool for discovering network paths in an InfiniBand east-west fabric. It provides a comprehensive view of the fabric’s topology, enabling administrators to trace paths between compute nodes and ensure seamless data flow.

MLNX_OFED (Mellanox OpenFabrics Enterprise Distribution) is an NVIDIA-tested and packaged version of the OpenFabrics Enterprise Distribution (OFED) for Linux. It provides the necessary drivers and tools to support InfiniBand and Ethernet interconnects using the same RDMA (Remote Direct Memory Access) and kernel bypass APIs. MLNX_OFED enables high-performance networking capabilities essential for AI clusters, including support for up to 400Gb/s InfiniBand and RoCE (RDMA over Converged Ethernet).​

Reference Extracts from NVIDIA Documentation:

"MLNX_OFED is an NVIDIA tested and packaged version of OFED that supports two interconnect types using the same RDMA (remote DMA) and kernel bypass APIs called OFED verbs – InfiniBand and Ethernet."​

"Up to 400Gb/s InfiniBand and RoCE (based on the RDMA over Converged Ethernet standard) over 10/25/40/50/100/200/400GbE are supported."​

For optimal performance in multi-node, multi-GPU AI workloads:

NVLink provides high-speed, low-latency communication between GPUs within the same node.

InfiniBand offers efficient, scalable communication between nodes in a data center. Combining these technologies ensures both intra-node and inter-node communication needs are effectively met.​

Virtual Routing and Forwarding (VRF) is the most effective method to achieve network segmentation and isolation in a multi-tenant environment.

From the NVIDIA Cumulus Linux Documentation – VRF Section :

"VRF allows multiple instances of routing tables to coexist within the same switch, effectively isolating traffic between tenants or departments."

Each department can:

Operate in its own VRF domain

Have independent routing tables

Maintain strict separation of Layer 3 paths

Incorrect Options:

A (Port Mirroring) – Used for traffic monitoring, not isolation.

C (ACLs) – Useful for fine-grained filtering, but not scalable tenant isolation.

D (LLDP) – Used for neighbor discovery, not security or isolation.

Comprehensive and Detailed 150 to 250 words of Explanation From NVIDIA AI Networking Topics:

The correct answer is A. Cyber-AI platform . NVIDIA’s official UFM Cyber-AI documentation states that the UFM Cyber-AI platform determines a data center’s unique “vital signs” and uses them to identify performance degradation, component failures, and abnormal usage patterns . This directly matches the requirement for proactive detection of issues in a large InfiniBand fabric. NVIDIA also documents that Cyber-AI provides Link Anomaly detection and Link Failure Prediction , where machine learning models analyze monitoring information and predict future link failures 1 to 24 hours in advance . In addition, NVIDIA’s Cyber-AI materials describe integration with AI cybersecurity capabilities to help detect and mitigate security threats , which makes Cyber-AI the only option in this list that combines predictive analytics with security-oriented intelligence. By contrast, UFM Telemetry focuses on collecting and presenting real-time network data such as bandwidth, congestion, latency, and errors, while UFM Enterprise is the broader management platform and Host Agent is not the AI analytics engine for predictive failure and threat detection. Therefore, for AI-driven analytics, predictive maintenance, anomaly detection, and security-focused analysis in InfiniBand environments, Cyber-AI platform is the verified NVIDIA answer.

NVIDIA NetQ is a network operations tool that provides real-time visibility and automated validation of the network fabric. It helps in identifying misconfigurations, monitoring network health, and ensuring that the fabric meets the required specifications for AI workloads.​

Comprehensive and Detailed 150 to 250 words of Explanation From NVIDIA AI Networking Topics:

The correct answer is B. nv set qos roce enable on . In NVIDIA Cumulus Linux, RoCE configuration is managed under the QoS hierarchy in NVUE because RoCE depends on lossless Ethernet behavior, priority flow control, congestion management, and related quality-of-service mechanisms. NVIDIA’s official Cumulus Linux documentation shows that the RoCE feature is configured through nv set qos roce , and in newer command references this appears explicitly as nv set qos roce enable on|off . This confirms both the command structure and the ordering of the keywords: nv set → qos → roce → enable on .

The other options are incorrect because they do not follow valid NVUE syntax. NVIDIA documents NVUE commands in a strict object-based order, where the top-level feature group comes immediately after nv set . Since RoCE is implemented as part of the QoS framework, qos must appear before roce . Commands such as nv set roce qos enable on , nv qos roce enable on , and nv roce qos enable on do not match the official NVIDIA command model. Therefore, for enabling RoCE on a Cumulus switch in an NVIDIA AI networking environment, the verified command is nv set qos roce enable on .

In DPU Mode , the ARM cores on BlueField own the NIC data path , while still allowing the host system to access the DPU OS (via OOB or virtio) .

From NVIDIA BlueField Documentation:

"In DPU Mode, the data path is offloaded to the BlueField Arm cores, enabling advanced security and networking functions, while still allowing host access to the BlueField OS."

This is different from:

NIC Mode : Data path controlled by host, ARM cores inactive.

Separated Host Mode : Complete isolation; host cannot access DPU OS.

Restricted Mode : Limited host access to DPU OS, but without full offload capabilities.

NVIDIA NetQ is a highly scalable network operations toolset that provides visibility, troubleshooting, and validation of networks in real-time. It delivers actionable insights and operational intelligence about the health of data center networks—from the container or host all the way to the switch and port—enabling a NetDevOps approach.

NetQ can be used as the functional test platform for the network CI/CD in conjunction with NVIDIA Air. Customers benefit from testing the new configuration with NetQ in the NVIDIA Air environment (“digital twin”) and fix errors before deploying to their production. ​