> For the complete documentation index, see [llms.txt](https://asus-isg-aidc.gitbook.io/guide/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://asus-isg-aidc.gitbook.io/guide/latest/cli/commandreference/ib-sat.md).

# IB Fabric SAT

| Developer | Last modified |
| --------- | ------------- |
| AIDC Team | 2026/07/17    |

## Table of Contents

* [1. Overview](#overview)
* [2. Subcommands](#subcommands)
  * [2.1 `check`](#ib-sat-check)
* [3. Quick Summary](#quick-summary)
* [4. Usage Examples](#usage-examples)
* [5. Report Output](#report-output)

***

## 1. Overview <a href="#overview" id="overview"></a>

{% hint style="warning" %}
**⚠️ Prerequisite: `aidc-cli init` required first**

This command depends on settings saved by [`aidc-cli init`](/guide/latest/cli/commandreference/init.md). Before running it, make sure you have completed:

* [`aidc-cli init inventory`](/guide/latest/cli/commandreference/init.md#inventory) — render the node inventory from `aidc.csv`
* SSH access to the nodes — [`aidc-cli deploy install`](/guide/latest/cli/commandreference/deploy.md) sets this up during OS deployment; for nodes that already have an OS, run [`aidc-cli deploy access-key`](/guide/latest/cli/commandreference/deploy.md) or fill the `password` column in `aidc.csv`

For the full setup sequence, see the [Getting Started](/guide/latest/cli/gettingstarted.md) guide.
{% endhint %}

The `ib-sat` command runs an InfiniBand fabric SAT (Site Acceptance Test) readiness check from the IB management node. It inspects the fabric topology — spine and leaf switches and the HCA ports on compute nodes — and produces a readiness report in `.txt`, `.md`, and `.json` formats.

```bash
aidc-cli ib-sat <subcommand> [flags]
```

{% hint style="info" %}
This check runs on the IB management node and analyses the whole fabric. It is distinct from [`sat`](/guide/latest/cli/commandreference/sat.md), which validates hosts against golden hardware/firmware/OS baselines.
{% endhint %}

***

## 2. Subcommands <a href="#subcommands" id="subcommands"></a>

### 2.1 `check` — Run IB Fabric SAT Readiness Check <a href="#ib-sat-check" id="ib-sat-check"></a>

Runs the fabric readiness check and writes `.txt`, `.md`, and `.json` reports. Switch roles and the expected HCA-port count can be auto-detected or pinned explicitly.

```bash
aidc-cli ib-sat check [flags]
```

| Flag              | Short | Required |              Default             | Description                                                    |
| ----------------- | :---: | :------: | :------------------------------: | -------------------------------------------------------------- |
| `--prefix`        |  `-p` |     ❌    | `<report_path>/ib-sat/<task_id>` | Output prefix path for the `.txt`, `.md`, and `.json` reports. |
| `--spine-keyword` |   —   |     ❌    |            auto-detect           | Keyword to identify spine switches in the IB topology.         |
| `--leaf-keyword`  |   —   |     ❌    |            auto-detect           | Keyword to identify leaf switches in the IB topology.          |
| `--expected-hca`  |   —   |     ❌    |  `0` (auto-detect from majority) | Expected number of HCA ports per compute node.                 |
| `--skip-pattern`  |   —   |     ❌    |                 —                | Regex pattern to skip internal/virtual HCAs by name.           |
| `--stdout`        |  `-o` |     ❌    |              `false`             | Stream the report as JSON to stdout.                           |

{% hint style="info" %}
With `--expected-hca 0` the check infers the expected port count from the majority of nodes, so heterogeneous or partially-cabled fabrics are flagged rather than assumed correct. Switch roles auto-detect by default; override `--spine-keyword` / `--leaf-keyword` only when your naming convention is not recognised.
{% endhint %}

***

## 3. Quick Summary <a href="#quick-summary" id="quick-summary"></a>

| Subcommand | Description                                                             |
| ---------- | ----------------------------------------------------------------------- |
| `check`    | Run an InfiniBand fabric SAT readiness check on the IB management node. |

***

## 4. Usage Examples <a href="#usage-examples" id="usage-examples"></a>

```bash
# Run the readiness check with auto-detection and print the report to the terminal
aidc-cli ib-sat check --stdout

# Pin the expected HCA-port count per node
aidc-cli ib-sat check --expected-hca 8

# Use explicit switch-naming keywords and a custom report prefix
aidc-cli ib-sat check \
  --spine-keyword spine \
  --leaf-keyword leaf \
  --prefix /var/lib/aidc/reports/ib-sat/run-2026-06-29

# Skip internal/virtual HCAs by name pattern
aidc-cli ib-sat check --skip-pattern 'mlx5_bond|i40e'
```

***

## 5. Report Output <a href="#report-output" id="report-output"></a>

With `--stdout` / `-o`, the command streams the report as **JSON** to the terminal (the same data is also written to the `.json` report file alongside the `.txt` and `.md` versions).

The example below is **abbreviated** — it shows a small, healthy 2-tier fabric so the shape of every section is visible. In a real run the repetitive maps (`hca_counts`, `leaf_info`, `spine_leaf`) hold one entry per node/switch, and `link_info.links` holds one entry per link.

```json
{
  "total_sw": 4,
  "n_spine": 2,
  "n_leaf": 2,
  "n_hca": 32,
  "n_node": 4,
  "spine_ports": [64],
  "leaf_ports": [64],
  "sminfo": "sminfo: sm lid 1 sm guid 0xa088c20300ac4cfc, activity count 12345678 priority 15 state 3 SMINFO_MASTER",
  "topo_type": "2-tier Fat-Tree (Leaf-Spine)",
  "link_speeds": ["4xNDR"],
  "expected_hca": 8,
  "hca_counts": {
    "cn001": 8,
    "cn002": 8,
    "cn003": 8,
    "cn004": 8
  },
  "anomalous_nodes": {},
  "leaf_info": {
    "iba-lf01": { "up": 2, "down": 16, "ports": null },
    "iba-lf02": { "up": 2, "down": 16, "ports": null }
  },
  "spine_leaf": {
    "iba-sp01": { "count": 2, "connected": ["iba-lf01", "iba-lf02"] },
    "iba-sp02": { "count": 2, "connected": ["iba-lf01", "iba-lf02"] }
  },
  "missing_spine_leaf": {},
  "total_missing": 0,
  "blocking": [],
  "is_nonblocking": true,
  "link_info": {
    "links": [
      {
        "lguid": "", "lport": "", "rguid": "", "rport": "33",
        "rname": "iba-lf01", "speed": "106.25 Gbps", "width": "4X",
        "state": "Active", "raw": "4X        106.25 Gbps Active/  LinkUp"
      }
    ],
    "degraded": [],
    "speed_dist": { "106.25 Gbps": 32 },
    "width_dist": { "4X": 32 },
    "total": 32
  },
  "n_degraded": 0,
  "link_speed_dist": { "106.25 Gbps": 32 },
  "link_width_dist": { "4X": 32 },
  "link_total": 32,
  "port_errors": {
    "all_errors": [],
    "by_node": {},
    "critical": {},
    "warning": {},
    "total_errors": 0,
    "nodes_with_errors": 0
  },
  "n_error_nodes": 0,
  "n_critical_err": 0,
  "n_warning_err": 0,
  "ibdiagnet": {
    "available": true,
    "raw_summary": "",
    "stages": [
      { "name": "Discovery", "status": "PASSED", "warnings": 0, "errors": 0 },
      { "name": "Links Check", "status": "PASSED", "warnings": 0, "errors": 0 },
      { "name": "Subnet Manager", "status": "PASSED", "warnings": 0, "errors": 0 },
      { "name": "Speed / Width checks", "status": "PASSED", "warnings": 0, "errors": 0 }
    ],
    "ber_issues": [],
    "cable_info": [],
    "dup_guids": [],
    "dup_lids": [],
    "routing_issues": [],
    "link_issues": [],
    "overall_status": "PASS",
    "pass_count": 12,
    "fail_count": 0,
    "warn_count": 0,
    "skip_count": 0
  },
  "ibdiagnet_available": true
}
```

Key sections:

| Field                                                 | Meaning                                                                                                          |
| ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| `n_spine` / `n_leaf` / `n_node` / `n_hca`             | Fabric scale: spine switches, leaf switches, compute nodes, and total HCA ports.                                 |
| `topo_type` / `link_speeds`                           | Detected topology type and the link speeds present in the fabric.                                                |
| `expected_hca` / `hca_counts` / `anomalous_nodes`     | Expected HCA ports per node, the per-node count, and nodes that deviate from the expected count.                 |
| `spine_leaf` / `missing_spine_leaf` / `total_missing` | Spine-to-leaf connectivity, any missing spine→leaf links, and their total count.                                 |
| `blocking` / `is_nonblocking`                         | Leaf switches whose uplinks are insufficient for a non-blocking fabric; `is_nonblocking` is the overall verdict. |
| `link_info` / `n_degraded`                            | Per-link speed/width detail plus degraded-link (e.g. `2X` width) accounting.                                     |
| `port_errors` / `n_error_nodes`                       | Port error counters aggregated per node, split into critical and warning.                                        |
| `ibdiagnet`                                           | `ibdiagnet` stage results and alerts (duplicate GUIDs/LIDs, BER, routing). Requires MLNX\_OFED.                  |

{% hint style="info" %}
The report is **read-only**: it collects fabric state and never changes switch or host configuration. A fabric is SAT-ready when there are no missing spine→leaf links, no blocking leaves, no degraded links, no port errors, and `ibdiagnet` reports no failures.
{% endhint %}

**See also:** [SAT Baselines and Validation](/guide/latest/cli/commandreference/sat.md), [Network Validation](/guide/latest/cli/commandreference/networkvalidation.md).