ACI Startup Discovery

The clustered Cisco Application Policy Infrastructure Controller (APIC) provides DHCP, bootstrap configuration, and image management to the fabric for automated startup and upgrades.

The Cisco Nexus ACI fabric software is bundled as an ISO image, which can be installed on the Cisco APIC server through the KVM interface on the Cisco Integrated Management Controller (CIMC). The Cisco Nexus ACI Software ISO contains the Cisco APIC image, the firmware image for the leaf node, the firmware image for the spine node, default fabric infrastructure policies, and the protocols required for operation.

The ACI fabric bootstrap sequence begins when the fabric is booted with factory-installed images on all the switches. The Cisco Nexus 9000 Series switches that run the ACI firmware and APICs use a reserved overlay for the boot process. This infrastructure space is hard-coded on the switches. The APIC can connect to a leaf through the default overlay, or it can use a locally significant identifier.

The ACI fabric uses an infrastructure space, which is securely isolated in the fabric and is where all the topology discovery, fabric management, and infrastructure addressing is performed. ACI fabric management communication within the fabric takes place in the infrastructure space through internal private IP addresses. This addressing scheme allows the APIC to communicate with fabric nodes and other Cisco APICs in the cluster. The APIC discovers the IP address and node information of other Cisco APICs in the cluster using a Link Layer Discovery Protocol–based discovery process.

The following describes the APIC cluster discovery process:

• Each APIC in the Cisco ACI uses an internal private IP address to communicate with the ACI nodes and other APICs in the cluster. The APIC discovers the IP address of other APICs in the cluster through an LLDP-based discovery process.
• APICs maintain an appliance vector (AV), which provides a mapping from an APIC ID to an APIC IP address and a universally unique identifier (UUID) of the APIC. Initially, each APIC starts with an AV filled with its local IP address, and all other APIC slots are marked as unknown.
• When a switch reboots, the policy element (PE) on the leaf gets its AV from the APIC. The switch then advertises this AV to all its neighbors and reports any discrepancies between its local AV and the neighbors’ AVs to all the APICs in its local AV.

Using this process, the APIC learns about the other APICs in the ACI through switches. After these newly discovered APICs in the cluster have been validated, they update their local AV and program the switches with the new AV. Switches then start advertising this new AV. This process continues until all the switches have the identical AV and all APICs know the IP address of all the other APICs.

The ACI fabric is brought up in a cascading manner, starting with the leaf nodes directly attached to the APIC. LLDP and control-plane IS-IS convergence occurs in parallel to this boot process. The ACI fabric uses LLDP- and DHCP-based fabric discovery to automatically discover the fabric switch nodes, assign the infrastructure VXLAN tunnel endpoint (VTEP) addresses, and install the firmware on the switches. The fabric uses an IS-IS (Intermediate System to Intermediate System) environment utilizing Level 1 connections within the topology for advertising loopback addresses called the Virtual extensible LAN tunnel endpoints (VTEPs), which are used in the integrated overlay and advertised to all other nodes in the fabric for overlay tunnel use. Prior to this automated process, a minimal bootstrap configuration must be performed on the Cisco APIC. After the APICs are connected and their IP addresses assigned, the APIC GUI can be accessed by entering the address of any APIC into a web browser. The APIC GUI runs HTML5 and eliminates the need for Java to be installed locally.