ACI Domains, Leaf Interface, Leaf Switch Policies, Profiles & Selectors

We need to consider the design and implementation for domains, access ports and switch selector profiles carefully. Depending on the type of domain and/or type of port required this will make a difference to the design and configuration of these configuration sets. If we have a bunch of access ports with the same profiles in terms of things like CDP or LLDP then we are able to reuse profiles for all access ports for the same switches but where we require port channel (PC) or virtual port channel (vPC) ports this requires profile per server (or more accurately – for all ports in a channel for each connected device). The diagram shows the object relationships for leaf connected devices from the domain (Physical/VMM/[L2/L3-Out]) to switch and interface selection.

Starting with the Leaf Policy Group (top right green in diagram), there are three variants of this policy. This policy allows applying additional policies to the configured interfaces such as enabling/disabling CDP or LLDP, setting a link negotiation policy or applying a policy for port channels. The first type of a Leaf Policy Group is Access Port, this as it suggests allows the policy to be applied to individual access ports (i.e. non PC/vPC). As access ports don’t have specific configuration like PC/vPC’s do (i.e. min/max links, mac pinning/LACp active) this policy type can be applied to multiple access ports simultaneously. The other two types of Leaf Policy Groups are PC and vPC, these type have basically the same rule in that as they apply to a single port channel, this policy can only be applied to one set of ports in a PC/vPC. This policy uses the Port Channel Policy applied to it which states min/max links so therefore only the max links (interfaces) can be applied to this policy. So a Access Leaf Policy Group type can be applied to many interfaces simultaneously, a PC/vPC Leaf Policy Group type can only be applied to simultaneously to the interfaces in the port channel bundle. This is important as if you have a lot of single connected bare metal hosts you can reuse the same Access Leaf Policy Group type ultimately bound to a single switch (Access Leaf Policy Group per Switch), whereas the PC/vPC policies will be created per port channel bundle (PC/vPC Leaf Policy Group per Switch/Switch-Pair). The (Managed Object) MO or class name for this policy is either infraAccPortGrp (Access) or infraAccBndlGrp (PC/vPC).

This brings us to the leaf interface profiles and access port selectors. The terms interface and port refer to the same thing… an interface or a port depending on what you prefer but either way the same. The Leaf Interface Profile contains nothing more that a list of Access Port Selectors. An Access Port Selector (or Interface Selector as it is labelled in the Leaf Interface Profile GUI) – yes the names interface and port have been swapped around here and not consistent in use. It can be confusing to begin with looking for an Interface Selector but in come screens is called Access Port Selector, just keep ‘port’ and ‘interface’ inter-changeable in your head! So an Access Port Selector specified a ‘type’, this type is ‘ALL’ or ‘range’, specifying range means you will provide a range of ports (i.e. 1/23-1/27), ‘ALL’ means you don’t need to provide any ports and all ports on the switches will be included in this Access Port Selector. If you specify ‘range’ and provide a range of ports, these are stored in a object called infraPortBlk referenced by the Access Port Selector. Each Access Port Selector (and therefore its ports) then reference as Leaf Policy Group (discussed above), so a Leaf Interface Profile can have many Access Port Selectors (port blocks/ranges) with each utilising a different Leaf Policy Group. This is shown below with the Leaf Interface Profile (‘VMware-VPC-Interface-Profile’) having several Interface Selectors (Access Port Selectors), one for each vPC connected ESXi server and using different Policy Groups as discussed due vPC being used to connect the servers.

So at this stage we have discussed the Leaf Policy Group which is referenced by the Interface/Port profiles and selectors, we now need to add some switches in. We have only specified the ports (i.e. 1/10-1/11, 1/20-1/25, etc) but not defined any switches. Of course we need to state which switches the ports we are referencing are on, we do this by creating another set of profiles for switches very much like we did for the interfaces. Again we have the same model, we have a Leaf (switch) Profile which has one or more Leaf (switch) Selectors which use a Node Block to store the reference to the switches. The diagram below shows this relationship in a different view from the main model diagram and the similar objects.

The Leaf Profile references a Leaf Interface Profile, this is where we have the link between the interfaces selected in the Access Port Selector (and Port Block) and the selected switches. This needs a little thought as if we have selected port (interface) range 1/10-1/15 in the Access Port Selector, then for example selected switches 101 & 102 in the Leaf Selector which is pat of the Leaf Profile which we have configured to reference the Leaf Interface Profile (still with me !) then we have switches 101 & 102 and interface range 1/10-1/15 configured in this chain of objects. This means we have configured switch 101 with ports 1/10-1/15 and switch 102 with ports 1/10-1/15 in this policy set. ACI leaf switches are usually deployed in pairs and utilised in pairs for example for vPC. This configuration is still valid for PC or individual access ports but it is worth thinking about what you are committing to when specifying a port range and a switch range, the ports you specify will have the same policy applied to all those ports on all the switches you specify.

We may have a leaf switch pair for external connectivity (routers for WAN/Internet, non ACI DC) or services (i.e. Load Balancers, Firewalls, IPS, etc.), some of these devices may not require redundant connectivity and may require difference interface policies such as LLDP instead of CDP, or link negotiation differences. If we use the previous example of port range 1/10-1/15 on both 101 & 102 switches which use the same Leaf Policy Group which has link negotiation turned off. Connecting a device to switch 101-1/10 which requires no link negotiation works as expected, but if we now connect another device to switch 102-1/10 which requires link negotiation, this will fail as the policy states link negotiation is off for all switch ports connected to this policy. In this situation consideration of the design on the policies and the use of switches and ports would suggest that a pair of switches as just shown that is likely to have differing connectivity requirements would benefit from separate policy groups and each Leaf Selector having only one switch therefore creating a Leaf Profile per switch. The flip side to this is where we have a requirement which is consistent across the two switches and the port ranges selected. For example VMware ESXi hosts using vPC.

We can graph two example designs. The first one having two switches supporting both ESXi and Hyper-V hosts, these hosts are connecting to the fabric using vPC. Ranges of ports have been allocated for these hosts obn both switches. Two policy groups have been created – one for each host type.

The second using two switches but having different requirements of the same ports on different switches. Devices connecting to these switches don’t have redundant requirements like PC/vPC and therefore are basic access ports.

Documentation of the design and intended use of the model is important with ACI and it ever has been with other architectures to ensure that the implementation and future changes stay consistent with the original intention. To ensure consistency in implementation using tools like Python (Cobra), Ansible and the APIC REST interface ensures that the scripts have the design intention written into them excepting parameters for additions like server names and types. So a script would be specific for an addition of an ESXi host which would require the host name and port range the host is to use. The script would ensure the other configuration parts are consistent.