ACITroubleshooting

ACI Contracts (Troubleshoot & Verification)

This article steps though finding contracts for a VRF and then analyzing one from the list we find. Following this process will arm us with the information we need to do this in the reverse when we have specific issues between EPGs.

This article touches on pcTags and VNIDs, For more information on pcTags, pcTag ranges and VNIDs see this article.

Starting with a a chosen VRF, get the VNID of the VRF

leaf-01-201# show vrf TEN_DEVOPS:VRF_DEVOPS detail extended
VRF-Name: TEN_DEVOPS:VRF_DEVOPS, VRF-ID: 12, State: Up
    VPNID: unknown
    RD: 201:2785280

    Max Routes: 0  Mid-Threshold: 0
    Encap: vxlan-2785280
    Table-ID: 0x8000000c, AF: IPv6, Fwd-ID: 0x8000000c, State: Up
    Table-ID: 0x0000000c, AF: IPv4, Fwd-ID: 0x0000000c, State: Up

Get the VRF Prefix Table for VRF VNID ‘ 2785280‘, we are concerned with the Class ID in the output as this is a pcTag of an EPG in this VRF. Class number 15 is a drop all. For more information on pcTags, ranges and VNIDs see this article. As a reminder, 0-15 (Internal), Global Scope (16-16385), Local VRF Scope (16386-65535).

leaf-01-201# vsh -c 'show system internal policy-mgr prefix' | egrep 'Remote|^2785280'
Vrf-Vni VRF-Id Table-Id Table-State  VRF-Name                    Addr                                Class Shared Remote Complete
2785280 12     0xc           Up     TEN_DEVOPS:VRF_DEVOPS                           10.0.0.1/32  19      True   True   False
2785280 12     0xc           Up     TEN_DEVOPS:VRF_DEVOPS                            0.0.0.0/0   15      True   True   False
2785280 12     0x8000000c    Up     TEN_DEVOPS:VRF_DEVOPS                                 ::/0   15      True   True   False
2785280 12     0xc           Up     TEN_DEVOPS:VRF_DEVOPS                         172.28.0.0/16  10935   True   True   False
2785280 12     0xc           Up     TEN_DEVOPS:VRF_DEVOPS                         172.27.0.0/16  10935   True   True   False
2785280 12     0xc           Up     TEN_DEVOPS:VRF_DEVOPS                      192.168.9.192/28  10935   True   True   False
2785280 12     0xc           Up     TEN_DEVOPS:VRF_DEVOPS                         172.29.0.0/16  10935   True   True   False
2785280 12     0xc           Up     TEN_DEVOPS:VRF_DEVOPS                           10.0.0.2/32  10930   True   True   False
2785280 12     0xc           Up     TEN_DEVOPS:VRF_DEVOPS                         10.30.40.0/24  20      True   True   False

View the rules in place for any Class numbers greater than 15, 0-15 are reserved internally but be aware of them as they do effect traffic.

dev-leaf-01-201# show zoning-rule scope 2785280 src-epg 10935
+---------+--------+--------+----------+----------------+---------+---------+------+----------+------------------------+
| Rule ID | SrcEPG | DstEPG | FilterID |      Dir       |  operSt |  Scope  | Name |  Action  |        Priority        |
+---------+--------+--------+----------+----------------+---------+---------+------+----------+------------------------+
|   4490  | 10935  | 10933  |    32    | uni-dir-ignore | enabled | 2785280 |      |  permit  |     fully_qual(7)      |
|   4442  | 10935  |   0    | implicit |    uni-dir     | enabled | 2785280 |      | deny,log | shsrc_any_any_deny(12) |
|   4449  | 10935  | 10933  |    34    | uni-dir-ignore | enabled | 2785280 |      |  permit  |     fully_qual(7)      |
+---------+--------+--------+----------+----------------+---------+---------+------+----------+------------------------+

dev-leaf-01-201# show zoning-rule scope 2785280 dst-epg 10935
+---------+--------+--------+----------+--------+---------+---------+------+--------+---------------+
| Rule ID | SrcEPG | DstEPG | FilterID |  Dir   |  operSt |  Scope  | Name | Action |    Priority   |
+---------+--------+--------+----------+--------+---------+---------+------+--------+---------------+
|   4437  | 10933  | 10935  |    31    | bi-dir | enabled | 2785280 |      | permit | fully_qual(7) |
|   4497  | 10933  | 10935  |    33    | bi-dir | enabled | 2785280 |      | permit | fully_qual(7) |
+---------+--------+--------+----------+--------+---------+---------+------+--------+---------------+

Contract Filter Entries

Using the output from ‘ show zoning-rule scope 2785280 …’ , we see that we have two filters, HTTP & HTTPS both applied to contracts with ‘Reverse Filter Ports’ enabled as we see the reverse rule in the output too.

Endpoint Groups

Leading on from the last example and to complete the work, from the same rules above take the SrcEPG & DstEPG ID’s. These are uniquely 19 & 10933 which are the EPG pcTags.

apic-01# moquery -c fvAEPg -f 'fv.AEPg.pcTag=="10933"'
Total Objects shown: 1

# fv.AEPg
name                 : EPG_GENERAL
annotation           :
childAction          :
configIssues         :
configSt             : applied
descr                :
dn                   : uni/tn-TEN_DEVOPS/ap-AP_DEVOPS/epg-EPG_GENERAL
exceptionTag         :
extMngdBy            :
floodOnEncap         : disabled
fwdCtrl              :
hasMcastSource       : no
isAttrBasedEPg       : no
isSharedSrvMsiteEPg  : no
lcOwn                : local
matchT               : AtleastOne
modTs                : 2020-12-22T07:52:13.070+02:00
monPolDn             : uni/tn-common/monepg-default
nameAlias            :
pcEnfPref            : unenforced
pcTag                : 10933
prefGrMemb           : exclude
prio                 : unspecified
rn                   : epg-EPG_GENERAL
scope                : 2785280
shutdown             : no
status               :
triggerSt            : triggerable
txId                 : 1729382256910336179
uid                  : 15374

The pcTag ‘10933’ was found querying the IEPG (Internal EPG) MO class but not for pcTag ‘10935’. So this is likely a EEPG (External EPG in a L3Out).

-apic-01# moquery -c fvAEPg -f 'fv.AEPg.pcTag=="10935"'
No Mos found

Querying the MO ‘ l3extInstP ‘ which is the MO class for an EEPG in a L3out, we find the EEPG ‘ EPG_K8_NODES ‘.

apic-01# moquery -c l3extInstP -f 'l3.extInstP.pcTag=="10935"'
Total Objects shown: 1

# l3ext.InstP
name                 : EPG_K8_NODES
annotation           :
childAction          :
configIssues         :
configSt             : applied
descr                :
dn                   : uni/tn-TEN_K8_C1/out-L3O_K8_C1/instP-EPG_K8_NODES
exceptionTag         :
extMngdBy            :
floodOnEncap         : disabled
isSharedSrvMsiteEPg  : no
lcOwn                : local
matchT               : AtleastOne
mcast                : no
modTs                : 2021-04-25T10:37:54.744+02:00
monPolDn             : uni/tn-common/monepg-default
nameAlias            :
pcTag                : 10935
prefGrMemb           : exclude
prio                 : unspecified
rn                   : instP-EPG_K8_NODES
scope                : 2949120
status               :
targetDscp           : unspecified
triggerSt            : triggerable
txId                 : 3458764513821012100
uid                  : 15374

Therefore we have found that the L3Out EEPG is providing a contract permitting HTTP & HTTPS to the IEPG ‘EPG_K8_NODES’. Armed with this information you can see how you can reverse this process to work backwards if you want to troubleshoot between two EPGs from the start.

Simon Birtles

I have been in the IT sector for over 20 years with a primary focus on solutions around networking architecture & design in Data Center and WAN. I have held two CCIEs (#20221) for over 12 years with many retired certifications with Cisco and Microsoft. I have worked in demanding and critical sectors such as finance, insurance, health care and government providing solutions for architecture, design and problem analysis. I have been coding for as long as I can remember in C/C++ and Python (for most things nowadays). Locations that I work without additional paperwork (incl. post Brexit) are the UK and the EU including Germany, Netherlands, Spain and Belgium.