Console
    Contact Us Start free

    Troubleshoot MACsec

    This page describes how to troubleshoot MACsec for Cloud Interconnect.

    Cloud Interconnect displays an error when I attempt to create a new key

    If you have an existing MACsec key without a start time and you attempt to create a new key, Cloud Interconnect displays an error. To resolve the error, update the existing key's start time .

    MACsec is operationally down on my Cloud Interconnect connection

    You successfully enabled MACsec on your Cloud Interconnect connection and on your on-premises router, but the MACsec session displays that it is operationally down on your Cloud Interconnect connection links. The issue could be caused by one of the following:

    • The active keys on your on-premises router and Google's edge routers don't match.
    • A MACsec protocol mismatch exists between your on-premises router and Google's edge router.

    To resolve the MACsec state, do the following:

    1. To verify that MACsec is enabled on your Cloud Interconnect connection, select one of the following options:

      Console

      1. In the Google Cloud console, go to the Cloud Interconnect Physical connectionstab.

        Go to Physical connections

      2. Select the Cloud Interconnect connection that you want to view.

      3. On the MACsectab, verify that MACsec configurationdisplays one of the following:

        • Enabled, fail open:MACsec encryption is enabled on the link. If MACsec encryption isn't established between both ends, then the link operates without encryption.

        • Enabled, fail closed:MACsec encryption is enabled on the link. If MACsec encryption isn't established between both ends, then the link fails.

      gcloud

       gcloud compute interconnects describe INTERCONNECT_CONNECTION_NAME

      Replace INTERCONNECT_CONNECTION_NAME with the name of your Cloud Interconnect connection.

      The output is similar to the following. Verify that macsecEnabled: true is displayed:

        adminEnabled 
 : 
  
 true 
 availableFeatures 
 : 
 - 
  
 IF_MACSEC 
 circuitInfos 
 : 
 - 
  
 customerDemarcId 
 : 
  
 fake-peer-demarc-0 
  
 googleCircuitId 
 : 
  
 LOOP-0 
  
 googleDemarcId 
 : 
  
 fake-local-demarc-0 
 creationTimestamp 
 : 
  
 '2021-10-05T03:39:33.888-07:00' 
 customerName 
 : 
  
 Fake Company 
 description 
 : 
  
 something important 
 googleReferenceId 
 : 
  
 '123456789' 
 id 
 : 
  
 '12345678987654321' 
 interconnectAttachments 
 : 
 - 
  
 https://www.googleapis.com/compute/v1/projects/my-project1/regions/us-central1/interconnectAttachments/interconnect-123456-987654321-0 
 interconnectType 
 : 
  
 IT_PRIVATE 
 kind 
 : 
  
 compute#interconnect 
 labelFingerprint 
 : 
  
 12H17262736_ 
 linkType 
 : 
  
 LINK_TYPE_ETHERNET_10G_LR 
 location 
 : 
  
 https://www.googleapis.com/compute/v1/projects/my-project1/global/interconnectLocations/cbf-zone2-65012 
 macsec 
 : 
  
 failOpen 
 : 
  
 false 
  
 preSharedKeys 
 : 
  
 - 
  
 name 
 : 
  
 key1 
  
 startTime 
 : 
  
 2023-07-01T21:00:01.000Z 
 macsecEnabled 
 : 
  
 true 
 name 
 : 
  
  INTERCONNECT_CONNECTION_NAME 
 
 operationalStatus 
 : 
  
 OS_ACTIVE 
 provisionedLinkCount 
 : 
  
 1 
 requestedFeatures 
 : 
 - 
  
 IF_MACSEC 
 requestedLinkCount 
 : 
  
 1 
 selfLink 
 : 
  
 https://www.googleapis.com/compute/v1/projects/my-project1/global/interconnects/ INTERCONNECT_CONNECTION_NAME 
 
 selfLinkWithId 
 : 
  
 https://www.googleapis.com/compute/v1/projects/my-project1/global/interconnects/12345678987654321 
 state 
 : 
  
 ACTIVE

    2. To check the Cloud Interconnect port status, MACsec operational state, and the active key name, use one of the following options:

      Console

      1. In the Google Cloud console, go to the Cloud Interconnect Physical connectionstab.

        Go to Physical connections

      2. Select the Cloud Interconnect connection that you want to view.

      3. In Link circuit info, verify that Link statedisplays Activefor all links.

      4. Verify that MACsec key namedisplays a key name for all links, and that each key name displays MACsec on this link is upor MACsec on this link is down.

      gcloud

       gcloud compute interconnects get-diagnostics INTERCONNECT_CONNECTION_NAME 
\
    --project= PROJECT_NAME

      Replace PROJECT_NAME with the name of your Google Cloud project.

      The output is similar to the following. Verify that links.lacpStatus.state displays ACTIVE , that links.macsec.ckn displays a value, and that links.operationalStatus displays LINK_OPERATIONAL_STATUS_UP :

        bundleAggregationType 
 : 
  
 BUNDLE_AGGREGATION_TYPE_STATIC 
 bundleOperationalStatus 
 : 
  
 BUNDLE_OPERATIONAL_STATUS_UP 
 links 
 : 
 - 
  
 circuitId 
 : 
  
 LOOP-0 
  
 googleDemarc 
 : 
  
 fake-local-demarc-0 
  
 lacpStatus 
 : 
  
 googleSystemId 
 : 
  
 '00:11:22:33:44:55' 
  
 neighborSystemId 
 : 
  
 '55:44:33:22:11:00' 
  
 state 
 : 
  
 ACTIVE 
  
 macsec 
 : 
  
 ckn 
 : 
  
 0101010189abcdef...0123456789abcdef 
  
 operational 
 : 
  
 false 
  
 operationalStatus 
 : 
  
 LINK_OPERATIONAL_STATUS_UP 
  
 receivingOpticalPower 
 : 
  
 state 
 : 
  
 OK 
  
 value 
 : 
  
 -2.49 
  
 transmittingOpticalPower 
 : 
  
 state 
 : 
  
 OK 
  
 value 
 : 
  
 -0.88 
 macAddress 
 : 
  
 00:11:22:33:44:55

      If there isn't a value displayed for links.macsec.ckn , then contact Google Cloud Support for help.

    3. To verify the active key's CAK and CKN values, and the key's start time, select one of the following options:

      Console

      1. On the MACsectab, go to the Pre-shared keyssection, and then click Viewbeside the active key. If a CKN value isn't displayed, contact Google Cloud support for help.

      2. In the Pre-shared keyssection, verify that the start time listed for the active key matches the start time on your on-premises router. Do one of the following:

        • If the values don't match, refer to your router's manual to update the values on your router, and then verify whether a MACsec session can be established.

        • If the values match but the MACsec session is still operationally down on the link, then continue to the next step.

      gcloud

      1. Run the gcloud compute interconnects get-diagnostics command to display the active key's CKN value.

        If you have more than one key configured, then the key with the latest start time that isn't in the future is selected as the active key. Google's edge routers reject any new MACsec sessions that attempt to use old keys.

      2. Get the MACsec configuration and then note the CAK value and the key's start time that correspond to the previously displayed CKN value:

         gcloud compute interconnects macsec get-config INTERCONNECT_CONNECTION_NAME

        Replace INTERCONNECT_CONNECTION_NAME with the name of your Cloud Interconnect connection.

        The output is similar to the following; look for the ckn :

          preSharedKeys 
 : 
 - 
  
 name 
 : 
  
 key1 
  
 ckn 
 : 
  
 0101010189abcdef...0123456789abcdef 
  
 cak 
 : 
  
 0123456789abcdef...0123456789abcdef 
  
 startTime 
 : 
  
 2023-07-01T12:12:12Z 
 - 
  
 name 
 : 
  
 key2 
  
 ckn 
 : 
  
 0202020289abcdef...0123456789abcdef 
  
 cak 
 : 
  
 0123456789abcdef...0123456789abcdef 
  
 startTime 
 : 
  
 2023-08-01T12:12:12Z

      3. Verify that the active CKN, CAK, and start times on your on-premises router match the values that MACsec for Cloud Interconnect displays. Do one of the following:

        • If the values don't match, refer to your router's manual to update the values on your router, and then verify whether a MACsec session can be established.

        • If the values match but the MACsec session is still operationally down on the link, then continue to the next step.

    4. View metrics to determine whether packets are dropping on the ingress or egress of the Cloud Interconnect connection. For information about viewing metrics, see Monitor connections .

      To determine next steps, do the following:

      • If network/interconnect/link/macsec/received_errors_count is incrementing, then the packets are dropping at the ingress Cloud Interconnect connection due to errors. This indicates that a protocol mismatch exists between your on-premises router and Google's edge routers. Check your on-premises router's logs to troubleshoot.

      • If any of the following counters are incrementing, then contact Google Cloud Support for further assistance:

        • network/interconnect/link/macsec/received_dropped_packets_count
        • network/interconnect/link/macsec/send_errors_count
        • network/interconnect/link/macsec/send_dropped_packets_count

      • If none of the following counters are incrementing, then it indicates that packets are dropping at the egress of your on-premises router. Check your on-premises router's logs to troubleshoot.

        • network/interconnect/receive_errors_count
        • network/interconnect/received_unicast_packets_count
        • network/interconnect/link/macsec/received_control_packets_count
        • network/interconnect/link/macsec/received_data_packets_count
        • network/interconnect/link/macsec/received_errors_count
        • network/interconnect/link/macsec/received_dropped_packets_count

    MACsec is operational and is experiencing packet loss

    You successfully enabled MACsec for Cloud Interconnect and MACsec is operationally up, but you are experiencing packet loss.

    If your MACsec connection is operational but the Cloud Interconnect's Link Aggregation Control Protocol (LACP) status is Detached , verify that Secure Channel Identifier (SCI) is enabled on your on-premises router. For more information, see Configure your on-premises router .

    View metrics to determine if packets are dropping on the ingress or egress of the Cloud Interconnect connection. For information about viewing metrics, see Monitor connections . If the Cloud Interconnect connection does not show any packet errors or loss, then proceed to checking the MACsec routers:

    • If network/interconnect/link/macsec/received_errors_count is incrementing, then the packets are dropping at the ingress Cloud Interconnect connection due to errors. This indicates that a protocol mismatch exists between your on-premises router and Google's edge routers. Check your on-premises router's logs to troubleshoot.

    • If any of the following counters are incrementing, then contact Google Cloud Support for further assistance:

      • network/interconnect/link/macsec/received_dropped_packets_count
      • network/interconnect/link/macsec/send_errors_count
      • network/interconnect/link/macsec/send_dropped_packets_count

    • If none of the following counters are incrementing, then it indicates that packets are dropping at the egress of your on-premises router. Check your on-premises router's logs to troubleshoot.

      • network/interconnect/receive_errors_count
      • network/interconnect/received_unicast_packets_count
      • network/interconnect/link/macsec/received_control_packets_count
      • network/interconnect/link/macsec/received_data_packets_count
      • network/interconnect/link/macsec/received_errors_count
      • network/interconnect/link/macsec/received_dropped_packets_count

    Troubleshoot MACsec issues while fail-open behavior is enabled

    If you enable MACsec for Cloud Interconnect with fail-open behavior, then your Cloud Interconnect connection continues forwarding traffic even if a MACsec session can't be successfully established. We strongly recommend that you avoid using fail-open behavior on production Cloud Interconnect connections to avoid transmitting packets as clear text.

    To determine the configuration and state your MACsec connection, do the following:

    1. To verify the state of your Cloud Interconnect connection, select one of the following options:

      Console

      1. In the Google Cloud console, go to the Cloud Interconnect Physical connectionstab.

        Go to Physical connections

      2. Select the Cloud Interconnect connection that you want to view.

      3. On the MACsectab, verify that MACsec configurationdisplays Enabled, fail open.

      gcloud

       gcloud compute interconnects describe INTERCONNECT_CONNECTION_NAME

      Replace INTERCONNECT_CONNECTION_NAME with the name of your Cloud Interconnect connection.

      The output is similar to the following; look for macsec failOpen set to true and macsecEnabled set to true :

        availableFeatures 
 : 
 - 
  
 IF_MACSEC 
 adminEnabled 
 : 
  
 true 
 circuitInfos 
 : 
 - 
  
 customerDemarcId 
 : 
  
 fake-peer-demarc-0 
  
 googleCircuitId 
 : 
  
 LOOP-0 
  
 googleDemarcId 
 : 
  
 fake-local-demarc-0 
 creationTimestamp 
 : 
  
 '2021-10-05T03:39:33.888-07:00' 
 customerName 
 : 
  
 Fake Customer 
 description 
 : 
  
< something 
> googleReferenceId 
 : 
  
 '123456789' 
 id 
 : 
  
 '12345678987654321' 
 interconnectAttachments 
 : 
 - 
  
 https://www.googleapis.com/compute/prod/projects/my-project1/regions/us-central1/interconnectAttachments/interconnect-123456-123456789-0 
 interconnectType 
 : 
  
 IT_PRIVATE 
 kind 
 : 
  
 compute#interconnect 
 labelFingerprint 
 : 
  
 42WmSpB8rSM= 
 linkType 
 : 
  
 LINK_TYPE_ETHERNET_10G_LR 
 location 
 : 
  
 https://www.googleapis.com/compute/prod/projects/my-project1/global/interconnectLocations/cbf-zone2-65012 
 macsec 
 : 
  
 failOpen 
 : 
  
 true 
  
 preSharedKeys 
 : 
  
 - 
  
 name 
 : 
  
 key3 
  
 startTime 
 : 
  
 '2023-07-01T21:00:01.000Z' 
 macsecEnabled 
 : 
  
 true 
 name 
 : 
  
  INTERCONNECT_CONNECTION_NAME 
 
 operationalStatus 
 : 
  
 OS_ACTIVE 
 provisionedLinkCount 
 : 
  
 1 
 requestedFeatures 
 : 
 - 
  
 IF_MACSEC 
 requestedLinkCount 
 : 
  
 1 
 selfLink 
 : 
  
 https://www.googleapis.com/compute/prod/projects/my-project1/global/interconnects/ INTERCONNECT_CONNECTION_NAME 
 
 selfLinkWithId 
 : 
  
 https://www.googleapis.com/compute/prod/projects/my-project1/global/interconnects/ INTERCONNECT_CONNECTION_NAME 
/12345678987654321 
 state 
 : 
  
 ACTIVE

      In this example, macsec.failopen displays true and macsecEnabled displays true .

    2. To check the Cloud Interconnect connection's port status, MACsec operational state, and the active key name, select one of the following options:

      Console

      1. In the Google Cloud console, go to the Cloud Interconnect Physical connectionstab.

        Go to Physical connections

      2. Select the Cloud Interconnect connection that you want to view.

      3. In Link circuit info, verify that Link statedisplays Activefor all links.

      4. Verify that MACsec key namedisplays a key name for all links, and that each key name displays MACsec on this link is upor MACsec on this link is down.

      gcloud

       gcloud compute interconnects get-diagnostics INTERCONNECT_CONNECTION_NAME 
\
    --project= PROJECT_NAME

      Replace PROJECT_NAME with the name of your Google Cloud project.

      The output is similar to the following; look for the bundleOperationalStatus set to BUNDLE_OPERATIONAL_STATUS_UP , the state set to ACTIVE , and the macsec ckn operational set to false :

         
 bundleAggregationType 
 : 
  
 BUNDLE_AGGREGATION_TYPE_LACP 
  
 bundleOperationalStatus 
 : 
  
 BUNDLE_OPERATIONAL_STATUS_UP 
  
 links 
 : 
  
 - 
  
 circuitId 
 : 
  
 LOOP-0 
  
 googleDemarc 
 : 
  
 fake-local-demarc-0 
  
 lacpStatus 
 : 
  
 googleSystemId 
 : 
  
 '00:11:22:33:44:55' 
  
 neighborSystemId 
 : 
  
 '55:44:33:22:11:00' 
  
 state 
 : 
  
 ACTIVE 
  
 macsec 
 : 
  
 ckn 
 : 
  
 0101010189abcdef...0123456789abcdef 
  
 operational 
 : 
  
 false 
  
 operationalStatus 
 : 
  
 LINK_OPERATIONAL_STATUS_UP 
  
 receivingOpticalPower 
 : 
  
 state 
 : 
  
 OK 
  
 value 
 : 
  
 -2.49 
  
 transmittingOpticalPower 
 : 
  
 state 
 : 
  
 OK 
  
 value 
 : 
  
 -0.88 
  
 macAddress 
 : 
  
 00:11:22:33:44:55

      In this example:

      • bundleOperationalStatus: BUNDLE_OPERATIONAL_STATUS_UP indicates that the bundle is operationally up.
      • links.lacpStatus.state: ACTIVE indicates that the LACP member link is up.
      • links.macsec.operational: false indicates that MACsec is operationally down.

      In this case, because fail-open behavior is enabled, the LACP control packets are not dropped.

      If there isn't a value displayed for links.macsec.ckn , then contact Google Cloud Support for help.

      The gcloud compute interconnects get-diagnostics command displays the active key's CKN value. If you have more than one key configured, then the key with the latest start time is selected as the active key. Google's edge routers reject any new MACsec sessions that attempt to use the older keys.

    3. To get the MACsec configuration, and then note the CAK value and the key's start time that correspond to the previously displayed CKN value, select one of the following options:

      Console

      1. On the MACsectab, go to the Pre-shared keyssection, and then click Viewbeside the active key. If the key's CAK and CKN values aren't displayed, contact Google Cloud support for help.

      2. In the Pre-shared keyssection, verify that the start time listed for the active key matches the start times on your on-premises router.

      gcloud

      1. Run the following command:

         gcloud compute interconnects macsec get-config INTERCONNECT_CONNECTION_NAME

        The output is similar to the following; look for the preSharedKeys name ckn :

          preSharedKeys 
 : 
 - 
  
 name 
 : 
  
 key1 
  
 ckn 
 : 
  
 0101010189abcdef...0123456789abcdef 
  
 cak 
 : 
  
 0123456789abcdef...0123456789abcdef 
  
 startTime 
 : 
  
 2023-07-01T12:12:12Z 
 - 
  
 name 
 : 
  
 key2 
  
 ckn 
 : 
  
 0202020289abcdef...0123456789abcdef 
  
 cak 
 : 
  
 0123456789abcdef...0123456789abcdef 
  
 startTime 
 : 
  
 2023-08-01T12:12:12Z

      2. Verify that the active CKN, CAK, and start times on your on-premises router match the values that MACsec for Cloud Interconnect displays.

      3. Do one of the following:

        • If the values don't match, refer to your router's manual to update the values on your router, and then verify if a MACsec can be established now.

        • If the values match but the MACsec session is still operationally down on the link, then continue to the next step.

    4. View metrics to observe packet counters for your Cloud Interconnect connection. For more information about viewing metrics, see Monitor connections .

      When MACsec fail-open behavior is enabled, the following counters increment:

      • network/interconnect/sent_unicast_packets_count
      • network/interconnect/received_unicast_packets_count

      When MACsec fail-open behavior is enabled, the following counters don't increment:

      • network/interconnect/link/macsec/received_control_packets_count
      • network/interconnect/link/macsec/received_data_packets_count
      • network/interconnect/link/macsec/sent_control_packets_count
      • network/interconnect/link/macsec/sent_data_packets_count

      To determine next steps, do the following:

      • If network/interconnect/link/macsec/received_errors_count is incrementing, then the packets are dropping at the ingress Cloud Interconnect connection due to errors. This indicates that a protocol mismatch exists between your on-premises router and Google's edge routers. Check your on-premises router's logs to troubleshoot.

      • If any of the following counters are incrementing, then contact Google Cloud Support for further assistance:

        • network/interconnect/link/macsec/received_dropped_packets_count
        • network/interconnect/link/macsec/send_errors_count
        • network/interconnect/link/macsec/send_dropped_packets_count

      • If none of the following counters are incrementing, then it can indicate that packets are dropping at the egress of your on-premises router. Check your on-premises router's logs to troubleshoot.

        • network/interconnect/receive_errors_count
        • network/interconnect/received_unicast_packets_count
        • network/interconnect/link/macsec/received_errors_count
        • network/interconnect/link/macsec/received_dropped_packets_count

    What's next?
    Create a Mobile Website
    View Site in Mobile | Classic
    Share by: