Troubleshooting SLES pay-as-you-go registration

This document describes how to resolve issues you might encounter when you connect Compute Engine virtual machine (VM) instances running pay-as-you-go (PAYG) SUSE Linux Enterprise Server (SLES) to the SUSE Subscription Management Tool (SMT) repository.

Before you begin

• Ensure that the VM has an associated service account .
• Ensure that the Service Metadata API is accessible from the VM.
• Ensure network connectivity from the VM to the respective Region Servers and SMT Servers
• Use the sc-repocheck tool to automatically troubleshoot the issues.
• Check the steps described in the SUSE PAYG troubleshooting guide.
• If you haven't already, set up authentication . Authentication verifies your identity for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:

  Select the tab for how you plan to use the samples on this page:

  Console

  When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

  gcloud

  1. Install the Google Cloud CLI. After installation, initialize the Google Cloud CLI by running the following command:

     gcloud  
     init

     If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity .

  2. Set a default region and zone .

Network issues

Unresolvable domain name

You might encounter the following issues if the VM can't connect to the smt-gce.susecloud.net SMT server:

 SUSEConnect error: SocketError: getaddrinfo: Name or service not known 

 ping: unknown host smt-gce.susecloud.net 

These issues are likely caused by an incorrect resolution of the SMT server domain name smt-gce.susecloud.net . This domain is not globally resolvable, so you must set its IP address according to the VM region, by doing the following:

Check the /etc/hosts file to make sure it contains an entry with the smt-gce.susecloud.net domain.

 cat  
/etc/hosts  
 | 
  
grep  
-i  
smt 

The output looks similar to the following, but the IP address might be different:

 # Added by SMT registration do not remove, retain comment as well
108.59.80.221   smt-gce.susecloud.net   smt-gce 

If the /etc/hosts file doesn't contain the same lines as the preceding example, do the following:

1. Find an IP address that corresponds with your VM's region from the list of SUSE SMT IP addresses .

2. Edit the file to add the SUSE SMT IP address and any other information that is missing.

Network unavailability

You may encounter the following errors due to network unavailability, even if the VM is able to resolve Compute Engine Update Server domain name:

 Unexpected exception.
Not ready to read within timeout. 

 Repository 'SLE-Module-Adv-Systems-Management12-Pool' is invalid.
Repository 'SLE-Module-Adv-Systems-Management12-Updates' is invalid. 

The following are some examples of errors in the /var/log/cloudregister log file you may find during the investigation:

WARNING:Unable to remove client registration from server
WARNING:HTTPSConnectionPool(host='smt-gce.susecloud.net', port=443): Max retries exceeded with url: /connect/systems (Caused by NewConnectionError(' : Failed to establish a new connection: [Errno 110] Connection timed out',)) 

INFO:Region server arguments: ?regionHint=europe-central2
ERROR:No response from: [('34.118.112.80', None), ('34.116.251.218', None), ('34.116.224.144', None)]

To find out more about the cause of the issue, perform a network connectivity test. The following example shows how to test an HTTPS connection using cURL :

 curl  
-sSI  
-m  
 5 
  
-o  
/dev/null  
 \ 
  
-w  
 'Response code (>0 is OK): %{http_code}\n' 
  
 \ 
  
 'https://smt-gce.susecloud.net' 
 

The output of the command contains an HTTP response code or an error message. The following are common responses and errors:

• Successful response:

   Response code (>0 is OK): 200 
  

• Request timeout error:

   Response code (>0 is OK): 000
  curl: (28) Connection timed out after 5001 milliseconds 
  

• Unresolvable domain error:

   Response code (>0 is OK): 000
  curl: (6) Could not resolve host: smt-gce.susecloud.net 
  

In certain scenarios, such as strict host firewall rules, the default IP address associated with the smt-gce.susecloud.net domain might not be available. To ensure that the issue is not only related to the current IP address, perform a network connectivity tests for alternate regional servers. Retrieve the list of regional servers by doing the following:

To find the full list of SUSE server IPs for Google Cloud, view the following documents:

• Region Servers
• SMT Servers

The network unavailability may be due to VM misconfiguration. In case of issues it is necessary to perform network diagnostics to identify the root cause.

Registration failed

You might encounter the following error if you have VMs that have a private IP address in Cloud NAT:

 ERROR:  Registration failed: Registering system to registration proxy https://smt-gce.susecloud.net
command '/usr/bin/zypper --non-interactive refs Python_3_Module_x86_64' failed
Error: zypper returned 4 with 'Problem retrieving the repository index file for service 'Python_3_Module_x86_64':
Timeout exceeded when accessing 'https://smt-gce.susecloud.net/services/2045/repo/repoindex.xml?credentials=Python_3_Module_x86_64'. 

To resolve this issue, review the Cloud NAT configuration to verify that the minimum ports per VM instanceparameter is set to at least 256.

For more information, check the Registration and zypper failed for Compute Engine instances behind Cloud NAT SUSE support bulletin.

No response

If your VM experiences problems communicating with update and region servers, you may observe the following errors:

• SUSEConnect error:

   SUSEConnect error: Errno::ETIMEDOUT: Connection timed out - connect(2) for "smt-gce.susecloud.net" port 443 
  

• zypper error:

   Error retrieving metadata for 'SLE-Module-Adv-Systems-Management12-Pool':
  Not ready to read within timeout.
  ... 
  

These errors can be caused by the absence of a response from update and region servers. To verify if this is the case, check the /var/log/cloudregister logs for similar content:

INFO:Region server arguments: ?regionHint=europe-central2
INFO:Using API: regionInfo
INFO:Region server arguments: ?regionHint=europe-central2
INFO:Getting update server information, attempt 1
INFO:   Using region server: 130.211.242.136
ERROR:  No response from: 130.211.242.136
INFO:   Using region server: 35.187.193.56
ERROR:  No response from: 35.187.193.56
INFO:   Using region server: 162.222.182.90
ERROR:  No response from: 162.222.182.90
INFO:   Using region server: 130.211.88.88
ERROR:  No response from: 130.211.88.88
ERROR:  None of the servers responded
ERROR:  Attempted: [IPv4Address('130.211.242.136'), IPv4Address('35.187.193.56'), IPv4Address('162.222.182.90'), IPv4Address('130.211.88.88')]
...
...
...
ERROR:Request not answered by any server after 3 attempts
ERROR:Exiting without registration

To resolve this issue, try one or more of the following:

• Confirm that the VM has an external IP address or that the Virtual Private Cloud subnet uses a NAT (either Cloud NAT or custom solution).

• If you modified the default network routing rules, such as limiting public Internet access or routing traffic through an on-premises network, add routes manually for SMT IPs through the default gateway of Compute Engine, by doing the following:

  1. Go to the Routespage in the Google Cloud console.

     Go to the Routes page

  2. Under the Route Managementtab look for a route that includes the SUSE SMT IP addresses and verify that it has the Compute Engine default gateway set as the next hop.

  3. If the route is missing, you can add it by clicking on Create Routeand entering the necessary information.

• If you're using an internal passthrough Network Load Balancer, for example with additional intermediary network software (such as firewalls, custom NATs, etc.), make sure that the load balancer is being used as the next hop for VM traffic, by doing the following:

  1. Go to the VM instancespage in the Google Cloud console.

     Go to the VM instances page

  2. Click the name of the VM you want to check. The VM detailspage opens.

  3. In the Network interfacessection, click View details.

  4. In the Firewall and routes detailssection locate the route that defines the path to the selected IP address range.

  5. Click the name of the route and confirm that internal passthrough Network Load Balancer or its IP address is set as the next hop.

  If there is no route that defines the path to the selected IP address range, or if the next hop of the route is different from internal passthrough Network Load Balancer, then set up internal passthrough Network Load Balancer as the next hop .

• If you're using an internal passthrough Network Load Balancer, confirm that it's located in the same region as the VM.

  1. Go to the VM instancespage in the Google Cloud console.

     Go to the VM instances page

  2. Locate the VM you want to check and note down its region.

  3. Go to the Load balancingpage in the Google Cloud console.

     Go to the Load balancing page

  4. Locate the internal passthrough Network Load Balancer used and check if it is in the same region as the VM.

  5. If the VM and the internal passthrough Network Load Balancer aren't in the same region, enable global access .

OS configuration issues

Unknown registration status

If you don't know whether or not your pay-as-you-go (PAYG) SUSE Linux Enterprise Server (SLES) is registered, run the following command:

 sudo  
SUSEConnect  
--status-text 

The output contains the version and registration status of the SUSE products, including SUSE Linux Enterprise Server.

 Installed Products:
------------------------------------------

  SUSE Linux Enterprise Server 12 SP5
  (SLES/12.5/x86_64)

  Registered

------------------------------------------
... 

If the status is Not Registered , start from the re-registration process to fix the issue.

Incorrect base product symlink

You may encounter the following errors if the base product link points to an incorrect product file:

ERROR:Unable to obtain product information from server "108.59.85.41,None"
        Unprocessable Entity
        {"type":"error","error":"Unmet product dependencies, activate one of these products first: SUSE Linux Enterprise Server 12 x86_64...
        ...
Unable to register modules, exiting.

This error is caused by an incorrect product file (i.e. sle-module-toolchain.prod ) being pointed to by the /etc/products.d/baseproduct symbolic link.

To resolve this issue, update the symlink at /etc/products.d/baseproduct to point to the appropriate base product file, by doing the following:

1. Navigate to the /etc/products.d directory

      
    cd 
     
   /etc/products.d 
   

2. Run the following command replacing SLES.prod with SLES_SAP.prod if SLES for SAP is installed:

      
   sudo  
   ln  
   -sf  
   SLES.prod  
   baseproduct 
   

Instance identity information unavailability

You may encounter the following errors if the instance identity information is not available for the VM:

ERROR:Data collected from stderr for instance data collection "b'Unable to access instance identity information\n'"

To access the instance metadata for identity tokens all VMs must be associated with a service account .

For more information, read the Public Cloud Infrastructure Update .

To check that the VM is relevant to this situation, run the following command on the VM:

 curl  
-s  
-H  
 'Metadata-Flavor: Google' 
  
 \ 
  
 'http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity?audience=test' 
 

Example of a successful response with an identity token:

eyJhbGciOiJSUzI1NiIsImtpZCI6IjkzOTd0MDQxSHQ2NDNxNzkzUjY1MDIwNzEyMjZPNnppaTdqNTl3eTciLCJ0eXAiOiJKV1QifQ.eyJhdWQiOiJ0ZXN0IiwiYXpwIjoiMjY1MDIwMDUyMzgzMjYyNTk0ODU2IiwiZXhwIjoxNjgzNzEyNTQzLCJpYXQiOjE2ODM3MTI4NjQsImlzcyI6Imh0dHBzOi8vYWNjb3VudHMuZ29vZ2xlLmNvbSIsInN1YiI6IjQ1NjA2MzQ5MDg5Mzc0Njg3ODI5NyJ9.EpzQ3NZ8mKStdpH10fL34qsKG0rjQEflzvLJLm2tVNX4xBJAkMhi8lcs5InUEY-QMK3njgbzdzNtD1fXoIfKoeWsqkA8vG3NkBz5zqRrtaB2STcO14H5tjIdTBsrCtET447tRXlGG5cvgMcWnRDZG92-jUZEpWki_Ri4T69X5-bBWkfE2Thm3oSUW4fScdeVOEmOgWnzD2jeVqQ_2YniywvpkT-rLzKfN-5AgN66zgBfXqJVTC90KFMebfiaOoL7z6ZSM9AjZGf45QEMZjxjd-Xzyee6ZWK8s0RE3hJlytb3zYcLt3tJwQ1WhnrC2ToJ-ZmKxxK3xKDLCvCQ6Ny5to

If the metadata returned is not a token but an error message such as the following, the VM is affected:

 {
  "error": "invalid_request",
  "error_description": "Service account not enabled on this instance"
} 

To remediate this issue, perform the following steps:

1. Stop the VM:

    gcloud  
   compute  
   instances  
   stop  
    VM_NAME 
    
   

2. Add a service account to the VM:

    gcloud  
   compute  
   instances  
   set-service-account  
    VM_NAME 
     
    \ 
     
   --service  
   account  
    SERVICE_ACCOUNT 
     
    \ 
     
   --no-scopes 
   

3. Start the VM:

    gcloud  
   compute  
   instances  
   start  
    VM_NAME 
    
   

4. After adding the missing service account, run the following command from the VM to re-register the SLES:

    sudo  
   registercloudguest  
   --force-new 
   

   Check details in the re-registration section.

Registration behind proxies

You might encounter an issue if your VMs configured to utilize any kind of proxying software. The following example demonstrates an attempt to register SLES using an HTTP proxy.

ERROR: Baseproduct registration failed
ERROR: Registering system to registration proxy https://smt-gce.susecloud.net

Announcing system to https://smt-gce.susecloud.net ...
SUSEConnect error: Net::HTTPFatalError: 503 "Service Unavailable"

SUSE on Compute Engine does not provide official support for operating system registration when performed through intermediaries that modify the original communication, such as proxies of man-in-the-middle (MITM) or non-transparent types.

The official solution to resolve this issue is to Set up Cloud NAT and route VM traffic through it.

Common workarounds

Re-registration

In some cases, a re-registration approach can be used to work around registration issues.

To force a new registration use the following command:

 sudo  
registercloudguest  
--force-new 

If successful, the following line will be output.

 Registration succeeded 

Details of re-registration process can be found in the /var/log/cloudregister .

INFO:Forced new registration
INFO:Clean current registration server: ('108.59.80.221', None)
...
INFO:Starting new HTTP connection (1): 169.254.169.254
INFO:Region server arguments: ?regionHint=us-central1
INFO:Using region server: 130.211.242.136
INFO:Starting new HTTPS connection (1): 130.211.242.136
INFO:Starting new HTTPS connection (1): 108.59.80.58
INFO:Modified /etc/hosts, added: 108.59.80.58   smt-gce.susecloud.net   smt-gce
...
INFO:Starting new HTTPS connection (1): 108.59.80.58
DEBUG:"GET /api/health/status HTTP/1.1" 200 None
INFO:Current update server will be used: "('108.59.80.58', None)"
INFO:Starting new HTTPS connection (1): smt-gce.susecloud.net
DEBUG:"POST /connect/systems/products/migrations HTTP/1.1" 422 None
INFO:Registration: /usr/sbin/SUSEConnect --url https://smt-gce.susecloud.net --product sle-module-containers/12/x86_64 --instance-data /var/lib/cloudregister/9c982106-78de-48fe-a662-20383da4c760

 INFO:Forced new registration
INFO:Using API: regionInfo
INFO:Starting new HTTP connection (1): 169.254.169.254
INFO:Region server arguments: ?regionHint=us-central1
INFO:Using region server: 130.211.242.136
INFO:Starting new HTTPS connection (1): 130.211.242.136
ERROR:No response from: 130.211.242.136
INFO:Using region server: 130.211.88.88
INFO:Starting new HTTPS connection (1): 130.211.88.88
ERROR:No response from: 130.211.88.88
INFO:Using region server: 146.148.73.14
INFO:Starting new HTTPS connection (1): 146.148.73.14
ERROR:No response from: 146.148.73.14
ERROR:None of the servers responded
ERROR:  Attempted: ['130.211.242.136', '130.211.88.88', '146.148.73.14']
ERROR:Exiting without registration 

Deregistration

In some cases, such as major release upgrade, you may encounter the following errors because the system is already registered to SUMA:

Can't get available migrations from server: SUSE::Connect::ApiError: The requested products 'SUSE Manager Client Tools for SLE 12 x86_64' are not activated on the system.

 This system is managed by SUSE manager. 

Resolve the issue, by doing the following:

1. Remove the SUSE Manager Client Tools module as described in the Adding or deleting modules and extensions with SUSEConnect guide.

2. Deregister from SUMA by following the How to deregister a SUSE Manager Client guide.

3. Run the following commands from VM to cleanup old registration:

      
   sudo  
   SUSEConnect  
   --cleanup && 
    \ 
     
   sudo  
   registercloudguest  
   --clean && 
    \ 
     
   sudo  
   rm  
   -f  
   /etc/SUSEConnect && 
    \ 
     
   sudo  
   rm  
   -f  
   /etc/zypp/ { 
   repos,services,credentials } 
   .d/* && 
    \ 
     
   sudo  
   rm  
   -f  
   /var/lib/cloudregister/* && 
    \ 
     
   sudo  
   rm  
   -rf  
   /var/cache/zypp/* && 
    \ 
     
   sudo  
   rm  
   -rf  
   /var/cache/cloudregister/* && 
    \ 
     
   sudo  
   sed  
   -i  
    '/^# Added by SMT reg/,+1d' 
     
   /etc/hosts 
   

4. Run the following command to register the system again:

      
   sudo  
   registercloudguest  
   --force-new 
   

   Check details in the re-registration section.

5. When the registration process is done, refresh the services and repositories, and check if all the expected repositories for the system provided by the SMT server are present:

      
   sudo  
   zypper  
   ref  
   -s && 
    \ 
     
   sudo  
   zypper  
   ls && 
    \ 
     
   sudo  
   zypper  
   lr  
   -U