How to make your machine trust your self-signed certificates?

If you ever tried to set up a web application on your local machine with a secure connection (using HTTPS), you likely generated self-signed TLS certificates. When you create your own self-signed certificate, or even when the application you're using generates the certificate itself, your operating system (OS) will likely not trust the certificate. Consequently, also your other applications will not trust the certificate.

For instance, when trying to use the OpenStack command line application (which is written in Python) we get the following error when connecting to our OpenStack instance that uses a self-signed certificate:

user@ubuntu:~$ openstack image list
Failed to discover available identity versions when contacting https://10.0.0.45:5000. Attempting to parse version from URL.
Could not find versioned identity endpoints when attempting to authenticate. Please check that your auth_url is correct. SSL exception connecting to https://10.0.0.45:5000: HTTPSConnectionPool(host='10.0.0.45', port=5000): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))

The error tells us that Python could not verify the certificate that was used by our OpenStack instance. The reason for this is that Python only trusts certificates that are signed by a Certificate Authority (CA) which is trusted by the OS. However, we can make the OS trust your certificate, and thereby also using the OpenStack CLI without errors related to the self-signed certificate.

Solution

Disclaimer: the following part assumes that you have sudo privileges on your machine. If not, the commands will not work, and you will have to use other workarounds which depend on your situation. Some examples are provided below.

Various blogs and forum discussions concerning this topic can be found online [1]. However, it is easy to get lost in the specific issues of some forum posts. Therefore we'll explain more in-depth what steps you can take to get rid of the errors caused by the use of non-trusted self-signed certificates.

Trusting the Certificate Authority

We will not dive into the theory concerning the chain of trust built upon certificate authorities, but at least you should understand the basic concept. The certificate that your web application provides to your browser or CLI tool may contain a signature. The entity that generated this signature is called a Certificate Authority (CA). The presence of this signature tells the browser that this particular CA understates that the given certificate really belongs to the website that you are browsing. It is up to the browser to decide whether it trusts this statement. The outcome of this decision depends on whether the browser trusts the CA. If the CA's own certificate is known by the browser, because it is present in a set of trusted CA's, then the browser will also trust the certificates that were signed by this CA.

If we want our OS and applications to trust our self-signed certificate, then we need them to trust the CA that signed our certificate. If you generated the certificate yourself, you likely created a CA certificate along the way. If you used another tool for this, or the certificate was generated by the application you're using, then you may find the CA certificate among the generated certificates. The exact location of the certificate depends on the application at hand. In the case of an OpenStack instance that was installed using Kolla Ansible, the certificates will be located in the folder /etc/kolla/certificates/ and the CA certificate in a subfolder named /etc/kolla/certificates/ca/.

Preparation

We will use the CLI tool update-ca-certificates. First, you should locate the certificate of the CA that was used to sign the certificate of the web application that you're using. In our case with OpenStack, the CA certificate is located on the OpenStack server at /etc/kolla/certificates/ca/root.crt.

Put the CA certificate in the right place

We create a symlink in the folder /usr/local/share/ca-certificates/ that points to our certificate:

sudo ln /etc/kolla/certificates/ca/root.crt /usr/local/share/ca-certificates/KollaAnsible_OpenStack_CA.crt -s

Note that we have to use sudo, since the folder /usr/ is write-protected. Ensure that you use the .crt extension for the name of the symlink! If you change the extension, for example to .pem, the following steps will not work as intended.

Update the list of trusted certificates

To make our OS aware of your newly added CA certificate, run the update-ca-certificates tool in the terminal:

sudo update-ca-certificates

It should show something similar to the following output:

Updating certificates in /etc/ssl/certs...
rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

As we can see, a certificate, i.e. our CA certificate, has been added. You can now go ahead and connect again to the web application with the self-signed certificate. Normally, you should no longer receive errors concerning the verification.

Workarounds for non-sudo users

In some circumstances you may not have access to a user with sudo permissions. In those cases, you will have to use application specific methods for making the application at hand trust your self-signed certificate.

In local development situations you may even consider disabling certificate validation. Be aware, however, that if an attacker can replace the certificate with their own, you will not easily notice, since you will also trust their certificate. Therefore, disabling certificate validation in any context outside local development should not be done.

OpenStack CLI tool

The OpenStack CLI tool has a --os-cacert parameter for providing it with your CA certificate [2]:

openstack --os-cacert /etc/kolla/certificates/ca/root.crt

The OpenStack CLI tool can also use the environment variable OS_CACERT for validating self-signed TLS certificates of the OpenStack instance [2]. You can configure environment variables in various ways.

By putting assigning the environment variable on the same line as the one where the openstack command is run, the environment variable will only be available during execution of this command:

OS_CACERT=/etc/kolla/certificates/ca/root.crt openstack

To configure the environment variable during the current terminal session, you can use the export command:

export OS_CACERT=/etc/kolla/certificates/ca/root.crt

Similarly, we can add this line to our openrc.sh file (or admin-openrc.sh, depending on what you named it) with the login credentials used by the OpenStack CLI client. This way, the certificate will always be used when loading these credentials.

Disabling certificate validation

Although there should be no need to disable certificate validation if you have the CA certificate used for signing the self-signed certificate of your OpenStack instance, it is possible to disable validation anyway. This is done by running the openstack command with the --insecure flag [2] or by setting the environment variable OS_INSECURE=1 when using the OpenStack provider for Terraform [3].

Curl

When using curl you can include the CA certificate with the --cacert parameter [4]:

curl --cacert /etc/kolla/certificates/ca/root.crt https://10.0.0.45

Certificate validation can be disabled with the --insecure flag.

Wget

When using wget you can include the CA certificate with the --ca-certificate parameter [4]:

wget --ca-certificate /etc/kolla/certificates/ca/root.crt https://10.0.0.45

Certificate validation can be disabled with the --no-check-certificate flag.

Python requests library

The Python requests library allows for including your CA certificate as the value of the parameter verify [6]:

requests.get("https://10.0.0.45", verify="/etc/kolla/certificates/ca/root.crt")

Certificate validation can be disabled by setting verify=False instead.

Conclusion

When using a self-signed TLS certificate, we may bump into errors related to the fact the certificate could not be verified or validated. There are generally two options to cope with this:

• Add the CA that signed your certificate as a CA that is trusted by the OS. This requires sudo permissions.
• Configure your client application to trust the CA that signed your certificate. This depends on the application at hand. We showed how to do this for the OpenStack CLI tool, curl, wget and the Python requests library.

Bibliography

[1] T. Paranhos Lima, 'Python - How to make requests to URLs with self-signed certificates'. Accessed: Nov. 17, 2023. [Online]. Available: writeloop.dev/posts/python-requests-to-urls-with-self-signed-certificates

[2] ‘OpenStack Command Line’, OpenStack. Accessed: Nov. 17, 2023. [Online]. Available: docs.openstack.org/python-openstackclient/latest/cli/man/openstack.html

[3] ‘Terraform OpenStack Provider’, Terraform. Accessed: Nov. 17, 2023. [Online]. Available: registry.terraform.io/providers/terraform-provider-openstack/openstack/latest/docs

[4] ‘SSL Certificate Verification’, Curl. Accessed: Nov. 17, 2023. [Online]. Available: curl.se/docs/sslcerts.html

[5] ‘GNU Wget 1.21.1-dirty Manual’, GNU. Accessed: Nov. 17, 2023. [Online]. Available: www.gnu.org/software/wget/manual/wget.html

[6] ‘Advanced Usage’, Requests. Accessed: Nov. 17, 2023. [Online]. Available: docs.python-requests.org/en/latest/user/advanced

This blog post is licensed under CC BY-SA 4.0