Updating certificates on Windows and Linux servers

Updating certificates on Windows and Linux servers

What you need to know when updating CT Suite certificate on Windows App Server
CT Suite uses a TLS Proxy called stunnel to terminate TLS connections made to the windows app server. Certificates have a lifetime that will expire and the length of lifetime of the cert depends on how it was created.
You can check when the certificate expires by taking the following steps:
1. Open a chrome browser on the server in question.
2. Browse to https://<FQDN of server>/
3. This will take you to the CT Suite web client login.
4. Once here, click on the padlock icon to the left of the URL in the browser.
5. Select the “Connection is secure” then the “certificate is valid” options.
6. That will show the certificate and when it expires similar to below.

7. If your cert is expiring soon, take the following steps to swap it.

Swapping Certificate and private key
Prerequisites for this process:
- You will need the new cert from the resource who handles certificates. The cert AND Private Key will need to be provided in PEM format.
- It is very important that the private key not be encrypted. If it is encrypted, stunnel will not be able to decrypt the private key and it will not work. One way to check to see if the pkey that was provided to you is encrypted is to open the pkey in notepad. When you have it open, it should say “-----BEGIN PRIVATE KEY-----”. If it says “-----BEGIN ENCRYPTED PRIVATE KEY----” it will not work and you will need to remove the encryption.
1. Find the directory where stunnel is installed. Normally it will be C:\Program Files (x86)\stunnel\config\
2. Once here, open the stunnel.conf file in notepad++
3. Search for all sections towards the bottom that are not commented out with a semicolon by searching for the word “cert”. The file name you see to the right of the cert config is the cert you will need to update.
4. Go back to file explorer and make a copy of the old cert name. For example, if the cert in the conf file is stunnel.pem, then make a copy of the stunnel.pem file.
5. Next, edit the pem file by opening in notepad++ and overwrite the file by copying in the new UNENCRYPTED Private KEY. Once that has been copied in, hit a carriage return and then paste in the Certificate. The key and cert should but up to each other like the below screenshot. Save the file.

6. Once the file is saved, you can then go run the stunnel “reload configuration” batch file by clicking “start” and then searching for stunnel and then select the option that shows below.
*Note*: this can be done during business hours and will not impact existing connections made via stunnel.

7. You will get a notification that the file has been reloaded like you see below.

8. Now that the file has been reloaded, you can close the chrome sessions you had open. Then reopen the browser and navigate to the same URL you did earlier https://<FQDN of server>/ and select the connection is secure pad lock and inspect the certificate to see if the new cert has been inherited and you can see the new date of expiration.

Roll Back Steps:
1. If you have some issues after the reloading of the new cert, you can rollback to the original cert by renaming the stunnel.pem copy that was made before the change. Once the old cert/pkey is back in it’s original filename, you can then run the stunnel reload config batch file once more. Once this is done, it will be back to the same cert that was in place before the change.


What you need to know when updating CTS Analytics Linux Servers with new certificates
CT Suite leverages either Elasticsearch or OpenSearch for the CT Suite Analytics servers. These servers run on Linux and will require you to import a pfx/p12 version of the cert and key in a single file. When swapping a cert, you will need the following tools available.
- Please note that you will want to do this work outside of normal business hours so that the CTS application doesn’t run into issues while you are updating the cert.
- WinSCP installed on your desktop that can reach the Analytics server
- Putty or some other SSH client
- Cert/Key in PFX/P12 format generated from internal CA
      * If encrypted, the passphrase for the encrypted pkey

1. First step is to name the file the same name as the old cert that is expiring. You can check what the name of the existing cert is by opening putty to the server you are swapping the cert for. You can login with ctiadmin/1Password!
2. Once in, run the following command: “nano /etc/elasticsearch/elasticsearch.yml” This will open a nano text editor to the config file, if you page all the way down, you will find the path and name of the cert in the “xpack.security.http.ssl.keystore.path.” In the below example, the cert lives in the “/etc/elasticsearch/certs/” path and the name is “ct.p12” These two items are important for the next steps.

3. Once you have the cert in pfx/p12 format, you can open up WinSCP and open a connection to the server you are looking to update. You can log into the Linux server with user/pwd: ctiadmin/1Password! You can drag the file from left to right to upload the cert. Once you upload, the file will be copied to the /home/ctiadmin/


4. Now you can go back to the SSH/putty session and run the following commands
- “sudo su”
- Enter in “1Password!”
- You are now in the server as root.
- Navigate to the certs folder: “cd /etc/elasticsearch/certs”
- Make a copy of the existing prod cert in case we need to roll back to it.
      o “cp cert.pfx cert-backup.pfx”
- Next, we need to copy the new cert from /home/ctiadmin/ that was copied up to the server using WinSCP.
      o **important**: Make sure that the cert name is the same as the old cert.
      o Command to copy new cert over the top of the old cert:
            ▪ “cp /home/ctiadmin/<certname>.pfx /etc/elasticsearch/certs
- You have now overwritten the old cert with the new one.
- The next step is for instances where the cert is encrypted. If this is the case, you will need to run the following commands to store the passphrase that will be used to decrypt the private key.
- If cert is password protected, we must enter the password into the ES keystore. Run commands below. Enter the password when prompted.      
      o “cd /usr/share/elasticsearch”
      o It is important to make sure you type the below section correctly so please check for typos before entering.
      o “bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password”
            ▪ The server will prompt you for the passphrase for the encrypted cert.
            Enter value for xpack.security.http.ssl.keystore.secure_password:
      o Repeat the steps with a slightly different command and enter in the same passphrase.
      o bin/elasticsearch-keystore add xpack.security.http.ssl.truststore.secure_password
            Enter value for xpack.security.http.ssl.truststore.secure_password:
- Now that you have the new passphrase entered and the cert in the /certs folder, you are ready to test.
- Run the following command: “service elasticsearch stop”
- Next: “service elasticsearch start”
- Wait ~30 seconds and execute: “service elasticsearch status”
      o This command will tell you if the service is running. It usually takes around 30 seconds to crash the service if something about the cert is not good. So if the service shows up and
running after 30-40 seconds, you should be good.
- Next step is to open a new browser and hit the FQDN for the analytics server (https://<fqdn>:9200) and enter in the user/pwd: elastic/cti@12345
- Check the cert in the browser to make sure it is showing the new valid date ranges.
- If you run into issues, you can roll back to the old cert by copying the old cert we made a backup of and then stop and start the Elasticsearch service.




    • Related Articles

    • Issue when taking Snapshots of App Servers

      Issue When a customer takes a snapshot of the windows servers, it can cause network partitions to occur between the RMQ nodes. Normally snapshots are taken to prepare for an upgrade or change in the system, so they don't happen all the time, but when ...

    • Chrome Throttling - Browser Throttling - Issue with Web Client Getting Disconnected from Web Socket

      Problem Agent screen freezing, or turning blank after some time of being idle OR Agent seeing "The Device<extension#> is currently out of service" when an agent has been idle for more than 5 minutes. Other signs of throttling is Agent is unable to ...

    • Release notes v3.2

      Upgrade Path Based on the version currently installed, your system may need incremental upgrades to your system before you finalize on version 3.2.67 The following table illustrates the upgrade path required, depending on your current version. If you ...

    • Release notes v3.3

      Upgrade Path Based on the version currently installed, your system may need incremental upgrades to your system before you finalize on version 3.3.0 The following table illustrates the upgrade path required, depending on your current version. If you ...

    • Communications Server running out of disk space

      If the following setting is not set, you risk having the server run out of disk space. Steps: Go into Advanced>Modules>XML CDR variable. Set that to false and also stop the service. This way we will not overrun the hard drive on these SIP servers.