Skip to main content

Managing Hotfixes

This section describes the management of Key Manager hotfixes.

The high-level workflow for applying a hotfix is following:

  1. Upload the hotfix file to Key Manager database.

  2. Patch all the Key Manager servers with the uploaded hotfix.

The high-level workflow for removing a hotfix is following:

  1. Remove the hotfix from a Key Manager server

  2. Delete the hotfix from the Key Manager database.

Applying a Hotfix​

  1. Copy the hotfix file to a Key Manager server.

    Hotfix files are named as follows:

    sshmgr-<version>-hotfix_<component>_<description>_<issue_id>.gz

    For example:

    sshmgr-3.0.0-hotfix_common_fix_tags_and_stats_UKM-1291.gz

    note

    Ensure the hotfix file is accessible and executable by the sshmgr user. For example you can put the hotfix file to the sshmgr home directory /var/lib/sshmgr, then ensure the hotfix file is owned and executable by the sshmgr user.

  2. Register the hotfix to your Key Manager system, by running the following on any Key Manager server (replace path/to/the/hotfix.gz with the path to the hotfix file):

    # ssh-mgr-controller --upload-hotfix=path/to/the/hotfix.gz

  3. Verify the Name of the hotfix from the GUI, on the System→Hotfixes page. The Name is typically in hotfix_<component>_<description> format. For example:

    hotfix_common_fix_tags_and_stats

  4. To apply the hotfix on a Key Manager server, run (replace hotfix_name with the Name of the hotfix):

    # ssh-mgr-controller --apply-hotfix=hotfix_name

    Then restart its services:

    # supervisorctl restart all

    To verify that the hotfix was applied, check the Key Manager GUI at System→Hotfixes Applied.

note

In production environments, we recommend first testing the hotfix on a single Key Manager Server. Contact support at SSH Communications Security to help you with the verification process.

  • Hotfixes named hotfix_frontend_* are to be applied to a front end first. You must specifically be connected to the fixed front-end server to verify the fix. This is especially important if when a load-balancer is in use.

  • Hotfixes named hotfix_backend_* are to be applied to a back end first. You must ensure that the operation is run by the patched server. You may do this by setting the Maximum Processes setting to 0 on all unpatched back-end servers for the duration of the verification.

  • Hotfixes named hotfix_common_* may be first applied to any Key Manager server.

After you have verified the hotfix, apply the hotfix to the rest of your Key Manager servers.

Removing a Hotfix from Key Manager​

To remove a hotfix from a Key Manager server:

  1. Connect to the Key Manager server that you want to remove a hotfix from. Run the following command (replace hotfix_name with the Name of the hotfix):

    # ssh-mgr-controller --unapply-hotfix=hotfix_name

    Then restart its services:

    # supervisorctl restart all

    To verify that the hotfix was removed, check the Key Manager GUI at System→Hotfixes Applied.

  2. Hotfixes that are not applied to any Key Manager servers may be deleted from the system via the Key Manager GUI, from System→Hotfixes.

Troubleshooting Hotfix Management​

Symptom: Applying the hotfix fails with a checksum mismatch error​

If you encounter an error like the following when applying a hotfix:

Exception: File /opt/sshmgr/api/v2/operations.pyc's SHA checksum 
on disk 227957ee4fa02838e9ebbc7832e301df9f6978b2 does not match 
what we expected to override: f2e536bef2ed21552e21af703935b6d259f49dc5

The hotfix is not intended for the current version of Key Manager, and can not be applied. Contact support at SSH Communications Security for additional solutions.