Host Discovery with Offline Scans

The Key Manager host utility can be manually executed on target hosts to gather scan data. Scan data can then be manually imported to the Key Manager system. This method of scanning is called offline scanning. Offline scanning allows you to scan a host without having to deploy it to the managed environment.

In certain environments, setting up offline scanning may be significantly easier and faster than deploying a host to the managed environment, and in such cases this method can be used to quickly gain information about your hosts, before deciding which of your hosts need be deployed to Key Manager. Note that offline scanning only scans hosts, and does not enable management actions on hosts.

The high-level steps for setting up offline scanning are the following:

Upload the Key Manager host utility to all the hosts that you want to scan.
Install and configure the scan-data-import tool on an arbitrary host, or run the pre-installed scan-data-import tool on a Key Manager front end.

After setup, you can perform offline scanning as follows:

Run the Key Manager host utility on the target host(s) as root to generate scan data.
Run the scan-data-import tool to import previously-generated scan data into Key Manager.

Steps for setting up and performing offline scanning are described in more detail in the following subsections.

Prerequisites

note

Offline scanning feature must be enabled in your Key Manager system. On Red Hat and CentOS based Key Manager systems, certain support packages will need to be installed (see the PrivX Key Manager Installation Manual).

Check and execute the following steps before setting up offline scanning:

You will need the host utility package, which contains the files for the Key Manager host utility. The package also contains example tools for importing scan data. You can obtain the host utility package from the Customer Download Center at https://my.ssh.com.
Tools for importing scan data are provided as examples: they are intended for illustrating how to use the Key Manager API endpoint scandata for importing scan data. You may use the example tools as reference or as a basis for implementing your own tools.
If you are going to use the example tools (scan-data-batch-import and scan-data-batch-report) for importing scan data, the machine on which the tools are to be used must be installed with Python 3.5 or later. Furthermore, the Python 3 installation on that machine must include the aiohttp library version 1.0.5, the ujson library version 1.35, and the requests library. These can be added to your Python 3 installation, for example, using pip:
```
# pip3 install aiohttp==1.0.5
# pip3 install ujson==1.35
# pip3 install requests
```
Hosts using the offline-scan method have the following additional requirements:
- Unix platforms must have a minimum supported version of Python 1.5 or later.
- Windows platforms must be installed with the following:
  - .Net version 4 (Client Profile or Full Framework).
  - CLR version 4.
These instructions describe how to set up scan-data importing using the provided import tools. You will need to adapt the instructions when using your own tools for scan-data import.

Offline-Scan Setup

Set up the host utility in a way that you can execute it on all the hosts that you want to scan. Also set up the tools for importing scan data. These procedures are performed as follows:

Obtain the host utility files. These are available in the host utility package:

sshmgr-host-utility-*.zip

The package name indicates version numbers in the following format:

keymanagerversion-unixversion.windowsversion

Where keymanagerversion indicates the Key Manager version, unixversion indicates the version number of the Unix host-utility script, and windowsversion indicates the version number of the Windows host-utility script.

From the host-utility package, obtain the following host-utility scripts:

host utility for Unix hosts: ssh-mgr-host-utility.sh
host utility for Windows hosts: ssh-mgr-host-utility.exe

note

The Unix host utility ssh-mgr-host-utility.sh cannot process authorized key files larger than 2 MB by default. You can edit the file size limit using the UKM_SINGLE_FILE_SIZE_LIMIT environment variable when invoking the host utility. The value is defined in bytes.

Set up the host utility in a way that you can execute it on all the hosts that you want to scan. This can be accomplished by, for example, uploading the host utility to the target hosts. For each host, make sure to use the host-utility script that is intended for the host platform.

On Unix hosts, the host utility must be executable as root. To enable this, change the permissions of the host utility as follows (replace /path/to/ssh-mgr-host-utility.sh with the path of the host utility):
```
# chmod a+x /path/to/ssh-mgr-host-utility.sh
```
The host utility is now set up.

You will also need to set up a means to import the scan data. If you intend to use the example tools provided with Key Manager, perform the following:

Set up the API connection parameters using environment variables. The following environment variables must be set for the user who will run the scan-data-import tool:
- SSHMGR_URL: The URL prefix of a Key Manager front end that runs the Key Manager API. When using certificate authentication, the host must be the domain address specified in the server certificate. For example:
  
  https://frontend.example.com
- SSHMGR_TOKEN: Path to a Key Manager API-token file. For more information about generating API tokens, see Setting Up Client Connections to the API.
- SSHMGR_CA (optional): Path to the file containing the CA certificate of the Key Manager front end. Defaults to ~/crt/ca.crt. If set to ignore, insecure connections are used. Insecure connections are only recommended for local connections in evaluation environments.
Set up the tools required for importing scan data.

Copy the scan-data-batch-import tool and the scan-data-batch-report tools to the host that is to be used for importing scan data. The batch-import and batch-report tools are available in the host utility package:
- scan-data-batch-import tool: scan-data-batch-import.py
- scan-data-batch-report tool: scan-data-batch-report.py
note
On all Key Manager Servers, the example tools are available under the /opt/sshmgr/examples/ directory.
Ensure that the import tools can be executed by the user (replace user:group with the user and group of your choice, replace the /path/to/import-tool with the path to which you copied the scan-data-import script):
```
# chown user:group /path/to/import-tool
# chmod u+rx /path/to/import-tool
```
Repeat this step for each tool.

You should now have a working setup for offline scanning.

Performing Offline Scanning

For an overview on how offline scan functions, see Overview of the Offline Unix Scan Script and Windows Scan Utility.

After you have set up offline scanning (as per instructions in Offile Scan Setup), you can perform it as follows:

Run the Key Manager host utility on the hosts you want to scan. This is done as follows:

Running the host utility on Unix Machines

On the target Unix hosts, run the host utility with a command similar to the following (replace /path/to/ssh-mgr-host-utility.sh with the path of the host utility script, replace <scan_type> with the type of scan you want to perform, replace hostname-scan-results.txt with the path to which scan data will be generated):
```
# /path/to/ssh-mgr-host-utility.sh <scan_type> > hostname-scan-results.txt 2>&1
```
Running the host utility on Windows Machines

On the target Windows hosts, run the host utility with a command similar to the following (replace /path/to/ssh-mgr-host-utility.exe with the path of the host utility script, replace hostname-scan-results.json with the path to which scan data will be generated):
```
# /path/to/ssh-mgr-host-utility.exe scan-local > hostname-scan-results.json
```
For troubleshooting purposes you may run the following command instead. This runs the host utility, and also stores the error output (replace error-messages.txt with the name of the file to which you want to store the error output):
```
# /path/to/ssh-mgr-host-utility.exe \
scan-local > hostname-scan-results.json 2> error-messages.txt
```
note
Only type of scan available to the Windows host utility is scan-local.

Scan Types

The available scan types are:
- scan-local - Discovers host information, local users, and their locally-stored keys.
- scan-without-nfs - Includes scan-local, additionally discovers local keys that may belong to remote users.
- scan-full - Includes scan-without-nfs, additionally discovers potential remote users and their user keys from possible remote filesystems.
  
  note
  Note that scan-full may take considerably longer than other scan types. In environments where network users can freely access hosts, this may also add considerable amounts of redundant data into the Key Manager system.
Transfer the scan data from the scanned hosts to the machine where you have set up your import tools. Note that if you are using the example batch-import tools, all scan data must be placed under one single directory.
Run your tools for importing scan data. Here we only provide commands for running the example tools provided in the host utility package. If you are using your own tools, run those instead.

Run the scan-data-batch-import tool to import scan data. This is done with a command like the following (replace /path/to/scan-data-batch-import.py with the path to the scan-data-batch-import script, replace /path/to/scandata_directory with the directory containing your scan data):
```
$ /path/to/scan-data-batch-import.py /path/to/scandata_directory
```
The command returns the operation ID, similar to the following:
```
Created operation at https://keymanager.example.com/api/v3/operations/3/
Created tasks at:
        https://keymanager.example.com/api/v3/scandata-tasks/dd602bef-483f-48f0/
        https://keymanager.example.com/api/v3/scandata-tasks/d3c7d436-b2c4-4ab6/

        ...

Please run `scan-data-batch-report.py 999 <RESULT-DIRECTORY>` for details
```
To obtain the results of the import operation, run the scan-data-batch-report tool similarly to the following (replace /path/to/scan-data-batch-report.py with the path to the scan-data-batch-report script, replace /path/to/reports_directory with the path to the directory where you want the reports to be stored, replace 999 with the operation ID obtained previously):
```
$ /path/to/scan-data-batch-report.py 999 /path/to/reports_directory
```
After successfully importing scan data, the host(s) are available in Key Manager. Hosts that have been imported this way are initially placed in the externally-monitored (ext-monitored) state.

Deploying Offline-Scanned Hosts

You can enable management actions on offline-scanned hosts by deploying them. Offline-scanned hosts can be deployed in either of the following ways:

Add the host normally to the managed environment using agentless or agent-based connections, as described in Adding Agentless Hosts and Adding Agent-Based Hosts respectively.
Perform a Deploy action on the host. When an imported host is deployed this way, Key Manager authenticates to the host using host-deployment credentials and deploys the host using agentless connections. Note that deploying a host this way fails if no host-deployment credentials are set for the host (either in host-group settings, or in global host settings). For more information about host-deployment credentials, see Deployment Credentials.

After successful deployment, the state of the host should be monitored or managed (instead of ext-monitored).

caution

PuTTY discovery is only supported on hosts using offline scans. You will lose all visibility to PuTTY keys after host deployment.

Prerequisites​

Offline-Scan Setup​

Performing Offline Scanning​

Running the host utility on Unix Machines​

Running the host utility on Windows Machines​

Scan Types​

Deploying Offline-Scanned Hosts​

Prerequisites

Offline-Scan Setup

Performing Offline Scanning

Running the host utility on Unix Machines

Running the host utility on Windows Machines

Scan Types

Deploying Offline-Scanned Hosts