Importing Key-Activity Information Manually

Key Manager provides a key-activity-import tool that can be used to manually import key-activity information from specific log files, from a specified time period. Possible uses for the tool includes:

• Filling gaps in key-activity information: There may be some situation where key-activity scans fail to detect key activity from certain periods of time. Key-activity gaps can be caused by frequent log rotation and/or infrequent key-activity scanning. For example, when key-activity logs on the hosts in the managed environment are rotated to unknown locations before they are scanned.

  Key Manager raises alerts when it suspects that there are gaps in key-activity logs. When you encounter such alerts, you should use the key-activity-import tool to detect potentially missed key-activity entries.

• Replacing automatic key-activity scans: The key-activity import tool allows greater control over what key-activity information is imported, where it is imported from, and when it is imported.

Importing key-activity information involves the following tasks:

1. Obtain the log file that contains key-activity information.

2. If necessary, map log entries to corresponding host entries in Key Manager.

3. Run the key-activity-import tool to import key-activity information from the log file.

These tasks are explained in more detail in the following sections.

Prerequisites

The target hosts for which you want to import key-activity information must be part of the managed environment. Additionally, any keys with key activity must have been detected by regular host scans.

The key-activity-import tool is available on all Key Manager back ends. To run the key-activity-import tool, you must have root access to a Key Manager back end.

Key-activity information is recorded by the SSH server on each host. Note that OpenSSH and Quest SSH servers must be configured to log key-activity information, before any key-activity information can be imported from hosts that run these SSH server products.

Obtaining Key-Activity Log Files

Key-activity information is recorded by the SSH server on each host. You must obtain the log file that contains the SSH server entries.

Once you have the correct log file. Copy it to the Key Manager back end where you are going to run the key-activity-import tool.

Mapping Log Entries to Hosts

Mapping log entries to hosts is only necessary if the log file contains entries from multiple hosts. If the log file only contains log entries from a single host, you can skip this section.

1. On the Key Manager back end where you want to run the key-activity-import tool, create a mapping file that specifies what hosts each log entry belongs to. The mapping file should have the following format:

   {
     "hostname1_in_log" : "hostname1_in_keymanager",
     "hostname2_in_log" : "hostname2_in_keymanager",
     ...
     "hostnameN_in_log" : "hostnameN_in_keymanager"
   }

   The values hostname*in_log are to be replaced with the name of the host as specified in the log entries (syslog name). The corresponding hostname*in_keymanager values must be replaced with the name of the host as specified in the Key Manager system.

   Typically, the syslog name is given on each log entry, immediately after the time stamp.

   

The name of the host as specified by the Key Manager system, is displayed on the Hosts page in the Key Manager GUI, under the Hostname column. By default, Key Manager names hosts according to their FQDNs. If the host FQDN is not resolvable, Key Manager names hosts according to their IPs instead. When mapping log entries to hosts, be mindful that the defaults may not apply to all situations, as Key Manager administrators may set arbitrary names for hosts in Key Manager.

2. Save the file with the .json extension, with a file name and location of your choosing.

Key-Activity-Import Tool Usage

The key-activity-import tool is a Python script. The script is located at /opt/sshmgr/bin/ssh-mgr-import-key-activity-log on all Key Manager back ends.

The general syntax for running the key-activity-import tool is as follows:

/opt/sshmgr/bin/ssh-mgr-import-key-activity-log \
[options] [hostname] logfile

logfile

The absolute path to the log file from which you want to import key-activity information.

hostname

The name of the host for which the logfile specifies key-activity information. This must be the name of the host as specified in the Key Manager system.

hostname is used to indicate which host the key-activity entries in the logfile are to be associated to.

host names in the Key Manager system can be reviewed on the Hosts page in the GUI, under the Hostname column.

note

The hostname argument must not be used if the logfile contains log entries for more than one host. In such cases, log entries must be associated to hosts using a mapping file, with the option --hostname-map. For more information about using a mapping file, see Mapping Log Entried to Hosts.

The available options for the key-activity-import tool are:

-h, --help

Display the help message for the key-activity-import tool.

--hostname-map=HOSTNAME_MAP

Absolute path to the host-name mapping file, which specifies which host each log entry belongs to.

--timezone=TIMEZONE

The time zone used in the log entries, given as the offset from UTC in the ±hhmm format. For example, +0300 or -0500. This value is used only if the settings of the host do not specify a time zone.

--start-date=START_DATE

The date from which to start reading the key activity log. All key activity entries before this date are ignored. Provide the value in the %b dd hh:mm format, where %b is the abbreviated name of the month. The time of day is provided in the 24-hour-clock format. For example, Jan 01 14:32. Defaults to unlimited.

--end-date=END_DATE

The date at which to stop reading the key activity log. All key activity entries after this date are ignored. Provide the value in the same format as you would for --start-date. This option must always be provided (except when invoking the help message).

--update-gap

Imports all the key-activity entries from the given time period, regardless of whether the key-activity entry already exists in the Key Manager system. When using this option, you must provide both the --start-date and the --end-date options.

caution

--update-gap introduces duplicate key-activity entries if any of the imported entries already exists in the Key Manager system. Duplicate entries cause Key Manager to report corresponding login counts as greater values than what they should be. To prevent importing duplicate entries, make sure that you specify the time period so that no existing key-activity entries are included in it.

If you simply want to fill potential key-activity gaps (as reported by gap alerts), we recommend you use the --fill-gaps option instead.

--fill-gaps

Imports all the key-activity entries that occur during potential gaps within the given time period. Key Manager determines the time periods of potential gaps based on the periods listed in existing gap alerts. Gaps that occur partially within the given time period are only filled within the limits of the time period. Any gap alerts that belong (even partially) within the time period are automatically resolved.

--add-gap

Add a key-activity gap to the host. The timespan of the gap must be specified using the options --start-date and --end-date.

--year=YEAR

The year that is assigned to log entries where the timestamp does not specify year information. Defaults to the current year. Equivalent to setting --start-date and --end-data for a whole year.

--test-mode

Displays what entries would be imported, without actually importing anything. You can use this option to test your commands without making any changes to the Key Manager system.

--dry-run

Displays what entries would be imported, without actually importing anything. You can use this option to test your commands without making any changes to the Key Manager system.

-v, verbose

Enables verbose mode. When running in verbose mode, all the unknown and discarded key-activity entries are displayed.

Examples

This section provides examples about using the key-activity-import tool.

Filling Key-Activity Gaps

In this example we use the key-activity-import tool to fill key-activity gaps that have been reported by Key Manager. In the example scenario. Key Manager has raised the following alerts about potential gaps in key activity:

• Gap alert 1:

  • Target host: server01.example.com
  • Start time: Aug 01 13:00
  • End time: Aug 01 15:00

• Gap alert 2:

  • Target host: server01.example.com
  • Start time: Aug 02 16:00
  • End time: Aug 02 18:00

We have a log file that contains all the SSH server output from server01.example.com between Aug 01 01:00 and Aug 03 01:00 (we will call this file logfile during this example). We can use the key-activity-import tool to manually import any missing log entries from this file.

Additionally, we know that the log entries in logfile are time-stamped according to Central European Time (CET), which is at a +1-hour offset from UTC.

With this information, and with the described logfile, we can fill the key-activity gaps.

1. Gain root terminal access to a Key Manager back end. The remaining steps are to be performed on the Key Manager back end of your choice.

2. Download logfile to the Key Manager back end.

3. Fill the key-activity gaps with:

   # python /opt/sshmgr/bin/ssh-mgr-import-key-activity-log \
   --timezone="+0100" --start-date="Aug 01 13:00" --end-date="Aug 02 18:00" \
   --fill-gaps server01.example.com /path/to/logfile

   The --timezone option is used to indicate the UTC time-zone offset of the log entries in logfile. This argument is required to assign correct timestamps to imported key-activity entries.

   The options --start-date, --end-date, and --fill-gaps are used to restrict what key-activity entries are imported from logfile. In this example, all log entries that occur before Aug 01 13:00 or after Aug 02 18:00 are ignored. The --fill-gaps option further restricts what log entries are imported: Since we used the --fill-gaps option, the command only imports key-activity entries that occurred during the time periods specified in the gap alerts (from Aug 01 between 13:00 and 15:00, and from Aug 02 between 16:00 and 18:00). The gap alerts are automatically resolved.

   Immediately after the options, we specify the name of the host. Key Manager uses this information to associate key-activity entries to the correct host.

   The path to logfile is provided at the end of the command.

Importing Key Activity from Multiple-Host Log Files

In this example we import the latest key activity from a log file that contains key-activity entries from many different hosts. This example focuses on how to map log entries to corresponding hosts. We recommend that you familiarize yourself with the basics of host-name mapping (see Mapping Log Entries to Hosts) before reading this example.

Since logfile contains entries from multiple hosts, you must create a host-name-mapping file in order to associate each log entry to the correct host. For this you need to determine:

• The name of the hosts, as specified in the Key Manager system.

• The name of the same host, as specified in the log-file entries (syslog name of the host).

In the example scenario, we have a log file (called logfile in this example) that contains the latest key-activity information from 3 different hosts, which are named as follows in the Key Manager system:

• server01.example.com
• server02.example.com
• server03.example.com

We also know that all entries in logfile are timestamped according to Pacific Standard Time (PST), which is at a -8-hour offset from UTC. The time when this example is performed is Nov 22 13:00.

1. We need to determine which host each key-activity entry comes from.

   From the log entries in logfile, we can determine that the hosts are known under different names in logfile. For example, the following entry indicates that server01.example.com is probably named server01 in logfile:

   Nov 21 11:35:07 server01 sshd[1352]: \
   Accepted password for root from 10.1.64.236 port 50495 ssh2

   Additional log entries reveal the rest of the syslog names. In this example scenario, the syslog names and their corresponding host names in Key Manager are:

   • server01 : server01.example.com
   • server02 : server02.example.com
   • server03 : server03.example.com

2. After we have determined the correspondences between syslog names and host names in Key Manager, we can create the host-name-mapping file. In this example scenario, the contents of the host-name-mapping file look like the following:

   {
     "server01" : "server01.example.com",
     "server02" : "server02.example.com",
     "server03" : "server03.example.com"
   }

   We save the file as hostnamemap.json.

3. To import the latest key activity from logfile, we use the command:

   # /opt/sshmgr/bin/ssh-mgr-import-key-activity-log --timezone="-0800" \
   --hostname-map="hostnamemap.json" --end-date="Nov 22 13:00" \
   /path/to/logfile

   For each host, the command imports the key-activity entries that occurred after their last known key-activity entry. --end-date is set to current time, so we can be sure that the latest key-activity entries are imported (assuming that they exist in logfile).

   Key-activity entries are associated to appropriate hosts according to the host-name mappings defined in hostnamemap.json: Log entries registered for server01 are mapped to the host that is known as server01.example.com in the Key Manager system. Log entries for the other hosts are mapped similarly to their corresponding hosts.