Connect to NERSC’s Intel Data Center Tool for SSDs outputs
Processes and aggregates the output of Intel Data Center Tool for SSDs outputs
in the format generated by NERSC’s daily script. The NERSC infrastructure runs
ISDCT in a verbose way on every burst buffer node, then collects all the
resulting output text files from a node into a directory bearing that node’s nid
nid01234/*.txt). There is also an optional timestamp file contained
in the toplevel directory. Also processes .tar.gz versions of these collected
Dictionary subclass that self-populates with ISDCT output data
Load the output of a NERSC ISDCT dump.
Parameters: input_file (str) – Path to either a directory or a tar(/gzipped) directory containing the output of NERSC’s ISDCT collection script.
Calculate additional metrics not presented by ISDCT.
Calculates additional convenient metrics that are not directly presented by ISDCT, then adds the resulting key-value pairs to self.
Highlight differences between self and another NerscIsdct.
Subtract each counter for each serial number in this object from its counterpart in
old_isdct. Return the changes in each numeric counter and any serial numbers that have appeared or disappeared.
Dictionary containing the following keys:
- added_devices - device serial numbers which exist in self but not old_isdct
- removed_devices - device serial numbers which do not exist in self but do in old_isdct
- devices - dict keyed by device serial numbers and whose values are dicts of keys whose values are the difference between old_isdct and self
Wrapper around the filetype-specific loader.
Infer the type of input being given, dispatch the correct loading function, and populate keys/values.
Load ISDCT output from a tar(.gz).
Load a collection of ISDCT outputs as created by the NERSC ISDCT script. Assume that ISDCT output files each contain a single output from a single invocation of the isdct tool, and outputs are grouped into directories named according to their nid numbers (e.g., nid00984/somefile.txt).
Convert path to ISDCT output into a nid.
Given a path to some ISDCT output file, somehow figure out what the nid name for that node is. This encoding is specific to the way NERSC collects and preserves ISDCT outputs.
Parameters: path (str) – path to an ISDCT output text file Returns: Node identifier (e.g., nid01234) Return type: str
Merge ISDCT outputs into a single object.
Aggregates counters from each record based on the NVMe device serial number, with redundant counters being overwritten.
Parameters: parsed_counters_list (list) – List of parsed ISDCT outputs as dicts. Each list element is a dict with a single key (a device serial number) and one or more values; each value is itself a dict of key-value pairs corresponding to ISDCT/SMART counters from that device. Returns: Dict with keys given by all device serial numbers found in parsed_counters_list and whose values are a dict containing keys and values representing all unique keys across all elements of parsed_counters_list. Return type: dict
Coerce all keys into a similar naming convention.
Converts Intel’s mix of camel-case and snake-case counters into all snake-case. Contains some nasty acronym hacks that may require modification if/when Intel adds new funny acronyms that contain a mix of upper and lower case letters (e.g., SMBus and NVMe).
Parameters: key (str) – a string to normalize Returns: snake_case version of key Return type: str
Convert SMART values associated with one register into unique counters.
Take a buffer containing smart values associated with one register and create unique counters. Only necessary for older versions of ISDCT which did not output SMART registers in a standard “Key: value” text format.
Parameters: smart_buffer (dict) – SMART buffer as defined by parse_counters_fileobj() Returns: unique key:value pairs whose key now includes distinguishing device-specific characteristics to avoid collision from other devices that generated SMART data Return type: dict
Convert any output of ISDCT into key-value pairs.
Reads the output of a file-like object which contains the output of a single isdct command. Understands the output of the following options:
isdct show -smart(SMART attributes)
isdct show -sensor(device health sensors)
isdct show -performance(device performance metrics)
isdct show -a(drive info)
- fileobj (file) – file-like object containing the output of an ISDCT command
- nodename (str) – name of node corresponding to fileobj, if known
dict of dicts keyed by the device serial number.