tokio.tools.lfsstatus module¶
Given a file system and a datetime, return summary statistics about the OST fullness at that time
-
tokio.tools.lfsstatus.
_summarize_failover
(fs_data)[source]¶ Summarize failover data for a single time record
Given an fs_data dict, generate a dict of summary statistics. Expects fs_data dict of the form generated by parse_lustre_txt.get_failovers:
{ "abnormal_ips": { "10.100.104.140": [ "OST0087", "OST0086", ... ], "10.100.104.43": [ "OST0025", "OST0024", ... ] }, "mode": 1 }
Parameters: fs_data (dict) – a single timestamp and file system record taken from the output of nersc_lfsstate.NerscLfsOstMap.get_failovers Returns: summary metrics about the state of failovers on the file system Return type: dict
-
tokio.tools.lfsstatus.
_summarize_fullness
(fs_data)[source]¶ Summarize fullness data for a single time record
Given an fs_data dict, generate a dict of summary statistics. Expects fs_data dict of form generated by nersc_lfsstate.NerscLfsOstFullness:
{ "MDT0000": { "mount_pt": "/scratch1", "remaining_kib": 2147035984, "target_index": 0, "total_kib": 2255453580, "used_kib": 74137712 }, "OST0000": { "mount_pt": "/scratch1", "remaining_kib": 28898576320, "target_index": 0, "total_kib": 90767651352, "used_kib": 60894630700 }, ... }
Parameters: fs_data (dict) – a single timestamp and file system record taken from a nersc_lfsstate.NerscLfsOstFullness object Returns: summary metrics about the state of the file system fullness Return type: dict
-
tokio.tools.lfsstatus.
get_failures
(file_system, datetime_target, **kwargs)[source]¶ Get file system failures
Is a convenience wrapper for get_summary.
Parameters: - file_system (str) – Logical name of file system whose data should be retrieved (e.g., cscratch)
- datetime_target (datetime.datetime) – Time at which requested data should be retrieved
- cache_file (str) – Basename of file to search for the requested data
Returns: various statistics about the file system fullness
Return type:
-
tokio.tools.lfsstatus.
get_failures_lfsstate
(file_system, datetime_target, cache_file=None)[source]¶ Get file system failures from nersc_lfsstate connector
Wrapper around the generic get_lfsstate function.
Parameters: - file_system (str) – Lustre file system name of the file system whose data should be retrieved (e.g., snx11025)
- datetime_target (datetime.datetime) – Time at which requested data should be retrieved
- cache_file (str) – Basename of file to search for the requested data
Returns: Whatever is returned by get_lfsstate
-
tokio.tools.lfsstatus.
get_fullness
(file_system, datetime_target, **kwargs)[source]¶ Get file system fullness
Is a convenience wrapper for get_summary.
Parameters: - file_system (str) – Logical name of file system whose data should be retrieved (e.g., cscratch)
- datetime_target (datetime.datetime) – Time at which requested data should be retrieved
Returns: various statistics about the file system fullness
Return type: Raises: tokio.ConfigError
– When no valid providers are found
-
tokio.tools.lfsstatus.
get_fullness_hdf5
(file_system, datetime_target)[source]¶ Get file system fullness from an HDF5 object
Given a file system name (e.g., snx11168) and a datetime object, return summary statistics about the OST fullness.
Parameters: - file_system (str) – Name of file system whose data should be retrieved
- datetime_target (datetime.datetime) – Time at which requested data should be retrieved
Returns: various statistics about the file system fullness
Return type: Raises: ValueError
– if an OST name is encountered which does not conform to a naming convention from which an OST index can be derived
-
tokio.tools.lfsstatus.
get_fullness_lfsstate
(file_system, datetime_target, cache_file=None)[source]¶ Get file system fullness from nersc_lfsstate connector
Wrapper around the generic get_lfsstate function.
Parameters: - file_system (str) – Lustre file system name of the file system whose data should be retrieved (e.g., snx11025)
- datetime_target (datetime.datetime) – Time at which requested data should be retrieved
- cache_file (str) – Basename of file to search for the requested data
Returns: Whatever is returned by
tokio.tools.lfsstatus.get_lfsstate()
-
tokio.tools.lfsstatus.
get_lfsstate
(file_system, datetime_target, metric, cache_file=None)[source]¶ Get file system fullness or failures
Given a file system name (e.g., snx11168) and a datetime object
- locate and load the lfs-df (fullness) or ost map (failures) file
- find the sample immediately preceding the datetime (don’t find one that overlaps it
- return summary statistics about the OST fullness or OST failures
Parameters: - file_system (str) – Lustre file system name of the file system whose data should be retrieved (e.g., snx11025)
- datetime_target (datetime.datetime) – Time at which requested data should be retrieved
- metric (str) – either “fullness” or “failures”
- cache_file (str) – Basename of file to search for the requested data
Returns: various statistics about the file system fullness
Return type: Raises: ValueError
– ifmetric
does not contain a valid optionIOError
– when no valid data sources can be found for the given date