SPy Assets

Simple Asset Trees

class seeq.spy.assets.Tree(data, *, friendly_name=None, description=None, workbook='Data Lab >> Data Lab Analysis', datasource=None, convert_displays_to_sdl=True, quiet=None, errors=None, status=None, session: Session | None = None)

Bases: object

Manages an asset tree as a collection of item definitions in the form of a metadata DataFrame. Allows users to manipulate the tree using various functions.

Parameters:

data ({pandas.DataFrame, str}) –
Defines which element will be inserted at the root. If an existing tree already exists in Seeq, the entire tree will be pulled recursively. If this tree doesn’t already within the scope of the workbook, new tree elements will be created (by deep-copy or reference if applicable).

The following options are allowed:
1. A name string. If an existing tree with that name (case-insensitive) is found, all children will be recursively pulled in.
2. An ID string of an existing item in Seeq. If that item is in a tree, all children will be recursively pulled in.
3. spy.search results or other custom dataframes. The ‘Path’ column must be present and represent a single tree structure.
4. A filename or relative file path to a CSV file. The CSV file should have either a complete Name column or a complete ID column, and should specify the tree path for each item either in a ‘Path’ column formatted as “Root >> Next Level”:
  
  Path
  
  Name
  
  Root >> Next Level
  
  Item_Name
  
  or as a series of ‘Levels’ columns, e.g. “Level 1” and “Level 2” columns, where “Level 1” would be “Root” and “Level 2” would be “Next Level”:
  
  Level 1
  
  Level 2
  
  Name
  
  Root
  
  Next Level
  
  Item_Name
The ‘Level’ columns will be forward-filled.
friendly_name (str, optional) – Use this specified name rather than the referenced item’s original name.
description (str, optional) – The description to set on the root-level asset.
workbook (str, default 'Data Lab >> Data Lab Analysis') – The path to a workbook (in the form of ‘Folder >> Path >> Workbook Name’) or an ID that all pushed items will be ‘scoped to’. You can push to the Corporate folder by using the following pattern: ‘__Corporate__ >> Folder >> Path >> Workbook Name’. Scoped items will not be visible/searchable using the data panel in other workbooks. A Tree can be globally scoped (and therefore visible across all workbooks) by specifying workbook=None.
datasource (str, optional, default 'Seeq Data Lab') –
The name of the datasource within which to contain all the pushed items. Items inherit access control permissions from their datasource unless it has been overridden at a lower level. If you specify a datasource using this argument, you can later manage access control (using spy.acl functions) at the datasource level for all the items you have pushed.

If you instead want access control for your items to be inherited from the workbook they are scoped to, specify spy.INHERIT_FROM_WORKBOOK.
convert_displays_to_sdl (bool, default True) – If True, then if displays from a non-Seeq Data Lab datasource are found in a tree that is otherwise from a Seeq Data Lab datasource, then they will be replaced by displays from a Seeq Data Lab datasource when this tree is pushed.
quiet (bool, default False) – If True, suppresses progress output. This setting will be the default for all operations on this Tree. This option can be changed later using tree.quiet = True or by specifying the option for individual function calls. Note that when status is provided, the quiet setting of the Status object that is passed in takes precedence.
errors ({'raise', 'catalog'}, default 'raise') – If ‘raise’, any errors encountered will cause an exception. If ‘catalog’, errors will be added to a ‘Result’ column in the status.df DataFrame. The option chosen here will be the default for all other operations on this Tree. This option can be changed later using tree.errors = ‘catalog’ or by specifying the option for individual function calls.
status (spy.Status, optional) – If specified, the supplied Status object will be updated as the command progresses. It gets filled in with the same information you would see in Jupyter in the blue/green/red table below your code while the command is executed. The table itself is accessible as a DataFrame via the status.df property.
session (spy.Session, optional) – If supplied, the Session object (and its Options) will be used to store the login session state. This is useful to log in to different Seeq servers at the same time or with different credentials.

count(item_type=None)

Count the number of elements in the tree of each Seeq type. If item_type is not specified, then returns a dictionary with keys ‘Asset’, ‘Signal’, ‘Condition’, ‘Scalar’, and ‘Unknown’. If item_type is specified, then returns an int.

Parameters:: item_type ({'Asset', 'Signal', 'Condition', 'Scalar', 'Uncompiled Formula'}, optional) – If specified, then the method will return an int representing the number of elements with Type item_type. Otherwise, a dict will be returned.

property height: Property that gives the current height of the tree. This is the length of the longest item path within the tree.

insert(children=None, parent=None, *, name=None, formula=None, formula_parameters=None, roll_up_statistic=None, roll_up_parameters=None, friendly_name=None, errors=None, quiet=None, status=None)

Insert the specified elements into the tree.

Parameters:

children ({pandas.DataFrame, pandas.Series, dict, str, list, Tree}, optional) –
Defines which element or elements will be inserted below each parent. If an existing node already existed at the level in the tree with that name (case-insensitive), it will be updated. If it doesn’t already exist, a new node will be created (by deep-copy or reference if applicable).

The following options are allowed:
1. A basic string or list of strings to create a new asset.
2. Another SPy Tree.
3. spy.search results or other custom dataframes.
parent ({pandas.DataFrame, str, int, list}, optional) –
Defines which element or elements the children will be inserted below.

The following options are allowed:
1. No parent specified will insert directly to the root of the tree.
2. String name match (case-insensitive equality, globbing, regex, column values) will find any existing nodes in the tree that match.
3. String path match, including partial path matches.
4. ID. This can either be the actual ID of the tree.push()ed node or the ID of the source item.
5. Number specifying tree level. This will add the children below every node at the specified level in the tree (1 being the root node).
6. spy.search results or other custom dataframe.
name (str, optional) – An alternative to the children parameter for specifying a single name for an asset or calculation to be inserted.
formula (str, optional) – The formula for a calculated item. The formula and formula_parameters are used in place of the children argument.
formula_parameters (dict, optional) –
The parameters used in the specified formula. The following options are allowed:
1. The name of an item that is an asset sibling of this calculation.
2. The relative path to an item in this Tree (E.G. ‘.. >> Area E >> Temperature’).
3. The absolute path to an item in this Tree (E.G. ‘Example >> Cooling Tower 2 >> Area E >> Temperature’).
4. The ID of an item (including ones not in the Tree).
roll_up_statistic (str, optional) – The statistic to use when inserting a roll-up calculation. Valid options are Average, Maximum, Minimum, Range, Sum, Multiply, Union, Intersect, Counts, Count Overlaps, Combine With.
roll_up_parameters (str, optional) – A wildcard or regex string that matches all of the parameters for a roll-up calculation. The roll-up calculation will apply the function specified by roll_up_statistic to all parameters that match this string. For example, roll_up_statistic=’Average’, roll_up_parameters=’Area ? >> Temperature’ will calculate the average of all signals with path ‘Area ? >> Temperature’ relative to the location of the roll-up in the tree.
friendly_name (str, optional) – Use this specified name rather than the referenced item’s original name.
errors ({'raise', 'catalog'}, optional) – If ‘raise’, any errors encountered will cause an exception. If ‘catalog’, errors will be added to a ‘Result’ column in the status.df DataFrame. This input will be used only for the duration of this function; it will default to the setting on the Tree if not specified.
quiet (bool, optional) – If True, suppresses progress output. This input will be used only for the duration of this function; it will default to the setting on the Tree if not specified. Note that when status is provided, the quiet setting of the Status object that is passed in takes precedence.
status (spy.Status, optional) – If specified, the supplied Status object will be updated as the command progresses. It gets filled in with the same information you would see in Jupyter in the blue/green/red table below your code while the command is executed. The table itself is accessible as a DataFrame via the status.df property.

missing_items(return_type='print')

Identify elements that may be missing child elements based on the contents of other sibling nodes.

Parameters:: return_type ({'print', 'string', 'dict'}, default 'print') – If ‘print’, then a string that enumerates the missing items will be printed. If ‘string’, then that same string will be returned and not printed. If ‘dict’, then a dictionary that maps element paths to lists of their potential missing children will be returned.

move(source, destination=None, *, errors=None, quiet=None, status=None)

Move the specified elements (and all children) from one location in the tree to another.

Parameters:

source ({pandas.DataFrame, str}) –
Defines which element or elements will be moved.
1. String path match.
2. ID. This can either be the actual ID of the tree.push()ed node or the ID of the source item.
3. spy.search results or other custom dataframe.
destination ({pandas.DataFrame, str}; optional) –
Defines the new parent for the source elements.
1. No destination specified will move the elements to just below the root of the tree.
2. String path match.
3. ID. This can either be the actual ID of the tree.push()ed node or the ID of the source item.
4. spy.search results or other custom dataframe.
errors ({'raise', 'catalog'}, optional) – If ‘raise’, any errors encountered will cause an exception. If ‘catalog’, errors will be added to a ‘Result’ column in the status.df DataFrame. This input will be used only for the duration of this function; it will default to the setting on the Tree if not specified.
quiet (bool, optional) – If True, suppresses progress output. This input will be used only for the duration of this function; it will default to the setting on the Tree if not specified. Note that when status is provided, the quiet setting of the Status object that is passed in takes precedence.
status (spy.Status, optional) – If specified, the supplied Status object will be updated as the command progresses. It gets filled in with the same information you would see in Jupyter in the blue/green/red table below your code while the command is executed. The table itself is accessible as a DataFrame via the status.df property.

push(*, metadata_state_file: str | None = None, errors: str | None = None, quiet: bool | None = None, status: Status | None = None) → DataFrame

Imports the tree into Seeq Server.

metadata_state_filestr, optional: The file name (with full path, if desired) to a “metadata state file” to use for “incremental” pushing, which can dramatically speed up pushing of a large asset tree. If supplied, the metadata push operation uses the state file to determine what changed since the last time the metadata was pushed and it will only act on those changes.
errors{‘raise’, ‘catalog’}, optional: If ‘raise’, any errors encountered will cause an exception. If ‘catalog’, errors will be added to a ‘Result’ column in the status.df DataFrame. This input will be used only for the duration of this function; it will default to the setting on the Tree if not specified.
quietbool, optional: If True, suppresses progress output. This input will be used only for the duration of this function; it will default to the setting on the Tree if not specified. Note that when status is provided, the quiet setting of the Status object that is passed in takes precedence.
statusspy.Status, optional: If specified, the supplied Status object will be updated as the command progresses. It gets filled in with the same information you would see in Jupyter in the blue/green/red table below your code while the command is executed. The table itself is accessible as a DataFrame via the status.df property.

Returns:: A DataFrame with the metadata for the items pushed, along with any errors and statistics about the operation. See spy.push() documentation for further details on this returned DataFrame.
Return type:: pandas.DataFrame

remove(elements, *, errors=None, quiet=None, status=None)

Remove the specified elements from the tree recursively.

Parameters:

elements ({pandas.DataFrame, str, int}) –
Defines which element or elements will be removed.
1. String name match (case-insensitive equality, globbing, regex, column values) will find any existing nodes in the tree that match.
2. String path match, including partial path matches.
3. ID. This can either be the actual ID of the tree.push()ed node or the ID of the source item.
4. Number specifying tree level. This will add the children below every node at the specified level in the tree (1 being the root node).
5. spy.search results or other custom dataframe.
errors ({'raise', 'catalog'}, optional) – If ‘raise’, any errors encountered will cause an exception. If ‘catalog’, errors will be added to a ‘Result’ column in the status.df DataFrame. This input will be used only for the duration of this function; it will default to the setting on the Tree if not specified.
quiet (bool, optional) – If True, suppresses progress output. This input will be used only for the duration of this function; it will default to the setting on the Tree if not specified. Note that when status is provided, the quiet setting of the Status object that is passed in takes precedence.
status (spy.Status, optional) – If specified, the supplied Status object will be updated as the command progresses. It gets filled in with the same information you would see in Jupyter in the blue/green/red table below your code while the command is executed. The table itself is accessible as a DataFrame via the status.df property.

select(within=None, *, condition=None, start=None, end=None, errors=None, quiet=None, status=None) → Tree

Select a subtree based on the specified parameters. Selects nodes of this Tree - creating a sub-tree - based on one or more Seeq Conditions present in the tree and evaluated during the given date range. If the within argument is given, then only the related nodes are evaluated and possibly selected.

Parameters:

within ({pandas.DataFrame, list, str}, optional) –
Specifies a pattern by which to select elements from the tree. If an element of the tree matches this pattern, then itself, its descendants, and its ancestors will be included in the resulting subtree

The following rules apply:

1) If not specified, the selection will return all items matching the given condition, start, and end criterion. 2) String name match (case-insensitive equality, globbing, regex) will find any existing nodes in the tree that match. 3) String path match, including partial path matches. 4) ID. This can either be the actual ID of the pushed node or the ID of the source item. 5) spy.search results or other custom dataframe.
condition ({pandas.DataFrame, list, str}, optional) – If specified, then only assets that contain a condition matching this argument that have at least one capsule present in the start-end range specified will be included in the subtree.
start ({str, pd.Timestamp}, optional) – The start date for which to evaluate the specified condition(s).
end ({str, pd.Timestamp}, optional) – The end date for which to evaluate the specified condition(s).
errors ({'raise', 'catalog'}, optional - default 'catalog') – If ‘raise’, any errors encountered will cause an exception. If ‘catalog’, errors will be added to a ‘Result’ column in the status.df DataFrame. This input will be used only for the duration of this function; it will default to the setting on the Tree if not specified.
quiet (bool, optional) – If True, suppresses progress output. This input will be used only for the duration of this function; it will default to the setting on the Tree if not specified. Note that when status is provided, the quiet setting of the Status object that is passed in takes precedence.
status (spy.Status, optional) – If specified, the supplied Status object will be updated as the command progresses. It gets filled in with the same information you would see in Jupyter in the blue/green/red table below your code while the command is executed. The table itself is accessible as a DataFrame via the status.df property.

Returns:

A new Tree containing the items matching the given selection criterion.

Return type:

Tree

property size: Property that gives the number of elements currently in the tree.

summarize(ret=False)

Generate a human-readable summary of the tree.

Parameters:: ret (bool, default False) – If True, then this method returns a string summary of the tree. If False, then this method prints the summary and returns nothing.

visualize(subtree=None, print_tree=True)

Prints an ASCII visualization of this tree to stdout.

subtreestr, optional: Specifies an asset in the tree. Only the part of the tree below this asset will be visualized.
print_tree: bool, optional: True (default) to print the tree visualization, False to return it as a string

property workbook: Property that gives the current workbook of the tree. Returns None if tree is globally scoped

Asset Tree Templates: Classes

class seeq.spy.assets.Asset(context: BuildContext, definition: dict | DataFrame | Series = None, *, parent=None)

Bases: _AssetBase

A class derived from Asset can have @Asset.Attribute and @Asset.Component decorated functions that are executed as part of the call to build() which returns a list of definition dicts for the asset.

classmethod Attribute(): This decorator appears as @Asset.Attribute on a function with a class that derives from Asset.

classmethod Component(): This decorator appears as @Asset.Component on a function with a class that derives from Asset.

all_assets()

All asset instances in the entire tree.

Note that, during the INSTANTIATING build phase (the first), this function returns a blank list because not all asset objects have been instantiated yet.

Returns:: A list of all asset instances in the entire tree.

build_component(template, metadata, component_name) → List[dict]

Builds a single component by instantiating the supplied template and building it with the supplied metadata.

Parameters:

template ({Asset}) – An asset class that is used to construct the attributes and sub- components.
metadata (pd.DataFrame) – The metadata DataFrame containing all rows relevant to the sub-component.
component_name (str) – The name of the sub-component. This will be used as the Name property of the instantiated asset (unless template is a Mixin.)

Returns:

A list of definitions (comprising one item). Note that this list will be empty in the first build phase (where self.context.phase equals BuildPhase.INSTANTIATING), as only the asset objects will have been instantiated.

Return type:

list(dict)

build_components(template, metadata: DataFrame, column_name: str) → List[dict]

Builds a set of components by identifying the unique values in the column specified by column_name and then instantiating the supplied template for each one and building it with the subset of metadata for that column value.

Useful when constructing a rich model whereby a root asset is composed of unique components, possibly with further sub-components. For example, you may have a Plant asset that contains eight Refigerator units that each have two associated Compressor units.

Parameters:

template ({Asset}) – An asset class that is used to construct the attributes and sub- components.
metadata (pd.DataFrame) – The metadata DataFrame containing all rows relevant to all (sub-)components of this asset. The DataFrame must contain the column specified by column_name.
column_name (str) – The name of the column that will be used to discover the unique (sub-)components of this asset. For example, if column_name= ‘Compressor’, then there might be values of ‘Compressor A12’ and ‘Compressor B74’ in the ‘Compressor’ column of the metadata DataFrame.

Returns:

A list of definitions for each component. Note that this list will be empty in the first build phase (where self.context.phase equals BuildPhase.INSTANTIATING), as only the asset objects will have been instantiated.

Return type:

list(dict)

Examples

Define a Refrigerator template that has Compressor subcomponents.

>>> class Refrigerator(Asset):
>>>     @Asset.Attribute()
>>>     def Temperature(self, metadata):
>>>         return metadata[metadata['Name'].str.endswith('Temperature')]
>>>
>>>     @Asset.Component()
>>>     def Compressor(self, metadata):
>>>         return self.build_components(Compressor, metadata, 'Compressor')
>>>
>>> class Compressor(Asset):
>>>
>>>     @Asset.Attribute()
>>>     def Power(self, metadata):
>>>         return metadata[metadata['Name'].str.endswith('Power')]

property fqn: The Fully-Qualified Name of this object, which includes both the Path and the Name using the usual >> separators.

is_ancestor_of(asset): Tests if this asset instance (self) is a parent or grandparent etc of the specified asset (looking all the way up the tree). :param asset: The asset that might be above self in the hierarchy :return: True if self is above asset

is_child_of(asset): Tests if this asset instance (self) is a direct child of the specified asset. :param asset: The asset that might be self’s parent :return: True if self is a direct child of asset

is_descendant_of(asset): Tests if this asset instance (self) is a child or grandchild etc of the specified asset (looking all the way down the tree). :param asset: The asset that might be below self in the hierarchy :return: True if self is below asset

is_parent_of(asset): Tests if this asset instance (self) is a direct parent of the specified asset. :param asset: The asset that might be self’s child :return: True if self is a direct parent of asset

property parent: The parent Asset object of this Asset or Mixin, if it exists.

class seeq.spy.assets.ItemGroup(_list=None)

Bases: list

Represents a list of asset tree items that can be rolled up into a calculation

pick(criteria: dict) → ItemGroup

Filters items based on given criteria and returns them in a new ItemGroup

Parameters:: criteria (dict) – An dictionary mapping property names to property values

Examples

Return an ItemGroup composed of all items in item_group that have type Signal:

>>> item_group.pick({'Type': 'Signal'})

roll_up(statistic)

Returns a definition for a roll-up calculation that takes all items in this group as inputs.

Examples

Calculate the average of all items in item_group:

>>> item_group.roll_up('Average')

class seeq.spy.assets.Mixin(context, definition, parent)

Bases: _AssetBase

A Mixin is nearly identical to an Asset, but it adds attributes/components to an otherwise-defined Asset. The definition argument that is passed into the constructor should be the definition of the otherwise-defined Asset.

This allows asset tree designers to add a set of “special” attributes to a particular instance of an asset without “polluting” the main asset definition with attributes that most of the instances shouldn’t have.

Asset Tree Templates: Functions

seeq.spy.assets.build(model, metadata, *, workbooks=None, errors=None, quiet=None, status: Status = None)

Utilizes a Python Class-based asset model specification to produce a set of item definitions as a metadata DataFrame.

Parameters:

model ({ModuleType, type, List[type]}) – A Python module, a spy.assets.Asset or list of spy.asset.Assets to use as the model for the asset tree to be produced. Follow the spy.assets.ipynb Tutorial to understand the structure of your module/classes.
metadata ({pd.DataFrame}) – The metadata DataFrame, usually produced from calls to spy.search(), that will be used as the “ingredients” for the asset tree and passed into all Asset.Attribute() and Asset.Component() decorated class functions.
workbooks ({list}) – The set of workbooks (usually a single AnalysisTemplate object and one or more TopicTemplate objects) to be used in @Display and @Document functions.
errors ({'raise', 'catalog'}, default 'catalog') – If ‘raise’, any errors encountered will cause an exception. If ‘catalog’, errors will be added to a ‘Result’ column in the status.df DataFrame.
quiet (bool, default False) – If True, suppresses progress output.
status (spy.Status, optional) – If specified, the supplied Status object will be updated as the command progresses. It gets filled in with the same information you would see in Jupyter in the blue/green/red table below your code while the command is executed. The table itself is accessible as a DataFrame via the status.df property.

Returns:

A DataFrame with the metadata for the items that were built and are meant to be pushed, along with any errors in the Build Result column about the operation.

Additionally, the following properties are stored on the “spy” attribute of the output DataFrame:

Property	Description
func	A str value of ‘spy.assets.build’
kwargs	A dict with the values of the input parameters passed to spy.assets.build to get the output DataFrame
context	An object with the intermediate outputs of the build process, sometimes useful for debugging
status	A spy.Status object with the status of the spy.assets.build call

Return type:

pandas.DataFrame

seeq.spy.assets.brochure(model: module | Type[_AssetBase] | List[Type[_AssetBase]], output: str = 'dict') → dict | str

Generates a description of a template’s requirements and attributes for the purposes of advertising what the template offers and how it is useful.

Docstrings for each requirement/attribute/component/etc are included in the output.

Parameters:

model ({ModuleType, type, List[type]}) – A Python module, a spy.assets.Asset or list of spy.asset.Assets representing the model to inspect to produce the brochure.
output (str, default 'dict') –
The desired format of the output. Can be one of the following:
- ’dict’ - A dictionary of the template and all of its members, including the documentation in Markdown format.
- ’json’ - The same as ‘dict’ but converted to a JSON string.
- ’html’ - An HTML document (in string form) where the documentation has been converted from Markdown to HTML.

Returns:

A dictionary or string according to the specified “output” argument.

Return type:

{dict, str}

seeq.spy.assets.prepare(metadata, *, root_asset_name=None, old_asset_format=None)

Modifies (in place) a metadata DataFrame that represents an existing tree, usually obtained by doing spy.search(recursive=True), and creates “Build Path” and “Build Asset” columns that are suitable for typical use of spy.assets.build().

Parameters:

metadata (pd.DataFrame) – The metadata DataFrame, usually produced from calls to spy.search(recursive=True) on an existing tree, that will be used as the “ingredients” for spy.assets.build().
root_asset_name (str, optional) – An optional new name to use as the root of the tree. You will typically want to specify this so that your new tree is differentiated from the existing tree.
old_asset_format (bool, optional) –

Path	Name
Root >> Next Level	Item_Name

Level 1	Level 2	Name
Root	Next Level	Item_Name