import os
import shutil
from seeq import spy

# Set the compatibility option so that you maximize the chance that SPy will remain compatible with your notebook/script
spy.options.compatibility = 189

# Log into Seeq Server if you're not using Seeq Data Lab:
spy.login(url='http://localhost:34216', credentials_file='../credentials.key', force=False)

spy.workbooks

The spy.workbooks module provides functions for importing and exporting workbooks. A workbook is either a Workbench Analysis (colored green in the Seeq user interface) or an Organizer Topic (colored blue).

This functionality is helpful to:

  • Move content between two Seeq servers

  • Manage content by exporting and committing to a version control system like Git

The process typically starts by searching for some content that you have created in Seeq and exporting it. However, since this documentation needs some pre-built content to illustrate how it works, there is a pre-built-and-exported set of workbooks alongside this documentation notebook. So we’ll go in a non-typical order of operations in this example.

The Export Format

When content is exported from Seeq, each workbook is encapsulated in its own folder, including all its worksheets, calculated item definitions and all dependencies, journal/document images and anything else that is necessary to import that data into another server. Content is written to disk as either JSON, HTML or image files as appropriate. References to datasource items are also catalogued during export and default datasource maps are created that facilitate identification of equivalent signals/conditions/scalars on the destination system so that the imported content gets “hooked up” to the right data.

Main Actions

There are five main operations you can perform on workbooks:

  • search for workbooks whose content you want to pull

  • pull those workbooks into Workbook in-memory Python objects

  • save the Workbook Python objects to disk in the export format described above

  • load Workbook Python objects from disk into memory

  • push in-memory Workbook Python objects into a Seeq Server

As mentioned, we’re going to go out-of-order for illustration purposes: load, push, search, pull, save.

Importing

This set of documentation comes with an Example Export folder that contains an Analysis and a Topic for illustration purposes. First we load it into memory:

workbooks = spy.workbooks.load('Support Files/Example Export.zip')
workbooks

Now that the workbook definitions are in memory, we can push them into Seeq.

spy.workbooks.push(workbooks, path='SPy Documentation Examples >> My Import', errors='raise')

The workbooks have been imported into Seeq in a My Import folder with you as the owner. Refresh Seeq Workbench in your browser and take a look.

Exporting

In Seeq Workbench, try changing the name of the Example Analysis workbook to something like My First Analysis Export so that you can tell that your changes get exported.

Now we will search for the workbooks we want to export. The syntax for a workbook search query is very similar to an item metadata search via spy.search():

workbooks_df = spy.workbooks.search({
    'Path': 'SPy Documentation Examples >> My Import'
})

workbooks_df

As you can see, the spy.workbooks.search() command returns a metadata DataFrame with the properties of the workbooks. We can now use that to pull:

workbooks = spy.workbooks.pull(workbooks_df)
workbooks

These are the same type of in-memory Python objects that we had when we executed spy.workbooks.load(). Now we can save them to disk:

if os.path.exists('../My First Export'):
    shutil.rmtree('../My First Export')

spy.workbooks.save(workbooks, '../My First Export')

In the parent folder of this documentation notebook, you’ll find a new My First Export folder that contains similar files to the Example Export folder that’s part of the documentation.

Inspecting Worksheets

With the in-memory Python objects that result from spy.workbooks.pull() or spy.workbooks.load(), you can inspect the worksheets to see what is displayed on them. For example, let’s look at what’s in the Details Pane of the second worksheet of Example Analysis:

worksheet_items = [w for w in workbooks if w.name == 'Example Analysis'][0].worksheets[1].display_items
worksheet_items

Now you can call spy.pull() to pull data for the items in the worksheet.

spy.pull(worksheet_items, start='2019-01-01T00:00:00', end='2019-01-02T00:00:00')

Note that if you just wanted the full metadata for the items, you could execute spy.search(worksheet_items[['ID']]).

Re-importing and Labels

If you push a set of workbooks more than once, then by default you will simply overwrite the existing workbooks with the saved content. This can be useful when you are “backing up” content to disk, perhaps for the purposes of version control.

You can choose to push and supply a label, which will create a separate copy of all of the imported items instead of modifying the existing ones. This is useful when you want to import something that you are iterating on prior to affecting the “published” version. For example, let’s push our workbooks with the label of In Development:

spy.workbooks.push(workbooks, path='SPy Documentation Examples >> My Development Folder', label='In Development')

If you refresh Seeq Workbench, you’ll notice that there is now a My Development Folder and a separate copy of the Topic and Analysis that is independent of the original – including all calculated items.

Pushing with the same value for the label argument will overwrite the content for that label. Change the label again if you want yet another separate copy.

Importing to a Different Seeq Server

You may wish to copy content to a new/different Seeq Server by exporting and then importing. For example, you might have a development server where you iterate on content and a production server that you publish to when finished.

In order to accomplish this, you’ll do one of two actions:

  • If you’re using the SPy module within Seeq Data Lab, you’ll copy the exported folder to the other version of Seeq Data Lab and then push it from there.

  • If you’re using the SPy module with your own Python set up, you’ll log in to the other server and push it.

Detailed Help

All SPy functions have detailed documentation to help you use them. Just execute help(spy.<func>) like you see below.

help(spy.workbooks.search)

help(spy.workbooks.pull)

help(spy.workbooks.save)

help(spy.workbooks.load)

help(spy.workbooks.push)