Managing ScienceLogic Libraries

This chapter describes how to manage ScienceLogic Libraries in Skylar One (formerly SL1).

Viewing the List of ScienceLogic Libraries

The ScienceLogic Library Manager page (System > Customize > ScienceLogic Libraries) displays a list of all available ScienceLogic Libraries. From this page, you can download libraries, import new libraries, delete existing libraries, or view and edit a list of Execution Environments that you can associate with a library.

For each library, the page displays the following information:

To sort the list of ScienceLogic Libraries, click on a column heading. The list will be sorted by the column value, in ascending order. To sort by descending order, click the column heading again. The Last Edited column sorts by descending order on the first click; to sort by ascending order, click the column heading again.

• Library Name. The name of the ScienceLogic Library.
• Version. The version number of the ScienceLogic Library.
• System. Displays True if the ScienceLogic Library is included in the latest release of Skylar One and is aligned with the "System" environment. Displays False if the ScienceLogic Library is included in an imported PowerPack or manually installed by an administrator.
• Type. The file type used to bundle and deliver the ScienceLogic Library. Supported types include the following:

• py_package (.tar format). A py_package is a tar file that contains Python packages and metadata files (name, version, description, etc.). The py_package format is typically used by more experienced developers to build complex solutions. ScienceLogic recommends using the py_package format because it is easier to test and maintain.
• py_directory (.tar format). A py_directory is a tar file that includes Python files and is typically used by novice developers to share short code snippets between Dynamic Applications or Skylar One stacks. This format does not let you manage items such as dependencies, version, or test, and is not recommended.

Older versions of Skylar One bundled ScienceLogic Libraries in a py_directory format. That format has since been deprecated. Current ScienceLogic Libraries should be bundled only in a py_package format.

• Dependencies. A comma-separated list of other ScienceLogic Libraries on which the selected library is dependent, if applicable. These additional libraries are included in the selected library's py_package.tar file.
• Description. A single-line description of the ScienceLogic Library's intended use.
• Python Version. The Python compatibility version.
• Edited By. The name of the ScienceLogic user who created or last edited the ScienceLogic Library.
• Last Edited. The date and time the ScienceLogic Library was created or last edited.

Downloading a ScienceLogic Library

From the ScienceLogic Library Manager page, you can download ScienceLogic Libraries. Libraries that are included in Skylar One are bundled in a py_package format.

ScienceLogic-authored libraries are copyrighted content and should not be modified without the express permission of ScienceLogic.

Older versions of Skylar One bundled ScienceLogic Libraries in a py_directory format. That format has since been deprecated. Current ScienceLogic Libraries should be bundled only in a py_package format.

To download a ScienceLogic Library:

1. Go to the ScienceLogic Library Manager page (System > Customize > ScienceLogic Libraries).
2. Click the save icon () of the ScienceLogic Library you want to download.
3. The library file is downloaded to your computer.

Importing a ScienceLogic Library

In addition to the ScienceLogic Libraries that are included in the latest version of Skylar One or included in a PowerPack, you can use the ScienceLogic Library Manager page to import additional libraries.

To import a library:

1. Go to the ScienceLogic Library Manager page (System > Customize > ScienceLogic Libraries).
2. Click the Actions menu and then select Import ScienceLogic Library.
3. The Import ScienceLogic Library dialog box appears. Use the Browse button to navigate to the ScienceLogic Library file you want to import and then click the Import button.
4. The imported ScienceLogic Library appears on the ScienceLogic Library Manager page.

Creating a ScienceLogic Library

You can create your own library to import into Skylar One in a py_package library file saved in uncompressed .tar format. This section describes the requirements for a ScienceLogic-compatible library.

Important Notes on Creating ScienceLogic Libraries

Older versions of Skylar One bundled ScienceLogic Libraries in a py_directory format. That format has since been deprecated. Current ScienceLogic Libraries should be bundled only in a py_package format.

The following are important items to note when creating a ScienceLogic library:

• Library files must be saved in uncompressed .tar format.
• For best results, use GNU tar rather than bsdtar.
• You can only include wheel files (.whl) as additional dependencies. You cannot include any additional packages in your library file that require compilation or linking because Python build tools are not included in the Skylar One platform.
• Any libraries that are bundled as dependencies in your py_package file will not be used if a newer version of that library exists on the appliance unless you have a restriction in your package specifying a given version to use.
• For example, if your library "my-library" requires the Python community package "requests" version 2.25.1, you would create the library "requests" version "2.25.1" first. You would then import the requests library into Skylar One and align it and "my-library" to your Execution Environment. At runtime, both libraries, at those specific versions, will be used.
• For an alternative example, if your library "my-library" requires the Python community package "requests", but the version is not important, you would include the "requests" package as a ".whl" file in your "my-library" library, adding it to the "manifest.txt" file. After aligning "my-library" to your Execution Environment, it will be available at runtime. The "requests" package included in the "manifest.txt" will be available during the install of "my-library", but may not be used if a newer acceptable package is found for install.
• For a py_package library, an optional "metadata.json" file can be included but it must include the requires_python element, otherwise an error will be displayed upon import.

Library File Type: py_package

The py_package file type is the preferred library file type used to bundle Python packages and metadata files. A py_package must have the metadata files described below along with the .whl file(s), and it must be saved in uncompressed .tar format. You must create this tar file with Red Hat Enterprise Linux or a similar operating system to ensure compatibility. Tar files created on MacOS are not supported.

All metadata files included in the package must be at the root level of the directory.

Alternately, you can include a "metadata.json" file instead of the following .txt files, but the "metadata.json" file must include the requires_python element, otherwise an error will be displayed upon import.

A py_package tar file must include the following files:

• __init__.py. This file is required for the directory to be used as a Python package. It identifies this as a package.
• name.txt. This text file specifies the name of the library and must match the name of the package being installed.
• version.txt. This text file specifies the version of the library and must match the version of the package being installed.
• type.txt. This text file specifies that the library is a py_package type library.
• manifest.txt. This text file includes the Python package name and version (as specified in the name.txt and version.txt files), followed by a list of additional supporting files that must be deployed to the internal package repository to use the library. For example, if the library is dependent on another package, that other package must be listed in the manifest file.

The ScienceLogic library and the Python package must have the same name, and this name must be the first entry in the manifest file.

• Wheel (.whl) file. This is a standard wheel file that you create. See Python documentation for more information about creating wheel files. Wheel files must be compatible with the version of Python used in the library. Wheel files must be able to work in Red Hat Enterprise Linux or a similar operating system, but they do not need to be created in Red Hat.

ScienceLogic supports Python 3-compatible libraries and wheel files. Legacy Python 2 support is being deprecated, and ScienceLogic strongly encourages users to use Python 3 when creating libraries. However, if you are creating a Python 2 library, then its corresponding wheel file must also be Python 2-compatible.

Manually Building a ScienceLogic Library

To manually build a ScienceLogic library, follow this example:

1. Use the sample python library called silo_demo-0.0.7-py2.py3-none-any.whl:

   • Create a project folder for your library, for example, myproject.
   • Copy your project's python wheel file into your new project folder.

mkdir myproject

cp silo_demo-0.0.7-py2.py3-none-any.whl myproject/

2. Download a complete set of python dependencies for your python package. In this example case, there is a dependency on requests. To download the dependency package, you need to change the project directory to download the package and any transitive dependencies using the Python binary path in your execution environment.

   Any non-pure Python dependencies included in your ScienceLogic Library must be built distributions (wheels) that do not require a build stage. Skylar One does not include compilation tools for source distributions.

cd myproject

python3 -m pip download --find-links=. silo_demo==0.0.7

To build these packages, ScienceLogic recommends that you are on Oracle Linux or Redhat based x86-64 installation.

Execute the python3 -m pip download command with a python3.6 interpreter to operate correctly with version 11.3 or later of Skylar One. Some Linux distributions default the command python to python2.7.

• The corresponding request python package and all its dependencies will download into your project folder.

Looking in links: .

Processing ./silo_demo-0.0.7-py2.py3-none-any.whl

File was already downloaded /ScienceLogic/myproject/silo_demo-0.0.7-py2.py3-none-any.whl

Collecting requests

Using cached requests-2.29.0-py3-none-any.whl (62 kB)

Collecting certifi>=2017.4.17

Using cached certifi-2022.12.7-py3-none-any.whl (155 kB)

Collecting urllib3<1.27,>=1.21.1

Using cached urllib3-1.26.15-py2.py3-none-any.whl (140 kB)

Collecting charset-normalizer<4,>=2

Using cached charset_normalizer-3.1.0-cp39-cp39-macosx_10_9_x86_64.whl (124 kB)

Collecting idna<4,>=2.5

Using cached idna-3.4-py3-none-any.whl (61 kB)

Saved ./requests-2.29.0-py3-none-any.whl

Saved ./certifi-2022.12.7-py3-none-any.whl

Saved ./charset_normalizer-3.1.0-cp39-cp39-macosx_10_9_x86_64.whl

Saved ./idna-3.4-py3-none-any.whl

Saved ./urllib3-1.26.15-py2.py3-none-any.whl

Successfully downloaded silo_demo requests certifi charset-normalizer idna urllib3

• List the dependencies to ensure they are correct.

 

[myproject]$ ls -l

-rw-r--r--@ 1 dev staff 4524 May 2 11:31 silo_demo-0.0.7-py2.py3-none-any.whl

-rw-r--r-- 1 dev staff 62499 May 3 16:31 requests-2.29.0-py3-none-any.whl

-rw-r--r-- 1 dev staff 155255 May 3 16:31 certifi-2022.12.7-py3-none-any.whl

-rw-r--r-- 1 dev staff 124711 May 3 16:31 charset_normalizer-2.0.4-cp36-cp36mu-manylinux1_x86_64.whl

-rw-r--r-- 1 dev staff 61538 May 3 16:31 idna-3.4-py3-none-any.whl

-rw-r--r-- 1 dev staff 140881 May 3 16:31 urllib3-1.26.15-py2.py3-none-any.whl

Skylar One comes pre-installed with the proper versions of wheel, setuptools, and pip wheels for each release. Do not include these in your ScienceLogic library as it will cause a deployment failure.

3. Create the metadata.json file. Ensure that you are in your project folder and use your favorite editor tool to create the metadata.json file.

   The following example includes the minimum set of metadata required by Skylar One. The information supplied corresponds to the principle python library you set up in step 1. The environment deployer will pip install the principle python library based on the name and version metadata, so they must be accurate.

   Update the following fields with your information:

• meta_version. Leave this as default "0.2"
• description. A precise description of the ScienceLogic library
• manifest. Contains the complete list of wheel files from Step 2
• name. The precise name of the principle python package used by pip to install the python package. In this example, the name is silo_demo.
• type. Leave this as default "py_package"
• version. The precise version of the principle python package used by pip to install the python package. In this example, the version is 0.0.7.
• requires_python The specific versions of python you will use. For python2, use "~=2.7" and for python3, use "~=3.6". You should follow PEP440 version specifiers.

For more information about the PEP440 version specifiers, see the PEP440 webpage.

{

"meta_version": "0.2",

"fields": {

"description": "A demo ScienceLogic library",

"manifest": ["certifi-2022.12.7-py3-none-any.whl", "charset_normalizer-2.0.4-cp36-cp36mu-manylinux1_x86_64.whl", "idna-3.4-py3-none-any.whl", "requests-2.29.0-py3-none-any.whl", "silo_demo-0.0.7-py2.py3-none-any.whl", "urllib3-1.26.15-py2.py3-none-any.whl"],

"name": "silo_demo",

"type": "py_package",

"version": "0.0.7",

"requires_python": "~=3.6"

}

}

4. Validate the metadata.json file. Validating your JSON files ensures there aren't any errors in the pairing of braces and syntax. To validate your metadata.json file type:

   python -m json.tool metadata.json

5. Create the ScienceLogic library archive.

Give your archive a unique name so that it is easy to recognize and distinguish from other library archives. Since it is possible to support multiple python runtime versions, ScienceLogic recommends that you include the python version(s) supported in the library archive you create.

Use the following naming convention as a guide:

<lib name> + <lib version>. <runtime version/s>.tar

The following commands must be run from inside the directory you created in step 1. For the following examples, that is the myproject directory.

For example:

• Create the .tar archive:

tar cvf silo_demo+0.0.7.py36.tar *

• Confirm the creation of the .tar archive by listing List the .tar files in the directory:

ls -l *.tar

-rw-r--r-- 1 em7admin em7admin 480256 Nov 29 16:28 silo_demo+0.0.7.py36.tar

• Check the contents of the .tar archive:

tar tvf silo_demo+0.0.7.py36.tar

-rw-r--r-- 1 em7admin em7admin 161120 Nov 29 16:12 certifi-2022.9.24-py3-none-any.whl

-rw-r--r-- 1 em7admin em7admin 39748 Nov 29 16:12 charset_normalizer-2.0.4-cp36-cp36mu-manylinux1_x86_64.whl

-rw-r--r-- 1 em7admin em7admin 61538 Nov 29 16:12 idna-3.4-py3-none-any.whl

-rw-r--r-- 1 em7admin em7admin 479 Nov 29 16:25 metadata.json

-rw-r--r-- 1 em7admin em7admin 62843 Nov 29 16:12 requests-2.28.1-py3-none-any.whl

-rw-r--r-- 1 em7admin em7admin 4524 Nov 29 16:12 silo_demo-0.0.7-py2.py3-none-any.whl

-rw-r--r-- 1 em7admin em7admin 140572 Nov 29 16:12 urllib3-1.26.13-py2.py3-none-any.whl

CAUTION: Depending on your development environment, your .tar archiving tool may generate an archive that is incompatible with the Linux distribution used by Skylar One. If this occurs, try extracting ( tar xvf myarchive.tar ) or just listing the archive ( tar tvf my archive.tar ) on your Skylar One appliance before archiving to ensure compatibility with the following steps.

CAUTION: Only the contents of the directory you created in step 1 should be listed when you execute the previous command. If the directory itself is included in the output, then the .tar archive will not work correctly when imported into Skylar One. If this occurs, see the following knowledge base article for more information, including steps to resolve the issue: https://support.sciencelogic.com/s/article/14578.

6. Import the ScienceLogic library. For instructions, see Importing a ScienceLogic Library.

7. Create a Skylar One Execution Environment.

• To create your Python3 Execution Environment, go to the Environment Manager page (System > Customize > ScienceLogic Libraries > Actions > Execution Environments) and click New.
• In the Environment Name field, select python3.
• For this example, add the Environment Name Snippet Framework Demo py36 and click Save. The compatible libraries are displayed and available to the python3 execution environment.
• Click the lightning bolt () to include all of the libraries in this environment.

The string found in requires_python will follow the same convention used for python packages.

A ScienceLogic library with requires_python>=3.6 would be compatible with any execution environment greater than or equal to 3.6.

• Make note of the environment GUID for the following steps. In this example, the environment GUID was:

0117E1C041D8CE0C69A8391C02E6DC3B

8. Align your ScienceLogic library to the Skylar One Execution Environment.

• Return to the Environment Manager page (System > Customize > ScienceLogic Libraries > Actions > Execution Environments). You should see that your ScienceLogic library is available for alignment to your execution environment.
• Click the lightning bolt () to add your library to the environment.

9. Test your ScienceLogic library.

• Log on to the collector and deploy the ScienceLogic Execution Environment.

sudo -u s-em7-core /opt/em7/bin/python3 /opt/em7/lib/python3/sl_envs/env_manager.pyc --debug 0117E1C041D8CE0C69A8391C02E6DC3B

• Activate your new environment.

source /opt/em7/envs/0117E1C041D8CE0C69A8391C02E6DC3B/bin/activate

• From the python command line, try importing your new package and executing its API.

[uenv-3869008638788123352]$ python

Python 3.6.8 (default, Nov 18 2021, 10:07:16)

[GCC 4.8.5 20150623 (Red Hat 4.8.5-44.0.3)] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> from silo.demo import demo_class

>>>

ScienceLogic is working to minimize the development effort through the use of Snippet Framework and low code specifications. Unless your collection environment is unique, you may not need to develop content such as manually building your ScienceLogic library.

Converting a py_directory ScienceLogic Library

Older versions of Skylar One bundled ScienceLogic Libraries in a Python 2-compatible py_directory format. That format has since been deprecated. Current ScienceLogic Libraries should be bundled only in a Python 3-compatible py_package format.

To ensure your ScienceLogic Libraries continue operating as intended, you can convert Python 2 py_directory ScienceLogic Libraries to a wheel for use in Python 3-compatible py_package ScienceLogic Libraries.

Typically, a py_directory package consists of a few Python modules in a project folder that is only one level deep and consists only of Python content with no C extensions.

To perform this conversion, follow the example in this section, which uses a ScienceLogic Library called clevon. The original contents of this library looked like this: 

tar tvf clevon_v_1.4_~=2.7.tar

drwxr-xr-x  0 user staff       0 Oct 18  2023 clevon/
-rw-r--r--  0 user staff   11341 Oct 18  2023 clevon/clevon_globals.py
-rw-r--r--  0 user staff    1017 Oct 18  2023 clevon/misc.py
-rw-r--r--  0 user staff    1425 Oct 18  2023 clevon/interface.py
-rw-r--r--  0 user staff       0 Oct 18  2023 clevon/__init__.py
-rw-r--r--  0 user staff    1975 Oct 18  2023 clevon/graphing.py
-rw-r--r--  0 user staff    2430 Oct 18  2023 clevon/have_children.py
-rw-r--r--  0 user staff    2399 Oct 18  2023 clevon/math_lesson.py
-rw-r--r--  0 user staff       4 Oct 17  2023 clevon/version.txt
-rw-r--r--  0 user staff      33 Oct 18  2023 clevon/description.txt
-rw-r--r--  0 user staff     463 Oct 18  2023 clevon/chores.py

1. To convert this format, you must create a new project folder (project_folder, for this example) with the following structure:

   LICENSE.txt
   README.md
   pyproject.toml
   src/
     clevon/
       clevon_globals.py
       misc.py
       interface.py
       __init__.py
       graphing.py
       have_children.py
       math_lesson.py
       chores.py

   The LICENSE.txt and README.md files can be empty. Also, in this example, the metadata version.txt and description.txt files were removed from the original py_directory contents.

2. The contents of the pyproject.toml file in the above example should look similar to this:

   [build-system]
   requires = ["setuptools"]
   build-backend = "setuptools.build_meta"
   
   [project]
   name = "clevon"
   version = "1.6"
   authors = [
     { name="ScienceLogic, Inc.", email="author@sciencelogic.com" },
   ]
   description = "A Crucible test package"
   readme = "README.md"
   requires-python = ">=3.6"
   classifiers = [
       "Programming Language :: Python :: 3",
       "License :: OSI Approved :: MIT License",
       "Operating System :: OS Independent",
   ]

3. Run the builder to package the wheel:

   cd project_folder

   python3 -m build

   When the build finishes successfully, you should receive a message similar to this:

   Successfully built clevon-1.6.tar.gz and clevon-1.6-py3-none-any.whl

   If the build does not work, you might need to first install the build package into your environment:

   python3 -m pip install build

4. As part of the previous step, a dist directory should have been created that contains the built wheel. Confirm that the artifacts have been created:

   ls -l dist
   total 48
   -rw-r--r--  1 dayn.harum  staff  9764 Sep  9 09:25 clevon-1.6-py3-none-any.whl
   -rw-r--r--  1 dayn.harum  staff  8618 Sep  9 09:25 clevon-1.6.tar.gz

5. Confirm that you can install, import, and uninstall the package:

   cd dist
   dist % python3 -m pip install --no-index --find-links=. clevon==1.6
   Looking in links: .
   Processing ./clevon-1.6-py3-none-any.whl
   Installing collected packages: clevon
   Successfully installed clevon-1.6
   
   dist % python3
   Python 3.9.5 (default, May 18 2021, 12:31:01)
   [Clang 10.0.0 ] :: Anaconda, Inc. on darwin
   Type "help", "copyright", "credits" or "license" for more information.
   >>> import clevon
   >>> clevon.__path__
   ['/Users/user/miniconda3/lib/python3.9/site-packages/clevon']
   >>> from clevon import graphing
   >>>
   
   python3 -m pip uninstall clevon==1.6
   Found existing installation: clevon 1.6
   Uninstalling clevon-1.6:
     Would remove:
       /Users/user/miniconda3/lib/python3.9/site-packages/clevon-1.6.dist-info/*
       /Users/user/miniconda3/lib/python3.9/site-packages/clevon/*
   Proceed (Y/n)? Y
     Successfully uninstalled clevon-1.6

6. After you have successfully built the wheel, you can follow the steps in the section Manually Building a ScienceLogic Library to include it in a ScienceLogic Library.

Deleting a ScienceLogic Library

The ScienceLogic Library Manager page allows you to delete ScienceLogic Libraries that are not currently in use.

Skylar One will not allow you to delete ScienceLogic Libraries that are aligned to execution environments that are being used by any Dynamic Applications, credential tests, or Run Book Actions. You also cannot delete libraries that are aligned to the "System" environment.

To delete a library:

1. Go to the ScienceLogic Library Manager page (System > Customize > ScienceLogic Libraries).
2. Select the ScienceLogic Library that you want to delete.
3. Click the Select Actions menu in the lower right of the page and select DELETE ScienceLogic Library, and then click the Go button.
4. The selected ScienceLogic Library is deleted from the ScienceLogic Library Manager page.