***********
Performance
***********
The Snippet Framework was written with performance in mind. However
there are things you can do as a Snippet Argument developer to
help improve the performance of your Dynamic Application. Below
are some scenarios how you can instruct the Snippet Framework
to be more performant.

Utilizing De-Duplication
------------------------
The Snippet Framework performs de-duplication between shared
:ref:`step paths <glossary_step_path>`. Collections are grouped
based on these paths which removed performing the same operation
which increases the overall performance of
the Snippet Framework. When the step paths diverge,
the data is cloned into the divergent collections. This enables
the Snippet Framework to have performance closer to writing
a pythonic snippet.

In this example, we will use a small data set but this concept
applies to large data sets too. With the provided data set, we want
to extract out ``1_1`` and ``1_2`` but allow the framework to reduce the
amount of duplicated calls to ``key1``.

.. code-block:: json

 {
   "key1": {
     "1_1": 11,
     "1_2": 12
   },
   "key2": {
     "2_1": 21
   }
 }

The most performant way to write this utilizing JMESPath would
be to reduce the amount of times you iterate through ``key1``
by having that its own step. This reduces running the lookups
for ``key1`` for each collection which increases the overall
performance and reduces the memory footprint while executing.

.. code-block:: yaml

  low_code:
    version: 2
    steps:
        - gather_data
        - json
        - jmespath:
            value: "'key1'"
        - jmespath:
            value: "'1_1'"

.. code-block:: yaml

  low_code:
    version: 2
    steps:
        - gather_data
        - json
        - jmespath:
            value: "'key1'"
        - jmespath:
            value: "'1_2'"

As we can see while using this, the value nested under ``key1``
is only accessed once. If this was a large data set or used a complex
JMESPath expression, this would result in a performance boost
as it will not re-run the same JMESPath expression.

.. graphviz::  ../_static/graph_generator_dedup.dot

Large Data Sets
---------------
The Snippet Framework can process large data sets without
issues. However as a Snippet Argument developer you can use
more performant steps or arguments. For example, if you are using
a parser to convert text you should think about the search and memory
implications. Refer to `Python Time Complexity <https://wiki.python.org/moin/TimeComplexity>`_
when considering result data types.

Caching Across Dynamic Applications
-----------------------------------
The Snippet Framework can cache information between Dynamic Applications
through the use of a :ref:`Cacher <glossary_cacher>` step. This
step enables data to be saved and referenced at a later time which
can increase performance across Dynamic Applications by reusing
previously computed values.

A cache step acts as both a cache writer and reader within your Snippet
Arguments. If a valid cache entry is not found, it will run all the steps
preceding the cache step and write the result to cache. If a valid cache
entry is found, it starts processing at the step following the cache step
using the cache entry as the result.

When using a cache step, determine what steps will be required between
Dynamic Applications and select where in the Snippet Argument to cache the
data. A Snippet Argument can use multiple cacher steps throughout
the execution plan.

This functionality should be used when a network request's result must be
transformed, and the transformation time is costly. For example, if an HTTP
request's result is a large json payload and the steps to transform the data
takes about 2 minutes. Let's say the transformed data is needed for 5 Dynamic
Applications then without caching, each Dynamic Application would have to spend
2 minutes running through the transformation steps. By using caching, one
Dynamic Application will run the 2 minute transformation and the other
Dynamic Applications will reuse the newly cached data.

.. note::

   Examples of caching across Dynamic Applications can be found in
   the following Dynamic Applications that ship with the PowerPack:

    * Caching: Auto-Key Example 1
    * Caching: Auto-Key Example 2
    * Caching: User-Defined Key Example 1
    * Caching: User-Defined Key Example 2