Sysrepo performance

Sysrepo python2 performance is tested with the python file //TODO (add link after pull request). It runs multiple times each tested operations using data files of a different size. Then it prints the statistics.

The output of measurement consists of tables for three different data sizes.

                Data file 1 list instance in datastore startup 
Operation                               |    ops/sec |   items/op | ops performed |  items/sec |  test time.
---------------------------------------------------------------------------------------------------
                       Get item one leaf|      35304 |          1 |          5000 |      35304 |       0.14

                     Get item first leaf|      44664 |          1 |          5000 |      44664 |       0.11

      Get item (including session start)|       9246 |          1 |          5000 |       9246 |       0.54

                 Get all items of a list|      45372 |          1 |          5000 |      45372 |       0.11

                 Get items iter all list|      20784 |       5000 |          5000 |  103921681 |       0.24

             Get subtrees ietf-if config|      13480 |      45000 |          5000 |  606614716 |       0.37

                    Get subtree one leaf|      34398 |          1 |          5000 |      34398 |       0.15

   Get subtree (including session start)|       9694 |          1 |          5000 |       9694 |       0.52

                    Get subtree all leaf|      37827 |          1 |          5000 |      37827 |       0.13

             Get subtrees ietf-if config|      13838 |      50000 |          5000 |  691899653 |       0.36

                   Set & delete one list|      15824 |          6 |          5000 |      94946 |       0.32

                  Set & delete 100 lists|         38 |        600 |          5000 |      22713 |     132.08

                  Commit one leaf change|         56 |          1 |          5000 |         56 |      88.83



                Data file 20 list instance in datastore startup 
Operation                               |    ops/sec |   items/op | ops performed |  items/sec |  test time.
---------------------------------------------------------------------------------------------------
                       Get item one leaf|      19033 |          1 |          5000 |      19033 |       0.26

                     Get item first leaf|      45896 |          1 |          5000 |      45896 |       0.11

      Get item (including session start)|       4720 |          1 |          5000 |       4720 |       1.06

                 Get all items of a list|      22554 |          1 |          5000 |      22554 |       0.22

                 Get items iter all list|       9759 |     100000 |          5000 |  975941096 |       0.51

             Get subtrees ietf-if config|       1433 |     900000 |          5000 | 1290091350 |       3.49

                    Get subtree one leaf|      20841 |          1 |          5000 |      20841 |       0.24

   Get subtree (including session start)|       4669 |          1 |          5000 |       4669 |       1.07

                    Get subtree all leaf|      21918 |         20 |          5000 |     438354 |       0.23

             Get subtrees ietf-if config|       1186 |     905000 |          5000 | 1073128286 |       4.22

                   Set & delete one list|      13368 |          6 |          5000 |      80211 |       0.37

                  Set & delete 100 lists|         34 |        600 |          5000 |      20659 |     145.21

                  Commit one leaf change|         46 |          1 |          5000 |         46 |     108.20



                Data file 100 list instance in datastore startup 
Operation                               |    ops/sec |   items/op | ops performed |  items/sec |  test time.
---------------------------------------------------------------------------------------------------
                       Get item one leaf|       7273 |          1 |          5000 |       7273 |       0.69

                     Get item first leaf|      43253 |          1 |          5000 |      43253 |       0.12

      Get item (including session start)|       1339 |          1 |          5000 |       1339 |       3.73

                 Get all items of a list|       6266 |          1 |          5000 |       6266 |       0.80

                 Get items iter all list|       2846 |     500000 |          5000 | 1423079220 |       1.76

             Get subtrees ietf-if config|        240 |    4500000 |          5000 | 1081715583 |      20.80

                    Get subtree one leaf|       6295 |          1 |          5000 |       6295 |       0.79

   Get subtree (including session start)|       1131 |          1 |          5000 |       1131 |       4.42

                    Get subtree all leaf|       6809 |        100 |          5000 |     680921 |       0.73

             Get subtrees ietf-if config|        270 |    4505000 |          5000 | 1218274828 |      18.49

                   Set & delete one list|       4695 |          6 |          5000 |      28167 |       1.07

                  Set & delete 100 lists|         25 |        600 |          5000 |      15136 |     198.21

                  Commit one leaf change|         28 |          1 |          5000 |         28 |     177.95

End

ops performed = number of repetitions

ops/sec = (test time) / (op performed)

items/op = how many items was retrieved by the operation

items/sec = how many items was retrieved within one second

Shown output was created using a virtual machine running on a laptop:

Description of the tested operation:

Get item one leaf

example-module

retrieves one leaf from the last list instance in data file using get_item

Get item first leaf

example-module

retrieves the first node in data file using get_item

Get item incl session start

example-module

starts the session, retrieves one leaf from the last list instance in data file using get_item and stops the session

Get items all lists

example-module

retrieves one node from each list instance in data file using get_items

Get items iter all lists

example-module

retrieves one node from each list instance in data file usign get_items_iter

Get items ietf-if config

ietf-interfaces

retrieves whole configuration using get_items_iter

Get subtree one leaf

example-module

retrieves one leaf from the last list instance in data file using get_subtree

Get subtree incl session start

example-module

retrieves the first node in data file using get_subtree

Get subtrees all lists

example-module

retrieves one node from each list instance in data file using get_subtrees

Get subtrees ietf-if config

ietf-interfaces

retrieves whole configuration using get_subtrees

Set & delete 100 lists

example-module

sets 100 list instances with 3 leaves using set_item and them deletes them (one by one) using delete_item

Commit one leaf change

example-module

creates and deletes one leaf from the last list instance in the data file

The demo of this test can be played below:

https://asciinema.org/a/88312

NOTE: asciinema has a feature that it can squash the recording when there are no changes in the terminal present. Because of that the asciinema video is 2:18 minutes long but in reality it is 16:20 minutes long. The breaks in time occur when the test takes longer than 5 seconds.

python2/sysrepo-performance (last edited 2016-10-06 16:06:54 by mnovakovic)