Skip to content

IVAAP Scalability Supplemental



IVAAP Deployment Operations Guide
Scalability Supplemental




IVAAP 2025.1

Introduction

This document serves as a guide for set-up, configuration, and deployment of all components and component changes to achieve scalability in IVAAP.

To achieve scalability in IVAAP, a new 3rd party backing service has been added. The Infinispan server will handle all of the caching for the IVAAP backend nodes and scheduled tasks. These additions to IVAAP allows for higher performance when monitoring real-time data, as well as general performance increases by reducing overhead of all internal communication and cluster activities.

Helm Configuration

To enable infinispan, Values.ivaapBackendNodes.infinispan.enabled Values.ivaapBackendNodes.realtimeEnabled and must both be set to true.

Once enabled, configuration can be found at the bottom of values.yaml, under Values.configmap.backendinfinispan: 

  backendinfinispan:
    IVAAP_JCACHE_ENABLED: "false"
    # ----- If IVAAP_JCACHE_ENABLED = true, set the envars below following deployment documentation.
    # IVAAP_INFINISPAN_CLIENT_HOTROD_AUTH__USERNAME: "Ivaapuser"
    # IVAAP_INFINISPAN_CLIENT_HOTROD_AUTH__PASSWORD_ENCRYPTED: f1B5Cw7mwc+Kdd36XZXnlg==
    # IVAAP_INFINISPAN_CLIENT_ENCRYPTION_KEY:: "shrank.Salary5"
    # ----- Below are examples for deployer and admin users - refer to deployment guide for steps on how to create these users.
    # IVAAP_INFINISPAN_DEPLOYER_USER_CREDENTIALS: |
    #     Ivaapuser=scram-sha-1\:BYGcIAzOEPEN3b8lpDxiWMAktGrZDzonv8fVogf7GVi5EZl/Aw\=\=;scram-sha-256\:BYGcIAxfDp7OIFiYv/6kE70SuTNXwYL09mMcBSEa5GmeCdrHo4f1Zhnk7rmpX29C6w\=\=;scram-sha-384\:BYGcIAx73MYDX2sCVxPjN0rC2V2MOHKx+/MEjI5cRkWIiV7iMx6V/mYnh2JhDyQdsXdRUMADwOslW6R2HYLcoRs\=;scram-sha-512\:BYGcIAxWR7ckt/v7jx+/LmCOS0gjgzY+kCT3pPqNYS6ieWpdHcMjlZKz8x+5kb481vu/xvB+NrXnGv/4+9OG4UQBb2iuSU8RxNDYG4jZ5ZNh;digest-md5\:AglJdmFhcHVzZXIHZGVmYXVsdNPsjpYwvLHo3YKK73ruRyE\=;digest-sha\:AglJdmFhcHVzZXIHZGVmYXVsdHBvc1SDdhG4Wdz8Nj4EInFL/7g2;digest-sha-256\:AglJdmFhcHVzZXIHZGVmYXVsdHNQU71b3743dchoy3ndXEQoRnlagWT7vcF+7UbMHTsd;digest-sha-384\:AglJdmFhcHVzZXIHZGVmYXVsdNQbtLiAnOdR1tvjEgrC1v3/DolyfNeSENnaCw1M6utG7ZbpWv26fA1YezRaazKluQ\=\=;digest-sha-512\:AglJdmFhcHVzZXIHZGVmYXVsdC0df0zp7rNZe6+20TgE/xsZXSznDg9VG6uil9p6PNqxHoJaOPpjUmZ8gRcyjsm0wks/2gPHdJj0RCIkH49kHfE\=;
    # IVAAP_INFINISPAN_ADMIN_USER_CREDENTIALS: |
    #     Ivaapadmin=scram-sha-1\:BYGcIAwRd0UDYHjlThc8A1Hr2WeYKALggq0vP0Ysz4APdrfsaw\=\=;scram-sha-256\:BYGcIAxgERNdoIxqKqDuFZgTw71fhl0gJf7OZejVe/Cbs5Q4KpKZ/Mdh3nwQgPB/5A\=\=;scram-sha-384\:BYGcIAzpBATsSURyndcX3Myidxdk+MGKvOvmz06aWfTaLfoJXe7sd9AEDlBEJw38EWDxZG0sGvAh+Q8994PMftw\=;scram-sha-512\:BYGcIAyfx3GqwstbkFBgDNKv1NCQxu2LAgdrvX94OhxzxittEEGsXEh1UTnX+XzMT2f9ev3yKdFiS3reo1jD9SZEo3ZapSz7juNFUh0/x+eG;digest-md5\:AgpJdmFhcGFkbWluB2RlZmF1bHRGtJdvUGUl6lqxZUsg4pzy;digest-sha\:AgpJdmFhcGFkbWluB2RlZmF1bHTeLEsNROrcUcKL6q6TtIDi/rRfkw\=\=;digest-sha-256\:AgpJdmFhcGFkbWluB2RlZmF1bHTuxqS7anEX+F29iMQ6FCrKjZQhDcTUBz0yrW8J8BoMiA\=\=;digest-sha-384\:AgpJdmFhcGFkbWluB2RlZmF1bHSOLSA6smivQ24d0+/+jMbQmdP/xUG0hMMMiEOTmlK/vH01QS5I1djeJuwiEZ8oIiA\=;digest-sha-512\:AgpJdmFhcGFkbWluB2RlZmF1bHTPO9JgyuMtW1w9lZLKHpBrGFEmyV6q4fYEHLJKSVpK4jknsxXh0UchRlDz5SuWDqdRs4DRwUSxVTk3/pVxCgo4;

The variables for this ConfigMap are primarily connection and user details. These values will be shared amongst all IVAAP components that need them.

This is also where the variables IVAAP_INFINISPAN_DEPLOYER_USER_CREDENTIALS and IVAAP_INFINISPAN_ADMIN_USER_CREDENTIALS will be configured once custom users are created. Details on this will be found in the section below named Custom Infinispan User Creation and Configuration.

To finish configuration, set IVAAP_JCACHE_ENABLED to true, then uncomment the rest of the variables under .Values.configmap.backendinfinispan.

  backendinfinispan:
    IVAAP_JCACHE_ENABLED: "true"
    IVAAP_INFINISPAN_CLIENT_HOTROD_AUTH__USERNAME: "Ivaapuser"
    IVAAP_INFINISPAN_CLIENT_HOTROD_AUTH__PASSWORD_ENCRYPTED: f1B5Cw7mwc+Kdd36XZXnlg==
    IVAAP_INFINISPAN_CLIENT_ENCRYPTION_KEY:: "shrank.Salary5"
    IVAAP_INFINISPAN_DEPLOYER_USER_CREDENTIALS: |
        Ivaapuser=scram-sha-1\:BYGcIAzOEPEN3b8lpDxiWMAktGrZDzonv8fVogf7GVi5EZl/Aw\=\=;scram-sha-256\:BYGcIAxfDp7OIFiYv/6kE70SuTNXwYL09mMcBSEa5GmeCdrHo4f1Zhnk7rmpX29C6w\=\=;scram-sha-384\:BYGcIAx73MYDX2sCVxPjN0rC2V2MOHKx+/MEjI5cRkWIiV7iMx6V/mYnh2JhDyQdsXdRUMADwOslW6R2HYLcoRs\=;scram-sha-512\:BYGcIAxWR7ckt/v7jx+/LmCOS0gjgzY+kCT3pPqNYS6ieWpdHcMjlZKz8x+5kb481vu/xvB+NrXnGv/4+9OG4UQBb2iuSU8RxNDYG4jZ5ZNh;digest-md5\:AglJdmFhcHVzZXIHZGVmYXVsdNPsjpYwvLHo3YKK73ruRyE\=;digest-sha\:AglJdmFhcHVzZXIHZGVmYXVsdHBvc1SDdhG4Wdz8Nj4EInFL/7g2;digest-sha-256\:AglJdmFhcHVzZXIHZGVmYXVsdHNQU71b3743dchoy3ndXEQoRnlagWT7vcF+7UbMHTsd;digest-sha-384\:AglJdmFhcHVzZXIHZGVmYXVsdNQbtLiAnOdR1tvjEgrC1v3/DolyfNeSENnaCw1M6utG7ZbpWv26fA1YezRaazKluQ\=\=;digest-sha-512\:AglJdmFhcHVzZXIHZGVmYXVsdC0df0zp7rNZe6+20TgE/xsZXSznDg9VG6uil9p6PNqxHoJaOPpjUmZ8gRcyjsm0wks/2gPHdJj0RCIkH49kHfE\=;
    IVAAP_INFINISPAN_ADMIN_USER_CREDENTIALS: |
        Ivaapadmin=scram-sha-1\:BYGcIAwRd0UDYHjlThc8A1Hr2WeYKALggq0vP0Ysz4APdrfsaw\=\=;scram-sha-256\:BYGcIAxgERNdoIxqKqDuFZgTw71fhl0gJf7OZejVe/Cbs5Q4KpKZ/Mdh3nwQgPB/5A\=\=;scram-sha-384\:BYGcIAzpBATsSURyndcX3Myidxdk+MGKvOvmz06aWfTaLfoJXe7sd9AEDlBEJw38EWDxZG0sGvAh+Q8994PMftw\=;scram-sha-512\:BYGcIAyfx3GqwstbkFBgDNKv1NCQxu2LAgdrvX94OhxzxittEEGsXEh1UTnX+XzMT2f9ev3yKdFiS3reo1jD9SZEo3ZapSz7juNFUh0/x+eG;digest-md5\:AgpJdmFhcGFkbWluB2RlZmF1bHRGtJdvUGUl6lqxZUsg4pzy;digest-sha\:AgpJdmFhcGFkbWluB2RlZmF1bHTeLEsNROrcUcKL6q6TtIDi/rRfkw\=\=;digest-sha-256\:AgpJdmFhcGFkbWluB2RlZmF1bHTuxqS7anEX+F29iMQ6FCrKjZQhDcTUBz0yrW8J8BoMiA\=\=;digest-sha-384\:AgpJdmFhcGFkbWluB2RlZmF1bHSOLSA6smivQ24d0+/+jMbQmdP/xUG0hMMMiEOTmlK/vH01QS5I1djeJuwiEZ8oIiA\=;digest-sha-512\:AgpJdmFhcGFkbWluB2RlZmF1bHTPO9JgyuMtW1w9lZLKHpBrGFEmyV6q4fYEHLJKSVpK4jknsxXh0UchRlDz5SuWDqdRs4DRwUSxVTk3/pVxCgo4;

Infinispan Server

The Infinispan server will handle all of the caching for the IVAAP backend witsml nodes and tasks. This server will be delivered as a container to be deployed in the same namespace as IVAAP, and will be included in IVAAP’s helm charts.

# [ IVAAP Infinispan Caching ]
ivaapInfinispan:
  infinispan:
    repoName: ivaap/infinispan
    tag: infinispan-15.1.7.Final-fa466eec-20250623T152424Z

Infinispan Server Initialization

The scalability goals were achieved with an all-in-one Infinispan container that does not require a persisted volume. When the container spins up, the Infinispan server process is started via our entrypoint script. Our entrypoint script then checks a rest API to confirm when the server is up and ready. Once the server is ready, a java process runs to import all schemas and configurations into Infinispan to finish its initialization for use with IVAAP.

Infinispan User Defaults

The Infinispan container has built-in default users for IVAAP as an option to reduce configuration. These default users will be used if custom users are not configured.

There are two users for the Infinispan server; a deployer user and an admin user. The deployer user is the user that connects to the IVAAP backend, and is responsible for handling the data communication for caching. The admin user is used for administrative purposes, such as modifying schemas and configurations.

Below are the default credentials:

Deployer:  
Username: ivaapuser  
Password: IvaapUs3r  

Admin:  
Username: admin  
Password: Ivaap@dm1n

It is not recommended to use the default users in a production environment. For production, follow the steps in the next section to create and configure custom users.

Custom Infinispan User Creation and Configuration

The Infinispan server container’s entrypoint script checks for the following two environment variables to determine whether to use the default users or custom users.

IVAAP_INFINISPAN_ADMIN_USER_CREDENTIALS  
IVAAP_INFINISPAN_DEPLOYER_USER_CREDENTIALS

The container contains bash functions in ~/.bashrc to create the admin and deployer users. 

createDeployerUser() {
  USERNAME=$1
  local password
  read -s -p "Enter password: " PW
  echo
  echo "Creating new deployer user"
  /opt/ivaap/infinispan/bin/cli.sh user create $USERNAME -p $PW -g deployer
  echo -e "User created\nThe following output needs to be set for environment varable IVAAP_INFINISPAN_DEPLOYER_USER_CREDENTIALS:\n\n"
  cat /opt/ivaap/infinispan/server/conf/users.properties | grep $USERNAME
}

createAdminUser() {
  USERNAME=$1
  local password
  read -s -p "Enter password: " PW
  echo
  echo "Creating new admin user"
  /opt/ivaap/infinispan/bin/cli.sh user create $USERNAME -p $PW -g admin
  echo -e "User created\nThe following output needs to be set for environment varable IVAAP_INFINISPAN_ADMIN_USER_CREDENTIALS:\n\n"
  cat /opt/ivaap/infinispan/server/conf/users.properties | grep $USERNAME
}

To use these functions, it is easiest to spin up the Infinispan server container locally in a Linux CLI using docker, and override the entrypoint script with /bin/bash.

user@linux:~$ docker run -it –rm --entrypoint /bin/bash 245634265005.dkr.ecr.us-west-2.amazonaws.com/ivaap/infinispan:infinispan-15.1.7.Final-6170fae3-20250403T035255Z
d45907087d36:/opt/ivaap/infinispan$

The syntax for the functions are as follows:

# Create Deployer User
createDeployerUser USERNAME

# Create Admin User
createAdminUser USERNAME

Replace USERNAME with your desired username for the new user. This will prompt the user to enter the desired password for the new user.
Example:

user@linux:~$ docker run -it --rm --entrypoint /bin/bash 245634265005.dkr.ecr.us-west-2.amazonaws.com/ivaap/infinispan:infinispan-15.1.7.Final-6170fae3-20250403T035255Z
ec6b0028753f:/opt/ivaap/infinispan$ createDeployerUser myNewUser
Enter password: 
Creating new deployer user
User created
The following output needs to be set for environment varable IVAAP_INFINISPAN_DEPLOYER_USER_CREDENTIALS:


myNewUser=scram-sha-1\:BYGcIAybE/ID04262AJyCCmLvpd50bM08HJwDXa/ppgF2qzSoQ\=\=;scram-sha-256\:BYGcIAz1OWy5XtYiKX6AQJ5XNPHQqlfw5Lg1hnMJPSKKdT/h1d2pq9x6X4+nNMVpNw\=\=;scram-sha-384\:BYGcIAzQeqBg4/FLiO+JrVz3AmhpFDeQMuxazsV49V/u6pCJ8ggsSVBVv4vRkgyPxPjfmulI/dtEYm6rczwv7KA\=;scram-sha-512\:BYGcIAzj0Y4VWh2s1xZdKxUEhOt7Tj/D8UqQ+QERL3AW4uFAV2Cm1e0wlUuXUM8h78FRyBOv8BRXHDyrxs5Td+XWOb9jEjTwrD0YM4fjJDft;digest-md5\:AglteU5ld1VzZXIHZGVmYXVsdIwfmG+QwMxmAW+ry/mnjNM\=;digest-sha\:AglteU5ld1VzZXIHZGVmYXVsdHPpPv1USoiE4w/kTkQDmhPea9L5;digest-sha-256\:AglteU5ld1VzZXIHZGVmYXVsdEYyg06/ceBjYzmZQEuc4hHpv1cAzaxGrlTJPzZCIpaw;digest-sha-384\:AglteU5ld1VzZXIHZGVmYXVsdLcFQzVE3HsL1/pR8QoaiePED5YsreByzOgKfHtoQvLd+9pA+XJqvOyzGU6jUsRDFw\=\=;digest-sha-512\:AglteU5ld1VzZXIHZGVmYXVsdIkr/f4eOszU4z2s+gIyqLonfJ54vCEELTLmjEn35tliVChyqLMk5o3aooEifeEKffO4KddegY57QnC2x8vOQXM\=;

As mentioned in the output of this function, the environment variable IVAAP_INFINISPAN_DEPLOYER_USER_CREDENTIALS needs to be configured with the output from the command.

This output gets plugged into the users configuration file for Infinispan to use. Below is an example of this config in helm chart’s values.yaml:

configmap:
  backendinfinispan:
    IVAAP_INFINISPAN_DEPLOYER_USER_CREDENTIALS |
        myNewUser=scram-sha-1\:BYGcIAybE/ID04262AJyCCmLvpd50bM08HJwDXa/ppgF2qzSoQ\=\=;scram-sha-256\:BYGcIAz1OWy5XtYiKX6AQJ5XNPHQqlfw5Lg1hnMJPSKKdT/h1d2pq9x6X4+nNMVpNw\=\=;scram-sha-384\:BYGcIAzQeqBg4/FLiO+JrVz3AmhpFDeQMuxazsV49V/u6pCJ8ggsSVBVv4vRkgyPxPjfmulI/dtEYm6rczwv7KA\=;scram-sha-512\:BYGcIAzj0Y4VWh2s1xZdKxUEhOt7Tj/D8UqQ+QERL3AW4uFAV2Cm1e0wlUuXUM8h78FRyBOv8BRXHDyrxs5Td+XWOb9jEjTwrD0YM4fjJDft;digest-md5\:AglteU5ld1VzZXIHZGVmYXVsdIwfmG+QwMxmAW+ry/mnjNM\=;digest-sha\:AglteU5ld1VzZXIHZGVmYXVsdHPpPv1USoiE4w/kTkQDmhPea9L5;digest-sha-256\:AglteU5ld1VzZXIHZGVmYXVsdEYyg06/ceBjYzmZQEuc4hHpv1cAzaxGrlTJPzZCIpaw;digest-sha-384\:AglteU5ld1VzZXIHZGVmYXVsdLcFQzVE3HsL1/pR8QoaiePED5YsreByzOgKfHtoQvLd+9pA+XJqvOyzGU6jUsRDFw\=\=;digest-sha-512\:AglteU5ld1VzZXIHZGVmYXVsdIkr/f4eOszU4z2s+gIyqLonfJ54vCEELTLmjEn35tliVChyqLMk5o3aooEifeEKffO4KddegY57QnC2x8vOQXM\=;

In addition to this configuration, these user details will need to also be reflected in the following variables for the common/shared ConfigMap found in values.yaml:

configmap:
  backendinfinispan:
    IVAAP_INFINISPAN_CLIENT_HOTROD_AUTH__USERNAME: "myNewUser"
    IVAAP_INFINISPAN_CLIENT_HOTROD_AUTH__PASSWORD_ENCRYPTED: "b2zn3uzCMEm6NWqAx/Upmw=="
    IVAAP_INFINISPAN_CLIENT_ENCRYPTION_KEY: "myEncryptionKey"

Details on password encryption can be found in the next section.

Password Encryption

The Infinispan deployer user password is encrypted in the same way as other backend components. This can be found in the Encrypting Sensitive Passwords for IVAAP Java Components section of the configuration guide.

However, we have also added a function inside the Infinispan container to make this process easier.

encryptPW() {
  read -p "Enter desired encryption key: " KEY
  echo
  read -s -p "Enter desired password to encrypt: " PW
  echo
  echo -e "Encrypting PW using $KEY\n\nEncrypted Password:"
  java -cp /opt/ivaap/infinispan/conf/lib/com.interactive.ivaap.IVAAPCoreData-3.0-20250402.185422-168.jar com.interactive.ivaap.data.encryption.AesEncryption $KEY $PW | tail -n 1
  local ENCRYPTED_PW=$(java -cp /opt/ivaap/infinispan/conf/lib/com.interactive.ivaap.IVAAPCoreData-3.0-20250402.185422-168.jar com.interactive.ivaap.data.encryption.AesEncryption $KEY $PW | tail -n 1)
  echo
  echo -e "IVAAP_INFINISPAN_CLIENT_ENCRYPTION_KEY=$KEY\nIVAAP_INFINISPAN_CLIENT_HOTROD_AUTH__PASSWORD_ENCRYPTED=$ENCRYPTED_PW"
}

Similar to the functions to create users, it is easiest to spin up an Infinispan container locally for this process.
Example:

user@linux:~$ docker run -it --rm --entrypoint /bin/bash 245634265005.dkr.ecr.us-west-2.amazonaws.com/ivaap/infinispan:infinispan-15.1.7.Final-99c32b74-20250410T171105Z
3efc95840dcf:/opt/ivaap/infinispan$ encryptPW
Enter desired encryption key: myEncryptionKey

Enter desired password to encrypt: 
Encrypting PW using myEncryptionKey

Encrypted Password:
b2zn3uzCMEm6NWqAx/Upmw==

IVAAP_INFINISPAN_CLIENT_ENCRYPTION_KEY=myEncryptionKey
IVAAP_INFINISPAN_CLIENT_HOTROD_AUTH__PASSWORD_ENCRYPTED=b2zn3uzCMEm6NWqAx/Upmw==