Product:

Planning Analytics 2.0.9.19
Microsoft Windows 2019 server

Problem:
How log memory usage in Tm1?

Solution:

Add below to your tm1s-log.properties file in your TM1 applications config folder.

# memory logging configuration
log4j.logger.TM1.Server.Memory=INFO, S_Memory
log4j.appender.S_Memory=org.apache.log4j.SharedMemoryAppender
log4j.appender.S_Memory.MemorySize=1 MB
log4j.appender.S_Memory.MaxFileSize=10 MB
log4j.appender.S_Memory.MaxBackupIndex=10
#log4j.appender.S_Memory.Format=TM1Event
log4j.appender.S_Memory.TimeZone=Local
log4j.appender.S_Memory.File=tm1memory.log

 

This can give a message like this, when memory is finish on the server.

2024-08-22 13:04:13 local Shared memory reader PID 17780 started, for shmem “tm1s.exe-19096_3”
18512 [] ERROR 2024-08-22 13:04:12.676 TM1.Server.Memory Unable to allocate block from OS: Total Memory Allocated = 26772 Mb, Total Garbage Memory = 3 Mb, Allocating = 8388608 B
18512 [] WARN 2024-08-22 13:04:12.676 TM1.Server.Memory CreateNewGarbageBlocks() outOfMemory Exception <<< MEMORY_FATAL_LEVEL >>> – apifunc# “0”

You will get a separate log file for the memory records in the file tm1memory.log.

In new versions of TM1 you have a log file called TM1event.log, that you also can control from tm1s.cfg by adding this lines:

EnableTIDebugging=T
EventLogging=T
EventThreshold.ThreadRunningTime=3000
EventThreshold.ThreadWaitingTime=50

 

Set the EventLogging=F to not get the file created.

More Information:

https://www.ykud.com/blog/cognos/tm1-cognos/tm1s-log-properties/

https://www.ibm.com/support/pages/tm1-server-log-ends-logging-error-logger-layer-bad-log-message-size-bringing-shared-memory-reader-down

https://www.ibm.com/docs/en/planning-analytics/2.0.0?topic=monitoring-planning-analytics-tm1-logs

https://www.ibm.com/docs/en/planning-analytics/2.0.0?topic=logs-ops-logger 

https://www.ibm.com/docs/en/planning-analytics/2.0.0?topic=cubes-statsbycube 

https://www.performanceg2.com/tm1-control-cubes/ 

 

Below a suggested setup in tm1s-log.properties file.

# main logging file
log4j.logger.TM1=INFO, S1
# S1 is set to be a SharedMemoryAppender
log4j.appender.S1=org.apache.log4j.SharedMemoryAppender
# Specify the size of the shared memory segment
log4j.appender.S1.MemorySize=5 MB
# Specify the max filesize
log4j.appender.S1.MaxFileSize=20 MB
# Specify the max backup index
log4j.appender.S1.MaxBackupIndex=40
# Specify GMT or Local timezone
log4j.appender.S1.TimeZone=Local

# event logging configuration
# Meeds EventLogging=T in the tm1s.cfg
# https://www.ibm.com/support/knowledgecenter/SSD29G_2.0.0/com.ibm.swg.ba.cognos.tm1_inst.2.0.0.doc/c_eventlogging.html
#log4j.logger.Event=INFO, S_Event
#log4j.appender.S_Event=org.apache.log4j.SharedMemoryAppender
#log4j.appender.S_Event.MemorySize=1 MB
#log4j.appender.S_Event.MaxFileSize=10 MB
#log4j.appender.S_Event.MaxBackupIndex=10
#log4j.appender.S_Event.Format=TM1Event
#log4j.appender.S_Event.TimeZone=Local
#log4j.appender.S_Event.File=tm1event.log

# tm1top logging configuration only in version 2.0.7 and later
# Set up TopLogging=T in the tm1s.cfg
# https://www.ibm.com/support/knowledgecenter/SSD29G_2.0.0/com.ibm.swg.ba.cognos.tm1_op.2.0.0.doc/c_pa_top_logger.html
#log4j.logger.Top=INFO, S_Top
#log4j.appender.S_Top=org.apache.log4j.SharedMemoryAppender
#log4j.appender.S_Top.MemorySize=5 MB
#log4j.appender.S_Top.MaxFileSize=10 MB
#log4j.appender.S_Top.MaxBackupIndex=20
#log4j.appender.S_Top.TimeZone=Local
#log4j.appender.S_Top.Format=TM1Top
#log4j.appender.S_Top.File=tm1top.log

# Logins file -- records every time a user logins, can be used for license evaluation or just checking user activity
log4j.logger.TM1.Login=DEBUG, S_Login
log4j.appender.S_Login=org.apache.log4j.SharedMemoryAppender
log4j.appender.S_Login.MemorySize=5 MB
log4j.appender.S_Login.MaxFileSize=10 MB
log4j.appender.S_Login.MaxBackupIndex=20
log4j.appender.S_Login.TimeZone=Local
log4j.additivity.TM1.Login=false
log4j.appender.S_Login.File=tm1login.log


# memory logging configuration
log4j.logger.TM1.Server.Memory=INFO, S_Memory
log4j.appender.S_Memory=org.apache.log4j.SharedMemoryAppender
log4j.appender.S_Memory.MemorySize=5 MB
log4j.appender.S_Memory.MaxFileSize=20 MB
log4j.appender.S_Memory.MaxBackupIndex=10
log4j.appender.S_Memory.TimeZone=Local
log4j.appender.S_Memory.File=tm1memory.log


# it's good to have Locks details -- provides the names of the locked objects
# putting to a separate file to have a cleaner main log as PaW rollbacks requests a lot
log4j.logger.TM1.Lock.Exception=DEBUG, S_Lock
log4j.additivity.TM1.Lock.Exception=false
log4j.appender.S_Lock=org.apache.log4j.SharedMemoryAppender
log4j.appender.S_Lock.MemorySize=5 MB
log4j.appender.S_Lock.MaxFileSize=20 MB
log4j.appender.S_Lock.MaxBackupIndex=20
log4j.appender.S_Lock.TimeZone=Loca
log4j.appender.S_Lock.File=tm1lock.log

Product:
Planning Analytics 2.0.9.19 or newer
Microsoft Windows 2019 server

Issue:
How call a cmd/bat file without use the Executecommand, that is not around in the next version?

 

Solution:

In TM1 version 11 you can use ExecuteCommand(CommandLine, Wait)​;   https://www.wimgielis.com/tm1_executetempbatchfile_EN.htm 

In future version you have to do something else, probably you need to run some external schedule program that talk REST API to tm1 servers, like TM1Py (cubewise.com)  .. and then monitor the TM1 REST API to know when other external program should start.

From a external python script can you send email and start a TM1 TI process (with REST API) , but how do you activate them from inside TM1 TI without use of the ExecuteCommand?

Only option is ExecuteHttpRequest, described below: (but that means that the other application must have a REST API support)

This function executes an HTTP request. It supports HTTP methods GET, POST, PATCH, PUT and DELETE.

The HTTP response is cached in memory. A response consists of a status code, a number of headers and a body, any of which can be queried by using the corresponding TurboIntegrator functions: HttpResponseGetStatusCode, HttpResponseGetHeader, and HttpResponseGetBody.

The Planning Analytics Engine only keeps one HTTP response at a time. A new successful HTTP request execution update and overwrites the cached response.

The Planning Analytics Engine only keeps the first 100 KB of a response body and discards the rest. This prevents running out of evaluation memory for strings.

For convenience, the engine reuses the cookie it found in the previous responses in new requests when the Cookie header is absent.

Syntax

ExecuteHttpRequest(method, URL, option1, option2,
...);

Argument

Description

method The method of the HTTP request. Supported methods are
URL The URL where you want to execute the request. The URL must use the HTTP or HTTPS protocol.
option1, option2, … You can use these options in the request:

-u user
The basic user credential
-h header
The request header. A request can have multiple headers.
-d body
The request body. When started with an @ character, the body will be read from the file having the name found following the @ character.
-o filename
The file to which the response is written.
-c certificate_file
A custom CA certificate file.
-k
Instructs the server to not verify the TLS certificates.

https://www.ibm.com/docs/en/planning-analytics/2.0.0?topic=functions-executehttprequest

It may exist in some version of TM1.  Unclear in the IBM documentation.  Below links about Tm1 Rest Api

 

 

More Information:

https://www.ykud.com/blog/pa11-executehttprequest/ 

  • ExecuteCommand is going away in v12 (no interaction with OS in containers) so any external interaction (e.g. emails) will need another way to talk to something
  • TM1RunTI will not work in v12 either (as well as anything else using C API) and would leave a gap in PA orchestration capability, requiring some external tool to connect 2 PA databases. ExecuteHttpRequest would allow using TM1 Rest Api to achieve the same functionality

TM1/PA REST API Part 1: Introduction – Getting it Right

https://exploringtm1.com/managing-applications-command-line/ 

Send email/attachments | BIhints

Sending Email from IBM Cognos TM1 using Windows Powershell – Clear Insight | Total Performance Management

Using PowerShell in IBM Planning Analytics (aramar.co.uk)

Planning Analytics on Cloud – SMTP (send email) process using Powershell script (ibm.com)

How to use the Synchronize Function with RunProcess to Limit the Number of Threads (cubewise.com)

https://code.cubewise.com/blog/mastering-the-tm1-rest-api-with-postman/

https://github.com/Hubert-Heijkers/tm1-restapi-lab-dev-env-setup/blob/master/files/HOL-TM1SDK/postman_collections/TM1%20REST%20API.postman_collection.json

https://healthchecks.io/

https://www.febooti.com/products/automation-workshop/

https://n8n.io/integrations/

But does it exist a scheduler that talk REST API?   Power Automate can run Python or PowerShell code where you can write REST API (but Power Automate run IronPython version 2.7 or 3.4, that does not work well with latest tm1py).

https://learn.microsoft.com/en-us/power-automate/desktop-flows/requirements#sign-in-account-comparison

https://www.visualcron.com/comparelicenses.aspx#tasks

https://succeedium.com/teamone/

https://community.ibm.com/community/user/businessanalytics/blogs/vlad-didenko/2022/03/30/teamone-run-tm1-process-from-google-sheets

https://ironpython.net/blog/2014/12/07/pip-in-ironpython-275.html 

Product:

Python 3.12
Planning Analytics 2.0.9.19

Microsoft Windows 2019 server

Install Python, and tm1py.  https://code.cubewise.com/blog/installing-tm1py/

Issue:

How get a value from a dict object in python?

This code part will list the servers on localhost (from the tm1py utils lib)

(you need to adjust the code to make the things you want)

 

import csv

import http.client as http_client

import json

import ssl

from enum import Enum, unique

from io import StringIO

from typing import Any, Dict, List, Tuple, Iterable, Optional, Generator, Union, Callable

from urllib.parse import unquote

import os

import sys

from datetime import datetime

from TM1py.Services import TM1Service

import requests

from mdxpy import MdxBuilder, Member

from requests.adapters import HTTPAdapter

from TM1py.Exceptions.Exceptions import TM1pyVersionException, TM1pyNotAdminException, TM1pyNotDataAdminException, \

    TM1pyNotSecurityAdminException, TM1pyNotOpsAdminException, TM1pyVersionDeprecationException




try:

    import pandas as pd

    import numpy as np




    _has_pandas = True

except ImportError:

    _has_pandas = False




# --- parameters and settings

PORTS_TO_EXCLUDE = []

# TM1 connection settings (IntegratedSecurityMode = 1 )

ADDRESS = 'localhost'

USER = 'admin'

PWD = 'apple'




# ===== define a function to be called in the code

def get_all_servers_from_adminhost(adminhost='localhost', port=None, use_ssl=False) -> List:

    from TM1py.Objects import Server

    """ Ask Adminhost for TM1 Servers

    :param adminhost: IP or DNS Alias of the adminhost

    :param port: numeric port to connect to adminhost

    :param ssl: True for secure connection

    :return: List of Servers (instances of the TM1py.Server class)

    """




    if not use_ssl:

        conn = http_client.HTTPConnection(adminhost, port or 5895)

    else:

        conn = http_client.HTTPSConnection(adminhost, port or 5898, context=ssl._create_unverified_context())

    request = '/api/v1/Servers'

    conn.request('GET', request, body='')

    response = conn.getresponse().read().decode('utf-8')

    response_as_dict = json.loads(response)

    servers = []

    print (response_as_dict) # --- as inside function will not show ---

    for server_as_dict in response_as_dict['value']:

        server = Server(server_as_dict)

        servers.append(server)

    return servers




# ===== end of function by go back to beginning of line




# get TM1 models registered with the admin server

tm1_instances_on_server = get_all_servers_from_adminhost(ADDRESS, None, True)

# --- show the list of servers with data as dict

print (tm1_instances_on_server)




# --- for each item in list do this

for tm1_instance in tm1_instances_on_server:




        # get TM1 server information

        port = tm1_instance.http_port_number

        # --- show one value for that server from a dict

        print (tm1_instance.last_updated)

        print (port)




        if port in PORTS_TO_EXCLUDE:

            continue




        ssl = tm1_instance.using_ssl

        # --- connect to the tm1 server to get some more values

        tm1 = TM1Service(address=ADDRESS, port=port, user=USER, password=PWD, namespace='', gateway='', ssl=ssl)

# --- use the function 

        active_configuration = tm1.server.get_active_configuration()

        # print (active_configuration)

        print (active_configuration['Access']['HTTP'])
The last print line use more than one element to find the value to show.

print (tm1_instances_on_server) show the content of that variable, to be a list of dictionary’s.

All print statement in code, will be a line in the terminal window shown above. Use print to debug you code when you develop.

https://www.datacamp.com/tutorial/setting-up-vscode-python 

Solution:

print (active_configuration[‘Access’][‘HTTP’]) , you need to add the elements of the other lists inside the list, to get a value.

Traceback (most recent call last):
File “c:\temp\test2.py”, line 106, in <module>
print (active_configuration[‘Access’][‘Administration’])
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
KeyError: ‘Administration’   = you get a error if the key does not exist in the combo object of list and dict.

A Python dictionary (aka dict) is a series of key-value pairs referenced by key name. Dicts are delimited by curly braces. Key-value pairs are separated by commas. A key is separated from its value by a colon.

A list is an ordered collection of items, whereas a dictionary is an unordered data collection in a key: value pair. Elements from the list can be accessed using the index, while the elements of the dictionary can be accessed using keys.

Add this to the code to get a better view of the options:

from pprint import pprint

pprint (vars())

Above you can see the keys you can use to list the values in the print statement from the dict/list.

print (active_configuration[‘Access’][‘CAM’][‘ClientPingCAMPassport’]) will give you the value P0DT00H15M00S.

 

More Information:

https://packetpushers.net/blog/how-to-reference-nested-python-lists-dictionaries/ 

https://readthedocs.org/projects/tm1py/downloads/pdf/latest/ 

https://www.tm1forum.com/viewtopic.php?t=15531 

https://www.shiksha.com/online-courses/articles/difference-between-list-and-dictionary-in-python/ 

https://www.geeksforgeeks.org/difference-between-list-and-dictionary-in-python/

https://youtu.be/daefaLgNkw0?si=-RmMgixEKJtrtm4B

https://www.zyte.com/blog/json-parsing-with-python/ 

https://www.digitalocean.com/community/tutorials/python-pretty-print-json

Example of how to print a dict value:

import json
from types import SimpleNamespace

data = '{"name": "John Smith", "hometown": {"name": "New York", "id": 123}}'

# Parse JSON into an object with attributes corresponding to dict keys.
x = json.loads(data, object_hook=lambda d: SimpleNamespace(**d))
print(x.name, x.hometown.name, x.hometown.id)

https://www.tutorialspoint.com/how-to-print-all-the-keys-of-a-dictionary-in-python 

https://www.tutorialspoint.com/how-to-print-all-the-values-of-a-dictionary-in-python

https://www.altcademy.com/blog/how-to-print-a-dictionary-in-python/

https://favtutor.com/blogs/print-object-attributes-python

https://www.geeksforgeeks.org/how-to-create-a-list-of-object-in-python-class/

https://flexiple.com/python/python-print-list

https://blog.enterprisedna.co/python-how-to-print-a-list-6-ways-you-need-to-know/

https://code.visualstudio.com/docs/languages/python 

https://code.visualstudio.com/docs/python/python-tutorial

 

Product:
Planning Analytics 2.0.9.x TM1_version=TM1-AW64-ML-RTM-11.0.913.10-0
Microsoft Windows 2019 server
Python 3.11

Issue:
How tell a outside application that a TM1 TI process is finished?

Suggested solution:

Use TM1PY and a python script that checks a cube, where you write when the process have finished.

Create a cube for the flags

You have dimension FlagDimension and FlagValue, as shown above.

Create a TI process that you call from the epilog in your other process, to create the value in the cube. Looks like this:

When above TI process is run the value in the cell will increase with ONE, then you can see if the process has run more than once.

You have to come up with the pFlagName for each process, and use the same name in the Python code.

Install Python, and tm1py.  https://code.cubewise.com/blog/installing-tm1py/

Create a python script similar to below ( you need to change values to point to your tm1 server instance ):

# import the modules needed

import os

from TM1py.Services import TM1Service

# Parameters for connection - python is case sensitive

USER = "admin"

PWD = "apple"

namespace = ""

ADDRESS = "192.168.50.190"

gateway = ""

PORT = 12354

SSL =  True

#Connect to the TM1 instance

tm1 = TM1Service(address=ADDRESS, port=PORT, user=USER, password=PWD, ssl=SSL)

# where to check in cube

cube_name = 'CubeOfFlags'

elementname1 = "FlagName"

elementname2 = "FValue"

elementstr = elementname1 + "," + elementname2

values2 = tm1.cubes.cells.get_value (cube_name=cube_name, elements = elementstr, element_separator = ',')

# only to debug the code - it will show the value in terminal window

print(values2)

# in our example we use a bat file to run

# you need to use \\ in paths with python

testfilename = "c:\\temp\\testrun.cmd"

# execute the process if larger than zero

if values2 != None :

   if values2 > 0 :

      print ("this is more than zero")

      # get the process to run

      os.system (testfilename)

      # set the value to zero in the cube

      # so we do not run this process again

      # cellset to store the new data

      cellset = {}

      # Populate cellset with coordinates and value

      cellset[( elementname1 , elementname2 )] = 0  

   

      # send the cellset to TM1

      tm1.cubes.cells.write_values( cube_name , cellset)

     

# end of code

The line indent is important in python, that tells what code to run when the if statement is true.

To test we created a cmd file with this code – will create a txt file with the time in it.
When run it will create a txt file with this content:
You should be able to replace above with a cmd file that does what you want.
There are better ways to do this – please try out yourself.
Schedule the python script to check the cube every five minutes with a schedule program like:
IBM Cognos Command Center provides self-service process automation. Through a single interface, it enables you to view and run automated processes on an ad hoc basis, and diagnose and address issues much more quickly. The solution reduces the complexity of working in diverse software environments and brings you greater simplicity and control….
More Information:

 

If you use the command

time /T

that will print the time. (without the /T, it will try to set the time)

tm1.cubes.cells.get_value(cube_name: str, elements: str | Iterable = None, dimensions: List[str] = None, sandbox_name:

str = None, element_separator: str = ‘,’, hierarchy_separator: str = ‘&&’, hierarchy_element_separator: str = ‘::’, **kwargs) → str | float
Returns cube value from specified coordinates
Parameters
• cube_name – Name of the cube
• elements – Describes the Dimension-Hierarchy-Element arrangement – Example: “Hierarchy1::Element1 && Hierarchy2::Element4, Element9, Element2” – Dimensions are not specified! They are derived from the position. – The , separates the element-selections

• dimensions – List of dimension names in correct order
• sandbox_name – str
• element_separator – Alternative separator for the element selections
• hierarchy_separator – Alternative separator for multiple hierarchies
• hierarchy_element_separator – Alternative separator between hierarchy name and
element name

tm1.cubes.cells.write_value(value: str | float, cube_name: str, element_tuple: Iterable, dimensions: Iterable[str] = None, sandbox_name: str = None, **kwargs) → Response

Write value into cube at specified coordinates
Parameters
• value – the actual value
• cube_name – name of the target cube
• element_tuple – target coordinates
• dimensions – optional. Dimension names in their natural order. Will speed up the execution!
• sandbox_name – str

https://www.tutorialspoint.com/How-to-declare-a-variable-in-Python-without-assigning-a-value-to-it

https://code.cubewise.com/blog/loading-data-into-a-tm1-cube-with-tm1py/

https://www.learnpython.org/en/Dictionaries

https://diveintopython.org/learn/file-handling/zip

https://exploringtm1.com/cellincrementn-tm1-function-use-syntax/

Product:
Planning Analytics 2.0.9.x TM1_version=TM1-AW64-ML-RTM-11.0.913.10-0
Microsoft Windows 2019 server
Python 3.11

Issue:

Get error when run code below:

C:\Program Files\Python311>python.exe c:\temp\test.py

ERROR:
(‘Connection aborted.’, BadStatusLine(‘\x15\x03\x03\x00\x02\x02\n’))

 

Code that does not work:

from TM1py.Services import TM1Service

# Parameters for connection
user = "admin"
password = "apple"
namespace = ""
address = "192.168.50.190"
gateway = ""
port = 12354
# HTTPPortNumber=12354 for planning sample
ssl = "T"

if len(namespace.strip()) == 0:
namespace = None

if len(gateway.strip()) == 0:
gateway = None

try:
with TM1Service(
address=address,
port=port,
user=user,
password=password,
namespace=namespace,
gateway=gateway,
ssl=ssl) as tm1:
server_name = tm1.server.get_server_name()
print("Connection to TM1 established!! your Servername is: {}".format(server_name))
except Exception as e:
print("\nERROR:")
print("\t" + str(e))

https://code.cubewise.com/blog/check-connectivity-with-tm1/
Solution:

Change ssl = “T” to ssl = True to make the code work.

 

A small number of constants live in the built-in namespace. They are:

False
The false value of the bool type. Assignments to False are illegal and raise a SyntaxError.
True
The true value of the bool type. Assignments to True are illegal and raise a SyntaxError.
None
An object frequently used to represent the absence of a value, as when default arguments are not passed to a function. Assignments to None are illegal and raise a SyntaxError. None is the sole instance of the NoneType type.

 

More Information:

https://www.w3schools.com/python/python_variables.asp

https://github.com/cubewise-code/TM1py-samples

#import TM1py services
from TM1py.Services import TM1Service
from TM1py.Utils import Utils
#TM1 credentials
ADDRESS = "192.168.50.190"
PORT = 12354
USER = "admin"
PWD = "apple"
SSL = True

#Connect to the TM1 instance
tm1 = TM1Service(address=ADDRESS, port=PORT, user=USER, password=PWD, ssl=SSL)
# Cube view used 
cube_name = 'CubeOfFlags'
view_name = 'Default'
# get the data to a variable in python
values = tm1.cubes.cells.execute_view_values(cube_name=cube_name, view_name=view_name, private=False)
# extract first ten values
first_ten = list(values)[0:10]
# print first ten values
print(first_ten)

 

 

https://docs.python.org/3/library/constants.html

https://realpython.com/python-constants/

https://www.tutorialspoint.com/how-do-i-create-a-constant-in-python

https://learn.microsoft.com/en-us/troubleshoot/windows-server/networking/configure-ipv6-in-windows

Product:
Planning Analytics 2.0.9.19
Microsoft Windows 2019 server
Python 3.12

Issue:

When run below code, i get a error like:

C:\Python312>python.exe c:\proj1\test.py
Traceback (most recent call last):
File “c:\proj1\test.py”, line 4, in <module>
from distutils.util import strtobool
ModuleNotFoundError: No module named ‘distutils’

 

import getpass
from distutils.util import strtobool

from TM1py.Services import TM1Service

# Parameters for connection
user = input("TM1 User (leave empty if SSO): ")
password = getpass.getpass("Password (leave empty if SSO): ")
namespace = input("CAM Namespace (leave empty if no CAM Security): ")
address = input("Address (leave empty if localhost): ") or "localhost"
gateway = input("ClientCAMURI (leave empty if no SSO): ")
port = input("HTTP Port (Default 5000): ") or "5000"
ssl = strtobool(input("SSL (Default T or F): ") or "T")

if len(namespace.strip()) == 0:
namespace = None

if len(gateway.strip()) == 0:
gateway = None

try:
with TM1Service(
address=address,
port=port,
user=user,
password=password,
namespace=namespace,
gateway=gateway,
ssl=ssl) as tm1:
server_name = tm1.server.get_server_name()
print("Connection to TM1 established!! your Servername is: {}".format(server_name))
except Exception as e:
print("\nERROR:")
print("\t" + str(e))

 

Solution:

Change to python version 3.10 as python version 3.12 does not support this commands.

Python 3.12 no longer provides any standard-library distutils version at all. There is no longer any “stdlib” distutils, so it cannot be selected. Because distutils was deprecated in 3.10 and removed in 3.12.

You can still use distutils on Python 3.12+ by installing setuptools.

Run the command: python.exe -m pip install setuptools

setuptools provides a module _distutils_hack, which when loaded replaces sys.modules[‘distutils’] to point to setuptools._distutils; however, it only does this when SETUPTOOLS_USE_DISTUTILS is either unset or has the value ‘local’.

You can also avoid using the distutils functions.

 

More information:

https://code.cubewise.com/blog/check-connectivity-with-tm1/

https://marketplace.visualstudio.com/items?itemName=ms-python.python

https://realpython.com/python-zipfile/#creating-zip-files-sequentially

https://github.com/cubewise-code/tm1py

https://github.com/cubewise-code/TM1py-samples

Product:

Planning Analytics 2.0.9.3

Microsoft Windows 2019 server

issue:

How install tm1py to server that is not connected to internet?

https://code.cubewise.com/tm1py-help-content/installing-tm1py-on-a-server-with-no-internet-connection

Solution:

Download python to your laptop and to the server, and install python on both. https://www.python.org/downloads/

Run the python-3.11.3-amd64.exe to install the software.

Select Customize installation. (ensure you click “add python to path” )

Click Next

Select install python for all users, and change the path to a simpler one outside program files.

Click Close when done.

On your laptop (that is connected to internet) go to your python folder. Inside the command prompt enter:

pip install tm1py[pandas]

When done – run below command, to ensure you have the requirements:  (https://github.com/cubewise-code/tm1py)

pip install requests_negotiate_sspi

Create the folder c:\temp\offline. Move to that folder in command window and run the command:

pip freeze > C:/temp/offline/requirements.txt

Zip all the needed files with this command (run the command from the folder where you want the zip files to be created):

pip download tm1py[pandas]

The versions above listed, depend on the version you have installed of python and tm1py. You may get different versions.

If you get errors later, in most cases, it is because we have not succeeded to get the right files to this folder.

Now copy the files over to the server from the laptop. (copy all files to the same folder)

certifi==2022.12.7
charset-normalizer==3.1.0
idna==3.4
ijson==3.2.0.post0
mdxpy==1.2
numpy==1.24.2
pandas==2.0.0
pypiwin32==223
python-dateutil==2.8.2
pytz==2023.3
pywin32==306
requests==2.28.2
requests-negotiate-sspi==0.5.2
six==1.16.0
TM1py==1.10.2
tzdata==2023.3
urllib3==1.26.15

Above a example of the req.txt file content for tm1py needed files.

Copy the whl files in c:\python311\scripts over to the servers c:\python311\scripts folder (or the folder you are using like c:\temp\offline).

If you copy between virtual machines, in vbox, you access the local host drive with \\Vboxsvr\c_drive\temp.

How to create shared folder for virtual machine on VirtualBox

It is important that the files are in the same folder, otherwise you get:

C:\Python311\Scripts>C:\Python311\scripts\pip install –no-index –find-links C:/temp/offline -r requirements.txt
ERROR: Could not open requirements file: [Errno 2] No such file or directory: ‘requirements.txt’

C:\temp\offline>C:\Python311\scripts\pip install –no-index –find-links C:/temp/offline -r requirements.txt
Looking in links: c:/temp/offline
ERROR: Could not find a version that satisfies the requirement certifi==2022.12.7 (from versions: none)
ERROR: No matching distribution found for certifi==2022.12.7

C:\Python311\Scripts>C:\Python311\scripts\pip install –no-index –find-links -r requirements.txt
Looking in links: -r
WARNING: Location ‘-r’ is ignored: it is either a non-existing path or lacks a specific scheme.
ERROR: Could not find a version that satisfies the requirement requirements.txt (from versions: none)
HINT: You are attempting to install a package literally named “requirements.txt” (which cannot exist). Consider using the ‘-r’ flag to install the packages listed in requirements.txt
ERROR: No matching distribution found for requirements.txt

C:\Python311\Scripts>C:\Python311\scripts\pip install –no-index –find-links c:\python311\scripts -r requirements.txt
Looking in links: c:\python311\scripts
Processing c:\python311\scripts\certifi-2022.12.7-py3-none-any.whl
Processing c:\python311\scripts\charset_normalizer-3.1.0-cp311-cp311-win_amd64.whl
Processing c:\python311\scripts\idna-3.4-py3-none-any.whl
Processing c:\python311\scripts\ijson-3.2.0.post0-cp311-cp311-win_amd64.whl
Processing c:\python311\scripts\mdxpy-1.2.tar.gz
Installing build dependencies … error
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> [3 lines of output]
Looking in links: c:\python311\scripts
ERROR: Could not find a version that satisfies the requirement setuptools (from versions: none)
ERROR: No matching distribution found for setuptools
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Enter -vvv last to get more information about the issue;

pip install –no-index –find-links c:\python311\scripts -r requirements.txt -vvv

Maybe a solution, check what files you get issues with, if it is the files that end with tar.gz, then search on the laptop you have it installed on to find the whl file for the same function.  Search for mdxpy* if you get error at mdxpy-1.3.2.tar.gz.

You will find the file in a folder like C:\Users\Administrator\AppData\Local\pip\cache\wheels\f8\6d\90\4bfbb1a9cc872dbc6e440d1f4ec64cb896cc724e3a9d29fc92

copy the file mdxpy-1.3.2-py3-none-any.whl to the folder where you have the other files you need. (like c:\temp\offline)

Repeat for file TM1py-2.0.4-py3-none-any.whl, so you get a whl file from the cache to the other computer and place in the folder where you install from.

Try then again with the command similar to this:

c:\python312\scripts\pip install --no-index --find-links C:\tempd -r c:\tempd\req.txt

The clue is to have all the whl files you need in the folder where you install the things from.

If you get;

ERROR: Could not open requirements file: [Errno 2] No such file or directory: ‘requirements.txt’

Try this command after you copy the txt file to the scripts folder

C:\Python311\Scripts>pip install --no-index --find-links c:\python311\scripts -r requirements.txt -vvv

 

You can try to edit the txt file to have the number listed in the error, gives that the error goes away, but you get a new error.

C:\tempas>pip install –no-index –find-links c:\tempas\ -r requirements.txt
Looking in links: c:\tempas\
Processing c:\tempas\certifi-2023.11.17-py3-none-any.whl
Processing c:\tempas\charset_normalizer-3.3.2-cp311-cp311-win_amd64.whl
Processing c:\tempas\idna-3.6-py3-none-any.whl
ERROR: Could not find a version that satisfies the requirement ijson==3.2.0.post0 (from versions: 3.2.3)
ERROR: No matching distribution found for ijson==3.2.0.post0

When you try to install several times to the same computer, think that the pip program will load files from the local cache folder first, and therefor the installation can fail – even you are writing the correct command to do the work.

 

== The last Solution to try are below ==

To repeat the steps you need to do to install a python package on a air-gapped computer:

On the computer where you have internet contact;

Install the latest python package (python-3.12.4-amd64.exe ), install the same package on the air-gapped computer.

Ensure that during installation you select, install for all users and add to path. Then go to the command prompt and run:

pip install tm1py[pandas] streamlit

pip freeze > c:/tempe/req.txt

Go to the tempe folder and run below command

pip download streamlit

pip download tm1py[pandas]

Here you get the gz file, but it looks like you need the whl file. Copy that from the cache to the c:\tempe folder. You may need these files:

mdxpy-1.3.2-py3-none-any.whl

TM1py-2.0.4-py3-none-any.whl

ijson-3.3.0-cp312-cp312-win_amd64.whl

So you have all needed whl files in the folder. The version number can be different if you install a different version of tm1py.

Copy the \tempe\ folder over to the air-gapped computer and run command:

c:\python312\scripts\pip install  --no-index  --find-links  C:\tempe  -r  c:\tempe\req.txt

Hopefully it works easier for you to do the installation on your computer. (Below when the pip command works fine).

To check if tm1py is installed enter: pip show tm1py

Name: TM1py
Version: 1.10.2
Summary: A python module for TM1.
Home-page: https://github.com/cubewise-code/tm1py
Author: Marius Wirtz
Author-email: MWirtz@cubewise.com
License: MIT
Location: C:\Python311\Lib\site-packages
Requires: ijson, mdxpy, pytz, requests, requests-negotiate-sspi
Required-by:

How the latest ( https://www.python.org/downloads/ ) python install package steps look like:

More information:

Share Folders between Host and Guest OS in VirtualBox

https://bobbyhadz.com/blog/python-could-not-find-a-version-that-satisfies-the-requirement

https://bobbyhadz.com/blog/python-the-script-is-installed-in-which-is-not-on-path

Add python to path by using the keyboard shortcut (Win+Pause) to get to settings and under advance link, click on “Environment Variables” button.

https://www.activestate.com/resources/quick-reads/how-to-download-python-dependencies/

https://learn.microsoft.com/en-us/azure-data-studio/notebooks/notebooks-python-offline-installation

http://srikanthtechnologies.com/blog/python/installing_libraries_offline.aspx 

https://www.python.org/downloads/release/python-350/

https://cubewise-code.github.io/tm1py-tales/2022/tm1py-web-app.html

You can install streamlit via pip:

pip install streamlit

Once streamlit is installed, let us create a very basic web app.

As the first step, let us create a new Python script. Let us call it app.py.

Open app.py in your favourite IDE or text editor, then add these lines:

import streamlit as st

st.title("I love TM1py")

With that in place, we can run Streamlit from the command line:

https://en.wikipedia.org/wiki/Air_gap_(networking)

https://www.learnpython.org/en/Dictionaries

Product:
Planning Analytics Workspace 95
Microsoft Windows 2019 server

Issue:

In PAW administration page, for the agents, the version number is not showing the PAA agent version installed.

Version 2.0.93.1280

When you have installed PAA_Agent version 95. By command similar to this:

UpdatePAAAgent.bat “D:\Program Files\ibm\cognos\tm1_64”

Solution:

This value is from the file D:\Program Files\ibm\cognos\tm1_64\paa_agent\paaAgentCache\serversInfo.json

Test to erase the folder D:\Program Files\ibm\cognos\tm1_64\paa_agent\paaAgentCache in your lab environment, before you install the paa-agent, to see if that solves the issue. In most cases this does not work, and you have to reinstall PAL.

Our guess is that the different version of PAA Agent use different scripts/folders and therefor this file is not correct updated.

 

 

More Information:

https://www.ibm.com/docs/en/planning-analytics/2.1.0?topic=components-planning-analytics-administration-agent-local-only 

 

 

Product:
Planning Analytics 2.0.9.19
Microsoft Windows 2019 server

Issue:
We change the password for the system account that we use when calling TI processes inside TM1 instances, and now Tm1RunTi.exe does not work.

Solution:

Find the file that contain the password and key, run below command to create the new password files:

tm1crypt.exe  -keyfile  d:\arkiv\CognosUser_key.dat  -outfile  d:\arkiv\CognosUser_cipher.dat  -validate

(above will create the files in the folder d:\arkiv\, then you have to copy them to correct folder that you referee in your code)

You have to enter the new password for your service account twice.

#======================
# Parameters
#======================

sQuote = Char(34);

sRunTIPath=sQuote |'D:\Program Files\ibm\cognos\tm1_64\bin64\\TM1RunTI.exe'|sQuote ;
sPwdFile='CognosUser_cipher.dat';
sPwdKeyFile='CognosUser_key.dat';
sPasswordFile=sQuote |'D:\TM1 data\Tm1_Trigger\'|sPwdFile|sQuote ;
sPasswordKeyFile=sQuote |'D:\TM1 data\Tm1_Trigger\'|sPwdKeyFile|sQuote ;
sUserName='Cognos';
sUser=sQuote | sUserName | sQuote ;
sProcess = sQuote | pProcessName | sQuote ;
sInstance=sQuote|pInstance|sQuote;
sNameSpace=sQuote|'domain'|sQuote;
sAdminHost=sQuote|'servername'|sQuote;


#=====================
# Execute Command
#=====================

sCommand = sRunTIPath | ' -Process ' | sProcess | ' -Adminhost ' | sAdminHost | ' -Server ' | sInstance | ' -CAMNamespace ' | sNameSpace |
' -User ' | sUser | ' -passwordfile ' | sPasswordFile | ' -passwordkeyfile ' | sPasswordKeyFile ;
ExecuteCommand(sCommand , 0 );

 

Above in a prolog dialog will execute the TM1runTI.exe with the parameters you provide for TI process ( pProcessName ) to run and TM1 instance (pInstance) to use.

You need to update above TI code with the correct windows server name and Active Directory domain name, also point to the correct folder where you save the dat files, and the windows user account that you use for the access to the tm1 models.

 

More Information:

https://www.ibm.com/docs/en/planning-analytics/2.0.0?topic=encryption-run-tm1crypt-utility 

https://code.cubewise.com/blog/4-ways-to-speed-up-your-processes-in-ibm-tm1-and-planning-analytics/

 

Product:
Planning Analytics 2.0.9.19
Microsoft Windows 2019 server
Planning Analytics Workspace version 95

Problem:

Where is the files uploaded to?

When you try to create a new folder in PAW version  95, then you get a error like below, then something with the installation went wrong.

The following administration agent error occurred – {“errorMessage”:{},”query”:”/pakate-agent/v0/filemanager/Planning%20Sample/files”,”httpStatusCode”:500}

 

Suggestion:
The file manager will only access a folder under your applications data folder called model_upload like in C:\Program Files\ibm\cognos\tm1_64\samples\tm1\PlanSamp\model_upload. (i think you can edit this folder name in the paa-agent bootstrap.properties file.)  Try change this value:

MODEL_UPLOAD=model_upload

To upload a file go to PAW and WORKBENCH

Right click on the application you want to work with files and select file manager.

Click on the upload icon, and then drag the file to the area, and press upload button.

When the file is uploaded, close above dialog.

The file is put in a folder below your data folder, for your tm1 instance.

 

 

If the paa-agent not work as expected, then the files are created here, for all tm1 instances.

D:\Program Files\ibm\cognos\tm1_64\paa_agent\wlp\usr\servers\kate-agent\model_upload

Check the messages.log file above for more information why it does not work.

Reinstall PAA agent, for you to try resolving the issue.

The problem could be related to one or more old files in the paa_agent folder structure, test below steps.
1. Stop the “IBM Planning Analytics Administration Agent” service
2. Remove the the cache folders: “.logs” and “.classCache” in ../tm1_64/paa_agent/wlp/usr/servers
3. Remove the folder: “workarea” in ../tm1_64/paa_agent/wlp/usr/servers/kate-agent
4. Restart the “IBM Planning Analytics Administration Agent” service
Wait 5 minutes time as the Agent starts, before you test again.
Do not Remove the D:\Program Files\ibm\cognos\tm1_64\paa_agent\paaAgentCache folder, as it looks that it corrupts the PAA agent installation.

More Information:

https://www.ibm.com/docs/en/planning-analytics/2.0.0?topic=model-using-file-manager 

https://www.ibm.com/support/pages/planning-analytics-administration-agent-didnt-work-error-exception-javalangnullpointerexception 

Working with files and folders in File manager

​You can use the toolbar to create folders and sub-folders, upload files, refresh files, and search for a folder or file. Use the Actions button Actions button​ or select a file or folder to access options​such as Download, Delete, Cut, Copy, and Compress.

Deleting a file or folder
To delete an item, select the file or folder and click Delete. You can also click the Action button for the item and click Delete.

Warning: When you delete a folder or sub-folder, all files within that folder are also deleted.
Cut, copy, and paste files
You can move or copy files or folders to a different folder within the File manager. The Cut and Copy options are available from the Actions button or display in the toolbar when you select one or more items.
To cut or copy, select one or more items and click Cut or Copy. Your selections display in alphabetical order as a horizontal list for a quick review before you cut or copy the items.
Selected files for cutting or copying display in the File manager
After you review the list of items that you want to cut or copy, you can paste the items to a different folder. To paste a cut or copied item, go to the destination folder and click Paste.
Compress and expand files
Select one or more files or folders and click Compress to compress the items into a .zip file. You can compress a file and move it to a new location without using Cut. To do this, select a file and click Compress. Then, go to the destination folder and click Compress again.
To expand a compressed file, select the file and click Expand. As with compressing a file to a new location, you can expand a file to a new folder.