by ·Comments Off on Filebeat hang

Product:
FileBeat 8.19.3

Microsoft Windows 2022 server

Issue:

Have installed filebeat service in windows, to collect logs files to elastic, but it does not read any files. And when i try to stop the services it hangs.

After you stop the filebeat service with Task Manager, you need to erase the lock file in folder C:\ProgramData\filebeat\ to make it to read the yml file at next start of the service.

Solution:

Check the filebeat.yml file. The JSON format is sensitive to spaces and other formats.

In this case there was a row:
ignore_older: ‘7d’
that made the filebeat service to stop.

It only supports minutes and hours, so you need to enter like this:
ignore_older: ‘168h’

The ignore_older: ‘168h’ function will check the timestamp of the file, and not read files that was created more than 7 days ago.

The filebeat.yml file is in folder C:\Program Files\Filebeat on windows.

Below a example of a filebeat.yml file for use with TM1 logs files – you need to add spaces in the beginning of every line to get it to work.

# ============================== Filebeat inputs ===============================
filebeat.inputs:
- type: filestream
id: tm1server
enabled: true

paths:
- D:/TM1 folder/Logs/tm1server.log
fields_under_root: true
fields:
event:
dataset: audit.plain

- type: filestream
id: tm1s2
enabled: true
ignore_older: '168h'
paths:
- D:/TM1 folder/Logs/tm1s2*.log
exclude_lines:
- '^#'
include_lines:
- 'AD'
fields_under_root: true
fields:
event:
dataset: audit.plain

# ---------------------- beats state ----------------------
- type: filestream
id: beats-logs
enabled: true
paths:
- C:/ProgramData/filebeat/logs/filebeat*.ndjson
include_lines:
- 'Non-zero metrics in the last 30s'
fields_under_root: true
fields:
event:
dataset: beats.state
processors:
- dissect:
tokenizer: '%{}"@timestamp":"%{event.start}"'
field: message
target_prefix: ""
ignore_failure: true
setup.template.settings:
index.number_of_shards: 1
fields:
system:
env: prod
id: SystemTM1
fields_under_root: true
max_procs: 1
processors:
- add_host_metadata:
when.not.contains.tags: forwarded
- add_cloud_metadata: ~
output.logstash:
hosts: ["elasticservername.domain.com:9999"]
ssl.enabled: true
ttl: 5m
pipelining: 0

More Information:

Filebeat sends log files to Logstash or directly to Elasticsearch.

## Getting Started

To get started with Filebeat, you need to set up Elasticsearch on
your localhost first. After that, start Filebeat with:

./filebeat -c filebeat.yml -e

This will start Filebeat and send the data to your Elasticsearch
instance. To load the dashboards for Filebeat into Kibana, run:

./filebeat setup -e

For further steps visit the (https://www.elastic.co/guide/en/beats/filebeat/8.19/filebeat-installation-configuration.html) guide.

## Documentation

Visit (https://www.elastic.co/guide/en/beats/filebeat/8.19/index.html) for the full Filebeat documentation.

## Release notes

https://www.elastic.co/guide/en/beats/libbeat/8.19/release-notes-8.19.3.html

https://www.elastic.co/beats/filebeat

https://www.elastic.co/downloads/beats/filebeat

https://github.com/elastic/beats

by ·Comments Off on An authentication problem is preventing the dataflow from being refreshed

Product:

Microsoft Power Bi dataflow

Issue:

Get a error on all dataflow, Error: An authentication problem is preventing the dataflow from being refreshed. Please have the owner of the dataflow sign out of Power BI and sign back in and try again.

or error message like  {“code”:”DMTS_OAuthTokenRefreshFailedError”,”details”:[{“code”:”DM_ErrorDetailNameCode_UnderlyingErrorMessage”,”detail”:{“type”:1,”value”:”AADSTS135010: UserPrincipal doesn’t have the key ID configured} from the dataset that use the dataflow.

Can happen if you change you windows pincode login from microsoft, and use the same account to run the dataflow with.

Solution:

Go to the dataflow in your workspace in https://app.powerbi.com/

Go to settings

Click on data source credentials

Click on edit credentials

If the value is correct, click Sign in without a change.

Confirm the login question you will get, from you company.

Go to the workspace.

Try to refresh the dataflow again.

 

More information:

This error indicates that the cached credentials for the dataflow owner have expired, changed, or are no longer authorized to access the data source, causing a Power BI refresh failure. To resolve it, the owner must sign out of the Power BI service, clear browser cache, sign back in, and re-enter credentials in the dataflow settings.
Actionable Steps to Resolve:
  • Re-authenticate Dataflow Credentials: The dataflow owner should go to the Power BI Service, locate the dataflow, go to Settings > Data source credentials, and click “Edit credentials” to re-authenticate.
  • Sign Out/In: Sign out of app.powerbi.com completely, clear your browser cache/cookies, and sign back in to refresh token authentication.
  • Check Data Source: If using a Gateway, ensure the credentials in the gateway settings are valid.
  • Verify Permissions: Ensure the user who created the dataflow still has permissions to the underlying data sources (SQL, SharePoint, etc.).

 

https://www.fourmoo.com/2019/02/13/why-is-my-powerbi-dataflow-refresh-failing/ 

by ·Comments Off on Check cert in cognos

Product:
Cognos Analytics
Microsoft Windows 2019 server

Issue:

Does the cert expire in 2026 on old Cognos installations?

Solution:

Upgrade to later version of Cognos. The new version of Cognos Analytics and Planning Analytics have updated cert files as of IBM.

“IBM will provide updates SSL certificates as part of the next Planning Analytics Local 2.0.9 release. The best course of action is to configure your own SSL certificates for use with any version of TM1. You really do not need to wait until IBM provides updates certificates.”

If you have older version of TM1 client software and CA11 services, you may need to replace the certificates.

 

You can check the date of the cert in Cognos Analytics (CA11) with the IKEYMAN.EXE program, then open the camkeystore and check the date on your certificates as picture above show.
The case is that some certificates end date are updated, when you from inside Cognos Configuration program do a save.

Open the Cognos Configuration (as administrator) right click the root element –> “Test” then “Save” (re-save) and restart the service. Besides updating the cryptographic keys, you can also see transparency checking the details, during this process.

To check the Tm1 cert, copy C:\Program Files\ibm\cognos\tm1_64\bin64\ssl\ibmtm1.arm file to a c:\temp folder.
Rename the file to ibmtm1.crt. Right click on the ibmtm1.crt file and select OPEN.

There you see the certificate end date for your version of Tm1.

This document describes the process to renew and import updated certificates into Cognos Analytics, where Third-Party certificate authority has been enabled.

Objective

The purpose of this document, is to provide a straightforward process for obtaining and importing updated certificates into Cognos Analytics in order to minimize downtime, and reduce the need for a support case.
Depending on the certificate authority used by your organization, certificate renewal can typically be managed in one-of-two ways:
  1. The certificate authority allows you to resubmit your original Certificate Signing Request (CSR) to obtain a renewed certificate.
  2. The certificate authority requires that you present a new Certificate Signing Request (CSR) to obtain a renewed certificate.
Both of these methods can be achieved without repeating the original configuration steps to enable Third-Party certificate authority, and, with appropriate planning and backups in place, can be managed within a planned maintenance schedule.

Environment

Cognos Analytics 11.1.x and 11.2.x
All Operating System platforms
Third-Party certificate authority enabled

Steps

Part 1 – Back up the Cognos Analytics configuration.
  1. Stop Cognos Analytics
  2. Open Cognos Configuration
  3. Export the Configuration to plain text by clicking File –> Export As (recommended file name: decrypt.xml)
  4. Close Cognos Configuration
  5. Take a backup of the ‘configuration’ folder (recommended name: configuration-existingcerts)
Part 2 – Generate a Certificate Signing Request (CSR) if required.
  1. Open iKeyman (located in <COGNOS_HOME>\ibm-jre\jre\bin)
  2. Click Key Database File –> Open
  3. Navigate to <COGNOS_HOME>\configuration\certs
  4. Choose the CAMKeystore file and click Open
  5. Set the Key Database Type to PKCS12S2 and click OK
  6. Enter the keystore password (default is: NoPassWordSet) and click OK
  7. Click the current ‘encryption’ certificate, and click “Re-create Request”
  8. Provide a filename, and click OK
  9. Exit iKeyman
  10. Submit the generated CSR to your certificate authority for signing, and obtain your new certificate.
NOTE: If you need to have Cognos Analytics running while the certificate request is being processed, you can achieve this by renaming the ‘configuration’ folder to ‘configuration-csk’ and copying the ‘configuration-existingcerts’ folder back to ‘configuration’
Part 3 – Receiving the updated certificate
  1. Stop Cognos Analytics
  2. Ensure that the correct ‘configuration’ folder is in-place (if you used Step 2 of this process, ensure that the ‘configuration-csk’ folder is renamed to ‘configuration’
  3. Open iKeyman
  4. Open the <COGNOS_INSTALL>\configuration\certs\CAMKeystore file
  5. Click ‘Receive’
  6. Locate the newly received certificate, select, and click OK. You should receive a “Validation Successful” message
  7. Close iKeyman
  8. Open Cognos Configuration
  9. Save the configuration
  10. Start Cognos Analytics

 

 

More Information:

https://community.ibm.com/community/user/question/use-our-own-certificats 

For planning analytics this are the files used for certificates;

\bin64\ssl\ibmtm1.arm is the default certificate and it does not expire until 2035. ibmtm1.arm has been in use for a few years now, and even in 2020, its expiration was 2035. The “applixca” files in that folder are just there for some historical/nostalgic reason in my opinion, but I’m always focused on the latest releases, so there is that.

Inside of the \bin64\ssl\ibmtm1.kdb keystore, ibmtm1.arm has already been imported as “ibmtm1_server” and has been set as the default personal certificate. The “keystore” is what the components of PA will use to access the stored certificates and ibmtm1.sth is an encrypted password that PA uses to have access into the ibmtm1.kdb keystore while it is running.

New file \bin64\config.tm1admsrv.json  replaces the Cognos Configuration node that accepted TM1 Admin server settings. It defaults to using the \bin64\ssl\ibmtm1.kdb keystore and referring to the server certificate label ibmtm1_server that represents the imported ibmtm1.arm certificate IBM provided. A fresh install should be all set using ibmtm1.arm inside of ibmtm1.kdb.

You may have a \bin\tm1api.config or \bin64\tm1api.config where Architect and Perspectives have been installed on the users machines. This text file just refers to where Architect/Perspectives can find the keystore if it has been moved from \bin64\ssl\ibmtm1.kdb, if it is at a networked shared location, or it has been renamed. You likely don’t have this tm1api.config file if you never used custom certificates, but I mentioned it just in case. See link but ignore the stale top half. Most of the old Architect and Perspectives SSL Options are deprecated, so the top half is stale.

If you want to see the inside of the \bin64\ssl\ibmtm1.arm cert yourself, make a copy of it, and rename the copy to ibmtm1.crt. Then right-click on it and select “Open”…not “Install”. Microsoft will show you the date, etc. 2035.

Custom certificates for PA are not impossible, but not simple to implement.

 

NEW for 2.0.9.21/2.1.8/2.1.9 default \bin64\config.tm1admsrv.json file:

{
“tm1AdminNonSSLPortNumber”: 5495,
“tm1AdminSSLPortNumber”: 5498,
“tm1AdminHTTPPortNumber”: 5895,
“tm1AdminHTTPSPortNumber”: 5898,
“tm1AdminSupportNonSSLClients”: false,
    “tm1AdminKeyFile”: “./ssl/ibmtm1.kdb”,
    “tm1AdminKeyStashFile”: “./ssl/ibmtm1.sth”,
    “tm1AdminKeyLabel”: “ibmtm1_server”,
“tm1AdminTLSCipherList”: [],
“tm1AdminFIPSOperationMode”: 2,
“tm1AdminSupportPreTLSv12Clients”: false,
“tm1AdminNIST_SP800_131A_MODE”: false,
“tm1AdminIPVersion”: “IPv4”,
“tm1AdminActivityInterval”: 10,
“tm1AdminInactivityTimeout”: 10,
“tm1AdminRESTAPIToken”: “”
}

From the online documentation:

applixca.der
The original default certificate in DER format used for Java™ certificate stores.
applixca.pem
The original root authority certificate.
ibmtm1.arm
The default certificate file.
ibmtm1.crl
The certificate revocation list.
ibmtm1.kdb
The key database file, which contains the server certificate and trusted certificate authorities.
ibmtm1.rdb
The requested key pair and the certificate request data.
ibmtm1.sth
The key store, which contains passwords to the key database file.
tm1ca_v2.der
The updated default certificate.
tm1ca_v2.pem
The updated default root authority certificate.
tm1store
The Java certificate store containing the public root authority certificate.

https://www.ibm.com/support/pages/how-renew-third-party-ca-certificates-cognos-analytics 

https://community.ibm.com/community/user/discussion/what-happens-when-configurationscerts-expire 

https://community.ibm.com/community/user/groups/community-home/digestviewer/viewthread?GroupId=3067&MessageKey=5a98b967-835b-482b-998c-20bba9d68119&CommunityKey=8fde0600-e22b-4178-acf5-bf4eda43146b&tab=digestviewer&hlmlt=QT 

https://www.tm1forum.com/viewtopic.php?t=16287