Product:
Planning Analytics Workspace version 36
Microsoft Windows 2016 server
Problem:
After some time use, the user can not access the paw. They can not login.
This happen after the server have rebooted during a Microsoft Windows Patching.
To find out if your PAW installation is damage, check this steps:
When you run a docker ps command, you see services listed as restarting.
When you run a ./paw.ps1 stop command, you see services that does not stop, even after 20 minutes.
Error message:
503 The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
Follow this guide lines to troubleshoot:
https://www.ibm.com/support/pages/troubleshooting-planning-analytics-workspace-related-docker-issues
https://www.ibm.com/support/pages/how-stop-and-start-planning-analytics-workspace-command-line
If nothing helps do this:
– increase to 8 cpu cores on your Microsoft Windows 2016 box
– turn off your anti virus software (McAfee and Symantec)
– turn off Windows Defender Real-Time Protection
– uninstall latest windows patch applied before last reboot of Windows server
– run paw.ps1 command as is, to repair containers (if they exist with strange names)
– remove paw and install it again
– restore your paw database backup with the ./restore.ps1 script
If you do not have a backup of paw database content, you need to do this steps;
– set docker service to manual
– stop cognos and planning service
– reboot the windows server
– after windows have started fine, manually start the docker service
– in powershell, enter docker ps, to monitor that the containers have started
– enter the command ./paw.ps1 stop
– when all containers is stopped, run the ./backup.ps1 command to get the backup zip files created.
Remove paw
https://www.ibm.com/support/knowledgecenter/SSD29G_2.0.0/com.ibm.swg.ba.cognos.tm1_inst.2.0.0.doc/t_paw_uninstall.html
– copy the configuration files from folder d:\ibm\paw\config to a different temp folder
– copy the backup files from folder d:\ibm\paw\backup to a different temp folder
– run ./paw.ps1 stop to stop all containers
– run ./paw.ps1 down –rmi all -v
– run docker ps -a to list all containers
– run docker rm $(docker ps -a -q) to remove all containers
– run docker images to list all images still there
– run docker rmi <image-id> to remove any Hello-World containers
– run docker network ls to show the docker networks
– rename the d:\ibm\paw\log folder to log-old
– create a new empty d:\ibm\paw\log folder
– copy the config/paw.ps1 file to a temp folder
– remove the d:\ibm\paw\config\paw.ps1 file
In case you need to remove docker, follow the steps last on this page https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-docker/configure-docker-daemon
To install docker on Windows 2016, follow this steps https://docs.docker.com/ee/docker-ee/windows/docker-ee/
Run below command to test that docker works, before installing PAW.
docker container run hello-world:nanoserver-sac2016
Reinstall paw
– start the IBM Cognos Planning Analytics Services
– run ./Start.ps1 answer Yes to both questions.
– Inside the Administration web page, enter all the values (check in your old paw.ps1 file for values)
– click on validate
– click on update
– you may get a error, open the newly created d:\ibm\paw\config\paw.ps1 file in notepad++
– add manually the configuration values for $env:PAGatewayHTTPPort=”9580″ and $env:ProxyTimeoutSeconds=”600″ to the config/paw.ps1 file. Check from you backup config/paw.ps1 file what values you used last, when it was working.
https://www.ibm.com/support/knowledgecenter/SSD29G_2.0.0/com.ibm.swg.ba.cognos.tm1_inst.2.0.0.doc/c_paw_config_file.html
– Inside the Administration web page, click on restart
– wait for paw containers to start
– surf to your paw page, does it work, but the folders are missing?
– run the ./restore.ps1 d:\temp\backup\yyyy-mm-dd command to restore from your backup
– surf to your paw page, hopefully all is there.
If you still have issues, please contact IBM Cognos support for better instructions.
There can be new versions of other software that have issues with Docker containers;
https://github.com/docker/for-win/issues/5044
If the system goes down after a few minutes, try to stop the VMware Tools service to see if that helps.
More information:
To prevent this from happening, you must:
Always run the ./paw.ps1 stop script before you restart docker service or reboot the server.
Every week run the ./backup.ps1 script, to make a zip backup of the content inside PAW.
Exclude the docker folders from your anti virus software.
Turn off Real Time Protection.
Set the Docker service to Automatically (Delayed Start)
Ensure there is at least 100 GB free space on the hard disk for docker files.
Add ./shutdown.ps1 script to windows shutdown policy.
https://www.ibm.com/support/knowledgecenter/SSD29G_2.0.0/com.ibm.swg.ba.cognos.tm1_inst.2.0.0.doc/t_paw_shutdown_on_windows_mongo_db.html
More info about Anti-virus software:
https://docs.microsoft.com/en-us/windows-hardware/drivers/ifs/anti-virus-optimization-for-windows-containers
https://docs.docker.com/engine/security/antivirus
https://success.docker.com/article/endpoint-security-for-windows-containers
https://support.symantec.com/us/en/article.tech183201.html
https://support.symantec.com/us/en/article.tech246815.html
https://kc.mcafee.com/corporate/index?page=content&id=KB90041
https://github.com/Microsoft/hcsshim/issues/437
“We find out with premier support that normally when container starts NET_LUID index for an NDIS loopback interface is allocated from registry key HKLM\SYSTEM\ControlSet001\Services\NDIS\IfTypes\24\IfUsedNetLuidIndices and when container stops it will be released.
But if you restart computer where is running containers then those values will not be released and if you have multiple automatically starting containers and scheduled restarts then eventually that registry key value will run out of free indexes and you are not anymore able to start containers”
