Oracle are recommending we use out of place patching, and AutoUpgrade for patching. Until now we have used a custom patch download script and an ansible patching process. Let’s see how AutoUpgrade works for patching.

Documentation is in the Oracle Database Upgrade guide under AutoUpgrade Patching. As with many Oracle things, it takes some effort to understand how it is supposed to work.

Downloading Auto Upgrade

We download auto upgrade from Oracle Support document KB123450, or from the Oracle website So we can do:

I am looking into running Change Assistant automatically, but also with the option of manually running it if necessary. The problem with this is permissions. We are supposed to log in as our own users and not as admins if we can help it.

Inspiration

The clever people at psadmin.io came up with a method which is pretty good start. There are a couple of refinements to this I felt needed to be made. These mean that the solution is a bit more complicated than the above.

Oracle mentioned that PeopleSoft is supported in Containers. So, with great excitement I decided to try to get it working.

The general approach is as follows:

Install the build environment

This uses the DPK. We need to ensure Podman is installed, and the associated build tools. Then we run the DPK install to create the build environment:

My work laptop runs Windows. I would prefer to run Linux, but some programs I need only work on Windows. Another program I need, Ansible, does not support Windows. I get round this by not running it on Windows, but it would be nice to be able to develop on my laptop to avoid frustration with network slow downs etc.

VSCode supports dev containers. This is great, because development happens in a container. But the officially supported way of doing this is with Docker. Docker Desktop for Windows is not free. However, Podman desktop is, so we can use that.

The Problem

Our databases didn’t stop and start automatically with the OS. The oracle supplied scripts don’t cater for standby databases. Lastly we have one listener per database. Whether this is necessary or not is open to debate, but I would like to make our databases stop and start with the operating system. I’d also like the following features:

• Stop all the databases whether or not they are in the oratab on shutdown.
• Start all database that were stopped on startup.
• Deal with standby and logical standby databases.
• Deal with our unusual listener setup which has one listener per database instance.

Oracle provides scripts called dbstart and dbshut in $ORACLE_HOME/bin which are documented in the administrator guide, but they only start and stop databases in the oratab.

I have had a couple of instances where I have needed to look at which ports are open. On one occasion a firewall change meant I needed to check in a hurry whether ports I needed were open. On another instance, another team raised concerns with some of the TLS ciphers listening on some of the ports in our system.

I do not recommend port scanning across the internet. All these scans were completed within my employers infrastructure as part of my job. If you would like to try these commands and don’t have a similar job to me, I would suggest using devices on your home network such as a raspberry pi, or scanning a VM running on your laptop. The nmap security scanning book has a chapter on legal issues.

We noticed that our test PeopleSoft system was very slow on occasion, such that the load balancer decided it was broken. Sessions were redirected to the webpage we have for the system being in maintenance.

Since the webserver is Weblogic, it runs in a Java Virtual Machine (JVM). The first thing to check is how long the garbage collection pauses are. Fortunately I had garbage collection logging switched on, so I could see that they were over 100 seconds on occasion, which is far too long. This is what my garbage collection log parameters were set to (Java 11).

Normally newer versions of Operating systems have newer packages. But not RedHat when it comes to Ansible. On my workstation:

$ cat /etc/redhat-release 
Red Hat Enterprise Linux release 8.10 (Ootpa)
$ ansible --version
ansible [core 2.16.3]
  config file = /etc/ansible/ansible.cfg
  ...
  python version = 3.12.6
  jinja version = 3.1.2
  libyaml = True

But on the management server:

$ cat /etc/redhat-release 
Red Hat Enterprise Linux release 9.4 (Plow)
$ ansible --version
ansible [core 2.14.17]
  config file = /etc/ansible/ansible.cfg
  ...
  python version = 3.9.18
  jinja version = 3.1.2
  libyaml = True

Wait, what? The older operating system has a newer version of Ansible, and Python?