Online Patching (ADOP) in Oracle EBS R12.2

Online patching is supported by the capability of storing multiple application editions in the database, and the provision of a dual application tier file system. At any given point in time, one of these file systems is designated as run (part of the running system) and the other as patch (either being patched or awaiting the start of the next patching cycle).

Important Points About ADOP
-------------------------------------:

- Adop utility is put under $APPL_TOP_NE/ad/bin. It is a wrapper script which calls internally the perl script $AD_TOP/bin/adzdoptl.pl which does actual work of applying the patch.

-Adop will automatically set its environment as required, but it is the user’s responsibility to set the environment correctly for any other commands that may be run. Set the run edition environment whenever executing commands that you intend to affect the run edition.

- All the phases need to be completed and you can’t skip any of these. For example, if you try to skip prepare phase, you may get error message like “Apply phase can only be run while in a patching cycle, i.e. after prepare phase.”

- After an online patching cycle is started, you should not perform any configuration changes in the run edition file system. Any that are made will not be propagated and will therefore be lost after cutover is complete.

- You should not attempt to clone an Oracle E-Business Suite system while an online patching cycle is in progress.

- The prepare, apply, and fs_clone phases all require at least 10GB of free disk space. All other phases require 1GB of free space. A warning message will be displayed if less than the needed amount is available.

- The directories where you extracted the patches applied in a given patching cycle must be retained, in the same location and with the same contents, until the next prepare phase completes. This is also a requirement for patches applied in a hotpatch session.

- Maintenance Mode is not needed for online patching, and so Maintenance Mode is not available in Oracle E-Business Suite Release 12.2.

For applying a patch in R12.2 you need to use adop and run through all below phases in sequence mentioned below.

1) adop phase=prepare
2) adop phase=apply patches=<patch_number1>,<patch_number2> workers=<number_of_worker>
3) adop phase=finalize workers=<number_of_worker> (called automatically)
4) adop phase=cutover workers=<number_of_worker>
5) adop phase=cleanup (called automatically)

OR

Running all phases in single command:
adop phase=prepare,apply,finalize,cutover,cleanup patches=<patch_number1>,<patch_number2>


1) PREPARE PHASE DETAILS
----------------------------------------:

Used to start a new online patching cycle

How to execute:

A) Set the environment by executing (sourcing) the run file system environment file:
$ source <EBS install base>/EBSapps.env run

B) Verify envirionment
You can confirm that the environment is properly set by examining the relevant environment variables:
$ echo $FILE_EDITION
 run
$ echo $TWO_TASK
 dbSID

C) Download Patches
Download patches to be applied and place then in the $PATCH_TOP directory of your system. This directory is pre-created by the install in the non-editioned file system (fs_ne) and should not be changed.

Important: On a multi-node system with non-shared file systems, you must copy the patch files to each separate $PATCH_TOP directory, so that the patch files are available from the same location on all nodes.

D) Unzip the patch
$ unzip <patch>.zip

E) Run Prepare Command
Prepare the system for patching by running the following command to start a new patching cycle:

$ adop phase=prepare
Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do:

-Gets patching filesystem ready to apply a patch.
-Checks that ETCC has been run against the database Oracle Home and that all required patches are applied.
-synchronize filesystems - If any configuration changes have been made, this will cause the prepare phase to initiate a fs_clone of the current run filesystem to the patch filesystem.
-run autoconfig if necessary.
-check tablespaces for sufficient free space before starting.
-check APPS_FNDFS listener to make sure concurrent requests will launch.
-launch concurrent request - "Online Patching In Progress" (ADZDPATCH).
-synchronize snapshots between run and patch filesystems.
-Prepare the patch file system to ready for the patching
Create Patch Edition
Submit ADZDPATCH concurrent request to stop any conflict requests to submit from patch edition.
-Create session ID in ad_adop _sessions table with prepare_status as 'Y' at the end of prepare.
-Run $AD_TOP/patch/115/bin/txkADOPPreparePhaseSynchronize.pl to synchronize the patches which have been applied to the run APPL_TOP, but not the patch_top.
-During patching phase, adop will use this tns entry to connect to database.


2) APPLY PHASE DETAILS
------------------------------------:

In the apply phase, adop applies the specified patches to the system. Patches are applied to the patch edition of the database and file system.

How to execute:

Example:

$ adop phase=apply patches=1234,7891 workers=8, merge=yes

Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

Where 1234 and 7891 are the patch numbers

What it will do:

-Merging patches - ADOP determines correct options automatically. (if choosen merge option)
-Patch repository location.
-ADOP log directories.
-How to run ADOP unattended by passing the required passwords into the ADOP command.
-How to use adctrl with ADOP.
-Viewing the ADOP patch log file while ADOP is running.
-Apply - Actual patching activity
-Apply the specified patch to Patch File system.
-Update the session ID created by prepare with apply_status as 'P' at the end of apply.
-Insert entry in ad_adop_session_patches table with bug_number as the patch number.


3) FINALIZE PHASE DETAILS
-----------------------------------------:

The finalize phase will be executed while the application is still online. It is used to perform any remaining processing that is needed to ensure the system is ready for the fastest possible cutover.

Used to perform the final patching operations that can

How to execute:

$ adop phase=finalize

Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do:

-Compiles invalids objects in the patch edition.
-Determines DDL statements to execute at cutover.
-Validates that the system is ready for a cutover.
-Gathers statistics for performance.
-Finalize - To prepare DB and APPS for cutover

VERY IMPORTANT 1 : Up to this phase, you can run a special phase called abort, which will undo the changes made so far in the patching cycle. After cutover is complete, however, you cannot do this.

VERY IMPORTANT 2 : In an online patching cycle, the requisite JAR files are initially stored in the $APPL_TOP/admin/<SID>/out directory, and then uploaded into the database during the cutover phase. Therefore, the out directory must not be deleted at least until cutover (next phase) is complete.


4) CUTOVER PHASE DETAILS
-----------------------------------------:

Used to perform the transition to the patched environment. Shuts down application tier services, makes the patch edition the new run edition, and then restarts application tier services. This is the only phase the involves a brief downtime.

Important: No users should remain on the system during cutover, as there will be a short downtime period while the application tier services are restarted. Also, any third-party processes connected to the old run edition of the database should be shut down, or they will be terminated automatically.

How to execute:

$ adop phase=cutover

Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do:

-Once started, the abort phase can not be issued.
-Runs the finalize phase if it was missed.
-Cancels concurrent request - "Online Patching In Progress" (ADZDPATCH).
-Shuts down the internal concurrent manager.
-Shuts down the Nodemanager
-Stops any processes that may have been started on the patch filesystem.
-Stops the RUNNING applications tiers.
-Executes the stored cutover DDL statements prepared during the finalize phase.
-Loads jar files into the database.
-Synchronizes snapshots between run and patch filesystems.
-Broadcasts a message to local users that a cutover is taking place and to re-source their working environment.
-After flipping filesystems for run and patch editions, it will automatically start admin server and middle tier services.

Note:

1.When you run the cutover phase, you can no-longer issue an adop abort command. Your changes are permanent and you will need to restore you instance from backup prior to your adop cycle.

2. (All users must re-source the environment) This broadcast message is only received by unix users that are logged into the server. If anyone is connected though sqlplus or other from their desktop, they will not see this broadcast message.


5) CLEANUP PHASE DETAILS
----------------------------------------:

Important: If you fail to run the cleanup phase explicitly, it will be run automatically on the next prepare cycle, but this will cause a delay in starting your next online patching cycle.

This adop phase is used to remove obsolete code and data from old editions.

How to execute:

$ adop phase=cleanup

Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do:

-Various actions are performed during cleanup, including dropping (removing) obsolete: Crossedition triggers, Seed data, Editioned code objects (covered objects), Indexes, Columns, Editions.
-Three parameters used using cleanup_mode:
QUICK: cleanup performs removal of obsolete crossedition triggers and seed data.
STANDARD: cleanup performs removal of obsolete crossedition triggers and seed data, drops (removes) obsolete editioned code objects (covered objects).
(DEFAULT MODE)
FULL: cleanup performs maximum cleanup, which drops all obsolete code and data from earlier editions
-Cleanup_mode=full: does the same as standard mode, and also drops obsolete columns and old editions.
-If you execute adop phase=cleanup without specifying cleanup_mode=, then you get the default mode.


THERE ARE TWO SPECIAL PHASES:


A) ABORT PHASE DETAILS
--------------------------------------:

Abort PHASE is conditional phase. This phase cannot be specified with any other phase.

If for some reason either the prepare or apply phase failed or gave problems, you can abort the patching cycle at either of these points by running a special phase with the Command. The actions taken will be discarded (rollbacked).

IMPORTANT: This abort command is only available upto (but not including) the cutover phase. After cutover, the system is running on the new edition, and abort is no longer possible for that patching cycle.

How to execute:

The command to perform this operation is:

$ adop phase=abort

Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do:

- Confirms that there is an in-progress online patching cycle, so the abort call is therefore valid.
- Checks for the existence of a patch edition and drops one if it exists.
- Cancels the ADZDPATCH concurrent program, if it is running.
- Deletes the rows inserted for the pending session ID from the ad_adop_sessions and ad_adop_session_patches tables.

VERY IMPORTANT: After running abort, a full cleanup must be performed. The cleanup command is: adop phase=cleanup cleanup_mode=full). This will remove any columns that were added by the patch but are no longer needed because of the abort. If they are not removed, they may cause problems in a later patching cycle. Alternatively, you can run a combined command to abort the patching cycle and perform a full cleanup:

$ adop phase=abort,cleanup cleanup_mode=full
If any attempt was made to apply patches to the patch edition, after abort you must run the fs_clone phase (adop phase=fs_clone) to recreate the patch file system.


B) FS_CLONE PHASE DETAILS
------------------------------------------:

The fs_clone phase is a command (not related to adcfgclone.pl) that is used to synchronize the patch file system with the run file system. The fs_clone phase should only be run when mentioned as part of a specific documented procedure.

How to execute:

The fs_clone phase is run using the following command:

$ adop phase=fs_clone

Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do

-This is a separate phase, so we have a new session id.
-It is a stand-alone command used for file system cloning.
-Standard cloning (using adcfgclone.pl) cannot be used to synchronize the run and patch file systems.
-This special phase must be run from the run file system, and cannot be specified with any other phase.
-It should be before the next prepare phase is run.
-It is an alternate to PREPARE phase

FS_CLONE Phase - alternate to PREPARE phase ?

-Create a new patch file system by cloning the run file system.
-This method is useful if the APPL_TOPs have become very unsynchronized (meaning that there would be a large number of delta patches to apply).
-It is a heavyweight process, taking a backup of the entire current patch APPL_TOP and then cloning the run APPL_TOP to create a new patch APPL_TOP.
-A total of least 75 GB of free disk space is required. Also, you will need at least 25 GB of free space in your temporary directory (typically /tmp).
-As well as being resource-intensive, this method is very time-consuming and requires several manual actions by the user.
-It should therefore be used only when necessary.

Note:

There are total 5 columns when ADOP patching is running, they are:

-ADOP (C.Delta.7)
-Session ID: 16
-Node: mqmprod
-Phase: fs_clone
-Log:/u01/fs_ne/220121_0891.log