Application DBA Online Patching (ADOP) faq's

Online Patching Questions:


What is Online Patching?
Online patching is a new patching mechanism that allows the application of patches while the system is up and running, and the users are working as normal.

Which Oracle E-Business Suite releases can Online Patching be used with?
Online patching is used with Oracle E-Business Suite 12.2 and higher.

What types of patch are applied online?
All Oracle E-Business Suite Release 12.2 patches are applied online. This includes one-off patches, patch rollups, consolidated updates and security patches.

What is the Online Patching cycle?
The Online Patching cycle is a sequence of inter-related steps (phases) used to apply patches to an Oracle E-Business Suite system.

What tool is used to apply online patches?
The AD Online Patching (adop) command-line utility is used to manage the Online Patching cycle.

Is there any downtime in Online Patching?
There is a short period of downtime when the application tier services are shut down and restarted. The database remains open all the time. For more details of these topics, see the Online Patching Concepts section below.

Do I upgrade to Release 12.2 with Online Patching?
No. After you upgrade to Oracle E-Business Suite Release 12.2, you will enable Online Patching.

Once I upgrade to Release 12.2, can I still apply patches in the traditional way?
No. All patches for Release 12.2 will be online patches. The traditional, pre-12.2 method of applying patches will not work.

Can I still use adpatch in the Release 12.2 Online Patching environment?
No. While the Oracle tools may internally invoke the adpatch utility, direct user invocation of adpatch is not allowed.

Are there any plans to make Online Patching available with pre-12.2 releases?
No. Online patching can only be used with Release 12.2.


Online Patching Concepts:


What is the Online Patching infrastructure?
The Online Patching infrastructure is the Oracle E-Business Suite Release 12.2 mechanism that allows Oracle E-Business Suite patches to be applied online, while the system is running and users are working as normal. This infrastructure includes database objects and file system components.

Does Online Patching require the 11gR2 Oracle Database Edition Based Redefinition (EBR) feature?
Yes. Online patching depends on the Edition Based Redefinition (EBR) feature that was introduced in the Oracle 11gR2 Database. Most notably, EBR allows editioning of code objects in the database. To do this, it provides new object types such as editions, editioning views, and cross-edition triggers, all of which are part of the Online Patching infrastructure.

What are the phases that make up the Online Patching cycle?
The Online Patching cycle consists of the following phases:
Prepare a virtual copy (patch edition) of the running application (run edition).
Apply patches to the patch edition of the application.
Finalize the system in readiness for the cutover phase.
Cutover to the patch edition and make it the new run edition.
Cleanup obsolete definitions or data to recover space.

What downtime is required during an Online Patching cycle?
The cutover phase requires a short period of downtime (typically a few minutes) for transition tasks such as a restart of the application tier services.

Is any downtime required for the database tier?
No. In fact, the database needs to be up and running during each phase of the Online Patching cycle. However, non-Oracle E-Business Suite delivered database clients need to re-establish their database connection after the Online Patching cutover phase is performed. This includes connections from ODI Agent and Discoverer Server, as well as connection pools from Oracle SOA Suite and any other third party software connected to the Oracle E-Business Suite database.

How does Online Patching work on the application tier?
During Release 12.2 installation, Rapid Install will lay down two copies of the application tier file system. One of the copies will be labeled as the run file system, and the other as the patch file system. Subsequently, when a patch is applied, adop will:
Synchronize the contents of the run file system to the patch file system. This happens during the prepare phase.
Perform patching actions on the patch file system. This happens during the apply phase.
Finally, during the cutover phase, the adop utility restarts the application tier services. The patch file system is then promoted to be the new run file system, and the old run file system becomes the patch file system for the next patching cycle.
Note that a third file system, the non-editioned file system (fs_ne), is created to store files containing data that is needed across all file systems,such as log files.

How do I apply Oracle Fusion Middleware patches in Oracle E-Business Suite Release 12.2?
During the apply phase of an Online Patching cycle, you apply Oracle Fusion Middleware patches to the Oracle homes of the patch edition file system. Then, after the cutover phase is complete, you synchronize the file systems by performing an fs_clone operation. (Also see My Oracle Support Knowledge Document 1355068.1, as listed in Appendix A.)

Is it possible to apply patches in advance of the downtime period required by cutover?
Yes, you can apply online patches at any time in advance of a suitable cutover point, and then run in that state (with cutover pending) for as long as you need to.

Is Online Patching used in a test environment?
Yes. It is required in all Oracle E-Business Suite environments.

Can I use the patch edition for testing purposes?
As a specialized component of the Online Patching infrastructure, the patch edition is not supported for use as a test environment. You should continue to employ a separate, dedicated test environment.

Does the hotpatch option still exist in an Online Patching environment?
Yes, but it is only supported for use with patches that have been designed and tested to be applied in this way. This is because hotpatch mode applies changes to the run edition while this edition is in active use, which may result in one or more of the following issues:
Runtime transactions may fail due to invalid objects.
Runtime transactions may fail due to loss of PL/SQL package state.
Application code and database objects may be temporarily inconsistent.
Seed data may change, and may be temporarily inconsistent.
Tables that are patched will be temporarily inconsistent.
Code and data cached in application tier server memory may be inconsistent with changes made by the hotpatch.
Runtime processing may hold long-term locks on code or data, leading to execution failures in the hotpatch.
Therefore, you should not attempt to apply a patch in hotpatch mode unless explicitly directed to do so by the patch readme.

Can Online Patching be used with database technologies such as Active Dataguard and Flashback?
Yes. Online patching can be used alongside Active Dataguard and Flashback.

What are the key differences between the DBA_OBJECTS, DBA_OBJECTS_AE, and AD_OBJECTS tables?
DBA_OBJECTS shows object information for the current edition, but the STATUS column in this view may show the object as VALID even if the object actually needs to be compiled before use. This is a side effect of how the database handles objects in old editions that have been inherited into the current edition. The issue is tracked in database Bug 17777718.
DBA_OBJECTS_AE is similar to DBA_OBJECTS, but shows object information across all editions. This has the drawback of showing objects in old editions that are no longer accessible to the application.
AD_OBJECTS is the Oracle E-Business Suite workaround to the unreliable STATUS column in DBA_OBJECTS. AD_OBJECTS shows the correct status for each object visible in the current edition. It also shows whether the object is “actual” (a real object) in the current edition, or a "stub" object (the object definition was inherited from a previous edition). You can query AD_OBJECTS to locate objects that need to be recompiled before use:
SQL>select owner, object_name, object_type from ad_objects where status = 'INVALID' order by 1,2,3;
The same logic can be applied by running the script:
SQL>sqlplus apps/apps @$AD_TOP/sql/ADZDSHOWINVALID


Resource Requirements:


What are the space requirements for Online Patching?
Space required on the application tier file system is approximately double that of Release 12.1.3, to support the dual file system architecture. In the database there must be sufficient space to hold a duplicate copy of any code, seed data or table columns that are patched. Refer to Oracle E-Business Suite Installation Guide: Using Rapid Install for additional guidance.

Does Online Patching have any performance implications?
Online patching is designed to run efficiently alongside the running application. While an Online Patching cycle is in progress, there is a small additional processing overhead. The degree of this additional overhead can be managed by controlling the number of parallel workers used by Online Patching.

Will the editioning views on top of the data model have an impact on performance?
No. The editioning views are special views that were specifically designed not to have any performance impact on the applications.
Does Online Patching increase the network port requirements on an Oracle E-Business Suite instance?
Yes. Online patching requires an additional set of network ports for the Oracle WebLogic Server managed servers on the second file system. During the cutover phase, the managed servers run simultaneously on the patch file system and run file system for a brief period, in a rolling transition process.


Patching Features:


Can I apply multiple patches in an Online Patching cycle?
Yes. We recommend that if you have multiple patches to apply, you apply them in a single Online Patching cycle. This will minimize the overall time needed.

Is it possible to abort an Online Patching session?
Yes. Up to cutover, you can run the abort phase to undo the changes made so far in the patching cycle. It is not possible to back out patches once cutover is complete.

Is there any limitation on the locations of the patch and run file systems?
Both the patch and run file systems must be located on the same node (machine).

Is the shared APPL_TOP configuration supported with Online Patching?
Yes. A shared APPL_TOP configuration is supported and recommended for multi-node application tier implementations in Release 12.2.

Does the adop utility support non-interactive mode?
Yes. The adop utility can be executed non-interactively, using an input file to supply the parameters that you would have supplied on the command line. However, using this option means you will have to allow for the downtime of the cutover phase occurring automatically - perhaps at a time that is not convenient.

Is there a mechanism to communicate with users about the downtime during cutover?
Yes. You can use the Oracle Applications Manager Downtime Planning feature that allows you to publish alerts to your end users about planned downtime.

What is duplicated on the second file system used by Online Patching?
The second file system contains a copy of all the components that make up an application tier file system, including:
APPL_TOP - Oracle E-Business Suite code
INST_TOP- Instance Configuration Home
FMW_HOME - Oracle Weblogic Server Home and Oracle E-Business Suite Domain
ORACLE_HOME - Oracle Application Server Home, Forms, Reports
IAS_ORACLE_HOME - Oracle OHS Home
COMMON_TOP - Oracle E-Business Suite Java code, third-party libraries

How does adop work in a multi-node environment?
The adop Online Patching tool uses remote APIs and ssh login to execute patching operations on remote nodes in a multi-node environment. The node that launches adop becomes the ‘master’ node, and the remote nodes are referred to as ‘slaves’.

How do I determine the status of my Online Patching session?
You can run the adop -status command. This will display information that includes phases completed and the time taken. If you want additional details of operations performed, you can run the adop -status -detail command.

What is downtime mode and when can it be used?
To optimize the process of upgrading to E-Business Suite Release 12.2, the AD Delta 5 Release Update Pack introduced downtime mode, which is used as follows:

$ adop phase=apply patches=<patch_number> apply_mode=downtime

Downtime mode does not use an online patching cycle. The process of applying a patch in downtime mode completes more quickly than in online mode, but at the cost of increased system downtime. When applying Oracle E-Business Suite patches in this mode, adop will first confirm that the application tier services are down, and will then proceed to apply the patch to the run edition of the Oracle E-Business Suite database and file system.

Downtime mode is supported for:

All patching (including post-upgrade patching) that forms part of the Release 12.2 upgrade process and is completed before the system is scaled up, the application tier services are started, and users log in to the upgraded system.
Single-node development or test environments, where production support and high availability are not required.
Downtime mode allows the 12.2 upgrade process to be completed as quickly as possible. Once the upgrade is complete and users are online, all subsequent patching on a production system should use online mode, not downtime mode, unless the patch readme states otherwise.

Several restrictions apply to the use of downtime mode:

You cannot validate successful patch application before cutover to the updated code takes place.
There is no capability to abort a failed patch and return to the existing run edition.
Release 12.2 patches are not normally tested in downtime mode.
Use of downtime mode in a multi-node application tier environment is not tested or supported.

Note: Downtime mode is only supported for production use where explicitly documented, or otherwise directed by Oracle.


Applying Online Patches:


Should I source the environment before executing adop?
This is not required, as adop will automatically source the correct environment for each phase. However, you must source the correct environment when using most other admin scripts or tools.

Should I enable Maintenance Mode before patching?
This option is not applicable to Online Patching, and therefore not available in Release 12.2.

Do I need to stop services during an Online Patching cycle?
This is not required. The adop utility will automatically stop and restart the services during the cutover phase. Also see next question.

Which services need to be running for adop to work?
The adop utility requires that the database is up and running when the various Online Patching phases are being executed. However, adop will work if the application tier services are down. When necessary, it will start these services.

What can I do to reduce the time required for cutover?
It is important to distinguish between the time needed for the whole cutover phase, and the downtime period within the phase. The actual downtime (during which users cannot log in) is significantly shorter than the whole phase. To help reduce the overall time taken by cutover, you can do three things:
Run the finalize phase explicitly, to obviate the need for cutover to do so.
Shut down the concurrent managers before running cutover, to avoid having to wait for concurrent requests to complete. Alternatively, ensure no long-running concurrent jobs are submitted while a patching cycle is in progress.
Ensure you are using the maximum number of parallel workers your system will support.

I have already run the finalize step in this patching cycle. Why is cutover running the finalize phase again?
You can run the finalize phase at any point during the Online Patching cycle. If you do not run the finalize phase just before cutover, cutover will run it again to process the latest changes (such as those in an apply command).

What is fs_clone and how is it used?
The command adop phase=fs_clone is a special command that is used to copy the run file system to the patch file system. Also see previous question.

Will AutoConfig and adadmin maintenance tasks such as adrelink be performed online?
Yes, these maintenance tasks will be performed online. The relevant operations will be targeted to the patch file system, and should be performed during a patching cycle. They will not have any impact on the run file system.

Does Online Patching change the way data fix patches are applied to Oracle E-Business Suite 12.2?
Yes. Data fix patches (used to fix transactional data) require special handling. The patch readme will give full instructions.


Customizations and Third Party Integration:


How do I apply or patch my customizations in Oracle E-Business Suite Release 12.2?
You should apply your customizations to the patch edition during the apply phase of an Online Patching cycle. The changes will subsequently be propagated to the new run edition, along with the fixes in the patches that were applied during the patching cycle. For more information, refer to Section 1.4.1: Recommended Approaches for Deploying Custom Code in My Oracle Support Knowledge Document 1577661.1, Developing and Deploying Customizations in Oracle E-Business Suite Release 12.2.

If custom code is installed on a separate database schema, do I have to edition-enable my custom database schema?
The coding standards in the Oracle E-Business Suite Developer's Guide state that the first step to any custom application development is to register the custom Oracle schema with the Oracle E-Business Suite applications. The Online Patching enablement patch enables editioning on all the schemas registered with the application. If you follow this process, your schema will be edition-enabled automatically.

Are there any special considerations for creating custom patches that are compliant with Online Patching?
Yes. There are some special considerations for creating custom patches that are compliant with Online Patching. Refer to the Patching Standards section of Oracle E-Business Suite Developer's Guide.

How is a non Oracle E-Business Suite database schema able to access the Oracle E-Business Suite tables?
Any third-party schema, either from thid-party products or custom code, must access Oracle E-Business Suite tables via the synonyms in the APPS schema. Direct access to Oracle E-Business Suite tables may produce incorrect results.

Is there a utility to report Online Patching requirements violations?
Yes. There are actually three related utilities that you can use here:
Online Patching Enablement Readiness Report, which checks the system's preparedness for Online Patching enablement.
Online Patching Database Compliance Checker (ADZDDBCC.sql), which reports database objects standards violations.
Global Standards Compliance Checker (gscc.pl), which scans the file system for source files that violate the standards.

Can the Online Patching Readiness Report also be run after the Release 12.2 upgrade is complete?
Yes. The Online Patching Readiness Report can be run at any time before or after the upgrade to Release 12.2. This is particularly useful in checking for future violations.