How to Resize the Undo Tablespace

1. To Shrink the Undo Tablespace Size.
2. Add Space to the Undo Tablespace.

Solution:

1. To Shrink the Undo Tablespace Size.
Undo space once allocated will be available for reuse but will not be deallocated to the OS. The best way to shrink Undo tablespace is to switch to a new Undo tablespace and drop the old Undo tablespace. The steps are:

a) Create a new undo tablespace of the same size (larger or smaller) depending on your database requirements.

SQL> create undo tablespace undotbs02 datafile '/u01/oracle/prod/undotbs02.dbf' size 5000M;

b) Switch to the new Undo tablespace

ALTER SYSTEM SET UNDO_TABLESPACE = undotbs02 SCOPE=BOTH;

c) Check the status of the undo segments and determine if all the segments in the old undo tablespace are offline.

select tablespace_name , status , count(*) from dba_rollback_segs group by tablespace_name , status;

If there are Undo segments with status other than OFFLINE in the tablespace to be dropped, we need to wait till they become OFFLINE. You may have to wait for the duration of the tuned_undoretention (from v$undostat) to ensure all Undo segments have become OFFLINE.

select status,segment_name from dba_rollback_segs where status not in ("OFFLINE') and tablespace_name=<undo tablespace to be dropped>;

eg:

select status,segment_name from dba_rollback_segs where status not in ("OFFLINE') and tablespace_name='UNDOTBS1';

d). If all the Undo segments in the old Undo tablespace to the dropped is of status OFFLINE, then drop the tablespace.

select tablespace_name , status , count(*) from dba_rollback_segs group by tablespace_name , status;

Verify and then drop:

SQL>Drop tablespace <tablespace_name> including contents and datafiles;

eg:

SQL>Drop tablespace UNDOTBS1 including contents and datafiles;

2. Add Space to the Undo Tablespace.

For increasing / resize undo tablespace there are two options :
a) Resize the existing undo datafile
b) Add new undo datafile to the tablespace.

a) To resize the existing undo datafile:

select tablespace_name T_NAME,file_name, bytes/1024/1024 MB from dba_data_files where tablespace_name =(SELECT UPPER(value) FROM v$parameter WHERE name = 'undo_tablespace') order by file_name;

alter database datafile '<COMPLETE_PATH_OF_UNDO_DBF_FILE>' resize <SIZE>M;

Example:
alter database datafile '/u01/oracle/prod/undotbs02.dbf' resize 1500M;
 

b) Step to add a new datafile:

alter tablespace <UNDO tbs name> ADD DATAFILE '<COMPLETE_PATH_OF_UNDO_DBF_FILE>' size 20M;

Example:
alter tablespace UNDOTBS1 ADD DATAFILE '/u01/oracle/prod/undotbs02.dbf' size 20M;

How to resize redolog files in oracle

step.1

First see the size of the current  redo logs.           
SQL> select group#, bytes, status from v$log;

step.2

Retrieve all the log member names for the groups.                                         
SQL> select group#, member from v$logfile;       

step.3

Create 3 new log groups and name them groups 4, 5, and 6, with different size.                         
SQL> alter database add logfile group 4 '/oracle/dbs/log4PROD.dbf' size 10M;                               
SQL> alter database add logfile group 5 '/oracle/dbs/log5PROD.dbf' size 10M;   
SQL> alter database add logfile group 6 '/oracle/dbs/log6PROD.dbf' size 10M; 

step.4

Run a query to view the v$log status.                                 
SQL> select group#, status from v$log;                                     

step.5

Drop the old redolog files, make sure the status should be INACTIVE.
SQL> alter database drop logfile group 1;                                   
SQL> alter database drop logfile group 2;                                   
SQL> alter database drop logfile group 3;   

step.6

Verify the groups were dropped, and the new groups' sizes are correct.
SQL> select group#, bytes, status from v$log; 

step.7

Take a backup of the database, and delete the files associated with old redolog groups as they are no longer needed.
rm /usr/oracle/dbs/log1PROD.dbf
rm /usr/oracle/dbs/log2PROD.dbf 
rm /usr/oracle/dbs/log3PROD.dbf       

step.8

Monitor the alert.log for the times of redo log switches. Due to increased redo log size, the groups should not switch as frequently under the same load conditions.

Note:

You cannot drop CURRENT and ACTIVE redolog files.

WebLogic Server Cannot Be Started And Throws Following Exception "com.bea.wcp.sip.engine.server.setup.SipAnnotationParsingException"

Applies To:

Oracle WebLogic Server - Version 10.3.6 and later
Information in this document applies to any platform.
***Checked for relevance on28/01/2014***

Symptoms:

After successfully installing WebLogic 10.3.6, WebLogic Admin server cannot be started. The following exception is thrown and can be seen the the Admin Server log file when trying to start the WebLogic server :

####<Mar 19, 2012 2:41:33 PM EDT> <Error> <WLSS.Setup> <np-wlssb1.erp.temple.edu> <AdminServer> <[ACTIVE] ExecuteThread: '11' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <0000JOgFnm3EoIVX21m3Vj1FPrmf000002> <1332182493341> <BEA-331210> <Skip SIP related logic, because error occurs when parsing sip related annotations of "em"
com.bea.wcp.sip.engine.server.setup.SipAnnotationParsingException:
at com.bea.wcp.sip.engine.server.setup.SipAnnotationData.<init>(SipAnnotationData.java:155)
at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:96)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
Caused By: java.lang.OutOfMemoryError: PermGen space
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at weblogic.utils.classloaders.GenericClassLoader.loadClass(GenericClassLoader.java:179)
at weblogic.utils.classloaders.ChangeAwareClassLoader.loadClass(ChangeAwareClassLoader.java:43)
at com.bea.wcp.sip.engine.server.setup.SipAnnotationData.loadClassFromName(SipAnnotationData.java:430)
at com.bea.wcp.sip.engine.server.setup.SipAnnotationData.jarAnnotationParsing(SipAnnotationData.java:287)
at weblogic.application.internal.BaseDeployment$2.next(BaseDeployment.java:671)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)
at weblogic.application.internal.BaseDeployment.activate(BaseDeployment.java:212)
at weblogic.application.internal.EarDeployment.activate(EarDeployment.java:59)
at weblogic.application.internal.DeploymentStateChecker.activate(DeploymentStateChecker.java:161)
at weblogic.deploy.internal.targetserver.AppContainerInvoker.activate(AppContainerInvoker.java:79

Cause:

This issue is due to Out of memory situation in Perm Gen space.

The JVM has run out of space in the permanent generation.

Solution:

1)Increase the Perm Gen space in the java options of the Admin Server in setDomainEnv.sh.

It can be resized according to the requirement using -XX:MaxPermSize and -XX:PermSize JVM Options.

The following needs to be set :

USER_MEM_ARGS=-Xms2048m -Xms2048m -XX:PermSize=512 -XX:MaxPermSize=1024

The values can be set per the requirement

2) Try starting the server after that.

Reference metalink Doc ID 1439171.1

How To Check if a Patch is Applied in 12.2.x using SQL*PLUS 

Applies To:

Oracle Trade Management - Version 12.2 to 12.2.7 [Release 12.2]
Oracle Application Object Library - Version 12.2 to 12.2.7 [Release 12.2]
Oracle Applications DBA - Version 12.2 to 12.2.7 [Release 12.2Cloud to 12.2]
Oracle E-Business Suite Technology Stack - Version 12.2 to 2.2.7 [Release 12.2]
Oracle Applications Manager - Version 12.2 to 12.2.7 [Release 12.2Cloud to 12.2]
Information in this document applies to any platform.

Goal:

The purpose of this document is to explain how to check if a Patch is applied on an environment in Release 12.2.x.

Solution:

In eBusiness Suite (EBS) 12.2.x you cannot query the AD_BUGS table to check if patches have been applied.

The AD_BUGS table may have entries for patches that were applied but later the patching cycle was aborted (not really applied).

To check whether a patch is really applied use the AD_PATCH.IS_PATCH_APPLIED pl/sql function.

Using this API is an alternative method for users without access to Oracle Applications Manager's "Patching and Utilities" feature to determine if a certain patch is applied.

Usage:

select AD_PATCH.IS_PATCH_APPLIED(\'$release\',\'$appltop_id\',\'$patch_no\',\'$language\')
from dual;

Example sql using the APPL_TOP ID for a specific node in a multinode environment (1045 is the APPL_TOP ID):

SELECT adb.bug_number,ad_patch.is_patch_applied('R12', 1045, adb.bug_number)
FROM ad_bugs adb
WHERE adb.bug_number in (20034256);

or for single app tier installations:

select ad_patch.is_patch_applied('R12',-1,20034256) from dual;

expected results:
EXPLICIT = applied
NOT APPLIED = not applied / aborted

Note: If you are sure patch is applied, but showing as not applied then do the following workaround.

The cause is the snapshot is not updated, now it is on development's working in the following bug:

Bug: 23059082 - current view snapshot is not being updated during patch application.

The workaround is to updated the snapshot manually with the steps below:

1. Start adadmin after source the RUN FS env.
2. Select "2. Maintain Applications Files menu" in "AD Administration Main Menu".
3. In "Maintain Applications Files", select "4. Maintain snapshot information".
4. Select "2. Update current view snapshot" in the "Maintain Snapshot Information".
5. Select "1. Update Complete APPL_TOP" in the "Maintain Current View Snapshot Information".

Reference metalink Doc ID 1963046.1

FS_CLONE Phase - R12.2 Online Patching Cycle

-It is a stand-alone command used for filesystem cloning.
-Standard cloning (using adcfgclone.pl) cannot be used to synchronize the run and patch file systems.
-This special phase must be run from the run file system, and cannot be specified with any other phase.
-It should be before the next prepare phase is run.
-It is an alternate to PREPARE phase

FS_CLONE Phase - alternate to PREPARE phase ?

-Create a new patch file system by cloning the run file system.
-This method is useful if the APPL_TOPs have become very unsynchronized (meaning that there would be a large number of delta patches to apply).
-It is a heavyweight process, taking a backup of the entire current patch APPL_TOP and then cloning the run APPL_TOP to create a new patch APPL_TOP.
-A total of least 75 GB of free disk space is required. Also, you will need at least 25 GB of free space in your temporary directory (typically /tmp).
-As well as being resource-intensive, this method is very time-consuming and requires several manual actions by the user.
-It should therefore be used only when necessary.

FS_CLONE Phase – Possible Scenarios

Instance is 12.2.1 and we want to upgrade to 12.2.2

Method 1 : adop phase=prepare + adop phase=apply
adop phase=prepare
adop phase=apply patches=16207672

Start time : Nov 14 2013 08:41:29 End time : Nov 14 2013 16:47:08
Total Time : approximately 8 hours

adop phase=finalize
waiting for downtime approval from customer
adop phase=cutover ( 1 hour downtime )
Released the instance to customer
adop phase=cleanup

Method 2 : adop phase=fs_clone + adop phase=prepare

Here fs_clone phase will take 2 hours 36 Minutes

Start Time : Nov 16 2013 22:06:44 End Time : Nov 17 2013 00:43:29 Total Time : 2 hours 36 Minutes
prepare phase will take 16 Minutes
Start Time : Nov 17 2013 00:44:31 End Time : Nov 17 2013 01:01:01 Total Time : 16 Minutes
It takes approximately 3 hours

Conclusion :

Method 1 takes 8 hours but consumes less resource and less complexity.
Method 2 takes 3 hours and consumes more resource and more complexity.

Online Patching (ADOP) in Oracle EBS R12.2

Online patching is supported by the capability of storing multiple application editions in the database, and the provision of a dual application tier file system. At any given point in time, one of these file systems is designated as run (part of the running system) and the other as patch (either being patched or awaiting the start of the next patching cycle).

Important Points About ADOP
-------------------------------------:

- Adop utility is put under $APPL_TOP_NE/ad/bin. It is a wrapper script which calls internally the perl script $AD_TOP/bin/adzdoptl.pl which does actual work of applying the patch.

-Adop will automatically set its environment as required, but it is the user’s responsibility to set the environment correctly for any other commands that may be run. Set the run edition environment whenever executing commands that you intend to affect the run edition.

- All the phases need to be completed and you can’t skip any of these. For example, if you try to skip prepare phase, you may get error message like “Apply phase can only be run while in a patching cycle, i.e. after prepare phase.”

- After an online patching cycle is started, you should not perform any configuration changes in the run edition file system. Any that are made will not be propagated and will therefore be lost after cutover is complete.

- You should not attempt to clone an Oracle E-Business Suite system while an online patching cycle is in progress.

- The prepare, apply, and fs_clone phases all require at least 10GB of free disk space. All other phases require 1GB of free space. A warning message will be displayed if less than the needed amount is available.

- The directories where you extracted the patches applied in a given patching cycle must be retained, in the same location and with the same contents, until the next prepare phase completes. This is also a requirement for patches applied in a hotpatch session.

- Maintenance Mode is not needed for online patching, and so Maintenance Mode is not available in Oracle E-Business Suite Release 12.2.

For applying a patch in R12.2 you need to use adop and run through all below phases in sequence mentioned below.

1) adop phase=prepare
2) adop phase=apply patches=<patch_number1>,<patch_number2> workers=<number_of_worker>
3) adop phase=finalize workers=<number_of_worker> (called automatically)
4) adop phase=cutover workers=<number_of_worker>
5) adop phase=cleanup (called automatically)

OR

Running all phases in single command:
adop phase=prepare,apply,finalize,cutover,cleanup patches=<patch_number1>,<patch_number2>


1) PREPARE PHASE DETAILS
----------------------------------------:

Used to start a new online patching cycle

How to execute:

A) Set the environment by executing (sourcing) the run file system environment file:
$ source <EBS install base>/EBSapps.env run

B) Verify envirionment
You can confirm that the environment is properly set by examining the relevant environment variables:
$ echo $FILE_EDITION
 run
$ echo $TWO_TASK
 dbSID

C) Download Patches
Download patches to be applied and place then in the $PATCH_TOP directory of your system. This directory is pre-created by the install in the non-editioned file system (fs_ne) and should not be changed.

Important: On a multi-node system with non-shared file systems, you must copy the patch files to each separate $PATCH_TOP directory, so that the patch files are available from the same location on all nodes.

D) Unzip the patch
$ unzip <patch>.zip

E) Run Prepare Command
Prepare the system for patching by running the following command to start a new patching cycle:

$ adop phase=prepare
Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do:

-Gets patching filesystem ready to apply a patch.
-Checks that ETCC has been run against the database Oracle Home and that all required patches are applied.
-synchronize filesystems - If any configuration changes have been made, this will cause the prepare phase to initiate a fs_clone of the current run filesystem to the patch filesystem.
-run autoconfig if necessary.
-check tablespaces for sufficient free space before starting.
-check APPS_FNDFS listener to make sure concurrent requests will launch.
-launch concurrent request - "Online Patching In Progress" (ADZDPATCH).
-synchronize snapshots between run and patch filesystems.
-Prepare the patch file system to ready for the patching
Create Patch Edition
Submit ADZDPATCH concurrent request to stop any conflict requests to submit from patch edition.
-Create session ID in ad_adop _sessions table with prepare_status as 'Y' at the end of prepare.
-Run $AD_TOP/patch/115/bin/txkADOPPreparePhaseSynchronize.pl to synchronize the patches which have been applied to the run APPL_TOP, but not the patch_top.
-During patching phase, adop will use this tns entry to connect to database.


2) APPLY PHASE DETAILS
------------------------------------:

In the apply phase, adop applies the specified patches to the system. Patches are applied to the patch edition of the database and file system.

How to execute:

Example:

$ adop phase=apply patches=1234,7891 workers=8, merge=yes

Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

Where 1234 and 7891 are the patch numbers

What it will do:

-Merging patches - ADOP determines correct options automatically. (if choosen merge option)
-Patch repository location.
-ADOP log directories.
-How to run ADOP unattended by passing the required passwords into the ADOP command.
-How to use adctrl with ADOP.
-Viewing the ADOP patch log file while ADOP is running.
-Apply - Actual patching activity
-Apply the specified patch to Patch File system.
-Update the session ID created by prepare with apply_status as 'P' at the end of apply.
-Insert entry in ad_adop_session_patches table with bug_number as the patch number.


3) FINALIZE PHASE DETAILS
-----------------------------------------:

The finalize phase will be executed while the application is still online. It is used to perform any remaining processing that is needed to ensure the system is ready for the fastest possible cutover.

Used to perform the final patching operations that can

How to execute:

$ adop phase=finalize

Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do:

-Compiles invalids objects in the patch edition.
-Determines DDL statements to execute at cutover.
-Validates that the system is ready for a cutover.
-Gathers statistics for performance.
-Finalize - To prepare DB and APPS for cutover

VERY IMPORTANT 1 : Up to this phase, you can run a special phase called abort, which will undo the changes made so far in the patching cycle. After cutover is complete, however, you cannot do this.

VERY IMPORTANT 2 : In an online patching cycle, the requisite JAR files are initially stored in the $APPL_TOP/admin/<SID>/out directory, and then uploaded into the database during the cutover phase. Therefore, the out directory must not be deleted at least until cutover (next phase) is complete.


4) CUTOVER PHASE DETAILS
-----------------------------------------:

Used to perform the transition to the patched environment. Shuts down application tier services, makes the patch edition the new run edition, and then restarts application tier services. This is the only phase the involves a brief downtime.

Important: No users should remain on the system during cutover, as there will be a short downtime period while the application tier services are restarted. Also, any third-party processes connected to the old run edition of the database should be shut down, or they will be terminated automatically.

How to execute:

$ adop phase=cutover

Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do:

-Once started, the abort phase can not be issued.
-Runs the finalize phase if it was missed.
-Cancels concurrent request - "Online Patching In Progress" (ADZDPATCH).
-Shuts down the internal concurrent manager.
-Shuts down the Nodemanager
-Stops any processes that may have been started on the patch filesystem.
-Stops the RUNNING applications tiers.
-Executes the stored cutover DDL statements prepared during the finalize phase.
-Loads jar files into the database.
-Synchronizes snapshots between run and patch filesystems.
-Broadcasts a message to local users that a cutover is taking place and to re-source their working environment.
-After flipping filesystems for run and patch editions, it will automatically start admin server and middle tier services.

Note:

1.When you run the cutover phase, you can no-longer issue an adop abort command. Your changes are permanent and you will need to restore you instance from backup prior to your adop cycle.

2. (All users must re-source the environment) This broadcast message is only received by unix users that are logged into the server. If anyone is connected though sqlplus or other from their desktop, they will not see this broadcast message.


5) CLEANUP PHASE DETAILS
----------------------------------------:

Important: If you fail to run the cleanup phase explicitly, it will be run automatically on the next prepare cycle, but this will cause a delay in starting your next online patching cycle.

This adop phase is used to remove obsolete code and data from old editions.

How to execute:

$ adop phase=cleanup

Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do:

-Various actions are performed during cleanup, including dropping (removing) obsolete: Crossedition triggers, Seed data, Editioned code objects (covered objects), Indexes, Columns, Editions.
-Three parameters used using cleanup_mode:
QUICK: cleanup performs removal of obsolete crossedition triggers and seed data.
STANDARD: cleanup performs removal of obsolete crossedition triggers and seed data, drops (removes) obsolete editioned code objects (covered objects).
(DEFAULT MODE)
FULL: cleanup performs maximum cleanup, which drops all obsolete code and data from earlier editions
-Cleanup_mode=full: does the same as standard mode, and also drops obsolete columns and old editions.
-If you execute adop phase=cleanup without specifying cleanup_mode=, then you get the default mode.


THERE ARE TWO SPECIAL PHASES:


A) ABORT PHASE DETAILS
--------------------------------------:

Abort PHASE is conditional phase. This phase cannot be specified with any other phase.

If for some reason either the prepare or apply phase failed or gave problems, you can abort the patching cycle at either of these points by running a special phase with the Command. The actions taken will be discarded (rollbacked).

IMPORTANT: This abort command is only available upto (but not including) the cutover phase. After cutover, the system is running on the new edition, and abort is no longer possible for that patching cycle.

How to execute:

The command to perform this operation is:

$ adop phase=abort

Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do:

- Confirms that there is an in-progress online patching cycle, so the abort call is therefore valid.
- Checks for the existence of a patch edition and drops one if it exists.
- Cancels the ADZDPATCH concurrent program, if it is running.
- Deletes the rows inserted for the pending session ID from the ad_adop_sessions and ad_adop_session_patches tables.

VERY IMPORTANT: After running abort, a full cleanup must be performed. The cleanup command is: adop phase=cleanup cleanup_mode=full). This will remove any columns that were added by the patch but are no longer needed because of the abort. If they are not removed, they may cause problems in a later patching cycle. Alternatively, you can run a combined command to abort the patching cycle and perform a full cleanup:

$ adop phase=abort,cleanup cleanup_mode=full
If any attempt was made to apply patches to the patch edition, after abort you must run the fs_clone phase (adop phase=fs_clone) to recreate the patch file system.


B) FS_CLONE PHASE DETAILS
------------------------------------------:

The fs_clone phase is a command (not related to adcfgclone.pl) that is used to synchronize the patch file system with the run file system. The fs_clone phase should only be run when mentioned as part of a specific documented procedure.

How to execute:

The fs_clone phase is run using the following command:

$ adop phase=fs_clone

Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:

What it will do

-This is a separate phase, so we have a new session id.
-It is a stand-alone command used for file system cloning.
-Standard cloning (using adcfgclone.pl) cannot be used to synchronize the run and patch file systems.
-This special phase must be run from the run file system, and cannot be specified with any other phase.
-It should be before the next prepare phase is run.
-It is an alternate to PREPARE phase

FS_CLONE Phase - alternate to PREPARE phase ?

-Create a new patch file system by cloning the run file system.
-This method is useful if the APPL_TOPs have become very unsynchronized (meaning that there would be a large number of delta patches to apply).
-It is a heavyweight process, taking a backup of the entire current patch APPL_TOP and then cloning the run APPL_TOP to create a new patch APPL_TOP.
-A total of least 75 GB of free disk space is required. Also, you will need at least 25 GB of free space in your temporary directory (typically /tmp).
-As well as being resource-intensive, this method is very time-consuming and requires several manual actions by the user.
-It should therefore be used only when necessary.

Note:

There are total 5 columns when ADOP patching is running, they are:

-ADOP (C.Delta.7)
-Session ID: 16
-Node: mqmprod
-Phase: fs_clone
-Log:/u01/fs_ne/220121_0891.log

EBS R12.2 Operating System Environment Variables

Below are some of the important Operating system level environment variables related to R12.2.

$ echo $FILE_EDITION
<shows which file edition you are using, run or patch>

$ echo $RUN_BASE
<shows absolute path to run file system>

$ echo $PATCH_BASE
<shows absolute path to patch file system>

$ echo $NE_BASE
<shows absolute path to non-edition file system>

$ echo $APPL_TOP_NE
<non-editioned appl_top path. Equivalent to $NE_BASE/EBSapps/appl>

$ echo $LOG_HOME
<Application Instance Specific Log Directory>

$ echo $ADOP_LOG_HOME
<Online patching Specific Log Directory. Equivalent to $NE_BASE/EBSapps/log/adop>

$ echo $IAS_ORACLE_HOME
<FMW Web Tier Home Directory>

$ echo $ORACLE_HOME
< 10.1.2 ORACLE_HOME>

$ echo $CONTEXT_FILE
<Source for information populating template files (autoconfig)>

$ echo $EBS_DOMAIN_HOME
<WLS Deployment of Oracle E-Business Suite 12.2 Domain (instance specific)>

$ echo $ADMIN_SCRIPTS_HOME
<Shell scripts to control processes associated to the Applications Instance>

$ echo $EBS_ORACLE_HOME
<Oracle E-Business Suite 12.2 FMW Deployment directory>

$ echo $RW
<10.1.2 reports directory>

$ echo $HOSTNAME
<hostname without domain name>

$ echo $APPS_VERSION
<to get the EBS version>

ADOP Patching Fails Due To ETCC Error During The Apply Phase ' [ERROR] ETCC not run in the database node'

Applies To:

Oracle Applications DBA - Version 12.2 and later
Information in this document applies to any platform.

Symptoms:

E-Business Suite 12.2.x, Online Patching ADOP Utilities issues ETCC errors when applying a patch in Apply phase.

Receiving an ETCC error while executing the following ADOP command:

$ adop phase=apply apply_mode=downtime patches=21865610

ERROR
-----------:
Validating credentials.

Initializing.
Run Edition context :/d01/oracle/inst/fs1/inst/apps/odbdev_oadev/appl/admin/odbdev_oadev.xml
Patch edition context: /d01/oracle/inst/fs2/inst/apps/odbdev_oadev/appl/admin/odbdev_oadev.xml
Reading driver file (up to 50000000 bytes).
Patch file system free space: 45.86 GB

Validating system setup.

Node registry is valid.
[ERROR] ETCC not run in the database node odbdev
The EBS Technology Codelevel Checker needs to be run on the database node.
It is available as Patch 17537119.

Encountered the above errors when performing database validations.
Resolve the above errors and restart adop.


STEPS
------:

The issue can be reproduced at will with the following steps:

1. Running the Apply phase.

Cause:

The ETCC scripts had not been successfully run prior to applying the patch.

1. Patch error:

[ERROR] ETCC not run in the database node odbdev
The EBS Technology Codelevel Checker needs to be run on the database node.
It is available as Patch 17537119.

2. ETCC error:

[oracle@odbdev ~]$ checkMTpatch.sh

+===============================================================+
| Copyright (c) 2005, 2016 Oracle and/or its affiliates. |
| All rights reserved. |
| Oracle E-Business Suite Release 12.2 |
| Middle Tier Technology Codelevel Checker |
+===============================================================+

Using context file from currently set applications environment:
/d01/oracle/odbdev/db/tech_st/12.1.0/appsutil/odbdev_odbdev.xml
Unable to read the value of s_tools_oh from the context file
Verify location and rerun this script.

Solution:

To resolve the issue test the following steps in a development instance and then migrate accordingly:

1. Stop the instance and run AutoConfig.

2. After AutoConfig completes successfully, then run both the ETCC scripts according to Sections 2 & 3 Note 2008451.1 - How To Run The 12.2 EBS Technology Code Level Checker (ETCC):
checkDBpatch.sh
checkMTpatch.sh

3. Reapply the patch with ADOP:
$ adop phase=apply patches=xxxxxxxx

(Note: If the ETCC reports show that some patches need to be applied, at some point apply all of the patches identified in the ETCC reports.)

Reference metalink Doc ID 2190263.1

Resize /DEV/SHM Filesystem In Linux

Step 1: Open /etc/fstab with vi or any text editor of your choice

Step 2:  Locate the line of /dev/shm and use the tmpfs size option to specify your expected size

e.g.
tmpfs /dev/shm tmpfs defaults,size=1500m 0 0
or
tmpfs /dev/shm tmpfs defaults,size=2g 0 0

Step 3: To make change effective immediately, run this mount command to remount the /dev/shm filesystem:

mount -o remount /dev/shm

Step 4: Verify

# df -h

Filesystem            Size  Used Avail Use% Mounted on   

tmpfs                  2G  232M   16G   2% /dev/shm

RMAN 11G : Data Recovery Advisor - RMAN command line example 

What Is the Data Recovery Advisor?

The Data Recovery Advisor is a tool that helps you to diagnose and repair data failures and corruptions. The Data Recovery Advisor analyzes failures based on symptoms and intelligently determines optimal repair strategies. The tool can also automatically repair diagnosed failures.

The Data Recovery Advisor is available from Enterprise Manager (EM) Database Control and Grid Control. You can also use it via the RMAN command-line.

This DRA commands are available within RMAN:

List Failure     # lists the results of previously executed failure assessments. Revalidates existing failures and closes them, if possible.
Advise Failure   # presents manual and automatic repair options
Repair Failure   # automatically fix failures by running optimal repair option, suggested by ADVISE FAILURE. Revalidates existing failures when completed.
Change Failure # enables you to change the status of failures.

Restrctions:

Data Recovery Advisor supports single-instance databases. Oracle Real Application Clusters databases are not supported in 11.1.0.6 -> 11.1.0.8. Data Recovery Advisor cannot use blocks or files transferred from a standby database to repair failures on a primary database. Also, you cannot use Data Recovery Advisor to diagnose and repair failures on a standby database. However, the Data Recovery Advisor does support failover to a standby database as a repair option (as mentioned above).

Examples:

RMAN> list failure;
RMAN> advise failure;
RMAN> repair failure;
RMAN> change failure 522 closed;

Others Command:

RMAN> list failure low;
RMAN> list failure high;
RMAN> list failure critical;
RMAN> repair failure preview;
RMAN> change failure 522 priority low;

Password Not Supplied Error Occurs When Trying to Start WebLogic Using nohup ./startWeblogic.sh 

Applies To:

Oracle WebLogic Server - Version 8.1 and later
Information in this document applies to any platform.
***Checked for relevance on 19-Nov-2015***

Symptoms:

When attempting to start WebLogic Server using the command nohup ./startWeblogic.sh, the following error occurs:

weblogic.security.SecurityInitializationException: Authentication for user denied
  at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.doBootAuthorization(CommonSecurityServiceManagerDelegateImpl.java:966)
  at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1054)
  at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:873)
  at weblogic.security.SecurityService.start(SecurityService.java:141)
  at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
  Truncated. see log file for complete stacktrace
Caused By: javax.security.auth.login.FailedLoginException: [Security:090304]Authentication Failed: User javax.security.auth.login.LoginException: [Security:090301]Password Not Supplied
  at weblogic.security.providers.authentication.LDAPAtnLoginModuleImpl.login(LDAPAtnLoginModuleImpl.java:261)
  at com.bea.common.security.internal.service.LoginModuleWrapper$1.run(LoginModuleWrapper.java:110)
  at com.bea.common.security.internal.service.LoginModuleWrapper.login(LoginModuleWrapper.java:106)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  Truncated. see log file for complete stacktrace

Cause:

When using nohup there is not a chance to enter the username and password at the server startup command prompt. For that reason, if the boot.properties file does not exist in the security folder of the AdminServer, this error is shown in the nohup.out file and the server fails to start.

Solution:

To resolve the issue, follow these steps:

Go to the WL_HOME/user_projects/domains//servers/AdminServer folder.
Create a folder called "security" if it does not already exist.
Create a new file called boot.properties and put the following information inside it:
username=WLS_username
password=WLS_password
Save the changes and restart the servers using nohup startWeblogic.sh again.
Note that the values of boot.properties are automatically encrypted if you are running in development mode. If you are running in production mode, you will need to encrypt these values before starting the server. Use the weblogic.security.Encrypt utility to accomplish this.

Reference metalink Doc ID 1504258.1

Oracle RAC Administration Commands

Shutdown and Start sequence of Oracle RAC

STOP ORACLE RAC (11g)
1. emctl stop dbconsole
2. srvctl stop listener -n racnode1
3. srvctl stop database -d RACDB
4. srvctl stop asm -n racnode1 -f
5. srvctl stop asm -n racnode2 -f
6. srvctl stop nodeapps -n racnode1 -f
7. crsctl stop crs

START ORACLE RAC (11g)
1. crsctl start crs
2. crsctl start res ora.crsd -init
3. srvctl start nodeapps -n racnode1
4. srvctl start nodeapps -n racnode2
5. srvctl start asm -n racnode1
6. srvctl start asm -n racnode2
7. srvctl start database -d RACDB
8. srvctl start listener -n racnode1
9. emctl start dbconsole

-To start and stop oracle clusterware (run as the superuser) :

[root@node1 ~]# crsctl stop crs
[root@node1 ~]# crsctl start crs

-To start and stop oracle cluster resources running on all nodes :

[root@node1 ~]#  crsctl stop cluster -all
[root@node1 ~]#  crsctl start cluster -all

-To check the current status of a cluster :

[oracle@node1~]$ crsctl check cluster
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online

-To check the current status of CRS :

[oracle@node1 ~]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online

-To display the status cluster resources :

[oracle@node1 ~]$ crsctl stat res -t

-To check version of  Oracle Clusterware :

[oracle@node1 ~]$ crsctl query crs softwareversion
Oracle Clusterware version on node [node1] is [11.2.0.4.0]
[oracle@node1 ~]$
[oracle@node1 ~]$ crsctl query crs activeversion
Oracle Clusterware active version on the cluster is [11.2.0.4.0]
[oracle@node1 ~]$ crsctl query crs releaseversion
Oracle High Availability Services release version on the local node is [11.2.0.4.0]

-To check current status of OHASD (Oracle High Availability Services) daemon :

[oracle@node1 ~]$ crsctl check has
CRS-4638: Oracle High Availability Services is onli

-Forcefully deleting resource :

[oracle@node1 ~]$ crsctl delete resource testresource -f

-Enabling and disabling CRS daemons (run as the superuser) :

[root@node1 ~]# crsctl enable crs
CRS-4622: Oracle High Availability Services autostart is enabled.
[root@node1 ~]#
[root@node1 ~]# crsctl disable crs
CRS-4621: Oracle High Availability Services autostart is disabled.

-To check the status of Oracle CRS :

[oracle@node1 ~]$ olsnodes
node1
node2

-To print node name with node number :

[oracle@node1 ~]$ olsnodes -n
node1 1
node2 2

-To print private interconnect address for the local node :

[oracle@node1 ~]$ olsnodes -l -p
node1 192.168.1.101

-To print virtual IP address with node name :

[oracle@node1 ~]$ olsnodes -i
node1 node1-vip
node2 node2-vip
[oracle@node1 ~]$ olsnodes -i node1
node1 node1-vip

-To print information for the local node :

[oracle@node1 ~]$ olsnodes -l
node1
pl

-To print node status (active or inactive) :

[oracle@node1 ~]$ olsnodes -s
node1 Active
node2 Active
[oracle@node1 ~]$ olsnodes -l -s
node1 Active

-To print node type (pinned or unpinned) :

[oracle@node1 ~]$ olsnodes -t
node1 Unpinned
node2 Unpinned
[oracle@node1 ~]$ olsnodes -l -t
node1 Unpinned

-To print clusterware name :

[oracle@node1 ~]$ olsnodes -c
rac-scan

-To display global public and global cluster_interconnect :

[oracle@node1 ~]$ oifcfg getif
eth0  192.168.100.0  global  public
eth1  192.168.1.0  global  cluster_interconnect

-To display the database registered in the repository :

[oracle@gpp4 ~]$ srvctl config database
TESTRACDB

-To display the configuration details of the database :

[oracle@TEST4 ~]$ srvctl config database -d TESTRACDB
Database unique name: TESTRACDB
Database name: TESTRACDB
Oracle home: /home/oracle/product/11.2.0/db_home1
Oracle user: oracle
Spfile: +DATA/TESTRACDB/spfileTESTRACDB.ora
Domain:
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools: TESTRACDB
Database instances: TESTRACDB1,TESTRACDB2
Disk Groups: DATA,ARCH
Mount point paths:
Services: SRV_TESTRACDB
Type: RAC
Database is administrator managed

-To change  policy of database from automatic to manual :

[oracle@TEST4 ~]$ srvctl modify database -d TESTRACDB -y MANUAL

-To change  the startup option of database from open to mount :

[oracle@TEST4 ~]$ srvctl modify database -d TESTDB -s mount

-To start RAC listener :

[oracle@TEST4 ~]$ srvctl start listener

-To display the status of the database :

[oracle@TEST4 ~]$ srvctl status database -d TESTRACDB
Instance TESTRACDB1 is running on node TEST4
Instance TESTRACDB2 is running on node TEST5

-To display the status services running in the database :

[oracle@TEST4 ~]$ srvctl status service -d TESTRACDB
Service SRV_TESTRACDB is running on instance(s) TESTRACDB1,TESTRACDB2

-To check nodeapps running on a node :

[oracle@TEST4 ~]$ srvctl status nodeapps
VIP TEST4-vip is enabled
VIP TEST4-vip is running on node: TEST4
VIP TEST5-vip is enabled
VIP TEST5-vip is running on node: TEST5
Network is enabled
Network is running on node: TEST4
Network is running on node: TEST5
GSD is enabled
GSD is not running on node: TEST4
GSD is not running on node: TEST5
ONS is enabled
ONS daemon is running on node: TEST4
ONS daemon is running on node: TEST5

[oracle@TEST4 ~]$  srvctl status nodeapps -n TEST4
VIP TEST4-vip is enabled
VIP TEST4-vip is running on node: TEST4
Network is enabled
Network is running on node: TEST4
GSD is enabled
GSD is not running on node: TEST4
ONS is enabled
ONS daemon is running on node: TEST4

-To start or stop all instances associated with a database. This command also starts services and listeners on each node :

[oracle@TEST4 ~]$ srvctl start database -d TESTRACDB

-To shut down instances and services (listeners not stopped):

[oracle@TEST4 ~]$ srvctl stop database -d TESTRACDB

You can use -o option to specify startup/shutdown options.
To shutdown immediate database – srvctl stop database -d TESTRACDB -o immediate
To startup force all instances – srvctl start database -d TESTRACDB -o force
To perform normal shutdown – srvctl stop database -d TESTRACDB -i instance racnode1

-To start or stop the ASM instance on racnode01 cluster node :

[oracle@TEST4 ~]$ srvctl start asm -n racnode1
[oracle@TEST4 ~]$ srvctl stop asm -n racnode1

-To display current configuration of the SCAN VIP’s :

[oracle@test4 ~]$ srvctl config scan
SCAN name: vmtestdb.exo.local, Network: 1/192.168.5.0/255.255.255.0/eth0
SCAN VIP name: scan1, IP: /vmtestdb.exo.local/192.168.5.100
SCAN VIP name: scan2, IP: /vmtestdb.exo.local/192.168.5.101
SCAN VIP name: scan3, IP: /vmtestdb.exo.local/192.168.5.102

Refreshing  SCAN VIP’s with new IP addresses from DNS :

[oracle@test4 ~]$ srvctl modify scan -n your-scan-name.example.com

-To stop or start SCAN listener and the  SCAN VIP resources :

[oracle@test4 ~]$ srvctl stop scan_listener
[oracle@test4 ~]$ srvctl start scan_listener
[oracle@test4 ~]$ srvctl stop scan
[oracle@test4 ~]$ srvctl start scan

-To display the status of SCAN VIP’s and SCAN listeners :

[oracle@test4 ~]$ srvctl status scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is running on node test4
SCAN VIP scan2 is enabled
SCAN VIP scan2 is running on node test5
SCAN VIP scan3 is enabled
SCAN VIP scan3 is running on node test5

[oracle@test4 ~]$ srvctl status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is running on node test4
SCAN Listener LISTENER_SCAN2 is enabled
SCAN listener LISTENER_SCAN2 is running on node test5
SCAN Listener LISTENER_SCAN3 is enabled
SCAN listener LISTENER_SCAN3 is running on node test5

-To add/remove/modify SCAN :

[oracle@test4 ~]$ srvctl add scan -n your-scan
[oracle@test4 ~]$ srvctl remove scan
[oracle@test4 ~]$ srvctl modify scan -n new-scan

-To add/remove SCAN listener :

[oracle@test4 ~]$ srvctl add scan_listener
[oracle@test4 ~]$ srvctl remove scan_listener

-To modify SCAN listener port :

srvctl modify scan_listener -p <port_number>
srvctl modify scan_listener -p <port_number>  (reflect changes to the current SCAN listener only)

-To start the ASM instnace in mount state :

ASMCMD> startup --mount

-To shut down ASM instance immediately(database instance must be shut down before the ASM instance is shut down) :

ASMCMD> shutdown --immediate

Use lsop command on ASMCMD to list ASM operations :

ASMCMD > lsop

-To perform quick health check of OCR :

[oracle@test4 ~]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
Version                  :          3
Total space (kbytes)     :     262120
Used space (kbytes)      :       3304
Available space (kbytes) :     258816
ID                       : 1555543155
Device/File Name         :      +DATA
                                    Device/File integrity check succeeded
Device/File Name         :       +OCR
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check bypassed due to non-privileged user

-To dump content of OCR file into an xml :

[oracle@test4 ~]$ ocrdump testdump.xml -xml

-To add or relocate the OCR mirror file to the specified location :

[oracle@test4 ~]$ ocrconfig -replace ocrmirror ‘+TESTDG’
[oracle@test4 ~]$ ocrconfig -replace +CURRENTOCRDG -replacement +NEWOCRDG

-To relocate existing OCR file :

[oracle@test4 ~]$ ocrconfig  -replce ocr ‘+TESTDG’

-To add mirrod disk group for OCR :

[oracle@test4 ~]$ ocrconfig -add +TESTDG

-To remove OCR mirror :

ocrconfig -delete +TESTDG

-To remove the OCR or the OCR mirror :

[oracle@test4 ~]$ ocrconfig -replace ocr

[oracle@test4 ~]$ ocrconfig replace ocrmirror

-To list ocrbackup list :

[oracle@test4 ~]$ ocrconfig -showbackup

test5     2016/04/16 17:30:29     /home/oracle/app/11.2.0/grid/cdata/vmtestdb/backup00.ocr
test5     2016/04/16 13:30:29     /home/oracle/app/11.2.0/grid/cdata/vmtestdb/backup01.ocr
test5     2016/04/16 09:30:28     /home/oracle/app/11.2.0/grid/cdata/vmtestdb/backup02.ocr
test5     2016/04/15 13:30:26     /home/oracle/app/11.2.0/grid/cdata/vmtestdb/day.ocr
test5     2016/04/08 09:30:03     /home/oracle/app/11.2.0/grid/cdata/vmtestdb/week.ocr

-Performs OCR backup manually :

[root@testdb1 ~]# ocrconfig -manualbackup

testdb1     2016/04/16 17:31:42     /votedisk/backup_20160416_173142.ocr     0 

-Changes OCR autobackup directory

[root@testdb1 ~]# ocrconfig -backuploc /backups/ocr

-To verify the integrity of all the cluster nodes:

[oracle@node1]$ cluvfy comp ocr -n all -verbose
Verifying OCR integrity
Checking OCR integrity...

Checking the absence of a non-clustered configuration...
All nodes free of non-clustered, local-only configurations

Checking daemon liveness...

Check: Liveness for "CRS daemon"
  Node Name                             Running?               
  ------------------------------------  ------------------------
  node2                                yes                   
  node1                                yes                   
Result: Liveness check passed for "CRS daemon"

Checking OCR config file "/etc/oracle/ocr.loc"...
OCR config file "/etc/oracle/ocr.loc" check successful

Disk group for ocr location "+DATA/testdb-scan/OCRFILE/registry.255.903592771" is available on all the nodes
Disk group for ocr location "+CRS/testdb-scan/OCRFILE/registry.255.903735431" is available on all the nodes
Disk group for ocr location "+MULTIPLEX/testdb-scan/OCRFILE/registry.255.903735561" is available on all the nodes

Checking OCR backup location "/bkpdisk"
OCR backup location "/bkpdisk" check passed
Checking OCR dump functionality
OCR dump check passed

NOTE:
This check does not verify the integrity of the OCR contents. Execute 'ocrcheck' as a privileged user to verify the contents of OCR.
OCR integrity check passed
Verification of OCR integrity was successful. 

Weblogic Patching Activity On Different Versions

Weblogic Patching Activity On Different Versions

Weblogic 10.3.6 Patching With BSU.SH Utililty
+++++++++++++++++++++++++++++++++++:

Step.1

Set the below homes:
export MW_HOME=/u01/app/oracle/product/fmw11g
export WLS_HOME=$MW_HOME/wlserver_10.3
export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.41.x86_64
export PATH=$JAVA_HOME/bin:$PATH

Step.2

Copy the patches in $MW_HOME/utils/bsu/cache_dir and Unzip the patch and read the README.txt file

Step.3

Apply the patch

syntax:

./bsu.sh -install -patch_download_dir=$MW_HOME/utils/bsu/cache_dir -patchlist=FMJJ -prod_dir=$WLS_HOME -log=/tmp/weblogic_patching.log

Note:
If you get conflicts, you may have to remove previous patches, before attempting to apply the patch again.

Example:

[oracle@bsu]$ ./bsu.sh -install -patch_download_dir=$MW_HOME/utils/bsu/cache_dir -patchlist=FMJJ -prod_dir=$WLS_HOME -log=/tmp/weblogic_patching.log
Checking for conflicts........
Conflict(s) detected - resolve conflict condition and execute patch installation again
Conflict condition details follow:
Patch FMJJ is mutually exclusive and cannot coexist with patch(es): B25A

[oracle@FCISCOSTBAPP1 bsu]$

[oracle@bsu]# ./bsu.sh -remove -patchlist=B25A -prod_dir=$WLS_HOME -log=/tmp/weblogic_patching.log
Checking for conflicts......
No conflict(s) detected

Removing Patch ID: B25A.
Result: Success

After the patch is successfully applied, restart all WebLogic servers.

How to Check the version.

[oracle@bsu]$ . $WLS_HOME/server/bin/setWLSEnv.sh

[oracle@bsu]$ java weblogic.version


Weblogic 12c Patching With Opatch Utility
++++++++++++++++++++++++++++++++:

Step.1

Set the different home paths

a)set the patch_top directory
export PATCH_TOP=/u01/install/weblogic_patch
echo $PATCH_TOP
/u01/install/weblogic_patch

b)set the ORACLE_HOME
export ORACLE_HOME=/u01/app/fmw
echo $ORACLE_HOME
/u01/app/fmw

c) set the java_home
export JAVA_HOME=/u01/app/jdk1.7.0_15
echo $JAVA_HOME
/u01/app/jdk1.7.0_15

d) set the opatch path
export PATH=$ORACLE_HOME/OPatch:$PATH

Step.2

Make sure opatch and unzip will found (which opatch and which unzip)

Step.3

Unzipping the patches and read the README.txt and apply the patch using opatch utility.

syntax:
$opatch apply -jdk $JAVA_HOME

To check the how many patches are applied ?
$opatch lsinventory -jdk $JAVA_HOME

To rollback a weblogic patch
$ opatch rollback -id 15941858 -jdk $JAVA_HOME

Note: After applying a particular in weblogic.

Before Patch:

[oracle@rac1 ]$ java weblogic.version

WebLogic Server 10.3.6.0  Tue Nov 15 08:52:36 PST 2011 1441050
Use 'weblogic.version -verbose' to get subsystem information
Use 'weblogic.utils.Versions' to get version information for all modules

After Patch:

[oracle@rac1 ]$ java weblogic.version

WebLogic Server 10.3.6.0.8 PSU Patch for BUG18040640 THU MARCH 27 15:54:42 IST 2014
WebLogic Server 10.3.6.0  Tue Nov 15 08:52:36 PST 2011 1441050
Use 'weblogic.version -verbose' to get subsystem information

Note: Patching information will be stored in a file called patch-registry.xml under $MW_HOME/patch_wls1036/registry directory

[oracle@rac1 fmw]$ tail -f patch_wls1036/registry/patch-registry.xml
    <version>
      <version>10.3.6.0</version>
      <patchInstallEntry>
        <id>T5F1</id>
        <timestamp>2018-04-21+05:30</timestamp>
        <profile>Default</profile>
      </patchInstallEntry>
    </version>
  </product>
</patchRegistry>