Wednesday, December 27, 2017

How To Change The Weblogic Server Administrator Password


Applies To:

Oracle WebLogic Server - Version 8.1 and later

IF YOU KNOW CURRENT PASSWORD:

Start the Admin Server and log into /console.
Go to page: Home > Summary of Security Realms > myrealm > Users and Groups > weblogic. and click on tab Passwords.
Enter the new Password.
Restart the server.
If you get a weblogic.security.SecurityInitializationException error, perform these additional steps on every Managed Server (or eventually the Admin Server, too):

Go to folder DOMAIN_HOME/servers/AdminServer/security
Edit the boot.properties file and change the password to the value already entered on the Admin Console. Do this for all the servers in the domain.
Start the Admin Server (Weblogic Server will encrypt the password for you).
Optionally, you can force a Managed Server to connect to the embedded LDAP server on the Administration Server, instead of connecting to the local replicated LDAP server. Follow these steps:

Go to page: Domain > Security > Embedded LDAP page on the Admin Console.
Enable MasterFirst.
Restart the server.


IF YOU DON'T KNOW CURRENT PASSWORD:

If you forget your administrative password and cannot boot the server, the following procedure works for the default authenticator using the embedded LDAP server and only if you have not modified the global Admin role, which by default is granted to the Administrators group. For our example, it is assumed that your server name is AdminServer.  Important: Using the weblogic.security.utils.AdminAccount tool creates a new admin user, but you may lose your existing LDAP data, which includes user, groups, and policy data, so backing up your LDAP folder is recommended before executing the steps.


To reset the password, follow these steps:

Make sure Weblogic Server instance is stopped.

Make a backup of the LDAP folder of the admin server as well as managed servers (you may rename those folders):

$WL_HOME/user_projects/domains/$DOMAIN_NAME/servers/$SERVER_NAME/data/ldap

Set your environment variables by running setDomainEnv.sh (UNIX) or setDomainEnv.cmd (Windows). For UNIX only environments do as follows:
. ./setDomainEnv.sh (Notice the space between the dots) located in $DOMAIN_HOME/<servername>/bin

Create a new initialization file for the default authenticator by running the following command that creates a new DefaultAuthenticatorInit.ldift file in the $DOMAIN_HOME/security subdirectory:
java weblogic.security.utils.AdminAccount <tempadmin> <temppassword> $DOMAIN_HOME/security

Remove the initialized status file DefaultAuthenticatormyrealmInit.initialized from the $DOMAIN_HOME/servers/AdminServer/data/ldap/ subdirectory:
cd WL_HOME/user_projects/domains/base_domain/servers/AdminServer/data/ldap
rm DefaultAuthenticatormyrealmInit.initialized

NOTE: In some cases, it has been necessary to delete ldap directory for this process to work.

Go to folder DOMAIN_HOME/servers/AdminServer/security

Edit the boot.properties file and change the password to the value already used on step 5. Do this for all the servers in the domain.

Start Weblogic Server (Weblogic Server will encrypt the password for you).

IMPORTANT:

Keep in mind that we are not just changing the password for the Admin Console, but we are rather changing it for the Admin User (which may connect in many different ways to Admin Server).

Remember to use the new password (once successfully changed), when when connecting to WLST, start the managed servers or using weblogic.Admin utility.


Reference metalink Doc ID 1082299.1

Tuesday, December 26, 2017

Should We Use The Smart Update (BSU) cache_dir For Downloaded Patches Or Is Another Directory OK? 



Installation Instructions
---------------------------
- unzip downloaded .zip to {MW_HOME}/utils/bsu/cache_dir or any local directory
However, after applying the patch, we see that it is not getting listed in bsu.sh -view status unless and until the patch ID jar and patch_catalog_xxx.xml files are copied to the cache_dir directory. So is it required to copy the patch files to cache_dir?

Solution:

In order to apply the patch, you do not need to use cache_dir. However, as you noted, bsu looks into its cache_dir for the list of patches which are applied, downloaded, etc. So if you don't use cache_dir, bsu isn't going to find the patch where it is expecting to.

So the best practice is to use cache_dir. It may not be absolutely required, but it definitely works best to use cache_dir.

By the way, please also note that the patch-catalog_xxx.xml and patch jars for ALL the patches applied should be in the same place so that bsu can determine which patch-catalog_xxx.xml file is the most current. For that reason as well, you should always use cache_dir. (Please never rename any of the patch-catalog_xxx.xml files -- bsu relies on these files being named correctly.)


NOTE: This article describes use of Smart Update (bsu), which is the patching utility used with WebLogic Server 9.x to 12.1.1. For WLS 12.1.2 and higher, Smart Update has been replaced by the Oracle-standard OPatch utility and you should use OPatch instead.

Monday, December 25, 2017

R12 E-Business Suite Users Report Responsibilities Are Missing From The EBS Home Page, After Login The Responsibility Page Is Blank 


Symptoms:

-Intermittently responsibilities are missing from the Personal Home Page, or after user login the responsibilities homepage is blank.
(or)
-Missing responsibilities for a user in the navigator page during logon.
-After upgrading and patching, the System Administrator responsibility no longer appears in the Navigator.
-Some users cannot see/receive their open notifications due to missing responsibilities.
-Some users get notification e-mails even though their responsibilities are end-dated
-If a user is end-dated, a user's responsibility creates duplicate row
-Newly added responsibility is not displayed until Apache is bounced
-Intermitent issue: After resetting user passwords using the "Login Assistance " > "Forgot Password" link, users do not see their responsibilities in home page and get the following message:
"There are no active responsibilities available for this user"
"System Administrator" responsibility missing

Cause:

Workflow Roles are out of sync.

Solution:

To resolve the issue test the following steps in a development instance and then migrate accordingly:

1. Run the sync responsibility data into the workflow tables concurrent request via the following steps:

a. Navigate -> System Administrator > Concurrent > Request: Submit a New Request
b. Select request "Sync responsibility role data into the WF table"

2. Retest the user login and confirm the responsibilities now appear as expected.

3. If the issue still occurs, test the following set of additional steps to see if the problem is resolved:

a. Run request -> "Compile Security" - For parameter Everything, select Yes
b. Run request -> "Workflow Directory Services User/Role Validation" with: 'Fix dangling user/roles', 'Add missing user/role assignments' and Update WHO columns in WF tables set to "yes"
c. Run request -> "Synchronize WF LOCAL tables"
d. Run request -> "Create FND_RESP WF ROLES"
e. Run request -> "Sync responsibility role data into the WF table."
f. Follow MOS Doc ID 759038.1 How To Clear The Cache Using Functional Administrator.

4. Retest the user login and confirm the responsibilities now appear.

Reference metalink Doc ID 2259375.1

Friday, December 22, 2017

Introduction To Oracle Access Manager (OAM), Oracle Identity Manager (OIM) And Oracle Internet Directory (OID)


Oracle Access Manager (OAM)

Oracle Access Manager is a J2EE application typically deployed on a dedicated managed server in a Weblogic (Application Server) clustered environment. An enterprise typically has many applications for different purposes. Each application typically has its own authentication and authorization functionality.

OAM provides a single point to control all resource grants in an enterprise where multiple applications exist on different platform.

OAM provides:

-Single Sign On (SSO)
-Authentication
-Authorization
-Real time session management
-Auditing
-Policy Administration
-Flaws in conventional security model.

Individual authentication/authorization for each independent application in the enterprise. .net, J2EE, SAP, WebCenter etc. All application have their own authentication and authorization mechanism.

-Effective Security
-Cost
-Inconsistence
-Security Complainces
-Ease for users (Single Sign On)
-Governance, Support and Management
-One of the web server will have OAM-Agent. Other web servers will be redirected to this OAM-Agent via a reverse proxy. Hence, we don't need OAM-Agent on each Web Server.

The request goes to the OAM agent which redirects the request to OAM which in turn challenges the user for user/pwd. Once user/pwd is provided the OAM goes to the LDAP (AD or OID) to authenticate the user. Once the user is authenticated the webgate opens the gate to the underlying corresponding web server.


Oracle Identity Manager (OIM)

An Oracle identity management (OIM) provides a mechanism for implementing the user management aspects of a corporate policy. It can also be a means to audit users and their access privileges. The OIM enables enterprises to manage the entire user life cycle across all enterprise resources both within and beyond a firewall.

OIM application is deployed on Weblogic Managed Server in domain and There is second managed server for SOA Infra on same domain. OIM utilizes this SOA to process User Provisioning for workflow based approval. In real world all User provisioning has approval chain.

OIM server is a J2EE application. User provisioning is done in OIM. The OIM integrates this with all the other applications.

OIM does life cycle management of an identity (generally a user, e.g employee).

Lets take an example of an employee joining an organizaiton. He/She needs access to various applications in the organization. The HR typically creates the employee in HRMS on the joining date. The manager raises various user ids creations for this new employee for email, timesheet app, crm, leave mgmt app etc. With OIM this provisionting can be done automatically or manually at single point.

OIM provides a unified access control for all the applications in the enterprize. Once the employee quits, the manager need only to log onto OIM and delete (soft/hard) the employee from various applications.

OIM integrates with other application using SOA suite with respective JCA adapters.


Oracle Internet Directory (OID)

This is a directory of objects. For e.g in case of employees in an organization, this directory will hold employees details like name, designation, enterprize roles, applicaiton specific roles, security credentials like password, password reminder questions etc. An online directory is a specialized database that stores and retrieves collections of information about objects. The information can represent any resources that require management

-This is typically a single source of truth for information about employees in an organization.

-Various applications access OID to authenticate and authorize users. Typically, OID is integrated with OAM.

-OID is Oracle's LDAP implementation. Active Directory or AD is similar implementation for the same solution from Microsoft.

-OID generally uses oracle database for storage of all the said information above.

-The information in the directory is available to different clients, such as single sign-on solutions, email clients, and database applications. Clients communicate with a directory server by means of the Lightweight Directory Access Protocol (LDAP). Oracle Internet Directory is an LDAP directory that uses an Oracle Database for storage.

Thursday, December 21, 2017

ORA-20001: Invalid Column Name or Duplicate Columns/Column groups/expressions In method_opt


Symptoms:

Gather Schema Statistics" program reported following errors in request log files:

ORA-20001: invalid column name or duplicate columns/column groups/expressions in method_opt

Statistics Errors:
stats on table FND_CP_GSM_OPP_AQTBL is locked
Error #1: ERROR: While GATHER_TABLE_STATS:
object_name=GL.JE_BE_LINE_TYPE_MAP******
Error #2: ERROR: While GATHER_TABLE_STATS:
object_name=GL.JE_BE_LOGS***ORA-20001: invalid column name or duplicate columns/column
groups/expressions in method_opt***
Error #1: ERROR: While GATHER_TABLE_STATS:
object_name=GL.JE_BE_LINE_TYPE_MAP***ORA-20001: invalid column name or duplicate columns/column groups/expressions in method_opt***
Error #2: ERROR: While GATHER_TABLE_STATS:
object_name=GL.JE_BE_LOGS***ORA-20001: invalid column name or duplicate columns/column groups/expressions in method_opt***
Error #3: ERROR: While GATHER_TABLE_STATS:
object_name=GL.JE_BE_VAT_REP_RULES***ORA-20001: invalid column name or duplicate columns/column groups/expressions in method_opt***
Error #4: ERROR: While GATHER_TABLE_STATS:  object_name=FII.FII_FIN_ITEM_HIERARCHIES***ORA-20001: invalid column name or duplicate columns/column groups/expressions in method_opt***
groups/expressions in method_opt***

Cause:

1 ) There are duplicate rows on FND_HISTOGRAM_COLS table for JE_BE_LINE_TYPE_MAP table. Because of this problem, FND_STATS tries to gather histogram information using wrong command and it fails with ora-20001 errors.

The following SQL should return one row, not two:
SQL> show user
USER is "APPS"

SQL>select column_name, nvl(hsize,254) hsize from FND_HISTOGRAM_COLS where table_name = 'JE_BE_LINE_TYPE_MAP' order by column_name;

COLUMN_NAME HSIZE
------------------------------ ----------
SOURCE 254
SOURCE 254

SQL>
Since there are two rows in histograms table, FND_STATS creates following command to gather statistics on table 'JE_BE_LINE_TYPE_MAP' :


Solution:                                                                                                                                         

Find out all duplicates and/or obsolete rows in FND_HISTOGRAM_COLS and delete one of them logged in as the applsys user. Remember to take backup of the FND_HISTOGRAM_COLS table before deleting any data.

SQL> select table_name, column_name, count(*) from FND_HISTOGRAM_COLS group by table_name, column_name having count(*) > 1;

TABLE_NAME                     COLUMN_NAME                      COUNT(*)
------------------------------ ------------------------------ ----------
JE_BE_LOGS                     DECLARATION_TYPE_CODE                   2
JE_FR_DAS_010                  TYPE_ENREG                              2
JE_FR_DAS_010_NEW              TYPE_ENREG                              2
JE_BE_LINE_TYPE_MAP            SOURCE                                  2
JE_BE_VAT_REP_RULES            SOURCE                                  2
JE_BE_VAT_REP_RULES            LINE_TYPE                               2
JE_BE_VAT_REP_RULES            VAT_REPORT_BOX                          2
JG_ZZ_SYS_FORMATS_ALL_B        JGZZ_EFT_TYPE                           2

-- Use above results on the following SQL to delete duplicates

SQL> delete from FND_HISTOGRAM_COLS
where table_name = '&TABLE_NAME'
and  column_name = '&COLUMN_NAME'
and rownum=1;
Enter value for table_name: JE_BE_LOGS
old   2: where table_name = '&TABLE_NAME'
new   2: where table_name = 'JE_BE_LOGS'
Enter value for column_name: DECLARATION_TYPE_CODE
old   3: and  column_name = '&COLUMN_NAME'
new   3: and  column_name = 'DECLARATION_TYPE_CODE'

1 row deleted.

SQL> /
Enter value for table_name: JE_FR_DAS_010
old   2: where table_name = '&TABLE_NAME'
new   2: where table_name = 'JE_FR_DAS_010'
Enter value for column_name: TYPE_ENREG
old   3: and  column_name = '&COLUMN_NAME'
new   3: and  column_name = 'TYPE_ENREG'

1 row deleted.

SQL> /
Enter value for table_name: JE_FR_DAS_010_NEW
old   2: where table_name = '&TABLE_NAME'
new   2: where table_name = 'JE_FR_DAS_010_NEW'
Enter value for column_name: TYPE_ENREG
old   3: and  column_name = '&COLUMN_NAME'
new   3: and  column_name = 'TYPE_ENREG'

1 row deleted.

SQL> /

Repeat same till you delete all the duplicate rows and then commit.
SQL> select table_name, column_name, count(*)
from FND_HISTOGRAM_COLS
group by table_name, column_name
having count(*) > 1;

no rows selected
SQL>
SQL> select column_name, nvl(hsize,254) hsize
from FND_HISTOGRAM_COLS
where table_name = 'JE_BE_LINE_TYPE_MAP'
order by column_name;

COLUMN_NAME                         HSIZE
------------------------------ ----------
SOURCE                                254

SQL>
SQL> commit;

Commit complete.
SQL>


RC-40000: Unable To Startup The Database


Error:

RC-40000 will occur when you gave wrong password for adpreclone

[oracletest@jancln]$ adpreclone.pl dbTier

                     Copyright (c) 2002 Oracle Corporation
                        Redwood Shores, California, USA

                        Oracle Applications Rapid Clone

                                 Version 12.0.0

                      adpreclone Version 120.20.12010000.2

Enter the APPS User Password:*******
Running:
perl /u02/oracle/PROD/db/tech_st/11.1.0/appsutil/bin/adclone.pl java=/u02/oracle/PROD/db/tech_st/11.1.0/appsutil/jre mode=stage stage=/u02/oracle/PROD/db/tech_st/11.1.0/appsutil/clone component=dbTier method=CUSTOM dbctx=/u02/oracle/PROD/db/tech_st/11.1.0/appsutil/PROD_ruh-erpdb01.xml showProgress
APPS Password :

Beginning database tier Stage - Sat Apr  8 10:05:53 2017

/u02/oracle/PROD/db/tech_st/11.1.0/appsutil/jre/bin/java -Xmx600M -DCONTEXT_VALIDATED=false  -Doracle.installer.oui_loc=/u02/oracle/PROD/db/tech_st/11.1.0/oui -classpath /u02/oracle/PROD/db/tech_st/11.1.0/lib/xmlparserv2.jar:/u02/oracle/PROD/db/tech_st/11.1.0/jdbc/lib/ojdbc6.jar:/u02/oracle/PROD/db/tech_st/11.1.0/appsutil/java:/u02/oracle/PROD/db/tech_st/11.1.0/oui/jlib/OraInstaller.jar:/u02/oracle/PROD/db/tech_st/11.1.0/oui/jlib/ewt3.jar:/u02/oracle/PROD/db/tech_st/11.1.0/oui/jlib/share.jar:/u02/oracle/PROD/db/tech_st/11.1.0/oui/jlib/srvm.jar:/u02/oracle/PROD/db/tech_st/11.1.0/jlib/ojmisc.jar   oracle.apps.ad.clone.StageDBTier -e /u02/oracle/PROD/db/tech_st/11.1.0/appsutil/PROD_ruh-erpdb01.xml -stage /u02/oracle/PROD/db/tech_st/11.1.0/appsutil/clone -tmp /tmp -method CUSTOM    -showProgress
APPS Password :
Log file located at /u02/oracle/PROD/db/tech_st/11.1.0/appsutil/log/PROD_ruh-erpdb01/StageDBTier_04081005.log

AutoClone needs to start the database up to generate cloning information
  -     50% completed       RC-40000: Fatal: Unable to start up the database

ERROR while running Stage...
Sat Apr  8 10:06:19 2017

ERROR while running perl /u02/oracle/PROD/db/tech_st/11.1.0/appsutil/bin/adclone.pl java=/u02/oracle/PROD/db/tech_st/11.1.0/appsutil/jre mode=stage stage=/u02/oracle/PROD/db/tech_st/11.1.0/appsutil/clone component=dbTier method=CUSTOM dbctx=/u02/oracle/PROD/db/tech_st/11.1.0/appsutil/PROD_ruh-erpdb01.xml showProgress ...
Sat Apr  8 10:06:19 2017
[oracletest@jancln]$


adpreclone logfile message:


    Connection could not be obtained; returning null

-------------------ADX Database Utility Finished---------------

RC-40000: Unable to start up the database
Raised by oracle.apps.ad.clone.StageDatabase
RC-40000: Unable to start up the database
Raised by oracle.apps.ad.clone.StageDatabase


Cause:

Because of wrong credentials, apps user account has locked. You cannot connect apps until you have unlocked apps user account.

[oracletest@jancln]$ sqlplus / as sysdba

SQL*Plus: Release 11.1.0.7.0 - Production on Sat Apr 8 10:14:50 2017

Copyright (c) 1982, 2008, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> connect apps/apps
ERROR:
ORA-28000: the account is locked


Warning: You are no longer connected to ORACLE.
SQL> exit


Solution:


[oracletest@jancln]$ sqlplus / as sysdba

SQL*Plus: Release 11.1.0.7.0 - Production on Sat Apr 8 10:15:33 2017

Copyright (c) 1982, 2008, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL>
SQL> select username, account_status from dba_users where username like 'APPS';

USERNAME                       ACCOUNT_STATUS
------------------------------ --------------------------------
APPS                           LOCKED(TIMED)

SQL> alter user apps account unlock;

User altered.

SQL> select username, account_status from dba_users where username like 'APPS';

USERNAME                       ACCOUNT_STATUS
------------------------------ --------------------------------
APPS                           OPEN

SQL> exit

[oracletest@jancln]$ adpreclone.pl dbTier

                     Copyright (c) 2002 Oracle Corporation
                        Redwood Shores, California, USA

                        Oracle Applications Rapid Clone

                                 Version 12.0.0

                      adpreclone Version 120.20.12010000.2

Enter the APPS User Password:*******
Running:
perl /u02/oracle/PROD/db/tech_st/11.1.0/appsutil/bin/adclone.pl java=/u02/oracle/PROD/db/tech_st/11.1.0/appsutil/jre mode=stage stage=/u02/oracle/PROD/db/tech_st/11.1.0/appsutil/clone component=dbTier method=CUSTOM dbctx=/u02/oracle/PROD/db/tech_st/11.1.0/appsutil/PROD_ruh-erpdb01.xml showProgress
APPS Password :

Beginning database tier Stage - Sat Apr  8 10:21:14 2017

/u02/oracle/PROD/db/tech_st/11.1.0/appsutil/jre/bin/java -Xmx600M -DCONTEXT_VALIDATED=false  -Doracle.installer.oui_loc=/u02/oracle/PROD/db/tech_st/11.1.0/oui -classpath /u02/oracle/PROD/db/tech_st/11.1.0/lib/xmlparserv2.jar:/u02/oracle/PROD/db/tech_st/11.1.0/jdbc/lib/ojdbc6.jar:/u02/oracle/PROD/db/tech_st/11.1.0/appsutil/java:/u02/oracle/PROD/db/tech_st/11.1.0/oui/jlib/OraInstaller.jar:/u02/oracle/PROD/db/tech_st/11.1.0/oui/jlib/ewt3.jar:/u02/oracle/PROD/db/tech_st/11.1.0/oui/jlib/share.jar:/u02/oracle/PROD/db/tech_st/11.1.0/oui/jlib/srvm.jar:/u02/oracle/PROD/db/tech_st/11.1.0/jlib/ojmisc.jar   oracle.apps.ad.clone.StageDBTier -e /u02/oracle/PROD/db/tech_st/11.1.0/appsutil/PROD_ruh-erpdb01.xml -stage /u02/oracle/PROD/db/tech_st/11.1.0/appsutil/clone -tmp /tmp -method CUSTOM    -showProgress
APPS Password :
Log file located at /u02/oracle/PROD/db/tech_st/11.1.0/appsutil/log/PROD_ruh-erpdb01/StageDBTier_04081021.log

  |      0% completed

Completed Stage...
Sat Apr  8 10:21:26 2017
[oracletest@jancln]$

Automatic Tuning Of Undo_retention Causes Space Problems

Automatic Tuning Of Undo_retention Causes Space Problems


Check Whether the undo is automatically managed by the database by checking the following instance parameter: UNDO_MANAGEMENT=AUTO

Whether the undo tablespace is fixed in size:

SQL> SELECT autoextensible FROM dba_data_files WHERE tablespace_name='<UNDO_TABLESPACE_NAME>'

This returns "NO" for all the undo tablespace datafiles.
The undo tablespace is already sized such that it always has more than enough space to store all the undo generated within the undo_retention time, and the in-use undo space never exceeds the undo tablespace warning alert threshold (see below for the query to show the thresholds).
The tablespace threshold alerts recommend that the DBA add more space to the undo tablespace:

SQL> SELECT creation_time, metric_value, message_type, reason, suggested_action FROM dba_outstanding_alerts WHERE object_name='<UNDO_TABLESPACE_NAME>';

This returns a suggested action of: "Add space to the tablespace".

Or,

This recommendation has been reported in the past but the condition has now cleared:
SQL> SELECT creation_time, metric_value, message_type, reason, suggested_action, resolution FROM dba_alert_history WHERE object_name='<UNDO_TABLESPACE_NAME>';

The undo tablespace in-use space exceeded the warning alert threshold at some point in time. To see the warning alert percentage threshold, issue:

SQL> SELECT object_type, object_name, warning_value, critical_value FROM dba_thresholds WHERE object_type='TABLESPACE';

To see the (current) undo tablespace percent of space in use:

SQL> SELECT
         ((SELECT (NVL(SUM(bytes),0))
           FROM dba_undo_extents
           WHERE tablespace_name='<UNDO_TABLESPACE_NAME>'
           AND status IN ('ACTIVE','UNEXPIRED')) * 100)/       
         (SELECT SUM(bytes)
          FROM dba_data_files
          WHERE tablespace_name='<UNDO_TABLESPACE_NAME>')
         "PCT_INUSE"
     FROM dual;

Cause:

The cause of this problem has been identified in:
Bug:5387030 - AUTOMATIC TUNING OF UNDO_RETENTION CAUSING SPACE PROBLEMS
It is caused by a wrong calculation of the tuned undo retention value.
Bug:5387030 is fixed in RDBMS 11.1.

Solution:

To implement a solution for Bug:5387030, please execute any of the below alternative solutions:

Upgrade to 11.1 in which Bug:5387030 is fixed

OR

Apply patchset release 10.2.0.4 or higher in which Bug:5387030 is fixed.

OR

Download and apply interim Patch:5387030, if available for your platform and RDBMS release. To check for conflicting patches, please use the MOS Patch Planner Tool. If no patch is available, file a Service Request through My Oracle Support for your specific Oracle version and platform.

OR

Use any of the following workarounds:

1. Set the AUTOEXTEND and MAXSIZE attributes of each datafile of the undo tablespace in such a way that they are autoextensible and the MAXSIZE is equal to the current size (so the undo tablespace now has the AUTOEXTEND attribute but does not autoextend):
SQL> ALTER DATABASE DATAFILE '<datafile_flename>' AUTOEXTEND ON MAXSIZE <current_size>

With this setting, V$UNDOSTAT.TUNED_UNDORETENTION is not calculated based on a percentage of the undo tablespace size. Instead it is set to the maximum of (MAXQUERYLEN secs + 300) and UNDO_RETENTION.

2. Set the following instance parameter:
_smu_debug_mode=33554432

With this setting, V$UNDOSTAT.TUNED_UNDORETENTION is not calculated based on a percentage of the fixed size undo tablespace. Instead it is set to the maximum of (MAXQUERYLEN secs + 300) and UNDO_RETENTION.

3. Set the following instance parameter:
_undo_autotune = false

With this setting, V$UNDOSTAT (and therefore V$UNDOSTAT.TUNED_UNDORETENTION) is not maintained and the undo retention used is based on the UNDO_RETENTION instance parameter.

NOTE: This means you loose all advantages in having automatic undo management and is not an ideal long term fix.
NOTE: Even with the patch fix installed, the autotuned retention can still grow under certain circumstances. The fix attempts to throttle back how aggressive that autotuning will be. Options 2 and 3 may be needed to get around this aggressive growth in some environments.

Refernce metalink Doc ID 420525.1

Clearing Cache In Oracle E-Business Suite


Apache / iAS Cache clear for 11.5.9 or 11.5.10.x

- shutdown iAS server
- go to $OA_HTML (for 11.5.9) or $COMMON_TOP (for 11.5.10.x) directory
- backup the directory _pages and delete its contents by running for instance:

rm -rf $COMMON_TOP/_pages/*

- for modplsql caches remove contents of $IAS_ORACLE_HOME/Apache/modplsql/cache directory
- restart iAS server


Cache clear on middle tier for R12


Note: Clearing _pages directory is no longer a recommended solution.
Clearing the _pages in R12 creates blank login page issue, as in R12 the jsp files does not get compiled automatically.

From Forntend
--------------

- go to "Functional Administrator" responsibility

- select Core Services => Caching Framework => Global Configuration => Clear cache

From backend
------------

- adopmnctl.sh stopall

- rm -fr $INST_TOP/ora/10.1.3/j2ee/oacore/persistence/*

- rm -fr $INST_TOP/ora/10.1.3/j2ee/oafm/persistence/*

- rm -fr $INST_TOP/ora/10.1.3/j2ee/forms/persistence/*

- adopmnctl.sh startall


Cabo
-----

Images and style sheets can be corrupted or out of sync in the cabo caches,you may need to clear the related directories after backup:

- $OA_HTML/cabo/images/cache
- $OA_HTML/cabo/styles/cache

ERROR: ORA-00257: Archiver Error. Connect Internal Only, Until Freed.

ERROR: ORA-00257: Archiver Error. Connect Internal Only, Until Freed.


Cause:

The archiver process received an error while trying to archive a redo log. If the problem is not resolved soon, the database will stop executing transactions. The most likely cause of this message is the destination device is out of space to store the redo log file.

Action:

Check archiver trace file for a detailed description of the problem. Also verify that the device specified in the initialization parameter ARCHIVE_LOG_DEST is set up properly for archiving.alter database flashback on

There is two possible way to solution.
1. without increasing DB_RECOVERY_FILE_DEST_SIZE.
2. by increasing DB_RECOVERY_FILE_DEST_SIZE.

Without increasing DB_RECOVERY_FILE_DEST_SIZE.
1. Check whether the database is in archive log mode and automatic archiving is enabled.

 SQL> archive log list;
 Database log mode              Archive Mode
 Automatic archival             Enabled
 Archive destination            USE_DB_RECOVERY_FILE_DEST
 Oldest online log sequence     29
 Next log sequence to archive   31
 Current log sequence           31

2. If archive destination is defined by USE_DB_RECOVERY_FILE_DEST, find the archive destination by:

SQL> show parameter db_recovery_file_dest;
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_recovery_file_dest                string      C:\oracle\product\10.2.0/flash_recovery_area
db_recovery_file_dest_size           big integer 2G
Check what the value for db_recovery_file_dest_size.

3. Find the space used in flash recovery area by using following SQL:

select name, round(space_limit/1048576) As space_limit,round(space_used/1048576) As space_used
from v$RECOVERY_FILE_DEST;

4. If SPACE_USED is equal to SPACE_LIMIT of db_recovery_file_dest, move the archive logs to different destination.

5. Archive all the log files

SQL> alter system archive log all;
6. Just switch the logs to verify

SQL> alter system switch logfile;
7. DB_RECOVERY_FILE_DEST_SIZE is to delete (archive log) files from DB_RECOVERY_FILE_DEST if you are sure you have backups and the archived logs are no longer necessary.

$rman target /
RMAN>delete archivelog until time 'SYSDATE-1';
or,
RMAN>delete archivelog all;
By increasing DB_RECOVERY_FILE_DEST_SIZE.
1. See the path of flash recovery area.

SQL> show parameter db_recovery_file_dest;
2. Increase the Flash Recovery Area

SQL> ALTER SYSTEM SET db_recovery_file_dest_size='10G' SCOPE=BOTH;
Sytem Altered.

If User Complain For Long Running Request's, Do The Following.

If User Complain For Long Running Request's, Do The Following.


- Check the Statistics,History Runs of that program & Based on this timelines we need to decide the long running Request.

- Check The load on DB node to ensure that High Resource Usage is not the Main cause.i troubleshoot

- Check DB locks in The Databse To Ensure that This Session is not blocked by any other session.

- Check the CM status and ICM status.

- Check any Locks on DB particularly related to that program.

- Check any INVALID objects related to that program.

- Check any incompatible requests/programs are running for that program.

-Login as system administrator responsibility and query the long running request /program. Navigate to concurrent program define query that long running request in the same form select incompatible requests .

-If any incompatible request is running then find the user name who is running that program inform them regarding the incompatibilities


Concurrent Manager Troubleshooting


1. First check the CM is up or not by using below any one procedure.

i. ps –ef|grep FNDLIBR
ii. adcmctl.sh status apps/appsPWD
iii. Login as system administrator responsibilities go to the below navigation Concurrent -> manager -> Administer -> see the  Actual and target for all the managers

2. If CM in down check the internal manager Logfile in $APPLCSF/APPLLOG location for errors.

3. If any errors related to FNDFS then check the Application listener STATUS , if it is not running start it.

4. If CM is up/running then log file and output file are not able see the user from the front end then check the Application listener is status if it is not running then start it.

Concurrent Manager Not Starting After Cloning

Concurrent Manager Not Starting After Cloning


Solution:

- Shutdown the application services

EXEC FND_CONC_CLONE.SETUP_CLEAN;
COMMIT;
EXIT;

- Ran AutoConfig on all tiers, firstly on the DB tier and then the APPS tiers.

- Start the application services

Actually FND_CONC_CLONE.SETUP_CLEAN clears up the tables with nodes information and when we run autoconfig, it repopulates these tables with correct node information.

Blank Login Page Issue EBS R12


Solution:

- The first thing you need to check always is DB and listener is running fine.
- In case you are are not aware then I must tell you, its better to check if "apps" user is working fine.
- check is the Alert Log, as there may be issues related to Database.

Later comes the Apache logs. Location is "$LOG_HOME/ora/10.1.3/Apache".

The files are
error_log
access_log,
check the latest ones.

Following the steps to resolve the above issue:

1- Stop all application services       
$INST_TOP/admin/scripts/adstpall.sh

2- Remove any lock files from following folders:
$INST_TOP/ora/10.1.3/j2ee/oacore/persistence/oacore_default_group_1
$INST_TOP/ora/10.1.3/j2ee/oafm/persistence/oafm_default_group_1
$INST_TOP/ora/10.1.3/j2ee/forms/persistence/forms_default_group_1

3- Compile the JSP files with the following command
cd $FND_TOP/patch/115/bin
perl ojspCompile.pl --compile --flush -p 2

4 -Run Autoconfig first on the DB Tier, and then on APPS Tier

5 -Start appl services

$ADMIN_SCRIPTS_HOME/adstrtall.sh

Gathering Schema Stats ORA-20000: Index "FA"."FA_RX_LOV_U1" or Partition of Such Index Is Unusable


Symptoms:

When running Gathering Schema Statistics

The following error is returned..

ORA-20000: index "FA"."FA_RX_LOV_U1" or partition of such index is in unusab

SQL> EXEC FND_STATS.gather_schema_statistics (schemaname=>'ALL', estimate_percent=>dbms_stats.auto_sample_size, options=>'GATHER AUTO');
BEGIN FND_STATS.gather_schema_statistics (schemaname=>'ALL', estimate_percent=>dbms_stats.auto_sample_size, options=>'GATHER AUTO'); END;

*
ERROR at line 1:
ORA-20000: index "FA"."FA_RX_LOV_U1" or partition of such index is in unusable
state
ORA-06512: at "SYS.DBMS_STATS", line 26211
ORA-06512: at line 1
ORA-06512: at "APPS.FND_STATS", line 711
ORA-06512: at "APPS.FND_STATS", line 1836
ORA-06512: at "APPS.FND_STATS", line 1183
ORA-06512: at "APPS.FND_STATS", line 1275
ORA-06512: at "APPS.FND_STATS", line 815
ORA-06512: at line 1

The issue can be reproduced with Running command to gather schema stats for "ALL"

Cause:

The index is unusable.

The following select statements confirm this:

SQL> select owner, index_name, status, last_analyzed from dba_indexes where index_name = 'FA_RX_LOV_U1';

SQL> select index_owner, index_name, status, last_analyzed from dba_ind_partitions where index_name = 'FA_RX_LOV_U1';


Results from 1st SQL above.


OWNER   INDEX_NAME          STATUS         LAST_ANALYZED
--------   --------------------  --------------  ------------------
 FA          FA_RX_LOV_U1     UNUSABLE


Rows from 2nd SQL above.

No rows returned.

Solution:

1) Rebuild the index using the following command.

SQL> alter index fa.FA_RX_LOV_U1 rebuild online;

2) Re-run the gathering of schema statistics

Reference metalink Doc ID 2013352.1

Wednesday, December 20, 2017

Application DBA Online Patching (ADOP) faq's


Online Patching Questions:


What is Online Patching?
Online patching is a new patching mechanism that allows the application of patches while the system is up and running, and the users are working as normal.

Which Oracle E-Business Suite releases can Online Patching be used with?
Online patching is used with Oracle E-Business Suite 12.2 and higher.

What types of patch are applied online?
All Oracle E-Business Suite Release 12.2 patches are applied online. This includes one-off patches, patch rollups, consolidated updates and security patches.

What is the Online Patching cycle?
The Online Patching cycle is a sequence of inter-related steps (phases) used to apply patches to an Oracle E-Business Suite system.

What tool is used to apply online patches?
The AD Online Patching (adop) command-line utility is used to manage the Online Patching cycle.

Is there any downtime in Online Patching?
There is a short period of downtime when the application tier services are shut down and restarted. The database remains open all the time. For more details of these topics, see the Online Patching Concepts section below.

Do I upgrade to Release 12.2 with Online Patching?
No. After you upgrade to Oracle E-Business Suite Release 12.2, you will enable Online Patching.

Once I upgrade to Release 12.2, can I still apply patches in the traditional way?
No. All patches for Release 12.2 will be online patches. The traditional, pre-12.2 method of applying patches will not work.

Can I still use adpatch in the Release 12.2 Online Patching environment?
No. While the Oracle tools may internally invoke the adpatch utility, direct user invocation of adpatch is not allowed.

Are there any plans to make Online Patching available with pre-12.2 releases?
No. Online patching can only be used with Release 12.2.


Online Patching Concepts:


What is the Online Patching infrastructure?
The Online Patching infrastructure is the Oracle E-Business Suite Release 12.2 mechanism that allows Oracle E-Business Suite patches to be applied online, while the system is running and users are working as normal. This infrastructure includes database objects and file system components.

Does Online Patching require the 11gR2 Oracle Database Edition Based Redefinition (EBR) feature?
Yes. Online patching depends on the Edition Based Redefinition (EBR) feature that was introduced in the Oracle 11gR2 Database. Most notably, EBR allows editioning of code objects in the database. To do this, it provides new object types such as editions, editioning views, and cross-edition triggers, all of which are part of the Online Patching infrastructure.

What are the phases that make up the Online Patching cycle?
The Online Patching cycle consists of the following phases:
Prepare a virtual copy (patch edition) of the running application (run edition).
Apply patches to the patch edition of the application.
Finalize the system in readiness for the cutover phase.
Cutover to the patch edition and make it the new run edition.
Cleanup obsolete definitions or data to recover space.

What downtime is required during an Online Patching cycle?
The cutover phase requires a short period of downtime (typically a few minutes) for transition tasks such as a restart of the application tier services.

Is any downtime required for the database tier?
No. In fact, the database needs to be up and running during each phase of the Online Patching cycle. However, non-Oracle E-Business Suite delivered database clients need to re-establish their database connection after the Online Patching cutover phase is performed. This includes connections from ODI Agent and Discoverer Server, as well as connection pools from Oracle SOA Suite and any other third party software connected to the Oracle E-Business Suite database.

How does Online Patching work on the application tier?
During Release 12.2 installation, Rapid Install will lay down two copies of the application tier file system. One of the copies will be labeled as the run file system, and the other as the patch file system. Subsequently, when a patch is applied, adop will:
Synchronize the contents of the run file system to the patch file system. This happens during the prepare phase.
Perform patching actions on the patch file system. This happens during the apply phase.
Finally, during the cutover phase, the adop utility restarts the application tier services. The patch file system is then promoted to be the new run file system, and the old run file system becomes the patch file system for the next patching cycle.
Note that a third file system, the non-editioned file system (fs_ne), is created to store files containing data that is needed across all file systems,such as log files.

How do I apply Oracle Fusion Middleware patches in Oracle E-Business Suite Release 12.2?
During the apply phase of an Online Patching cycle, you apply Oracle Fusion Middleware patches to the Oracle homes of the patch edition file system. Then, after the cutover phase is complete, you synchronize the file systems by performing an fs_clone operation. (Also see My Oracle Support Knowledge Document 1355068.1, as listed in Appendix A.)

Is it possible to apply patches in advance of the downtime period required by cutover?
Yes, you can apply online patches at any time in advance of a suitable cutover point, and then run in that state (with cutover pending) for as long as you need to.

Is Online Patching used in a test environment?
Yes. It is required in all Oracle E-Business Suite environments.

Can I use the patch edition for testing purposes?
As a specialized component of the Online Patching infrastructure, the patch edition is not supported for use as a test environment. You should continue to employ a separate, dedicated test environment.

Does the hotpatch option still exist in an Online Patching environment?
Yes, but it is only supported for use with patches that have been designed and tested to be applied in this way. This is because hotpatch mode applies changes to the run edition while this edition is in active use, which may result in one or more of the following issues:
Runtime transactions may fail due to invalid objects.
Runtime transactions may fail due to loss of PL/SQL package state.
Application code and database objects may be temporarily inconsistent.
Seed data may change, and may be temporarily inconsistent.
Tables that are patched will be temporarily inconsistent.
Code and data cached in application tier server memory may be inconsistent with changes made by the hotpatch.
Runtime processing may hold long-term locks on code or data, leading to execution failures in the hotpatch.
Therefore, you should not attempt to apply a patch in hotpatch mode unless explicitly directed to do so by the patch readme.

Can Online Patching be used with database technologies such as Active Dataguard and Flashback?
Yes. Online patching can be used alongside Active Dataguard and Flashback.

What are the key differences between the DBA_OBJECTS, DBA_OBJECTS_AE, and AD_OBJECTS tables?
DBA_OBJECTS shows object information for the current edition, but the STATUS column in this view may show the object as VALID even if the object actually needs to be compiled before use. This is a side effect of how the database handles objects in old editions that have been inherited into the current edition. The issue is tracked in database Bug 17777718.
DBA_OBJECTS_AE is similar to DBA_OBJECTS, but shows object information across all editions. This has the drawback of showing objects in old editions that are no longer accessible to the application.
AD_OBJECTS is the Oracle E-Business Suite workaround to the unreliable STATUS column in DBA_OBJECTS. AD_OBJECTS shows the correct status for each object visible in the current edition. It also shows whether the object is “actual” (a real object) in the current edition, or a "stub" object (the object definition was inherited from a previous edition). You can query AD_OBJECTS to locate objects that need to be recompiled before use:
SQL>select owner, object_name, object_type from ad_objects where status = 'INVALID' order by 1,2,3;
The same logic can be applied by running the script:
SQL>sqlplus apps/apps @$AD_TOP/sql/ADZDSHOWINVALID


Resource Requirements:


What are the space requirements for Online Patching?
Space required on the application tier file system is approximately double that of Release 12.1.3, to support the dual file system architecture. In the database there must be sufficient space to hold a duplicate copy of any code, seed data or table columns that are patched. Refer to Oracle E-Business Suite Installation Guide: Using Rapid Install for additional guidance.

Does Online Patching have any performance implications?
Online patching is designed to run efficiently alongside the running application. While an Online Patching cycle is in progress, there is a small additional processing overhead. The degree of this additional overhead can be managed by controlling the number of parallel workers used by Online Patching.

Will the editioning views on top of the data model have an impact on performance?
No. The editioning views are special views that were specifically designed not to have any performance impact on the applications.
Does Online Patching increase the network port requirements on an Oracle E-Business Suite instance?
Yes. Online patching requires an additional set of network ports for the Oracle WebLogic Server managed servers on the second file system. During the cutover phase, the managed servers run simultaneously on the patch file system and run file system for a brief period, in a rolling transition process.


Patching Features:


Can I apply multiple patches in an Online Patching cycle?
Yes. We recommend that if you have multiple patches to apply, you apply them in a single Online Patching cycle. This will minimize the overall time needed.

Is it possible to abort an Online Patching session?
Yes. Up to cutover, you can run the abort phase to undo the changes made so far in the patching cycle. It is not possible to back out patches once cutover is complete.

Is there any limitation on the locations of the patch and run file systems?
Both the patch and run file systems must be located on the same node (machine).

Is the shared APPL_TOP configuration supported with Online Patching?
Yes. A shared APPL_TOP configuration is supported and recommended for multi-node application tier implementations in Release 12.2.

Does the adop utility support non-interactive mode?
Yes. The adop utility can be executed non-interactively, using an input file to supply the parameters that you would have supplied on the command line. However, using this option means you will have to allow for the downtime of the cutover phase occurring automatically - perhaps at a time that is not convenient.

Is there a mechanism to communicate with users about the downtime during cutover?
Yes. You can use the Oracle Applications Manager Downtime Planning feature that allows you to publish alerts to your end users about planned downtime.

What is duplicated on the second file system used by Online Patching?
The second file system contains a copy of all the components that make up an application tier file system, including:
APPL_TOP - Oracle E-Business Suite code
INST_TOP- Instance Configuration Home
FMW_HOME - Oracle Weblogic Server Home and Oracle E-Business Suite Domain
ORACLE_HOME - Oracle Application Server Home, Forms, Reports
IAS_ORACLE_HOME - Oracle OHS Home
COMMON_TOP - Oracle E-Business Suite Java code, third-party libraries

How does adop work in a multi-node environment?
The adop Online Patching tool uses remote APIs and ssh login to execute patching operations on remote nodes in a multi-node environment. The node that launches adop becomes the ‘master’ node, and the remote nodes are referred to as ‘slaves’.

How do I determine the status of my Online Patching session?
You can run the adop -status command. This will display information that includes phases completed and the time taken. If you want additional details of operations performed, you can run the adop -status -detail command.

What is downtime mode and when can it be used?
To optimize the process of upgrading to E-Business Suite Release 12.2, the AD Delta 5 Release Update Pack introduced downtime mode, which is used as follows:

$ adop phase=apply patches=<patch_number> apply_mode=downtime

Downtime mode does not use an online patching cycle. The process of applying a patch in downtime mode completes more quickly than in online mode, but at the cost of increased system downtime. When applying Oracle E-Business Suite patches in this mode, adop will first confirm that the application tier services are down, and will then proceed to apply the patch to the run edition of the Oracle E-Business Suite database and file system.

Downtime mode is supported for:

All patching (including post-upgrade patching) that forms part of the Release 12.2 upgrade process and is completed before the system is scaled up, the application tier services are started, and users log in to the upgraded system.
Single-node development or test environments, where production support and high availability are not required.
Downtime mode allows the 12.2 upgrade process to be completed as quickly as possible. Once the upgrade is complete and users are online, all subsequent patching on a production system should use online mode, not downtime mode, unless the patch readme states otherwise.

Several restrictions apply to the use of downtime mode:

You cannot validate successful patch application before cutover to the updated code takes place.
There is no capability to abort a failed patch and return to the existing run edition.
Release 12.2 patches are not normally tested in downtime mode.
Use of downtime mode in a multi-node application tier environment is not tested or supported.

Note: Downtime mode is only supported for production use where explicitly documented, or otherwise directed by Oracle.


Applying Online Patches:


Should I source the environment before executing adop?
This is not required, as adop will automatically source the correct environment for each phase. However, you must source the correct environment when using most other admin scripts or tools.

Should I enable Maintenance Mode before patching?
This option is not applicable to Online Patching, and therefore not available in Release 12.2.

Do I need to stop services during an Online Patching cycle?
This is not required. The adop utility will automatically stop and restart the services during the cutover phase. Also see next question.

Which services need to be running for adop to work?
The adop utility requires that the database is up and running when the various Online Patching phases are being executed. However, adop will work if the application tier services are down. When necessary, it will start these services.

What can I do to reduce the time required for cutover?
It is important to distinguish between the time needed for the whole cutover phase, and the downtime period within the phase. The actual downtime (during which users cannot log in) is significantly shorter than the whole phase. To help reduce the overall time taken by cutover, you can do three things:
Run the finalize phase explicitly, to obviate the need for cutover to do so.
Shut down the concurrent managers before running cutover, to avoid having to wait for concurrent requests to complete. Alternatively, ensure no long-running concurrent jobs are submitted while a patching cycle is in progress.
Ensure you are using the maximum number of parallel workers your system will support.

I have already run the finalize step in this patching cycle. Why is cutover running the finalize phase again?
You can run the finalize phase at any point during the Online Patching cycle. If you do not run the finalize phase just before cutover, cutover will run it again to process the latest changes (such as those in an apply command).

What is fs_clone and how is it used?
The command adop phase=fs_clone is a special command that is used to copy the run file system to the patch file system. Also see previous question.

Will AutoConfig and adadmin maintenance tasks such as adrelink be performed online?
Yes, these maintenance tasks will be performed online. The relevant operations will be targeted to the patch file system, and should be performed during a patching cycle. They will not have any impact on the run file system.

Does Online Patching change the way data fix patches are applied to Oracle E-Business Suite 12.2?
Yes. Data fix patches (used to fix transactional data) require special handling. The patch readme will give full instructions.


Customizations and Third Party Integration:


How do I apply or patch my customizations in Oracle E-Business Suite Release 12.2?
You should apply your customizations to the patch edition during the apply phase of an Online Patching cycle. The changes will subsequently be propagated to the new run edition, along with the fixes in the patches that were applied during the patching cycle. For more information, refer to Section 1.4.1: Recommended Approaches for Deploying Custom Code in My Oracle Support Knowledge Document 1577661.1, Developing and Deploying Customizations in Oracle E-Business Suite Release 12.2.

If custom code is installed on a separate database schema, do I have to edition-enable my custom database schema?
The coding standards in the Oracle E-Business Suite Developer's Guide state that the first step to any custom application development is to register the custom Oracle schema with the Oracle E-Business Suite applications. The Online Patching enablement patch enables editioning on all the schemas registered with the application. If you follow this process, your schema will be edition-enabled automatically.

Are there any special considerations for creating custom patches that are compliant with Online Patching?
Yes. There are some special considerations for creating custom patches that are compliant with Online Patching. Refer to the Patching Standards section of Oracle E-Business Suite Developer's Guide.

How is a non Oracle E-Business Suite database schema able to access the Oracle E-Business Suite tables?
Any third-party schema, either from thid-party products or custom code, must access Oracle E-Business Suite tables via the synonyms in the APPS schema. Direct access to Oracle E-Business Suite tables may produce incorrect results.

Is there a utility to report Online Patching requirements violations?
Yes. There are actually three related utilities that you can use here:
Online Patching Enablement Readiness Report, which checks the system's preparedness for Online Patching enablement.
Online Patching Database Compliance Checker (ADZDDBCC.sql), which reports database objects standards violations.
Global Standards Compliance Checker (gscc.pl), which scans the file system for source files that violate the standards.

Can the Online Patching Readiness Report also be run after the Release 12.2 upgrade is complete?
Yes. The Online Patching Readiness Report can be run at any time before or after the upgrade to Release 12.2. This is particularly useful in checking for future violations. 

Changes In 12.1.3 : While Doing Clone In Newly 12.1.3 Instance


- While running adcfgclone on appsTier it will ask hostname(FQDN).

- After providing all the inputs to adcfgclone.pl it will ask to start the services or not.

- When you are running cmclean.sql it will give notification to stop all the concurrent managers.




How To Configure Weblogic Username & Password In boot.properties?


After installing Weblogic need to give username and password each time you will start the Weblogic (./startWeblogic &)

To overcome this problem we have another way to configure the boot.preperties file in the security folder. Sometimes we dont have security folder available in Adminserver folder, so simply create boot.properties file with in security folder with username and password.

E.g.:

$ mkdir $DOMAIN_HOME/base_domain/servers/AdminServer/security
vi boot.properties
username=<username>
password=<password>

:wq!
$ cd $DOMAIN_HOME/base_domain/servers/AdminServer/security
$ cat > boot.properties
username=<username>
password=<password>

How To Start Weblogic Server In Background Even After Closing Terminal?

How To Start Weblogic Server In Background Even After Closing Terminal?


$nohup ./startWeblogic.sh &

hit the Enter command and open the nohup.out file and check the status.

Example:

tail -f nohup.out

Thursday, December 14, 2017

Active Data Guard 


The Active Data Guard Option is an evolution of Data Guard technology, it is designed for a specific purpose:

-To improve production database performance for critical transactions.
-Active Data Guard enables read-only access to a physical standby database while Redo Apply is active.
-Queries and reports can be offloaded from the production system to a synchronized physical standby database
-All queries at the standby database return up-to-date results.
-Unique corruption detection and automatic repair
-Offload read-only workloads to an up-to-date standby database
-Database rolling upgrades and standby-first patching using physical standby
-Zero data loss protection across any distance
-Enable incremental backups on an active standby
-Load balancing and service management across replicated databases
-An Active Data Guard Option license must be purchased in addition to Oracle Enterprise Edition in order to utilize these new capabilities

Benefits:

-Absolutely the best protection for Oracle Database
-Highest performance data recovery protection without compromises
-Zero data loss data recovery protection across any distance without impacting performance
-Comprehensive protection against planned and unplanned outages




Oracle Data Guard

Oracle Data Guard


Oracle Data Guard is the disaster recovery solution. Protects our production database from disasters, reducing the work load on it and more effective use it.

Simple Example:

Your primary database is running and you want to reduce downtime because of unplanned outages. You create a replica of this primary database (termed as standby database).You regularly ship redo generated in the primary database to standby database and apply it there. So that is our ‘Data Guard’ standby database and it is in a continuous state of recovery, validating and applying redo to remain in sync with the primary database.

Oracle Dataguard provides enhancements in different versions:

->ORACLE 8i

-Read-Only Standby Database
-Managed recovery
-Remote archiving redo log files

->ORALCE 9i

-“Zero Data Loss” Integration
-Data Guard Broker and Data Guard Manager GUI
-Swithcover and Failover operations
-Automatical synchronous
-Logical Standby Database
-Maximum Protection

->ORACLE 10g

-Real-Time Apply
-Forced support for Oracle RAC
-Fast-Start Failover
-sAsynchronous redo transfer
-Flashback Database

->ORACLE 11g

-Active Standby Database (Active Data Guard)
-Snapshot Standby
-Heterogeneous platform support (Production –Linux, Standby – Windows)
-Active Data Guard is another extension of physical standby, which is open in read-only mode, but applying the logs in the background. This keeps the database up-to-date and allows Real-Time Queries During Managed Recovery process.
-Zero data loss DR protection across any distance without impacting performance


DATA GUARD 11g SYNCHRONOUS REDO TRANSFER PROCESS ARCHITECTURE (SYNC)-ZERO DATA LOSS






Sync Process flow;

1 – The user initiates a transaction. This transaction is written to a redo buffer. When the user commit the transaction then the LGWR process writes it redo log file.

2 – LNS (logwriter Network Service) reports to RFS (Remote File Service) committed redo. RFS writes to standby redo log file. If we use physical standby, the MRP (Managed Recovery Process) will apply to standby database . In  Logical Standby this is made by LSP (Logical 

Standby Process) .

3 – RFS sends information to LNS that data is processed successfully. LNS transmits this information to LGWR . Finally, commit information is send to the user that initiated the transaction (transaction) .

Data transfer is ensured by the synchronous redo transfer. But there is a disadvantage. If a network failure occurs between production database (Primary) and the standby database  or  Primary database can not access  to the standby database then the primary database  will hang until standby response. In other words, the primary database can not serve. To avoid such situation, We need to use “NET_TIMEOUT” parameter. With this parameter you can determine the timeout period. In case of an outage, Primary waits until the timeout period and will continue to serve when timeout period expires. Default value of this parameter in 10g is 180s and in 11g is 30s.


DATA GUARD 11g ASYNCHRONOUS REDO TRANSFER PROCESS ARCHITECTURE (ASYNC)




Asynchronous reod transfer flow;

1 – The user initiates a transaction. This transaction is written to a redo buffer. When the user commit the transaction then the LGWR process writes it redo log file.

2 – LNS (logwriter Network Service) reports to RFS (Remote File Service) committed redo. RFS writes to standby redo log file. If we use physical standby, the MRP (Managed Recovery Process) will apply to standby database . In  Logical Standby this is made by LSP (Logical

Standby Process) .

3 – Once Redo Buffer is recycled, LNS  automatically reads redo log files and begins to send redo from log files.RFS doesn’t send information to LNS that data is processed successfully.

The most common used process architecture. Asynchronous redo transfer does not guarantee zero data loss. The system has recovered with minimal data loss.

Monday, November 20, 2017

Upgrade Oracle 11g R2 (11.2.0.4) Database to Oracle 12c (12.2.0.1)


Step 1:

Upgrade Path for Oracle Database to Oracle 12c Release 1 (12.1)

For example:

If you are upgrading from 11.2.0.1 or 11.1.0.6, then you must first upgrade to Oracle Database 11g Release 2 (11.2.0.2).
If you are upgrading from 10.2.0.2, 10.2.0.3, 10.2.0.4, or 10.1.0.5, then you must first upgrade to 10.2.0.5 or later.
For release 9.2.0.8, you must first upgrade to an intermediate Oracle Database release, as follows: 9.2.0.8 -> 11.2.0.2 or 11.2.0.3 -> 12.1

Upgrading 11.2.0.4 to 12.2.0.1, on Linux (x64_86) platform

Step 2 :

Requirements and recommendations for Source  database 

Either take a cold or hot backup of the source database (advisable to have cold backup).

a) Take a backup using RMAN
(or)
b) Perform Cold Backup  ( if  your database is in NOARCHIVELOG mode)


Step.3:

check source database before upgrade

-Ensure That No Files Need Media Recovery Before Upgrading
-Ensure That No Files Are in Backup Mode Before Upgrading
-Purge the Database Recycle Bin Before Upgrading
-Disable all batch and cron jobs
-location of datafiles, redo logs and control files. Also take a backup of all configuration files like listener.ora, tnsnames.ora, etc. from $ORACLE_HOME

Step.4:

Run the new preupgrade check script preupgrd.sql, in  11.2.0.4 environment – it will generate 3 files in $ORACLE_HOME/cfgtoollogs/$SID/preupgrade:

SQL> @/u01/app/oracle/product/12.1.0.2/rdbms/admin/preupgrd.sql

preupgrade.log
preupgrade_fixups.sql
postupgrade_fixups.sql

Step.5:

Verify the preupgrade.log and make necessary changes

Example of preupgrade.log:

-WARNING: --> Process Count may be too low
ALTER SYSTEM SET PROCESSES=300 SCOPE=SPFILE (bydefault 150)
-WARNING: --> Inactive DBIDs found in AWR
AWR contains inactive DBIDs which may need additional updating after upgrading.
-WARNING: --> Enterprise Manager Database Control repository found in the database In Oracle Database 12c, Database Control is removed during the upgrade. To save time during the Upgrade, this action can be done prior to upgrading using the following steps after copying rdbms/admin/emremove.sql from the new Oracle home
- Stop EM Database Control:
$> emctl stop dbconsole
Connect to the Database using the SYS account AS SYSDBA:
SET ECHO ON;
SET SERVEROUTPUT ON;
@emremove.sql Without the set echo and serveroutput commands you will not be able to follow the progress of the script.

-WARNING: --> Existing DBMS_LDAP dependent objects Database contains schemas with objects dependent on DBMS_LDAP package. Refer to the Upgrade Guide for instructions to configure Network ACLs. USER APEX_030200 has dependent objects.

-INFORMATION: -->
OLAP Catalog(AMD) exists in database Starting with Oracle Database 12c, OLAP is desupported. If you are not using the OLAP Catalog component and want to remove it, then execute the ORACLE_HOME/oraolap/admin/catnoamd.sql script before or after the upgrade.

-INFORMATION: --> Older Timezone in use

Please gather dictionary statistics 24 hours prior to upgrading the database.
To gather dictionary statistics execute the following command while connected as SYSDBA:
EXECUTE dbms_stats.gather_dictionary_stats;

Step.6:

Prepare spfile for the 12c upgrade according to the output from preupgrade.log:
(Please note: Best Practice would be to edit the init.ora for the upgrade manually. You could do so – the way i propose here is just a shortcut avoiding manual edit steps)
SQL> create pfile from spfile;


Step.7:

Execute the preupgrade_fixups.sql – it was created by preupgrd.sql in directory ORACLE_HOME/cfgtoollogs/$SID/preupgrade

SQL>@ORACLE_HOME/cfgtoollogs/$SID/preupgrade/preupgrade_fixups.sql

Please note that the preupgrade_fixups.sql script will still complain about PROCESSES being set too low. This is because I have used the ALTER SYSTEM command to adjust the parameter, but that adjustment will not take effect until the database is shutdown and restarted. Because I specified SCOPE=SPFILE, this parameter will be set correctly for the upgrade.

Step.8:

Shutdown the UPGRADE database:
SQL >shutdown immediate
SQL >exit

Step.9:

Copy the passwordfile and pfile/spfile from the 11g home to the 12c home.

$ cp /u01/app/oracle/product/11.2.0/dbs/spfileUPGR.ora /u01/app/oracle/product/12.1.0.2/dbs/
$ cp /u01/app/oracle/product/11.2.0/dbs/orapwUPGR /u01/app/oracle/product/12.1.0.2/dbs/

Step.10:

Set new ORACLE HOME of 12c location
export ORACLE_HOME=/u01/app/oracle/product/12.2.0.1/db_1/

Step.11:

SQL> startup nomount;
SQL> alter database mount;
SQL> alter database open upgrade;
SQL> exit

Step.12:

Run catctl.pl

Go to $ cd $ORACLE_HOME/rdbms/admin
$ $ORACLE_HOME/perl/bin/perl catctl.pl catupgrd.sql

This will take some time depending on your system. If you wonder about the RESTART phases: those happen if timing dependencies make it necessary to rerun a certain action. The logfiles will be written by default into the directory from which you started catctl.pl, $ORACLE_HOME/rdbms/admin Once the upgrade is finished it will shutdown the database and in the next phase restart it in normal mode.

logfile and upgrade summary reports are created in @ORACLE_HOME/cfgtoollogs/$SID

Note: It will create logfiles based on process in $ORACLE_HOME/rdbms/admin

example:

rw-r--r-- 1 oracle oinstall 5.4M Dec 18 10:41 catupgrd3.log
-rw-r--r-- 1 oracle oinstall 4.9M Dec 18 10:41 catupgrd2.log
-rw-r--r-- 1 oracle oinstall 4.2M Dec 18 10:41 catupgrd1.log
-rw-r--r-- 1 oracle oinstall 228M Dec 18 11:01 catupgrd0.log

Step.13:

Run postupgrade_fixups.sql which was created earlier.
@ORACLE_HOME/cfgtoollogs/$SID/preupgrade/postupgrade_fixups.sql

The database is using timezone datafile version 14 and the target 12.2.0.1.0 database ships with timezone datafile version 26. Oracle recommends using the most recent timezone data.  For further information, refer to My Oracle Support Note 1585343.1

Step.14

Run below scripts:

SQL> @?/rdbms/admin/utlu122s.sql
SQL> @?/rdbms/admin/catuppst.sql
Important: The catuppst.sql script is run as part of the upgrade process unless the upgrade returns errors during the process. Check the log file for "BEGIN catuppst.sql" to verify that catuppst.sql ran during the upgrade process. If catuppst.sql has not run, then proceed to run catuppst.sql as shown in this step. Warning messages are also displayed when running catctl.pl indicating that catuppst.sql was not run during the upgrade.

Run catuppst.sql, located in the $ORACLE_HOME/rdbms/admin directory, to perform upgrade actions that do not require the database to be in UPGRADE mode.

SQL> @catuppst.sql
This script can be run concurrently with utlrp.sql.
Run utlrp.sql to recompile any remaining stored PL/SQL and Java code in another session.

SQL> @utlrp.sql

SQL>@?/rdbms/admin/utlrp.sql

Step.15:

Set COMPATIBALE parameter value to 12.2.0, start the listener with new home and count the invalid objects

Step.16:

Restart the database and check the version of a database

What Happened When You Opened The Database In Startup Upgrade Mode


Basically startup upgrade will open the database by setting the below parameters in memory.(not for spfile)

ALTER SYSTEM enable restricted session;
ALTER SYSTEM SET _system_trig_enabled=FALSE SCOPE=MEMORY;
ALTER SYSTEM SET _undo_autotune=FALSE SCOPE=MEMORY;
ALTER SYSTEM SET undo_retention=900 SCOPE=MEMORY;
ALTER SYSTEM SET aq_tm_processes=0 SCOPE=MEMORY;
ALTER SYSTEM SET resource_manager_plan='' SCOPE=MEMORY;

It will be just a normal database startup, but it will make an environment for migrating the version.

Alert log will show all these details:

Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 2
Autotune of undo retention is turned on.
IMODE=BR
ILAT =18
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.3.0.
---------------------------------------------------
---------------------------------------------------
Mon Mar 25 19:54:01 2013
Database mounted in Exclusive Mode
Completed: ALTER DATABASE   MOUNT
Mon Mar 25 19:54:02 2013
ALTER DATABASE OPEN MIGRATE
Mon Mar 25 19:54:02 2013
-------------------------------------------------
-------------------------------------------------
Starting background process MMNL
MMNL started with pid=12, OS id=2536
Mon Mar 25 19:54:14 2013
ALTER SYSTEM enable restricted session;
Mon Mar 25 19:54:14 2013
ALTER SYSTEM SET _system_trig_enabled=FALSE SCOPE=MEMORY;
Autotune of undo retention is turned off.
Mon Mar 25 19:54:14 2013
ALTER SYSTEM SET _undo_autotune=FALSE SCOPE=MEMORY;
Mon Mar 25 19:54:14 2013
ALTER SYSTEM SET undo_retention=900 SCOPE=MEMORY;
Mon Mar 25 19:54:14 2013
ALTER SYSTEM SET aq_tm_processes=0 SCOPE=MEMORY;
Mon Mar 25 19:54:14 2013
ALTER SYSTEM SET resource_manager_plan='' SCOPE=MEMORY;
replication_dependency_tracking turned off (no async multimaster replication found)
Completed: ALTER DATABASE OPEN MIGRATE

Wednesday, November 15, 2017

Find And Replace In Vi editor


Syntax for Text Substitution inside the vi editor is,

:[range]s[ubstitute]/{pattern}/{string}/[Flags] [count]

Following are the possible Flags

[g] -> replace all occurances in the line.
[c] -> Confirm each substitution
[i] -> make case insensitive

Case 1: Replace a text with another in the whole file

:%s/oldText/newText/g

where %s indicates all lines in the file.

Case 2: Replace a text within a single line

:s/oldText/newText/g

No range was specified therefore it replaces text in the single line.

:s/I/oldText/newText/g

where I indicates case insensitive

Case 3: Replace a text within a range of lines

:1,10s/oldText/newText/g

replaces text between line 1 and 10

Case 4: Replace text in only the first N lines

:s/oldText/newText/g 5

replaces the text in 5 lines from the current cursor position.

Case 5: Replace the whole word and not the partial word

:s/\<his\>/her/

The standard replace will make the word history with herstory. Enclose the word with < and > to ensure the search is for the whole word

and not for partial one.

Case 6: Interactive Find and Replace

:%s/oldText/newText/gc

This will prompt for confirmation.

replace with newText (y/n/a/q/l/^E/^Y)?

y -> replace the highlighted word. After replace moves to the next word.
n -> does not replace the highlighted word. Moves to the next word.
a -> replace all the highlighted words.
l -> replaces the current highlighted word and terminates the find and replace.



pwd
/export/home/oratest/CLONE_24Jul2022

vi Db_config_bkp.sql

Add below contents in your file.


SQL>@Db_config_bkp.sql


Prompt
Prompt  === copying init.ora and spfiles  ===
Prompt  ========================================
Prompt

host cp $ORACLE_HOME/dbs/spfile$ORACLE_SID.ora /export/home/oratest/CLONE_24Jul2022

host cp $ORACLE_HOME/dbs/init$ORACLE_SID.ora /export/home/oratest/CLONE_24Jul2022

host cp $ORACLE_HOME/oraInst.loc /export/home/oratest/CLONE_24Jul2022

host cp $ORACLE_HOME/root.sh  /export/home/oratest/CLONE_24Jul2022

host cp $ORACLE_HOME/*.env  /export/home/oratest/CLONE_24Jul2022

host cp -R $ORACLE_HOME/network/admin/* /export/home/oratest/CLONE_24Jul2022

spool /export/home/oratest/CLONE_24Jul2022/Precloneinfo.log

alter database backup controlfile to trace as '/export/home/oratest/CLONE_24Jul2022/ctrl_$ORACLE_SID.sql';

create pfile='/export/home/oratest/CLONE_24Jul2022/initpfile.ora' from spfile;

Prompt
Prompt  === dblinks  ===
Prompt  ========================================
Prompt

begin
   dbms_metadata.set_transform_param (dbms_metadata.session_transform, 'SQLTERMINATOR', true);
   dbms_metadata.set_transform_param (dbms_metadata.session_transform, 'PRETTY', true);
end;
/

set lines 200
set pages 200
col DB_LINK for a50
col USERNAME for a20
col HOST for a50
select * from dba_db_links;

SELECT 'create '||DECODE(U.NAME,'PUBLIC','public ')||'database link '||CHR(10)
 ||DECODE(U.NAME,'PUBLIC',Null, U.NAME||'.')|| L.NAME||chr(10)
 ||'connect to ' || L.USERID || ' identified by '
 ||L.PASSWORD||' using ''' || L.host || ''''
 ||chr(10)||';' TEXT
 FROM sys.link$ L, sys.user$ U
 WHERE L.OWNER# = U.USER#;

Prompt
Prompt  === dba_directories  ===
Prompt  ========================================
Prompt

set pages 800
set lines 300
col DIRECTORY_NAME for a40
col DIRECTORY_PATH for a90
select * from dba_directories;

set pages 800
select 'create or replace directory '||DIRECTORY_NAME||' as '''||DIRECTORY_PATH||''';' from dba_directories;


archive log list

show parameter backgr

Prompt
Prompt  === temp_files  ===
Prompt  ========================================
Prompt

set lines 200
col FILE_NAME for a80;
select file_name,tablespace_name,bytes/1024/1024,status from dba_temp_files;

Prompt
Prompt  === data_files  ===
Prompt  ========================================
Prompt

select file_name,tablespace_name,bytes/1024/1024,status from dba_data_files;

Prompt
Prompt  === control_files  ===
Prompt  ========================================
Prompt
select name from v$controlfile;


Prompt
Prompt  === redolog_files  ===
Prompt  ========================================
Prompt
col MEMBER for a60
set pages 800
set lines 200
select * from v$logfile;

select * from v$log;


Prompt
Prompt  === all directories of db CRD files  ===
Prompt  ========================================
Prompt
select substr(name,1,instr(name,'/',-1)) from v$datafile
union
select substr(name,1,instr(name,'/',-1)) from v$tempfile
union
select substr(member,1,instr(member,'/',-1)) from v$logfile
union
select substr(name,1,instr(name,'/',-1)) from v$controlfile;


Prompt
Prompt  === status of CRD files  ===
Prompt  ========================================
Prompt
select distinct status from v$datafile
union
select distinct status from v$tempfile
union
select distinct status from v$logfile
union
select distinct status from v$controlfile;

Prompt
Prompt  === size and count of datafiles on each mount  ===
Prompt  ========================================
Prompt
set pages 200
set lines 300
col DATAFILES_DIR for a50
select substr(name, 1, instr(name, '/', -1)) datafiles_DIR, count(1) datafiles_count,sum(bytes/1024/1024/1024) size_in_GB from v$datafile group by rollup (substr(name, 1, instr(name, '/', -1))) order by 1;

Prompt
Prompt  === size and count of tempfiles on each mount  ===
Prompt  ========================================
Prompt
set pages 200
set lines 300
col TEMPFILES_DIR for a50
select substr(name, 1, instr(name, '/', -1)) tempfiles_DIR, count(1) tempfiles_count,sum(bytes/1024/1024/1024) size_in_GB from v$tempfile group by rollup (substr(name, 1, instr(name, '/', -1))) order by 1;

Prompt
Prompt  ===all file locations of db ===
Prompt  ========================================
Prompt
select distinct substr(name, 1, instr(name, '/', -1)) || '  -- DATAfiles' from v$datafile union
select distinct substr(name, 1, instr(name, '/', -1)) || '  -- TEMPfiles' from v$tempfile union
select distinct substr(member, 1, instr(member, '/', -1)) || '  -- LOGfiles' from v$logfile union
select distinct substr(name, 1, instr(name, '/', -1)) || '  -- Controlfiles' from v$controlfile;


select node_name from apps.fnd_nodes;

show parameter pfile

show parameter backgr

show parameter utl

Prompt *****Public synonym backup before import***********
set long 900000
select dbms_metadata.get_ddl(object_type=>'SYNONYM',name=>synonym_name,schema=>'PUBLIC') from all_synonymswhere owner='PUBLIC' and table_owner not in ('SYS')
Prompt
Prompt  === tablespace creation script  ===
Prompt  ========================================
Prompt


set long 900000
select dbms_metadata.get_ddl('TABLESPACE',tb.tablespace_name) from dba_tablespaces tb;

Prompt
Prompt  === user creation script  ===
Prompt  ========================================
Prompt


set long 9999999
select dbms_metadata.get_ddl('USER', username) || '/' usercreate from dba_users;

Prompt
Prompt  === username -- password script ===
Prompt  ========================================
Prompt


select 'alter user '|| name||' identified by values  '|| ''''|| password||''''||';' from SYS.USER$ WHERE password is not null;

Prompt
Prompt  === username -- its  Default tablespace  ===
Prompt  ========================================
Prompt
set lines 200 pages 9999
SELECT username, default_tablespace,temporary_tablespace,profile FROM dba_users order by 1;

select name,open_mode,log_mode,database_role,dbid from v$database;

col host_name fOR a36
col created fOR a16
col start_time fOR a16
col INSTANCE_NAME fOR a13
col OPEN_MODE fOR a10
set lines 280
select d.name db_name,i.INSTANCE_NAME,i.HOST_NAME,d.open_mode,d.database_role,to_char(d.created,'DD-MON-YY HH24:MI') created,to_char(i.startup_time,'DD-MON-YY HH24:MI') start_time from v$database d, gv$instance i;

spool off