How to Stop and Start Oracle RAC Services on Single Node

How to Stop and Start Oracle RAC Services on Single Node

1) Disable the automatic startup of the Oracle Clusterware software:

crsctl disable has

2) Stop the Oracle clusterware stack:

crsctl stop has

3) Once clusterware Services has been stopped we will restart those Services:

crsctl enable has

4) Enable the automatic startup of the Oracle Clusterware software:

crsctl start has

Check the status:

ps -ef|grep d.bin|wc -l

ps -ef|grep pmon

5) Start the Oracle clusterware stack:

crsctl start crs

6)Check the status of Oracle Clusterware Resources:

crsctl stat res -t

How to check the Grid Infrastructure version

How to check the Grid Infrastructure version

[root@DELLDB ~]#  crsctl query crs activeversion
Oracle Clusterware active version on the cluster is [11.2.0.4.0]

[root@DELLDB ~]#   crsctl query crs releaseversion 
Oracle High Availability Services release version on the local node is [11.2.0.3.0]

[root@DELLDB ~]#    crsctl query crs softwareversion
Oracle Clusterware version on node [DELLDBrac1] is [11.2.0.4.0]

What does crsctl query crs activeversion mean ?
If some nodes become unreachable in the middle of an upgrade, then you cannot complete the upgrade, because the upgrade script (rootupgrade.sh) did not run on the unreachable nodes. Because the upgrade is incomplete, Oracle Clusterware remains in the previous release version.

What does crsctl query crs releaseversion mean ?
Use the crsctl query crs releaseversion command to display the version of the Oracle Clusterware software stored in the binaries on the local node.

What does crsctl query crs softwareversion mean ?
Use the crsctl query crs softwareversion command to display latest version of the software that has been successfully started on the specified node.

Fixed ORA-00845: MEMORY_TARGET not supported on this system in Rac environment

Fixed ORA-00845: MEMORY_TARGET not supported on this system in Rac environment

Error:

[root@rac01 ~]# crsctl start cluster -all

CRS-2672: Attempting to start 'ora.crf' on 'rac01'

CRS-2672: Attempting to start 'ora.crf' on 'rac02'

CRS-2672: Attempting to start 'ora.asm' on 'rac02'

CRS-2672: Attempting to start 'ora.asm' on 'rac01'

CRS-2676: Start of 'ora.crf' on 'rac02' succeeded

CRS-2676: Start of 'ora.crf' on 'rac01' succeeded

CRS-5017: The resource action "ora.asm start" encountered the following error:
ORA-00845: MEMORY_TARGET not supported on this system
. For details refer to "(:CLSN00107:)" in "/u01/app/grid/diag/crs/rac02/crs/trace/ohasd_oraagent_grid.trc".
CRS-2674: Start of 'ora.asm' on 'rac02' failed
CRS-2679: Attempting to clean 'ora.asm' on 'rac02

Reason:
/dev/shm is also know as tmpfs i.e. temporary file system which keeps all the file system in virtual memory to speed up several processes.

Solution:

To increase the size
# mount -o remount,size=3G /dev/shm

Verify the size
# df -h
Filesystem   Size   Used   Avail  Use%  Mounted on
/dev/sda3   7.6G    4.4G   2.9G   61%   /
tmpfs       3G      1007M  2.1G   33%   /dev/shm
/dev/sda1   194M    25M    160M   14%   /boot

To make permanent changes to your file system update your fstab
# vi /etc/fstab
tmpfs  /dev/shm  tmpfs  defaults,size=3G  0 0

Update the new fstab file
# mount -a

ORA-01105: mount is incompatible with mounts by other instances – ORA-19808: recovery destination parameter mismatch

ORA-01105: mount is incompatible with mounts by other instances – ORA-19808: recovery destination parameter mismatch

sqlplus / as sysdba
SQL*Plus: Release 11.1.0.7.0 - Production on Mon Aug 6 10:45:16 2012
Copyright (c) 1982, 2008, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options

SQL> startup

ORACLE instance started.

Total System Global Area 1302769664 bytes
Fixed Size 2159392 bytes
Variable Size 989859040 bytes
Database Buffers 301989888 bytes
Redo Buffers 8761344 bytes
ORA-01105: mount is incompatible with mounts by other instances
ORA-19808: recovery destination parameter mismatch

Cause:

DB recovery destination was filled. Below errors were reported in alert.log file for the first instance:

Errors in file /appl/app/oracle/diag/rdbms/xleap/XLEAP1/trace/XLEAP1_mmon_25819.trc:
ORA-19815: WARNING: db_recovery_file_dest_size of 4096000 bytes is 100.00% used, and has 0 remaining bytes available.

Solution:

1. Clear the db_recovery_file_dest

OR

2. Increase the size of db_recovery_file_dest_size. Issued below statement on running instance to increase the size:

alter system set db_recovery_file_dest_size= <new-size> scope=both sid='*'

Reference Metalink Doc ID 1481209.1

11g Grid Control: Using 'opmnctl startall | status | stopall' in OMS_HOME Fails with "opmnctl: ORACLE_INSTANCE is not set"

11g Grid Control: Using 'opmnctl startall | status | stopall' in OMS_HOME Fails with "opmnctl: ORACLE_INSTANCE is not set"

Symptoms

Trying to start the 11.1.0.1 Grid Control with opmnctl command fails with the following errors.

[oraoem@erputdb2 bin]$ ./opmnctl status
opmnctl: ORACLE_INSTANCE is not set.

opmnctl status requires that ORACLE_INSTANCE be correctly set in your
environment.

Cause

In the 11.1.0.1 Grid Control release, "opmnctl" is not a valid command for starting the OMS as the 11g OMS is deployed in Weblogic server unlike 10g version which is deployed on an OC4J.

opmnctl utility should be used to start the OMS only in the 10g versions of Grid Control nad not in the 11g Grid Control version.

Solution

Grid Control 11.1.0.1 uses "emctl" command to start Grid Control:

cd OMS_HOME/bin
./emctl start oms

This will start the Grid Control 11.1.0.1 OMS as well as starting the necessary Weblogic processes needed by the OMS.

11g Grid Control: Using 'opmnctl startall | status | stopall' in OMS_HOME Fails with "opmnctl: ORACLE_INSTANCE is not set" [ID 1160335.1]

root.sh failure while running Oracle clusterware installation

root.sh failure while running Oracle clusterware installation

If the root.sh fails on a node and you try to re execute it after fixing the failure then it gives the following error:

CRS is already configured on this node for crshome=0 Cannot configure two CRS instances on the same cluster.Please deconfigure before proceeding with the configuration of new home.

In order to perform a cleanup you will need to execute the following two commands as the root user.

In all the nodes where the root.sh failed execute the following:

# perl $GRID_HOME/crs/install/rootcrs.pl -verbose -deconfig -force

On the last node of the cluster execute the following as the root user:

# perl $GRID_HOME/crs/install/rootcrs.pl -verbose -deconfig -force -lastnode

Now you will be in a state to re execute the root.sh command.

Enabling and disabling Archivelog mode

Enabling and disabling Archivelog mode

Solution:

First check in what mode you are.

Select NAME, CREATED, LOG_MODE, CHECKPOINT_CHANGE#, ARCHIVE_CHANGE# from V$DATABASE;

From NOARCHIVELOG -> AUTOMATIC ARCHIVELOG MODE

To turn on automatic archivelog mode for Oracle9i set the parameters.

alter system set log_archive_start=TRUE scope=spfile;
alter system set log_archive_max_processes=3 scope=spfile;
alter system set log_archive_min_succeed_dest=1 scope=spfile;
alter system set log_archive_trace=0 scope=spfile;
alter system set log_archive_format='arch%t_EDB1_%s.ARC' scope=spfile;
alter system set log_archive_dest_1='location=/oracle10/ARCHIVES/EDB1' scope=spfile;

You have to restart the database

sqlplus "/ as sysdba"

shutdown immediate;

startup mount;

alter database archivelog;

alter database open;

alter system switch logfile;

Check if the archive log file is created at the log_archive_dest_1

Depending on the version of Oracle some small differences might exist. For example at Oracle10g you don't have to set:

log_archive_start=TRUE and the log_archive_format must contain %s, %t and %r.

For RAC especially, because of the two instances lets say OTE1 and OTE2 the steps are a little bit different.

First you have to set the parameter cluster_database = FALSE

Then to startup mount only one instance (PTE1)

Then shutdown immediate this instance (PTE1)

Then change parameter cluster_database = TRUE

From AUTOMATIC ARCHIVELOG MODE -> NOARCHIVELOG

Remove all parameters from spfile that have to do with archiving, and restart the database. For example

ALTER SYSTEM RESET log_archive_format SCOPE=SPFILE SID='*';

You have to restart the database

sqlplus "/ as sysdba"

shutdown immediate;

startup mount;

alter database noarchivelog;

alter database open;

Deconfiguring Oracle Clusterware Without Removing Binaries

Deconfiguring Oracle Clusterware Without Removing Binaries

As root:

$cd $GRID_HOME/crs/install

$perl rootcrs.pl -deconfig -force (Repeat for all nodes)

On last node execute:

$perl rootcrs.pl -deconfig -force -lastnode

The -lastnode remove node cluster with the OCR and voting disks.