Oracle alerted us recently of a non public bug #6886239 "DBMS_SERVICE parameters are not added using srvctl add service. It affects 10g and 11gR1 databases, this is fixed in release 11.2 onwards. In Oracle RAC after setting up TAF although it does not fails and everything looks good, it does not configure correctly.
When you check the service configuration you find no values for failover method, type and retries, those values are needed for TAF to happen.
Creating a service
srvctl add service -d rac -s server_taf -r "rac1,rac2" -P BASIC
Start the service
srvctl start service -d rac -s server_taf
Checking that the services are running.
srvctl config service -d rac
ractest PREF: rac1 rac2 AVAIL:
server_taf PREF: rac1 rac2 AVAIL:
Getting the service ID value
sqlplus /nolog
Connect / as sysdba
SQL> select name,service_id from dba_services where name = 'server_taf';
NAME SERVICE_ID
---------------------------------------------------------------- ----------
server_taf 6
Checking the setup
SQL>col name format a15
col failover_method format a11 heading 'METHOD'
col failover_type format a10 heading 'TYPE'
col failover_retries format 9999999 heading 'RETRIES'
col goal format a10
col clb_goal format a8
col AQ_HA_NOTIFICATIONS format a5 heading 'AQNOT'
SQL>select name, failover_method, failover_type, failover_retries,goal, clb_goal,aq_ha_notifications
from dba_services where service_id = 6
NAME METHOD TYPE RETRIES GOAL CLB_GOAL AQNOT
--------------- ----------- ---------- -------- ---------- -------- -----
server_taf LONG NO
Adding the server side failover parameter to the service
Server side TAF method is BASIC. BASIC is the only value currently supported. This means that a new connection is established at failure time. It is not possible to pre-establish a backup connection. (which is to say, PRECONNECT is not supported)
SQL> execute dbms_service.modify_service (service_name => 'server_taf' -
, aq_ha_notifications => true -
, failover_method => dbms_service.failover_method_basic -
, failover_type => dbms_service.failover_type_select -
, failover_retries => 180 -
, failover_delay => 5 -
, clb_goal => dbms_service.clb_goal_long);
PL/SQL procedure successfully completed.
Check that the service can now see the values for methods, types and retries
SQL>select name, failover_method, failover_type, failover_retries,goal, clb_goal,aq_ha_notifications
from dba_services where service_id = 6
NAME METHOD TYPE RETRIES GOAL CLB_GOAL AQNOT
--------------- ----------- ---------- -------- ---------- -------- -----
server_taf BASIC SELECT 180 NONE LONG YES
Check that the listener has the service registered
lsnrctl services
Create a net service name. Here we have client load balancing between the two nodes.
SERVERTAF =
(DESCRIPTION =
(LOAD_BALANCE = yes)
(ADDRESS = (PROTOCOL = TCP)(HOST = dell01)(PORT = 1521))
(ADDRESS = (PROTOCOL = TCP)(HOST = dell02)(PORT = 1521))
(CONNECT_DATA =
(SERVICE_NAME = server_taf.za.oracle.com)
You are all set now :-
Tuesday, February 15, 2011
Tuesday, October 19, 2010
-- Questions about the Oracle GG Roadmap --
By Oracle Corp.
We now support the recyclebin on/true for Oracle 11g along with our DDL replication support, Oracle 10g still requires the recyclebin to be off/false.
2.- Does OGG support active-active in a 3+ node (multi master) configuration? Documentation shows 3 nodes in an active-active configuration, is 3+ supported? If so, any restrictions for DML or DDL for a 3+ active-active configuration?
1.- Will Oracle fix the recyclebin off issue when capturing DDL? If so, what OGG release and what RDBMS versions are expected to support a functioning recyclebin in the source?
We now support the recyclebin on/true for Oracle 11g along with our DDL replication support, Oracle 10g still requires the recyclebin to be off/false.
2.- Does OGG support active-active in a 3+ node (multi master) configuration? Documentation shows 3 nodes in an active-active configuration, is 3+ supported? If so, any restrictions for DML or DDL for a 3+ active-active configuration?
Our active-active capabilities apply to any number of nodes in the configuration, in other words, we are not limited to only two. Obviously you need to make special considerations for other factors such as the replication of tables that use sequences, replication of truncate or other DDL operations, for example. For best practices, I recommend for multi-node active-active environments, that user privileges/grants are set so that DDL operations can only originate from one node and not the others. Configure OGG to replicat those DDL operations to all downstream nodes in one direction only. Of course that means that other considerations need to plan in event that the node that allows DDL operations goes down or has a planned downtime for maintenance activities.
·
3.- When will OGG support source tables with Advanced Compression? 10.4.x docs says compressed tables are not supported for extract.
It is on the road-map to support advanced compression. In fact, a release defining feature is to integrate our transaction log reading functionality with the XStreamOut API. This integration will give us support for the variety of database compression features. This is scheduled to be part of the next major release.
4.- Will the SUPPRESSTRIGGERS param available for RDBMS 10.2.0.5 and 11.2.+ for OGG 11.1.x be supported in other RDBMS versions, mainly 10.2.0.4 and all 11G R1 RDBMS versions.
SUPPRESSTRIGGERS parameter support is the ultimate goal, but I am not clear which future release this is targeted to be included. Each database has different functionality so it might be a phased approach depending if it is possible and the development effort required to support it.
-- CRS will not start on one node (Fixed on 11.2.0.2 and Workaround below) --
Abstract: CORE DUMP OF OCSSD.BIN WHEN VOTING DISK IS NOT ACTIVATED.
Some explanation from development:
==================================
In 11.2, voting files are discovered, not hard-wired, so we look through a
list of files that are specified in the 'discovery string', e.g.
/dev/vdisk/*, and use all files that appear to be legitimate voting files,
i.e. they have a TOC (Table Of Contents), volume info block, etc. Since the
VG with the voting files is not online, the discovery does not see them at
all, so does not consider them as voting files and fails as a result of an
inability to find enough voting files.
=====================================
The workaround is to wait the disks are completely ONLINE after machine boot before cluster is started
(note 459169.1 CRS Does Not Startup Automatically After Node Reboot,
Manual Start is OK - but this node is up to 11.1 version)
or apply patch, backport is available for Solaris x86-64(64 bit) - please confirm the Operating System
please upload the patch inventory - I may need to raise backport for you
We're getting these errors on the other node when rebooting:
I did see in the boot messages on both hosts these items:
Johnston, Nathaniel [10:39 AM]:
Sep 24 16:05:15 mhddb-nb-2p.philadelphia.pa.bo.comcast.net root: Oracle HA daemon is enabled for autostart.
Johnston, Nathaniel [10:39 AM]:
Sep 24 16:05:18 mhddb-nb-2p.philadelphia.pa.bo.comcast.net unix: vn_rdwr failed with error 0x15
Sep 24 16:05:18 mhddb-nb-2p.philadelphia.pa.bo.comcast.net unix: kobj_load_module: read header failed
Johnston, Nathaniel [10:39 AM]:
Sep 24 16:05:19 mhddb-nb-2p.philadelphia.pa.bo.comcast.net root: exec /u01/app/grid/perl/bin/perl -I/u01/app/grid/perl/lib /u01/app/grid/bin/crswrapexece.pl /u01/app/grid/crs/install/s_crsconfig_mhddb-nb-2p_env.txt /u01/app/grid/bin/ohasd.bin "reboot"
Sep 24 16:05:19 mhddb-nb-2p.philadelphia.pa.bo.comcast.net root: exec /u01/app/grid/perl/bin/perl -I/u01/app/grid/perl/lib /u01/app/grid/bin/crswrap
Johnston, Nathaniel [10:40 AM]:
Sep 24 16:05:36 mhddb-nb-2p.philadelphia.pa.bo.comcast.net mDNSResponder (Engineering Build) (Nov 2 2009 05:02:07) [5272]: starting
Sep 24 16:05:37 mhddb-nb-2p.philadelphia.pa.bo.comcast.net mDNSResponder: Oracle mDNSResponder starting
======================================
WORKAROUND
======================================
| 11gR2 CRS doesn't startup after node reboot [ID 1050164.1] | |||||
| Modified 31-JAN-2010 Type PROBLEM Status PUBLISHED | |||||
In this Document
Symptoms
Changes
Cause
Solution
Applies to:
Oracle Server - Enterprise Edition - Version: 11.2.0.1.0 to 11.2.0.1.0 - Release: 11.2 to 11.2Generic Linux
Symptoms
- Installation of the 11gR2 Grid Infrastructure on a Linux cluster completed successfully
- OCR & Voting files located in ASM diskgroup
- using ASMLIB driver
- ASM disks are located on multipath devices (/dev/mapper/)
- following a node reboot CRS does not startup
- CSS daemon log shows the following message:
2010-01-13 09:04:15.075: [ CSSD][1150449984]clssnmvDDiscThread: using discovery string for initial discovery
2010-01-13 09:04:15.075: [ SKGFD][1150449984]Discovery with str::
2010-01-13 09:04:15.075: [ SKGFD][1150449984]UFS discovery with ::
2010-01-13 09:04:15.075: [ SKGFD][1150449984]OSS discovery with ::
2010-01-13 09:04:15.076: [ SKGFD][1150449984]Discovery with asmlib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: str ::
2010-01-13 09:04:15.076: [ SKGFD][1150449984]Fetching asmlib disk :ORCL:DATA1:
2010-01-13 09:04:15.076: [ SKGFD][1150449984]Fetching asmlib disk :ORCL:DATA2:
2010-01-13 09:04:15.076: [ SKGFD][1150449984]Fetching asmlib disk :ORCL:DATA3:
2010-01-13 09:04:15.076: [ SKGFD][1150449984]Fetching asmlib disk :ORCL:DATA4:
2010-01-13 09:04:15.077: [ SKGFD][1150449984]ERROR: -15(asmlib ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so op asm_open error Operation not permitted)
2010-01-13 09:04:15.077: [ SKGFD][1150449984]ERROR: -15(asmlib ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so op asm_open error Operation not permitted)
2010-01-13 09:04:15.077: [ SKGFD][1150449984]ERROR: -15(asmlib ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so op asm_open error Operation not permitted)
2010-01-13 09:04:15.077: [ SKGFD][1150449984]ERROR: -15(asmlib ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so op asm_open error Operation not permitted)
2010-01-13 09:04:15.077: [ CSSD][1150449984]clssnmvDiskVerify: Successful discovery of 0 disks
2010-01-13 09:04:15.077: [ CSSD][1150449984]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery
2010-01-13 09:04:15.077: [ CSSD][1150449984]clssnmvFindInitialConfigs: No voting files found
2010-01-13 09:04:15.077: [ CSSD][1150449984]###################################
2010-01-13 09:04:15.077: [ CSSD][1150449984]clssscExit: CSSD signal 11 in thread clssnmvDDiscThread
2010-01-13 09:04:15.077: [ CSSD][1150449984]###################################
2010-01-13 09:04:15.077: [ CSSD][1139960128]clssgmClientShutdown: total iocapables 0
2010-01-13 09:04:15.077: [ CSSD][1139960128]clssgmClientShutdown: graceful shutdown completed.
2010-01-13 09:04:15.077: [ CSSD][1150449984] - running the cluster verification utility returns the following messages:
/cluvfy stage -post crsinst -n racnode1
Performing post-checks for cluster services setup
Checking node reachability...
Node reachability check passed from node "racnode1"
Checking user equivalence...
User equivalence check passed for user "grid"
Checking time zone consistency...
Time zone consistency check passed.
ERROR:
Cluster manager integrity check failed
PRVF-5434 : Cannot identify the current CRS software version
UDev attributes check for OCR locations started...
UDev attributes check passed for OCR locations
UDev attributes check for Voting Disk locations started...
ERROR:
PRVF-5197 : Failed to retrieve voting disk locations
UDev attributes check failed for Voting Disk locations
Default user file creation mask check passed
Checking cluster integrity...
Cluster integrity check failed This check did not run on the following node(s):
racnode1
Checking OCR integrity...
Checking the absence of a non-clustered configuration...
All nodes free of non-clustered, local-only configurations
ERROR:
PRVF-5300 : Failed to retrieve active version for CRS on this node
OCR integrity check failed
Checking CRS integrity...
ERROR:
PRVF-5300 : Failed to retrieve active version for CRS on this node
CRS integrity check failed
OCR detected on ASM. Running ACFS Integrity checks...
Starting check to see if ASM is running on all cluster nodes...
PRVF-5137 : Failure while checking ASM status on node "racnode1"
Starting Disk Groups check to see if at least one Disk Group configured...
PRVF-5112 : An Exception occurred while checking for Disk Groups
PRVF-5114 : Disk Group check failed. No Disk Groups configured
Task ACFS Integrity check failed
Checking Oracle Cluster Voting Disk configuration...
ERROR:
PRVF-5434 : Cannot identify the current CRS software version
PRVF-5431 : Oracle Cluster Voting Disk configuration check failed
User "grid" is not part of "root" group. Check passed
Post-check for cluster services setup was unsuccessful on all the nodes.
Changes
Node was rebooted after install.Cause
The CSS daemon crashes because it cannot locate any Voting files in any of the discovered ASM disks, which is indicated by the following message in the CSS daemon log (/log//cssd/ocssd.log):
2010-01-13 09:04:15.077: [ CSSD][1150449984]clssnmvFindInitialConfigs: No voting files foundThis error is preceded by the following ASMLIB error:
2010-01-13 09:04:15.077: [ SKGFD][1150449984]ERROR: -15(asmlib ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so op asm_open error Operation not permitted)Solution
1. either edit the file /etc/sysconfig/oracleasm-_dev_oracleasm and change the lines:ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""ORACLEASM_SCANORDER="dm"
ORACLEASM_SCANEXCLUDE="sd"/usr/sbin/oracleasm configure -i -e -u user -g group -o "dm" -x "sd"2. stop & restart ASMLIB as user root using:
/usr/sbin/oracleasm exit
/usr/sbin/oracleasm init3. restart CRS or reboot node
The above steps need to be executed on all nodes
-- ASM unresponsive on 1 cluster node of 4 (Fixed on 11.2.0.2) --
By Rafael Orta
=======================================================
Bug 9276348 BugDB see Bug 9276348.-P Base Bug 9254569
Related Bugs : BugMatrix 9276348 DDR: BugDesc 9276348
Customer: INTERNAL - RACSYSTST Created: 11-JAN-10
Component: PCW Comp Ver: 11.2.0.2 Rel St: D Updated: 03-FEB-10
Sub Comp: RDBMS Ver: 11.2.0.2 By: HFLAKE
Status: 96,Closed, Duplicate Bug
Severity: 2,Severe Loss of Service Fixed In Ver:
O/S: 226 Linux x86-64
PL Group: Gen/Port: G Error #: - Pub: N
Hdr: 9276348 11.2.0.2 PCW 11.2.0.2 PRODID-5 PORTID-226 9254569
Abstract: LX64: CRS STACK FAILS TO COME UP ON FIRST NODE, CSS STARTS OK
=======================================================
3) Bug 9276348 is closed as duplicated of Bug 9254569 :
=======================================================
Bug 9254569 BugDB see Bug 9254569.-P No Base Bug
Related Bugs : BugMatrix 9254569 DDR: BugDesc 9254569
Customer: INTERNAL-RACSYSTST Created: 04-JAN-10
Component: PCW Comp Ver: 11.2 Rel St: D Updated: 13-SEP-10
Sub Comp: CRS RDBMS Ver: 11.2 By: ARU
Status: 80,Development to Q/A
Severity: 2,Severe Loss of Service Fixed In Ver: 11.2.
0.2
O/S: 233 Microsoft Windows Server 2003 (64-bit
PL Group: Gen/Port: G Error #: - Pub: N
Hdr: 9254569 11.2 PCW 11.2 CRS PRODID-5 PORTID-233
Abstract: MISSING DEPENDENCY IN OHASD: GIPC HA REQUIRES GIPCD
=======================================================
4) Bug 9254569 is fixed on release 11.2.0.2.
ACTION PLAN
===========
5) In order to fix the bug 9254569 please apply the patchset 11.2.0.2 on the Grid Infrastructure Oracle Home. Patchset 11.2.0.2 also contains another very important fixes for the Grid Infrastructure CRS.
6) For additional information please check the next note:
=)> Important Changes to Oracle Database Patch Sets Starting With 11.2.0.2 (Doc ID 1189783.1)
Bug 9276348 BugDB see Bug 9276348.-P Base Bug 9254569
Related Bugs : BugMatrix 9276348 DDR: BugDesc 9276348
Customer: INTERNAL - RACSYSTST Created: 11-JAN-10
Component: PCW Comp Ver: 11.2.0.2 Rel St: D Updated: 03-FEB-10
Sub Comp: RDBMS Ver: 11.2.0.2 By: HFLAKE
Status: 96,Closed, Duplicate Bug
Severity: 2,Severe Loss of Service Fixed In Ver:
O/S: 226 Linux x86-64
PL Group: Gen/Port: G Error #: - Pub: N
Hdr: 9276348 11.2.0.2 PCW 11.2.0.2 PRODID-5 PORTID-226 9254569
Abstract: LX64: CRS STACK FAILS TO COME UP ON FIRST NODE, CSS STARTS OK
=======================================================
3) Bug 9276348 is closed as duplicated of Bug 9254569 :
=======================================================
Bug 9254569 BugDB see Bug 9254569.-P No Base Bug
Related Bugs : BugMatrix 9254569 DDR: BugDesc 9254569
Customer: INTERNAL-RACSYSTST Created: 04-JAN-10
Component: PCW Comp Ver: 11.2 Rel St: D Updated: 13-SEP-10
Sub Comp: CRS RDBMS Ver: 11.2 By: ARU
Status: 80,Development to Q/A
Severity: 2,Severe Loss of Service Fixed In Ver: 11.2.
0.2
O/S: 233 Microsoft Windows Server 2003 (64-bit
PL Group: Gen/Port: G Error #: - Pub: N
Hdr: 9254569 11.2 PCW 11.2 CRS PRODID-5 PORTID-233
Abstract: MISSING DEPENDENCY IN OHASD: GIPC HA REQUIRES GIPCD
=======================================================
4) Bug 9254569 is fixed on release 11.2.0.2.
ACTION PLAN
===========
5) In order to fix the bug 9254569 please apply the patchset 11.2.0.2 on the Grid Infrastructure Oracle Home. Patchset 11.2.0.2 also contains another very important fixes for the Grid Infrastructure CRS.
6) For additional information please check the next note:
=)> Important Changes to Oracle Database Patch Sets Starting With 11.2.0.2 (Doc ID 1189783.1)
-- Backup Issue encountered after DB failover --
We encountered the backup issue on db. As per the metalink note 566635.1, there is an bug in 10.2. and it is fixed in 11.2
==================
Applies to:
Oracle Server - Enterprise Edition - Version: 10.2.0.2.0 This problem can occur on any platform.
Symptoms
Resync of the catalog for the Primary database fails with:
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of resync command on default channel at 05/06/2008
10:29:09
ORA-00001: unique constraint (RMAN.TF_P) violated Cause You are hitting bug 6653570
The root cause scenario in the bug is:
1) a primary and standby is created
2) on primary customer added a tempfile. This doesn't get created on standby.
It gets resynced to recovery catalog
3) Then when switchover happens, they add the tempfile on new primary and
resync it to recovery catalog. This resync marks the tempfile on primary (at
step 2) as dropped.
4) Then when the switchover happens again, the temp file that was dropped is
again resynced which is causing the problem.
Solution
Bug 6653570 is fixed in Oracle Database version 11.2
Please check metalink if the patch is available for your platform and version. If not and a one off is required, please contact Oracle support
The workaround for this issue is to recreate the temporary tablespace on the primary database.
-- 11.2.0.2 upgrade error --
When upgrading grid 11.2.0.1.0 to 11.2.0.2.0, you must make sure that patch 9413827 is installed on the 11.2.0.1.0 grid and database binaries PRIOR to starting the upgrade. The upgrade will continue without it until you get to the end when you should run root.sh and it will bomb out stating that that patch is not installed. In order to install it you will have to shut down the grid and asm home and apply the patch. This will of course mean that you have to kill the upgrade process. This causes a problem because after you apply the patch and restart the upgrade process, the installer states that there is no grid or asm installation on the machine. This is because it has wiped out a parameter in the oracle inventory for the grid installation. It removes the “ CRS=”true” “ parameter from the end of the line listing the grid home in the ~oraInventory/ContentsXML/inventory.xml file. The entry should look similar to this:
CRS="true">
If you run into this problem then manually edit the file and add the parameter back in and the installer will find the cluster and continue normally.
Wednesday, October 13, 2010
- Changing Private IP Addresses 11G R2 OCR / VD on ASM --
By Bill Toranto
I recently encountered this issue and thought everyone should be aware of it.
If any re IP-ing is to take place in your 11G R2 RAC clusters (mainly the Private IP), the Grid / CRS OCR needs to be updated prior to anything being done on the host. The ASM instance needs to be available in order for the OCR to be accessible.
IF the host is re IP-ed first, the Grid will need to be re installed using the new private IP address, since ASM will not mount & the OCR inaccessible.
Oracle does expect to have a fix for this very soon since they are getting a large number of reported issues related to this, and ASM OCR’s a relatively new implementation.
This may seem fundamental, but putting the OCR on ASM is a fairly new practice for most of us and previously could be modified after the host ip’s were modified since we used RAW devices.
I recently encountered this issue and thought everyone should be aware of it.
If any re IP-ing is to take place in your 11G R2 RAC clusters (mainly the Private IP), the Grid / CRS OCR needs to be updated prior to anything being done on the host. The ASM instance needs to be available in order for the OCR to be accessible.
IF the host is re IP-ed first, the Grid will need to be re installed using the new private IP address, since ASM will not mount & the OCR inaccessible.
Oracle does expect to have a fix for this very soon since they are getting a large number of reported issues related to this, and ASM OCR’s a relatively new implementation.
This may seem fundamental, but putting the OCR on ASM is a fairly new practice for most of us and previously could be modified after the host ip’s were modified since we used RAW devices.
Subscribe to:
Posts (Atom)
Posts
