Friday 9 August 2024

Target node/queue unavailable in R12

Today, we were facing following issue in  Concurrent Managers and some Other Managers, status field was showing message "Target node/queue unavailable" and Node Field was appearing Blank. This was our Production Instance.

After Checking Several Oracle Metalink ID's:

OAM Generic Collection Service shows State: "The target node/queue unavailable". [ID 393706.1] (it's given for 11i, but still worked for me in R12)

After Cloning all the Concurrent Managers do not start for the cloned Instance [ID 555081.1]

Conflict Resolution Manager Shows Target Node/Queue Unavailable [ID 732709.1]

Concurrent Managers Do Not Start After Cloning Nodes Not Updated In Conc_queues [ID 466532.1]

Summary of Possible Reasons and Solutions for the Problem Where All Concurrent Requests Stuck in Pending Phase [ID 182154.1]

Output Post Processor is Down with Actual Process is 0 And Target Process is 1 [ID 858813.1]

Solution which worked for me

SQL> set lines 200

SQL> column CONTROL_CODE format A15

SQL> select CONCURRENT_QUEUE_NAME, CONTROL_CODE , TARGET_NODE, NODE_NAME   from FND_CONCURRENT_QUEUES where concurrent_queue_name like 'OAMGCS_%';

CONCURRENT_QUEUE_NAME     CONTROL_CODE    TARGET_NODE       NODE_NAME
-----------------------------------------          -------------------          ---------------             ------------------
OAMGCS_ERPTSTAPP01                            E      ERPTSTAPP01

The standard codes used by Oracle Applications are as followings:

'A', -> 'Activate concurrent manager'

'D', -> 'Deactivate concurrent manager'

'E', -> 'Deactivated'

'N', -> 'Target node/queue unavailable'

'R', -> 'Restart concurrent manager'

'T', -> 'Terminate requests and deactivate manager'

'U', -> 'Update concurrent manager env inf.'

'V', -> 'Verify concurrent managers status'

'X', -> 'Terminated'

################STATUS_CODE Column############

A Waiting

B Resuming

C Normal

D Cancelled

E Error

F Scheduled

G Warning

H On Hold

I Normal

M No Manager

Q Standby

R Normal

S Suspended

T Terminating

U Disabled

W Paused

X Terminated

Z Waiting

################PHASE_CODE column############

C Completed

I Inactive

P Pending

R Running

################ This Solutions Not Work for me #################

SQL> update FND_CONCURRENT_QUEUES set control_code = null where concurrent_queue_name = 'OAMGCS_ERPTSTAPP01';

1 row updated.

SQL> update FND_CONCURRENT_QUEUES set TARGET_NODE='CBOS' where CONCURRENT_QUEUE_NAME='OAMGCS_ERPTSTAPP01';

1 row updated.

SQL> commit;

Commit complete.

SQL> set lines 200

SQL> col CONTROL_CODE format A15

SQL> select CONCURRENT_QUEUE_NAME, CONTROL_CODE , TARGET_NODE, NODE_NAME   from FND_CONCURRENT_QUEUES where concurrent_queue_name like 'OAMGCS_%';

CONCURRENT_QUEUE_NAME     CONTROL_CODE    TARGET_NODE       NODE_NAME
-----------------------------------------       ------------------------    ---------------------       -------------------
OAMGCS_ERPTSTAPP01                                                ERPTSTAPP01         ERPTSTAPP01

Stop and start Concurrent Managers using adcmctl.sh

The above Solution not worked for me and Concurrent Managers not came up.

################ Other Solutions (I've tried) ###################

Solution A worked for me:

Solution A:

CMCLEAN.SQL - Non Destructive Script to Clean Concurrent Manager Tables [ID 134007.1]

Run the cmclean.sql on admin node  

Start the managers and test.

The above Solution worked for me and Concurrent Managers came up without any issue.

Solution B:

sqlplus apps/pwd 

1. SQL> EXEC FND_CONC_CLONE.SETUP_CLEAN;

   SQL>COMMIT;

   SQL> EXIT;

Make sure following two sqls returns no rows

SQL> select node_name , node_mode, support_cp ,support_web , support_admin , support_forms  from FND_NODES;

SQL> select * from FND_OAM_CONTEXT_FILES;

2. Run AutoConfig on the database tier.

3. Run AutoConfig on the apps tier.    

4. Run cmclean.sql script .

SQL> sqlplus APPS/<Password>

SQL> select node_name , node_mode , support_cp ,support_web , support_admin , support_forms  from FND_NODES;

SQL> select CONCURRENT_QUEUE_NAME from FND_CONCURRENT_QUEUES where CONCURRENT_QUEUE_NAME like 'FNDSM%';

5. Start all application services and check whether managers are up & re-test.

Solution C:  

Apply this solution in last when the above one doesn’t work.

SQL> select node_name,target_node,control_code from fnd_concurrent_queues;

SQL> update apps.fnd_concurrent_queues set node_name = 'Node NAME' where node_name='Existing Node Name';

SQL> select NODE_NAME,NODE_MODE,STATUS from fnd_nodes;

SQL> select control_code,target_node,node_name,CONCURRENT_QUEUE_NAME from fnd_concurrent_queues;

SQL> UPDATE fnd_concurrent_queues set control_code = null;

SQL> select TARGET_NODE,NODE_NAME from fnd_concurrent_queues where node_name='<Existing Node Name>';

SQL> select TARGET_NODE,NODE_NAME from fnd_concurrent_queues where TARGET_NODE='<Existing Node Name>';

SQL> update fnd_concurrent_queues set NODE_NAME='<Node Name>' where NODE_NAME='<Source/Existing Node Name>';

SQL> update fnd_concurrent_queues set TARGET_NODE='<Node Name>' where TARGET_NODE='<Source/Exixting Node Name>';

SQL> UPDATE fnd_concurrent_queues set target_node = '<Node Name>';

SQL> UPDATE fnd_concurrent_queues set node_name = '<Node Name>';

SQL> Commit;

SQL> select control_code,target_node,node_name,CONCURRENT_QUEUE_NAME from fnd_concurrent_queues;

SQL>select TARGET_NODE,NODE_NAME from fnd_concurrent_queues where node_name='<Node Name>';

SQL>select TARGET_NODE,NODE_NAME from fnd_concurrent_queues where TARGET_NODE='<Node Name>';

How to identify/determine the maximum JVM Heap Size for OC4j components in R12 / 10gAS

Below command will help you to identify the maximum memory heap size than can be allocated to a JVM process.

java -mx[value]m -version

Start with a higher value like 4G or higher and slowly cut down to the maximum allowed value.

Sample

$ java -mx4096m -version

Invalid maximum heap size: -Xmx4096m

The specified size exceeds the maximum representable size.

Could not create the Java virtual machine.

As you can see my system can't support 4GB for a JVM.

Now lets reduce to a lesser value like 3GB

$ java -mx3072m -version

Invalid maximum heap size: -Xmx3072m

The specified size exceeds the maximum representable size.

Could not create the Java virtual machine.

As you can see my system can't support 3GB for a JVM.

Now again lets reduce to a lesser value like 2.5GB

$ java -mx2560m -version

java version "1.6.0_10"

Java(TM) SE Runtime Environment (build 1.6.0_10-b33)

Java HotSpot(TM) Server VM (build 11.0-b15, mixed mode)

Now its clear that the maximum allowed value is 2.5 GB.