Today, we were facing following issue in Concurrent Managers and some Other Managers, status field was showing message "Target node/queue unavailable" and Node Field was appearing Blank. This was our Production Instance.
After Checking Several Oracle Metalink ID's:
OAM Generic Collection Service shows State: "The target node/queue unavailable". [ID 393706.1] (it's given for 11i, but still worked for me in R12)
After Cloning all the Concurrent Managers do not start for the cloned Instance [ID 555081.1]
Conflict Resolution Manager Shows Target Node/Queue Unavailable [ID 732709.1]
Concurrent Managers Do Not Start After Cloning Nodes Not Updated In Conc_queues [ID 466532.1]
Summary of Possible Reasons and Solutions for the Problem Where All Concurrent Requests Stuck in Pending Phase [ID 182154.1]
Output Post Processor is Down with Actual Process is 0 And Target Process is 1 [ID 858813.1]
Solution which worked for me
SQL> set lines 200
SQL> column CONTROL_CODE format A15
SQL> select CONCURRENT_QUEUE_NAME, CONTROL_CODE , TARGET_NODE, NODE_NAME from FND_CONCURRENT_QUEUES where concurrent_queue_name like 'OAMGCS_%';
CONCURRENT_QUEUE_NAME CONTROL_CODE TARGET_NODE NODE_NAME
----------------------------------------- ------------------- --------------- ------------------
OAMGCS_ERPTSTAPP01 E ERPTSTAPP01
The standard codes used by Oracle Applications are as followings:
'A', -> 'Activate concurrent manager'
'D', -> 'Deactivate concurrent manager'
'E', -> 'Deactivated'
'N', -> 'Target node/queue unavailable'
'R', -> 'Restart concurrent manager'
'T', -> 'Terminate requests and deactivate manager'
'U', -> 'Update concurrent manager env inf.'
'V', -> 'Verify concurrent managers status'
'X', -> 'Terminated'
################STATUS_CODE Column############
A Waiting
B Resuming
C Normal
D Cancelled
E Error
F Scheduled
G Warning
H On Hold
I Normal
M No Manager
Q Standby
R Normal
S Suspended
T Terminating
U Disabled
W Paused
X Terminated
Z Waiting
################PHASE_CODE column############
C Completed
I Inactive
P Pending
R Running
################ This Solutions Not Work for me #################
SQL> update FND_CONCURRENT_QUEUES set control_code = null where concurrent_queue_name = 'OAMGCS_ERPTSTAPP01';
1 row updated.
SQL> update FND_CONCURRENT_QUEUES set TARGET_NODE='CBOS' where CONCURRENT_QUEUE_NAME='OAMGCS_ERPTSTAPP01';
1 row updated.
SQL> commit;
Commit complete.
SQL> set lines 200
SQL> col CONTROL_CODE format A15
SQL> select CONCURRENT_QUEUE_NAME, CONTROL_CODE , TARGET_NODE, NODE_NAME from FND_CONCURRENT_QUEUES where concurrent_queue_name like 'OAMGCS_%';
CONCURRENT_QUEUE_NAME CONTROL_CODE TARGET_NODE NODE_NAME
----------------------------------------- ------------------------ --------------------- -------------------
OAMGCS_ERPTSTAPP01 ERPTSTAPP01 ERPTSTAPP01
Stop and start Concurrent Managers using adcmctl.sh
The above Solution not worked for me and Concurrent Managers not came up.
################ Other Solutions (I've tried) ###################
Solution A worked for me:
Solution A:
CMCLEAN.SQL - Non Destructive Script to Clean Concurrent Manager Tables [ID 134007.1]
Run the cmclean.sql on admin node
Start the managers and test.
The above Solution worked for me and Concurrent Managers came up without any issue.
Solution B:
sqlplus apps/pwd
1. SQL> EXEC FND_CONC_CLONE.SETUP_CLEAN;
SQL>COMMIT;
SQL> EXIT;
Make sure following two sqls returns no rows
SQL> select node_name , node_mode, support_cp ,support_web , support_admin , support_forms from FND_NODES;
SQL> select * from FND_OAM_CONTEXT_FILES;
2. Run AutoConfig on the database tier.
3. Run AutoConfig on the apps tier.
4. Run cmclean.sql script .
SQL> sqlplus APPS/<Password>
SQL> select node_name , node_mode , support_cp ,support_web , support_admin , support_forms from FND_NODES;
SQL> select CONCURRENT_QUEUE_NAME from FND_CONCURRENT_QUEUES where CONCURRENT_QUEUE_NAME like 'FNDSM%';
5. Start all application services and check whether managers are up & re-test.
Solution C:
Apply this solution in last when the above one doesn’t work.
SQL> select node_name,target_node,control_code from fnd_concurrent_queues;
SQL> update apps.fnd_concurrent_queues set node_name = 'Node NAME' where node_name='Existing Node Name';
SQL> select NODE_NAME,NODE_MODE,STATUS from fnd_nodes;
SQL> select control_code,target_node,node_name,CONCURRENT_QUEUE_NAME from fnd_concurrent_queues;
SQL> UPDATE fnd_concurrent_queues set control_code = null;
SQL> select TARGET_NODE,NODE_NAME from fnd_concurrent_queues where node_name='<Existing Node Name>';
SQL> select TARGET_NODE,NODE_NAME from fnd_concurrent_queues where TARGET_NODE='<Existing Node Name>';
SQL> update fnd_concurrent_queues set NODE_NAME='<Node Name>' where NODE_NAME='<Source/Existing Node Name>';
SQL> update fnd_concurrent_queues set TARGET_NODE='<Node Name>' where TARGET_NODE='<Source/Exixting Node Name>';
SQL> UPDATE fnd_concurrent_queues set target_node = '<Node Name>';
SQL> UPDATE fnd_concurrent_queues set node_name = '<Node Name>';
SQL> Commit;
SQL> select control_code,target_node,node_name,CONCURRENT_QUEUE_NAME from fnd_concurrent_queues;
SQL>select TARGET_NODE,NODE_NAME from fnd_concurrent_queues where node_name='<Node Name>';
SQL>select TARGET_NODE,NODE_NAME from fnd_concurrent_queues where TARGET_NODE='<Node Name>';