How can you troubleshoot When all Users Can't Login to EBS by "Login failed. Please verify your login information or contact the system administrator" Even If Using a Right Password.
First check If all the services are up and running or not (Through WL console and backend).
Get the info about the recent changes or activities happened.
Check If tnsping SID is resolving from Application server.
If there is any issue with tns connectivity compare the tns file with backup and verify CDB level and PDB level remote and local listener parameters.
If there is 500--Internal Server Error error Check If you are able to connect to apps schema from App tier. Check If apps and applsys password is same or not. Validate applsys password in context file and dbc file. Check applsyspub user status in db_users.
Check If the status of index "FND_LOGINS_U1" was changed to unusable by unknown reason. If it is unusable then rebuild it by select owner,index_name,status from dba_indexes where index_name='FND_LOGINS_U1'; alter index applsys.FND_LOGINS_U1 rebuild;
What is fs_clone in adop ?
When a fs_clone is triggered the Patch file system is incrementally synchronized with the run edition file system.
What are all can be done in adop prepare phase ?
• Checks whether to perform a cleanup, which will be needed if the user failed to invoke cleanup after the cutover phase of a previous online patching cycle.
• Checks system configuration on each application tier node. A number of critical settings are validated to ensure that each application tier node is correctly registered, configured, and ready for patching.
• Checks for the existence of the “Online Patching In Progress” (ADZDPATCH) concurrent program. This program prevents certain predefined concurrent programs from being started, and as such needs to be active while a patching cycle is in progress (that is, while a database patch edition exists). If the ADZDPATCH program has not yet been requested to run, a request is submitted.
Note: ADZDPATCH is cancelled later on when the cutover phase is complete.
• Checks to see if the patch service has been created. adop requires that a special database service exists for the purpose of connecting to the patch edition. This service is created automatically, but its continued existence is validated on each prepare.
• During patching phase, adop will use this tns entry(SID_patch entry which has service name as ebs_patch) to connect to database.
• Invokes the TXK script $AD_TOP/patch/115/bin/txkADOPPreparePhaseSynchronize.pl to synchronize the patches which have been applied to the run APPL_TOP, but not the patch APPL_TOP.
• Checks the database for the existence of a patch edition, and creates one if it does not find one.
What are Database Editions and What objects can be included in DB editions ?
Database Editions are a new feature that allows you to have two versions of certain types of database objects with the same name, type and owner.
This is similar to a database schema, since you can have a same object (that is, an object with the same name and type) on two different schemas. Well now, the Edition level adds the schema to the equation, so now you can have two versions of a same object on the same schema.
Think about two versions of a database plsql function, version 1 and version 2, having both same name, type and being created on a same schema.
A database session context variable will define which session you’re working on, so depending on the session you’re connected to, you will access version 1 or version 2 of the plsql function.
The Online Patching architecture that Oracle defined works with two editions: one used for running purposes (that’s the code users are working with), and one for patching purposes (that’s the code that’s patched when you apply a patch). What this means basically is:
If you’re an end user, your environment is set with the running edition, so you access the running version of the code (let’s say that is the plsql function version 1)
If you’re a DBA applying a patch, your session is set to the patching edition, so you work with version 2 of the database function.
Editioned Objects
Unfortunately, object editioning doesn’t work with all object types. The reason behind this is that even though you can manage 2 versions of a piece of code with relative ease, it would be almost impossible to manage different versions of objects that keep data, like tables, since that would require managing different sets of data, one for each edition, which would need enormous amounts of storage and processing capacity.
So this is a hard constraint: object types that store data cannot be editioned. And there are many object types that store data, like indexes (they keep the values of the indexed attributes), or materialized views (the actually keep the data in the view), sequences (they keep the last value assigned), database links (they keep the target database information), etc.
Editionable objects are mostly those containing code like database packages, functions, procedures, views, synonyms, types, etc.
Also please consider that the “Potentially-Editioned” term is used to describe an object that belongs to one of the editionable types (like a database package) but is not editioned. So a “Potentially-Editioned” object “could” be editioned, but for any reason it’s not, like being installed on a non-editioned schema.
Non-Editioned Objects
Non-Editioned objects are those that cannot be editioned. This includes all objects that store data like tables, db links, materialized views, temporary tables, indexes, sequences, etc.
Effectively-Editioned Objects
A non-editionable object can never be editioned, though there are some workarounds available to apply “edition-like” features on those object types. Those are the effectively-editioned objects.
How to add a managed server ?
The below steps needs to be executed on run file system when there is no active ADOP cycle. During the next adop prepare, the Configuration Change Detector identifies that the addition has been made and the managed servers are automatically synced up from the run file system to the patch file system. The synchronization also gets done when fs_clone is executed.
Note: Managed server creation should be done only through the adProvisionEBS.pl script as mentioned in this section. Managed servers should not be created from the WebLogic Server Administration Console.
To add a new managed server of a specific service type, perform the following steps on the run file system:
Step 1:
Execute the following command to add a new managed server. This will create a managed server and add a new entry to the context file for starting and
For example, to add a managed server 'oacore_server2' of type 'oacore' with port 7203, run the following command:
$ perl $AD_TOP/patch/115/bin/adProvisionEBS.pl \
ebs-create-managedserver -contextfile=$CONTEXT_FILE \
-managedsrvname=oacore_server2 -servicetype=oacore \
-managedsrvport=7203 -logfile=$APPLRGF/TXK/addMS_oacoreserver2.log
Note: The port number for each managed server must be unique which means no two managed servers across the run and patch file systems can have the same port number.
Step 2:
Startup the managed server as follows:
$ sh ./admanagedsrvctl.sh start oacore_server2
Step 3:
Perform the following steps on all application tier nodes participating in the same cluster where this managed server is added:
Source the run file system.
Execute the following command to add details of the newly added managed servers into the OHS configuration files mod_wl_ohs.conf and apps.conf on the current node:
$ perl $FND_TOP/patch/115/bin/txkSetAppsConf.pl \
-contextfile= <context_file>\
-configoption=addMS \
-oacore=<host>.<domain>:port \
-oafm=<host>.<domain>:port \
-forms=<host>.<domain>:port \
-formsc4ws=<host>.<domain>:port
For example, if the managed server oacore_server2 has been added on host 'testserver' and domain
How can you change SYS password in Data Guard environment ?
The way to change the SYS password without breaking the redo transport service includes copying the primary database's password file to the standby server after changing the password.
What is Rebalancing in Oracle ASM ?
ASM enables you to change the storage configuration without having to take the database offline. It automatically rebalances—redistributes file data evenly across all the disks of the disk group—after you add disks to or drop disks from a disk group.
RAC
What is RAC ? What is the benefit of RAC over single instance database ?
In Real Application Clusters environments, all nodes concurrently execute transactions against the same database. Real Application Clusters coordinates each node's access to the shared data to provide consistency and integrity.
Benefits:
Improve throughput
High availability
Transparency
What is Oracle RAC One Node ?
Oracle RAC one Node is a single instance running on one node of the cluster while the 2nd node is in cold standby mode. If the instance fails for some reason then RAC one node detect it and restart the instance on the same node or the instance is relocate to the 2nd node incase there is failure or fault in 1st node. The benefit of this feature is that it provides a cold failover solution and it automates the instance relocation without any downtime and does not need a manual intervention. Oracle introduced this feature with the release of 11gR2 (available with Enterprise Edition).
Real Application Clusters
Oracle RAC is a cluster database with a shared cache architecture that overcomes the limitations of traditional shared-nothing and shared-disk approaches to provide a highly scalable and available database solution for all your business applications. Oracle RAC provides the foundation for enterprise grid computing.
Oracle's Real Application Clusters (RAC) option supports the transparent deployment of a single database across a cluster of servers, providing fault tolerance from hardware failures or planned outages. Oracle RAC running on clusters provides Oracle's highest level of capability in terms of availability, scalability, and low-cost computing.
One DB opened by multiple instances so the the db will be Highly Available if an instance crashes.
Cluster Software. Oracles Cluster ware or products like Veritas Volume Manager are required to provide the cluster support and allow each node to know which nodes belong to the cluster and are available and with Oracle Cluterware to know which nodes have failed and to eject then from the cluster, so that errors on that node can be cleared.
Oracle Clusterware has two key components Cluster Registry OCR and Voting Disk.
The cluster registry holds all information about nodes, instances, services and ASM storage if used, it also contains state information ie they are available and up or similar.
The voting disk is used to determine if a node has failed, i.e. become separated from the majority. If a node is deemed to no longer belong to the majority then it is forcibly rebooted and will after the reboot add itself again the the surviving cluster nodes.
Advantages of RAC (Real Application Clusters)
Availability – nodes can be added or replaced without having to shutdown the database
Scalability – more nodes can be added to the cluster as the workload increases
What is a virtual IP address or VIP ?
A virtual IP address or VIP is an alternate IP address that the client connections use instead of the standard public IP address. To configure VIP address, we need to reserve a spare IP address for each node, and the IP addresses must use the same subnet as the public network.
What is the use of VIP ?
If a node fails, then the node's VIP address fails over to another node on which the VIP address can accept TCP connections but it cannot accept Oracle connections.
Give situations under which VIP address failover happens:-
VIP addresses failover happens when the node on which the VIP address runs fails, all interfaces for the VIP address fails, all interfaces for the VIP address are disconnected from the network. Using virtual IP we can save our TCP/IP timeout problem because Oracle notification service maintains communication between each nodes and listeners.
What is the significance of VIP address failover ?
When a VIP address failover happens, Clients that attempt to connect to the VIP address receive a rapid connection refused error .They don't have to wait for TCP connection timeout messages.
What is voting disk ?
Voting Disk is a file that sits in the shared storage area and must be accessible by all nodes in the cluster. All nodes in the cluster registers their heart-beat information in the voting disk, so as to confirm that they are all operational. If heart-beat information of any node in the voting disk is not available that node will be evicted from the cluster. The CSS (Cluster Synchronization Service) daemon in the clusterware maintains the heart beat of all nodes to the voting disk. When any node is not able to send heartbeat to voting disk, then it will reboot itself, thus help avoiding the split-brain syndrome.
For high availability, Oracle recommends that you have a minimum of three or odd number (3 or greater) of votingdisks.
Voting Disk – is file that resides on shared storage and Manages cluster members. Voting disk reassigns cluster ownership between the nodes in case of failure.
The Voting Disk Files are used by Oracle Clusterware to determine which nodes are currently members of the cluster. The voting disk files are also used in concert with other Cluster components such as CRS to maintain the clusters integrity.
Oracle Database 11g Release 2 provides the ability to store the voting disks in ASM along with the OCR. Oracle Clusterware can access the OCR and the voting disks present in ASM even if the ASM instance is down. As a result CSS can continue to maintain the Oracle cluster even if the ASM instance has failed.
How many voting disks are you maintaining ?
By default Oracle will create 3 voting disk files in ASM.
Oracle expects that you will configure at least 3 voting disks for redundancy purposes. You should always configure an odd number of voting disks >= 3. This is because loss of more than half your voting disks will cause the entire cluster to fail.
You should plan on allocating 280MB for each voting disk file. For example, if you are using ASM and external redundancy then you will need to allocate 280MB of disk for the voting disk. If you are using ASM and normal redundancy you will need 560MB.
Why we need to keep odd number of voting disks ?
Oracle expects that you will configure at least 3 voting disks for redundancy purposes. You should always configure an odd number of voting disks >= 3. This is because loss of more than half your voting disks will cause the entire cluster to fail.
What are Oracle RAC software components ?
Oracle RAC is composed of two or more database instances. They are composed of Memory structures and background processes same as the single instance database.Oracle RAC instances use two processes GES(Global Enqueue Service), GCS(Global Cache Service) that enable cache fusion.Oracle RAC instances are composed of following background processes:
GTX0-j—Global Transaction Process
LMON—Global Enqueue Service Monitor
LMD—Global Enqueue Service Daemon
LMS—Global Cache Service Process
LCK0—Instance Enqueue Process
RMSn—Oracle RAC Management Processes (RMSn)
RSMN—Remote Slave Monitor
What are Oracle Clusterware processes for 10g ?
Cluster Synchronization Services (ocssd) — Manages cluster node membership and runs as the oracle user; failure of this process results in cluster restart.
Cluster Ready Services (crsd) — The crs process manages cluster resources (which could be a database, an instance, a service, a Listener, a virtual IP (VIP) address, an application process, and so on) based on the resource's configuration information that is stored in the OCR. This includes start, stop, monitor and failover operations. This process runs as the root user Event manager daemon (evmd) —A background process that publishes events that crs creates.
Process Monitor Daemon (OPROCD) —This process monitor the cluster and provide I/O fencing. OPROCD performs its check, stops running, and if the wake up is beyond the expected time, then OPROCD resets the processor and reboots the node. An OPROCD failure results in Oracle Clusterware restarting the node. OPROCD uses the hangcheck timer on Linux platforms.
RACG (racgmain, racgimon) —Extends clusterware to support Oracle-specific requirements and complex resources. Runs server callout scripts when FAN events occur.
What are Oracle database background processes specific to RAC ?
LMD—Global Enqueue Service Daemon
LMON—Global Enqueue Service Monitor
LCK0—Instance Enqueue Process
Oracle RAC instances use two processes, the Global Cache Service (GCS) and the Global Enqueue Service (GES). The GCS and GES maintain records of the statuses of each data file and each cached block using a Global Resource Directory (GRD). The GRD contents are distributed across all of the active instances.
What is Cache Fusion ?
Transfer of data across instances through private interconnect is called cachefusion. Oracle RAC is composed of two or more instances. When a block of data is read from datafile by an instance within the cluster and another instance is in need of the same block, It is easy to get the block image from the insatance which has the block in its SGA rather than reading from the disk. To enable interinstance communication Oracle RAC makes use of interconnects. The Global Enqueue Service(GES) monitors and Instance enqueue process manages the cache fusion.
What is SCAN ? (11gR2 feature)
Single Client Access Name (SCAN) is s a new Oracle Real Application Clusters (RAC) 11g Release 2 feature that provides a single name for clients to access an Oracle Database running in a cluster. The benefit is clients using SCAN do not need to change if you add or remove nodes in the cluster.
SCAN provides a single domain name via (DNS), allowing and-users to address a RAC cluster as-if it were a single IP address. SCAN works by replacing a hostname or IP list with virtual IP addresses (VIP).
Single client access name (SCAN) is meant to facilitate single name for all Oracle clients to connect to the cluster database, irrespective of number of nodes and node location. Until now, we have to keep adding multiple address records in all clients tnsnames.ora, when a new node gets added to or deleted from the cluster.
Single Client Access Name (SCAN) eliminates the need to change TNSNAMES entry when nodes are added to or removed from the Cluster. RAC instances register to SCAN listeners as remote listeners. Oracle recommends assigning 3 addresses to SCAN, which will create 3 SCAN listeners, though the cluster has got dozens of nodes.. SCAN is a domain name registered to at least one and up to three IP addresses, either in DNS (Domain Name Service) or GNS (Grid Naming Service). The SCAN must resolve to at least one address on the public network.
What are SCAN components in a cluster ?
2.SCAN IPs (3)
3.SCAN Listeners (3)
What is FAN ?
Fast application Notification as it abbreviates to FAN relates to the events related to instances,services and nodes. This is a notification mechanism that Oracle RAC uses to notify other processes about the configuration and service level information that includes service status changes such as, UP or DOWN events. Applications can respond to FAN events and take immediate action.
How to find location of OCR file when CRS is down ?
If you need to find the location of OCR (Oracle Cluster Registry) but your CRS is down.
When the CRS is down:
On Linux: /etc/oracle/ocr.loc
On Solaris: /var/opt/oracle/ocr.loc
When CRS is UP:
Set ASM environment or CRS environment then run the below command:
ocrcheck
In 2 node RAC, how many NIC's are r using ?
2 network cards on each clusterware node
Network Card 1 (with IP address set 1) for public network
Network Card 2 (with IP address set 2) for private network (for inter node communication between rac nodes used by clusterware and rac database)
In 2 node RAC, how many IP's are r using ?
## eth1-Public: 2
## eth0-Private: 2
## VIP: 2
How to find IP's information in RAC ?
# Do not remove the following line, or various programs
# that requires network functionality will fail.
127.0.0.1 localhost.localdomain localhost
## Public Node names
192.168.10.11 node1-pub.hingu.net node1-pub
192.168.10.22 node2-pub.hingu.net node2-pub
## Private Network (Interconnect)
192.168.0.11 node1-prv node1-prv
192.168.0.22 node2-prv node2-prv
## Private Network (Network Area storage)
192.168.1.11 node1-nas node1-nas
192.168.1.22 node2-nas node2-nas
192.168.1.33 nas-server nas-server
## Virtual IPs
192.168.10.111 node1-vip.hingu.net node1-vip
192.168.10.222 node2-vip.hingu.net node2-vip
What is difference between RAC ip addresses ?
Public IP address is the normal IP address typically used by DBA and SA to manage storage, system and database. Public IP addresses are reserved for the Internet.
Private IP address is used only for internal clustering processing (Cache Fusion) (aka as interconnect). Private IP addresses are reserved for private networks.
VIP is used by database applications to enable fail over when one cluster node fails. The purpose for having VIP is so client connection can be failover to surviving nodes in case there is failure.
Can application developer access the private ip ?
No. private IP address is used only for internal clustering processing (Cache Fusion) (aka as interconnect)
What is the role of daemon process in Oracle RAC ?
Diagnostic Daemon or Daemon process is a lightweight process, it uses the DIAG framework to monitor the health of the cluster. It captures information for later diagnosis in the event of failures. It will perform any necessary recovery if an operational hang is detected.
How to apply weblogic patches in 11g and 12c
After refresh weblogic console is not coming up. What will you check ?
filter rule parameter from deny to allow
How will you create an application node and What are the steps to do when you refresh multi node app ?
How to clear Temp Tablespace data ?
How many pds can be allocated in 19c ? and how many can be used for EBS ?
256 and 1 for EBS
What is the purpose of utlfile directory in 19c database ?
adzdpatch purpose in adop and where will it ends?
patch cycle creation
What files will be created or configured as part of adcfgclone ?
FMW_HOME, INST_TOP
What will be there in db_editions ?
Can there be same name for both CDB and PDB & What will be there in v$database name
Yes we can have same name and v$database always shows CDB name ?
In adop patch got failed in apply phase because of pre-req patch missing. What will you do ?
We will apply pre-req patch with options restart=no abondon=yes and then will apply the main patch with restart=yes and abondon=no
Database upgrade from 12c to 19c ?
Difference b/w 12c and 19c ?
What are wait_events ?
AD/TXK 14 upgrade feature?
EBS_APPLPTMP dba_directory is created
-- Upgrade Steps from 11g to 12c and 12c to 19c
-- How to minimise downtime for db upgrade
-- Multinode to multinode cloning with shared appl_top without shared appl_top
-- what are the entries are there in parse file
-- if it is a shared node how you will you add second node and if it is a non shared how you will you add second node((i) till DMZ it is shared and ii) your CM and web node is shared and DMZ is not shared ).
-- OPP process is going down. Requests are not processing. where will you check first.
-- How to add ASM disk. Explain in detail.
-- you have a two node RAC, when you try to start the DB with srvctl command, it get failed due to DB name not recognized. but when you try to start DB on both nodes with startup command it is working. what could be the reason.(what are the parameters you will check it out). (what is VIP in RAC, have you ever faced a scenario like where the actual node went down and it moved to another node).
Ans ) Troubleshooting when srvctl can't start RAC instance, but sqlplus can start it (Doc ID 844272.1)
-- Team is not able to open the log and out file from frontend. what could be the reason.
-- In Weblogic we have the inactive connection time out parameter. what purpose this parameter is being used.
-- Team is reporting request log and out files taking time to open. there is a delay. how you troubleshoot the issue.
-- pre requisite patch is missing in adop. how to apply the prerequisite patch.
Hot backup restore and cold backup restore
voting disk odd number ?
Why scan listener has 3 ips ?
what is the purpose of Virtual IP ?
How will you increase the mount space in OCI ?
What is the driver file patch we apply before upgrade to R12.2 ?
RMAN Level 0 and Level 1, Differential and Cumulative backups ?
What is Autoconfig 1st step from where it will create the templates.
What is the command to upgrade from 11g to 12c
Why the password should be same for apps and APPLSYS ?
No comments:
Post a Comment