IP Address Types in Oracle 11g RAC
Oracle 11g RAC utilizes several distinct IP address categories:
- Public IP: Used for normal cluster operations and client connections
- Virtual IP (VIP): Provides failover capabilities for application connections
- Single Client Access Name (SCAN) VIP: Allows clients to connect using a single name regardless of cluster node changes
- Private IP: Reserved for inter-node cluster communication
Common Scenarios Requiring IP Changes
- Data center migration with network infrastructure updates
- Transition from development to production IP addresses
- Network interface card bonding or unbonding operations
- Virtualization adapter type modifications
Environment Details
The demonstration environment uses VirtualBox with the following initial configuration:
#public ip
192.168.56.150 racnode1
192.168.56.152 racnode2
#private ip
10.10.10.11 racnode1-priv
10.10.10.12 racnode2-priv
#virtual ip
192.168.56.151 racnode1-vip
192.168.56.153 racnode2-vip
#scan ip
192.168.56.160 rac-cluster-scan
Target configuration after modification:
#public ip
192.168.1.50 racnode1
192.168.1.52 racnode2
#private ip
10.10.10.50 racnode1-priv
10.10.10.52 racnode2-priv
#virtual ip
192.168.1.51 racnode1-vip
192.168.1.53 racnode2-vip
#scan ip
192.168.1.60 rac-cluster-scan
Implementation Procedure
Step 1: Shutdown Database, Listeners, and CRS
Gracefully shut down the database, listeners, and Cluster Ready Services on all nodes:
[grid@racnode1 ~]$ srvctl stop database -d orcl -o immediate
[grid@racnode1 ~]$ srvctl stop listener
[grid@racnode1 ~]$ which crsctl
/opt/app/11.2.0/grid/bin/crsctl
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/crsctl stop crs
[root@racnode2 ~]# /opt/app/11.2.0/grid/bin/crsctl stop crs
Prevent automatic restart of listeners and database upon CRS startup:
--check and disable listener auto-start
srvctl status listener -n racnode1
srvctl status listener -n racnode2
srvctl disable listener -n racnode1
srvctl disable listener -n racnode2
--check and disable database auto-start
srvctl config database -db orcl
--execute as root:
cd /opt/app/11.2.0/grid/bin
./srvctl disable database -d orcl
Step 2: Update /etc/hosts Configuration
Create a backup of the hosts file on both nodes:
[root@racnode1 ~]# cp /etc/hosts /etc/hosts.bak
[root@racnode2 ~]# cp /etc/hosts /etc/hosts.bak
Modify the hosts file to reflect new public IP addresses, VIPs, and SCAN VIPs:
#public ip
192.168.1.50 racnode1
192.168.1.52 racnode2
#virtual ip
192.168.1.51 racnode1-vip
192.168.1.53 racnode2-vip
#scan ip
192.168.1.60 rac-cluster-scan
Private IP modifications will be handled separately in a later section.
Step 3: Modify Operating System Network Configuration
3.1 Document Current Public Network Interface
[root@racnode1 ~]# ifconfig -a
eth2 Link encap:Ethernet HWaddr 08:00:27:95:22:E3
inet addr:192.168.56.150 Bcast:192.168.56.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe95:22e3/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
[root@racnode2 ~]# ifconfig -a
eth2 Link encap:Ethernet HWaddr 08:00:27:1A:5A:7A
inet addr:192.168.56.152 Bcast:192.168.56.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe1a:5a7a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
3.2 Udpate VirtualBox Network Adapter
Change the adapter type from "Host-Only Adapter" to "Bridged Adapter" in VirtualBox settings.
3.3 Update Public IP Addresses
Modify network interface configuration files and restart interfaces:
vi /etc/sysconfig/network-scripts/ifcfg-eth2
ifdown eth2
ifup eth2
ifconfig eth2
ping 192.168.1.50
ping 192.168.1.52
Example configuration after modification:
[root@racnode1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth2
DEVICE=eth2
TYPE=Ethernet
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.1.50
NETMASK=255.255.255.0
[root@racnode1 ~]# ifconfig eth2
eth2 Link encap:Ethernet HWaddr 08:00:27:95:22:E3
inet addr:192.168.1.50 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe95:22e3/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Verify connectivity from another machine on the network before proceeding.
Step 4: Start CRS
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/crsctl start crs
[root@racnode2 ~]# /opt/app/11.2.0/grid/bin/crsctl start crs
Confirm database and listener resources remain offline:
ora.LISTENER.lsnr
OFFLINE OFFLINE racnode1
OFFLINE OFFLINE racnode2
ora.orcl.db
1 OFFLINE OFFLINE Instance Shutdown
2 OFFLINE OFFLINE Instance Shutdown
Step 5: Modify Public IP, VIP, and SCAN VIP
5.1 Update Public IP Configuration
--display current configuration:
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/oifcfg getif
eth2 192.168.56.0 global public
eth3 10.10.10.0 global cluster_interconnect
--remove old and set new values:
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/oifcfg delif -global eth2
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/oifcfg setif -global eth2/192.168.1.0:public
--verify changes:
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/oifcfg getif
eth3 10.10.10.0 global cluster_interconnect
eth2 192.168.1.0 global public
[root@racnode2 ~]# /opt/app/11.2.0/grid/bin/oifcfg getif
eth3 10.10.10.0 global cluster_interconnect
eth2 192.168.1.0 global public
5.2 Update VIP Configuration
Stop VIP resources:
[grid@racnode1 ~]$ srvctl stop vip -n racnode1
[grid@racnode1 ~]$ srvctl stop vip -n racnode2
Review current VIP settings:
[grid@racnode1 ~]$ olsnodes -s
racnode1 Active
racnode2 Active
[grid@racnode1 ~]$ srvctl config vip -n racnode1
VIP exists: /racnode1-vip/192.168.1.51/192.168.56.0/255.255.255.0/eth2, hosting node racnode1
[grid@racnode1 ~]$ srvctl config vip -n racnode2
VIP exists: /racnode2-vip/192.168.1.53/192.168.56.0/255.255.255.0/eth2, hosting node racnode2
Apply new VIP configuration:
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/srvctl modify nodeapps -n racnode1 -A 192.168.1.51/255.255.255.0/eth2
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/srvctl modify nodeapps -n racnode2 -A 192.168.1.53/255.255.255.0/eth2
Confirm VIP changes:
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/srvctl config vip -n racnode1
VIP exists: /racnode1-vip/192.168.1.51/192.168.1.0/255.255.255.0/eth2, hosting node racnode1
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/srvctl config vip -n racnode2
VIP exists: /racnode2-vip/192.168.1.53/192.168.1.0/255.255.255.0/eth2, hosting node racnode2
Restart VIP, listener, and database resources:
[grid@racnode1 ~]$ srvctl start vip -n racnode1
[grid@racnode1 ~]$ srvctl start vip -n racnode2
[grid@racnode1 ~]$ srvctl start listener
[grid@racnode1 ~]$ srvctl start database -d orcl
5.3 Verify LOCAL_LISTENER Paramter
Although Grid Infrastructure automatically updates LOCAL_LISTENER for VIP changes, Bug 22824602 may cause issues in certain scenarios. Manual verification is recommended:
su - grid
lsnrctl status listener_scan1
lsnrctl status
sqlplus /as sysasm
show parameter local_listener
--if correction needed:
alter system set local_listener='(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.1.51)(PORT=1521))' sid='+ASM1';
alter system set local_listener='(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.1.53)(PORT=1521))' sid='+ASM2';
alter system register;
su - oracle
sqlplus /as sysdba
show parameter local_listener
--if correction needed:
alter system set local_listener='(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.1.51)(PORT=1521))' sid='orcl1';
alter system set local_listener='(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.1.53)(PORT=1521))' sid='orcl2';
alter system register;
5.4 Update SCAN VIP
Review current SCAN VIP configuration:
[grid@racnode1 ~]$ srvctl config scan
SCAN name: rac-cluster-scan, Network: 1/192.168.1.0/255.255.255.0/eth2
SCAN VIP name: scan1, IP: /rac-cluster-scan/192.168.56.160
Stop SCAN listener and SCAN resources:
[grid@racnode1 ~]$ srvctl stop scan_listener
[grid@racnode1 ~]$ srvctl stop scan
--confirm status:
[grid@racnode1 ~]$ srvctl status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is not running
[grid@racnode1 ~]$ srvctl status scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is not running
Apply SCAN VIP modification using root privileges:
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/srvctl modify scan -n rac-cluster-scan
Restart SCAN resources:
[grid@racnode1 ~]$ srvctl modify scan_listener -u
[grid@racnode1 ~]$ srvctl start scan_listener
Verify SCAN VIP updates:
[grid@racnode1 ~]$ srvctl config scan
SCAN name: rac-cluster-scan, Network: 1/192.168.1.0/255.255.255.0/eth2
SCAN VIP name: scan1, IP: /rac-cluster-scan/192.168.1.60
[grid@racnode1 ~]$ srvctl config scan_listener
SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1521
Review complete cluster status:
[grid@racnode1 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA1.dg
ONLINE ONLINE racnode1
ONLINE ONLINE racnode2
ora.FRA1.dg
ONLINE ONLINE racnode1
ONLINE ONLINE racnode2
ora.LISTENER.lsnr
ONLINE ONLINE racnode1
ONLINE ONLINE racnode2
ora.OCR1.dg
ONLINE ONLINE racnode1
ONLINE ONLINE racnode2
ora.asm
ONLINE ONLINE racnode1 Started
ONLINE ONLINE racnode2 Started
ora.net1.network
ONLINE ONLINE racnode1
ONLINE ONLINE racnode2
ora.ons
ONLINE ONLINE racnode1
ONLINE ONLINE racnode2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE racnode1
ora.cvu
1 ONLINE ONLINE racnode2
ora.racnode1.vip
1 ONLINE ONLINE racnode1
ora.racnode2.vip
1 ONLINE ONLINE racnode2
ora.orcl.db
1 ONLINE ONLINE racnode1 Open
2 ONLINE ONLINE racnode2 Open
ora.scan1.vip
1 ONLINE ONLINE racnode1
Step 6: Modifying Private IP Addresses
Private IP modifications require different handling due to additional configuration storage beyond OCR.
6.1 Backup GPNP Profile
Starting with Oracle 11.2 Grid Infrastructure, private network configuration exists in both OCR and GPNP profile. Create backups before proceeding:
--on both nodes as grid user:
[grid@racnode1 ~]$ cd /opt/app/11.2.0/grid/gpnp/racnode1/profiles/peer/
[grid@racnode1 peer]$ cp -p profile.xml profile.xml.bk
[grid@racnode2 ~]$ cd /opt/app/11.2.0/grid/gpnp/racnode2/profiles/peer/
[grid@racnode2 peer]$ cp -p profile.xml profile.xml.bk
Verify cluster is operational:
[grid@racnode1 ~]$ olsnodes -s
racnode1 Active
racnode2 Active
6.2 Review Current Private Network Configuration
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/oifcfg getif
eth3 10.10.10.0 global cluster_interconnect
eth2 192.168.1.0 global public
In this scenario, the private subnet remains unchanged, so no oifcfg modifications are needed.
6.3 Disable and Stop Cluster Services
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/crsctl stop crs
[root@racnode2 ~]# /opt/app/11.2.0/grid/bin/crsctl stop crs
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/crsctl disable crs
[root@racnode2 ~]# /opt/app/11.2.0/grid/bin/crsctl disable crs
6.4 Update Operating System Private Interface
[root@racnode1 ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth3
[root@racnode1 ~]# ifdown eth3
[root@racnode1 ~]# ifup eth3
Update /etc/hosts with new private addresses:
#private ip
10.10.10.50 racnode1-priv
10.10.10.52 racnode2-priv
6.5 Enable and Start Cluster Services
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/crsctl enable crs
[root@racnode2 ~]# /opt/app/11.2.0/grid/bin/crsctl enable crs
[root@racnode1 ~]# /opt/app/11.2.0/grid/bin/crsctl start crs
[root@racnode2 ~]# /opt/app/11.2.0/grid/bin/crsctl start crs
6.6 Cleanup
Since the private subnet designation remains the same, oifcfg delif is not required.
Reference Documents
- How to Modify Public Network Information including VIP in Oracle Clusterware (Doc ID 276434.1)
- How to Modify Private Network Information in Oracle Clusterware (Doc ID 283684.1)
- How to Modify SCAN Setting or SCAN Listener Port after Installation (Doc ID 972500.1)
- How to update the IP address of the SCAN VIP resources (ora.scan{n}.vip) (Doc ID 952903.1)