Step by Step - Configure Direct NFS Client (DNFS) on Linux [ID 762374.1]

郑全

Step by Step - Configure Direct NFS Client (DNFS) on Linux [ID 762374.1]

2011-07-19 16:31:55| 分类： Oracle | 标签： |字号大中小 订阅

In this Document
Goal
Solution
References

---

Applies to:

Oracle Server - Enterprise Edition - Version: 11.1.0.6 to 11.2.0.1 - Release: 11.1 to 11.2
Generic Linux

Goal

The purpose of this document is to provide information about the Direct NFS client and Step by Step to configure Direct NFS client.

Note :-The following steps are purely provided for evaluative and educational purpose only.Any issues with configuring the NFS server/client on Linux ie PART A ,you need to contact your Vendor.Oracle will not provide support/troubleshoot PART A.

WARNING:
The DNFS Guide says to enable the init.ora param filesystem_io to enable direct I/O support.
Doing this, all DB access to all files will be via DIO.
There is no way to only enable direct IO for certain files and exclude others.
So ensure ALL the places where are located your database files (including pfile/spfile, logs,....) are configured to accept Direct IO.

Solution

Step by Step

To use Direct NFS Client, the NFS file systems must first be mounted and available over regular NFS mounts.

PART A -- SETTING UP THE NFS SERVER ON LINUX

Preparing NFS server for Oracle Direct NFS mount filesystem.

If you are already setup the NFS ,skip this step and go to PART B :-'Setup DNFS'

i) For setting up the NFS server we need to know the UID of the software owner and GID of the DBA group. The UID and GID should also be the same on all the cluster nodes.

To find out the UID and GID issue the id command as the Oracle software owner (e.g. oracle) on one of the cluster nodes,

# id uid=500(oracle) gid=500(dba) groups=500(dba)

In this case the UID is 500 and the GID is also 500.

ii) As root, create the directory for the oracle files on the NFS server and set the ownership of this directory to this UID and the GID,

# mkdir /oraclenfs
# chown 500:500 /oraclenfs

iii) Add this directory to the NFS exports file /etc/exports.

This file should now contain a line like this

/oraclenfs *(rw,sync,all_squash,anonuid=500,anongid=500)

The anonuid and anongid should contain the UID and GID we found for the oracle user and dba group on the cluster nodes; in this case 500 and 500.

iv) Make sure the NFS server will get started during boot of this server. I.E. for RedHat Linux this could be done like this;

chkconfig --level 345 nfs on

v) Now start the NFS server process on the NFS server. On RedHat Linux this could be done like this;

service nfs start

vi) If the new export directory is added to the /etc/exports file while the NFS server process was already running, restart the NFS server or re-export with the command “exportfs –a”.
Check if the oraclefs directory is exported correctly by issuing the exportfs –v command.

This command should return a line like this;

# [root@stgasm ~]# exportfs -v
/oraclenfs *(rw,wdelay,root_squash,all_squash,anonuid=500,anongid=500)

MOUNTING NFS ON THE CLIENT NODE / CLUSTER NODES ON LINUX

i) To be able to mount the NFS export, as the root user create an empty directory on each cluster node named /oradata1

ii) Make sure the NFS export is mounted on the cluster nodes during boot time by adding the following line to the /etc/fstab file on each cluster node;

stgasm:/oraclenfs /oradata1 nfs rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,actimeo=0,vers=3,timeo=600 0 0

iii) Mount the NFS export by executing the mount /oradata1 command on each server.

iv) Check if the NFS export is correctly mounted with the mount command.

This should return a line like this;

# mount
stgasm:/oraclenfs on /oradata1 type nfs (rw,bg,hard,intr,rsize=32768,wsize=32768,tcp,noac,nfsvers=3, timeo=600,addr=10.177.52.158)

PART B -- Configure Direct NFS Client (DNFS)

i) Configure oranfstab file

Direct NFS Client can use a new configuration file ‘oranfstab’ or the mount tab file (/etc/mtab on Linux) to determine the mount point settings for NFS storage devices.

This file is required only for configuring the Direct NFS for load balancing and specfic to single database.You can still enable the Direct NFS without configuring oranfstab file.DNFS will take mount point settings for NFS from /etc/mtab on Linux

In RAC,the oranfstab must be configured on all nodes and keep /etc/oranfstab file synchronized on all nodes.

(When the oranfstab file is placed in $ORACLE_HOME/dbs, the entries in the file are specific to a single database. In this case, all nodes running an Oracle RAC database use the same ORACLE_HOME/dbs/oranfstab file.

When the oranfstab file is placed in /etc, then it is globally available to all Oracle databases, and can contain mount points used by all Oracle databases running on nodes in the cluster, including single-instance databases. However, on Oracle RAC systems, if the oranfstab file is placed in /etc, then you must replicate the file /etc/oranfstab file on all nodes, and keep each /etc/oranfstab file synchronized on all nodes, just as you must with the /etc/fstab file.

For reference, Direct NFS searches for mount entries in the following order:
1. $ORACLE_HOME/dbs/oranfstab
2. /etc/oranfstab
3. /etc/mtab

configure the oranfstab as like below

$ORACLE_HOME/dbs/oranfstab
server: stgasm <=== NFS server Host name
path: 10.177.52.158 <--- First path to NFS server ie NFS server NIC
local: 10.177.52.151 <--- First client-side NIC
path: 10.177.52.159 <--- Second path to NFS server ie NFS server NIC (For load balance purpose)
local: 10.177.52.151 <--- Second client-side NIC (For load balance purpose)
export: /oraclenfs mount: /oradata1

ii) Enable the Direct NFS Client ODM Library

cd $ORACLE_HOME/lib

mv libodm11.so libodm11.so_bak

ln –s libnfsodm11.so libodm11.so

iii) Re-start the Database instance.

alert log shows Direct NFS client is enabled by checking the below message

Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 2.0

iv) Connect to DB instance and move existing datafiles to NFS drive or create tablespace on NFS drive to check DNFS is working or not

SQL> select * from v$dnfs_servers;

ID SVRNAME DIRNAME MNTPORT NFSPORT WTMAX RTMAX
-- -------------------- ---------- ---------- ---------- ---------- ----------
1 10.177.52.158 /oraclenfs 700 2049 0 0

PART C -- DNFS Workshop.

DNFS Workshop is located at: dnfs_workshop.pdf

References

http://www.oracle.com/technology/deploy/performance/pdf/directnfsclient_11gr1_twp.pdf
http://st-doc.us.oracle.com/11/111/install.111/b28263/stor_rac.htm?term=direct+nfs#BABFDIDE