on 2019/06/21 15:34
Document purpose | This Reference Architecture provides an overview of the architecture of a virtualized business continuity solution that uses the EMC Celerra unified storage platform and Oracle Database 11g on Linux using DNFS and NFS. Key elements, migration details, and hardware and software resources are among the topics discussed. Information in this document can be used:
|
Solution purpose | The purposes of this solution are to:
|
The business challenge | Midsize enterprises face the same challenges as their larger counterparts when it comes to managing database environments. These challenges include:
Unlike large enterprises, midsize enterprises are constrained by smaller budgets and cannot afford a custom, one-off solution. This makes the process of creating a database solution for midsize enterprises even more challenging than for large enterprises. Traditional database deployments have relied on utilizing fiber channel protocol storage infrastructure in a physically managed environment. Through significant technology innovations and close collaboration with both Oracle and VMware, EMC is able to offer new advanced solutions, incorporating flexible, tiered storage functionality and multiple protocols, as well as optimized use of virtualization. Customers’ choice to leverage different storage protocols, functionality, and virtualization should be based on their individual application requirements for utilization, performance, flexibility, and cost. EMC designed, tested, and documented this solution to specifically address the following challenges:
Focus on deploying Oracle over lower-cost NFS protocol environments to reduce cost and improve efficiency of networked storage
Align the technology strengths of networked storage with virtualization software to increase database server and storage utilization, with higher leverage at scale
Simplify database resource and infrastructure administration utilizing NFS and virtualization. Empower DBAs with enhanced solutions for backup, data movement, and data protection. This solution documents implementation procedures and best practices for deploying a single platform solution to quickly and effectively to achieve these benefits. |
The technology solution | This solution demonstrates how organizations can:
|
Introduction | This section describes the components of the solution, and explains some of the key terminology and concepts that are used in this document. |
Key terms defined | The following table describes several terms used in this document:
|
Solution attributes | The following table describes the solution attributes that are included in this solution:
|
Solution components | The following table describes the solution components that are included in this solution:
|
Overview | All database objects are stored on an NFS mount. In the case of Oracle Database 11g, datafiles, tempfiles, controlfiles, online redo logfiles, and archive logfiles are accessed using the DNFS protocol. The Oracle software stack is booted on virtualized hardware and incorporates a four-node VMware High Availability (HA) cluster. For the purpose of high availability, VMotion is used to migrate a virtual machine running the production database from one ESX server to another without any downtime. Each ESX server is connected to the production storage using a dedicated storage network; a separate set of dedicated TCP/IP networks is used for the cluster interconnect. Then based on the need, VMware DRS is used to automatically allocate resources to multiple virtual machines for load-balancing purposes. Two sites connected by a WAN are used in the solution, one site is used for production and the other site is used as a disaster recovery target. Oracle 11g or 10g for x86-64 is run on Red Hat Enterprise Linux or on Oracle Enterprise Linux on virtual machines. The solution also includes virtualized servers for use as test/dev and basic protect targets. Virtualization of the test/dev and disaster recovery (DR) target servers is supported using VMware ESX. The Replication Manager server is also virtualized. |
Production site | The production site consists of:
|
Disaster recovery target site | The disaster recovery target site consists of:
|
Storage layout | The following table describes how each Oracle file type and database object is stored for this solution:
For implementations using Oracle Database 11g, all files are accessed using DNFS. For implementations using Oracle Database 10g, all files are accessed using KNFS.
|
Network architecture | The design implements the following physical connections:
|
Introduction | Customers often request the ability to migrate a virtualized Oracle Database across storage protocols. In response to this, the Oracle Consulting (CSV) group has validated that customers who have an Oracle Database operating in a virtualized environment can migrate data from:
Customers might be interested in cross-protocol migration because testing and development activities can be supported on less expensive, simpler NFS storage, while production can be carried out on higher-performing ASM/FCP storage. Customers may also want to use the storage replication technologies available on a different protocol from the production database. For example, customers may wish to use RecoverPoint to support remote replication but have production on NFS. The migrations were performed with minimal performance impact and no downtime on the virtualized production database. |
SAN to NAS migration diagram | The following diagram is a high-level view of the migration component: |
Migrating an online Oracle Database from SAN to NAS | These steps were followed to perform the SAN to NAS migration operation:
|
The result, as stated above, is that the virtualized production FCP-mounted database can be migrated to NFS with minimal performance impact and no downtime. |
NAS to SAN migration diagram | The following diagram is a high-level view of the NAS to SAN migration component. |
Migrating an online Oracle Database from NAS to SAN | These steps were followed to perform the NAS to SAN migration operation:
|
The result, as stated above, is that the virtualized production NFS-mounted database can be migrated to FCP/ASM with minimal performance impact and no downtime.
Introduction | This section provides an overview of the technologies that are used in this solution. |
EMC Celerra unified storage platform | The Celerra is a remarkably versatile device. Celerra includes a world-class network-attached storage (NAS) array combined with the functionality and performance of the leading midrange storage area network (SAN) array. Celerra provides both NAS and SAN functionality and performance without compromise. The key features provided by the Celerra are described in the following table:
|
EMC Replication Manager | EMC Replication Manager manages EMC point-in-time replication technologies through a centralized management console. Replication Manager coordinates the entire data replication process – from discovery and configuration to the management of multiple application consistent disk-based replicas. Replication Manager can auto-discover your replication environment and enable streamlined management by scheduling, recording, and cataloging replica information including auto-expiration. Replication Manager integrates with the Oracle Database server and provides an easy interface to create and manage Oracle replicas. |
EMC Celerra SnapSure | SnapSure creates a logical point-in-time image (checkpoint) of a production file system (PFS) that reflects the state of the PFS at the point in time when the checkpoint is created. SnapSure can maintain up to 96 PFS checkpoints while allowing PFS applications continued access to the real-time data. How SnapSure worksThe principle of SnapSure is “copy on first write” (COFW). When a block within the PFS is modified, a copy containing the block’s original content is saved to a separate volume called the SavVol. Subsequent changes made to the same block in the PFS are not copied into the SavVol. The original blocks from the PFS (in the SavVol) and the unchanged PFS blocks (remaining in the PFS) are read by SnapSure according to a bitmap and blockmap data-tracking structure. These blocks combine to provide a complete point-in-time file system image called a checkpoint. |
VMware ESX | VMware ESX is the flagship virtualization server product from VMware and is the market leader in server virtualization. VMware ESX provides a high-performance, robust, fault-tolerant and high-availability virtualization solution. |
Oracle software stack | The Oracle software stack covered by this solution consists of:
|
Oracle DNFS | DNFS is a new feature introduced in Oracle Database 11g; it integrates the NFS client directly inside the database kernel instead of the operating system kernel. DNFS provides significant performance, manageability, and efficiency benefits over KNFS. Better performanceTransactions per second and user load are both higher with DNFS than with KNFS, and this enables organizations to gain more output from the same infrastructure. CPU costs on both the database server and the file server are lower. In addition, port scaling with DNFS is much better, enabling higher bandwidth and thus higher scaling. High availabilityLoad balancing and high availability (HA) are managed internally within the DNFS client. Concurrent I/OThe DNFS client performs concurrent I/O by bypassing the operating system. The benefits of this are:
Optimized for database workloadsDNFS is optimized for database workloads and supports asynchronous I/O, which is suitable for most databases. It delivers optimized performance by automatically load balancing across the available paths. Load balancing in DFNS is almost invariably superior to the conventional Linux kernel NFS (KNFS). |
VMware Distributed Resource Scheduler | VMware Distributed Resource Scheduler (DRS) is a cluster feature of VMware vCenter Server; it provides dynamic load balancing and resource sharing for multiple virtual machines across ESX servers. DRS uses VMotion as the underlying transport feature to move virtual machines from one ESX server to another. |
Reference architecture diagram | The following diagram depicts the overall physical architecture of the solution. |
Profile characteristics | For information on the validated environment profile and performance results, refer to the Proven Solution Guide for this solution. This information can be accessed on EMC Powerlink®, EMC.com, and the EMC|KB.WIKI. |
Hardware | The hardware used to validate the solution is listed below. |
Equipment | Quantity | Configuration |
EMC Celerra unified storage platforms (included an EMC CLARiiON CX4 back-end storage array) | 2 |
|
Gigabit Ethernet switches | 5 | 24 ports per switch |
VMware ESX HA cluster servers | 4 |
|
Virtualization server (VMware ESX) | 1 |
|
Software | The software used to validate the solution is listed below. |
Software | Version |
Oracle Enterprise Linux | 5.2 |
VMware ESX Server/vSphere | 4.0 |
Microsoft Windows Server 2003 Standard Edition | 2003 |
Oracle Database Standard Edition | 11g (11g Ver. 11.1.0.7.0) or 10g |
Quest Benchmark Factory for Databases | 5.8.1 |
EMC Celerra Manager Advanced Edition | 5.6 |
EMC Navisphere® Agent | 6.26.0.2.24 |
EMC Replication Manager | 5.2.2.0 |
EMC FLARE® | 04.28.000.5.504 |
EMC DART | 5.6.44-4 |
EMC Navisphere Management | 6.28 |
Summary | This section provides a summary of the solution and of the business challenges that it addresses. |
Reduced total cost of ownership | In any reasonable configuration, the database server's CPU is the most precious component of the entire architecture. Therefore, the over-arching principle of EMC's Oracle Database 11g and 10g solutions for midsize enterprises is to free up the database server's CPU (as well as memory and I/O channels) from utility operations such as backup and recovery, disaster recovery staging, test/dev, and cloning. The highest and best use of the database server’s CPUs is to parse and execute the SQL statements that are required by the application user. CPU usageThis solution reduces the load on the database server CPU by using:
|
Virtualization | The use of virtualization for the production database server provides manageability and ease-of-use advantages. In a scale-out context, virtualization can provide superior performance and scalability compared to physically booted configurations - even when using hardware identical to that used in the physically booted configuration. Utility servers, such as a test/dev target and basic protect target, are more easily and conveniently managed as virtual machines than as physically booted Oracle Database servers. The advantages of consolidation, flexible migration and so on, which are the mainstays of virtualization, apply to these servers as well. Reduced costsOne of the main challenges faced by the customer is to reduce cost by utilizing infrastructure effectively. Virtualization enables reduction in the number of servers and related IT hardware in the data center. The other main feature is the ability to move a running virtual machine production database from one physical sever to another physical server without any downtime. |
Distributed resource scheduling | VMware DRS provides the ability to distribute workload across multiple ESX servers by migrating virtual machines according to resource consumption and demand. VMware DRS dynamically manages pooled resources from multiple ESX servers. VMware DRS aggregates and centrally manages the resources of multiple hosts as resource pools. Different sets of rules and priorities can be defined for individual Oracle virtual machines based on the criticality. When an Oracle Database server virtual machine operating within a resource pool experiences increased utilization of resources, DRS will try to allocate the resources from the centralized pool based on the rules and priorities. |
High availability | VMotion The use of VMware VMotion to migrate an Oracle Database instance from one piece of hardware to another was achieved in our tests with virtually no performance impact and no downtime on the running Oracle Database instance. This provides a very high level of manageability and downtime reduction for tasks such as software and hardware upgrades. VMware HA clusterThe use of a VMware HA cluster provides further high-availability advantages to the virtualized solution. EMC has validated the use of a VMware HA cluster with Oracle Database 11g and 10g in a single-instance scale-out environment. In addition, a VMware HA cluster can be used to automate failover in the event of a hardware or software failure. Fault ToleranceThe use of VMware Fault Tolerance provides true zero downtime for Oracle VMware-based database servers. This means that a database server running on VMware can be protected from unplanned downtime. This is a significant improvement over the previous version of VMware and is provided on the HA cluster. The use of Fault Tolerance is presently limited to one vCPU, making Fault Tolerance only applicable to smaller Oracle Database servers. |
Improved performance | The Direct NFS (DNFS) client performs concurrent I/O by bypassing the operating system. The benefits of this are:
|
Ease of use | The use of DNFS simplifies network setup and management by eliminating administration tasks such as:
Load balancing and high availability (HA) are managed internally within the DNFS client. |
Business continuity | Advanced backup and recovery EMC Replication Manager is a comprehensive graphical application that provides Oracle storage replication using EMC storage technology. This eliminates the requirement for the customer to write scripts or to manually perform replication tasks. These tasks can now be fully automated and managed by Replication Manager. One of the solution components using Replication Manager in the solution presented in this reference architecture included Advanced Backup and Recovery using EMC SnapSure checkpoints. Test/devThe ability to deploy a writeable copy of the production database is required by many customers. The process of provisioning this writeable copy must create minimal, if any, performance impact on the production database server. Also, absolutely no downtime can be tolerated. The Test/Dev solution component documented here provides this using EMC Replication Manager for NFS with EMC SnapSure writeable checkpoint. |
Robust performance and scaling | The resiliency testing carried out by EMC ensures that the database configuration is reliable. High availability is used at every major layer of the solution, including the database server, NAS file server, and back-end CLARiiON CX4. By testing the fault tolerance of all of these layers, the ability of the application to withstand hardware failures with no downtime is assured. The performance testing carried out by EMC utilizes an industry-standard OLTP benchmark but does so without exotic tunings that are not compliant with best practices. In addition, real-world configurations that the customer is likely to deploy are used. This enables the customer to be reasonably assured that the configuration that they choose to run their application will do so predictably and reliably. |
Next steps | EMC can help accelerate assessment, design, implementation, and management while lowering the implementation risks and costs of an end-to- end solution for an Oracle Database 11g or 10g environment. To learn more about this and other solutions contact an EMC representative or visit http://www.emc.com/solutions/application-environment/oracle/solutions-for-oracle-database.htm |