Oracle Essentials Oracle Database 11g (57 page)

294

|

Chapter 12: Oracle and Hardware Architecture

Each subsequent release of Oracle has added more parallelized features to speed up the execution of queries as well as the tuning and maintenance of the database. For an extensive list of Oracle operations that can be parallelized, see the section
“What

Can Be Parallelized?” in Chapter
7.

Oracle’s parallel operations take advantage of available CPU resources. If you’re working with a system on which the CPU resources are already being completely consumed, this parallelism will not help improve performance; in fact, it could even hurt performance by adding the increased demands for CPU power required to manage the parallel processes. Oracle’s adaptive degree of parallelism automatically can reduce the degree of parallelism for an operation to prevent this situation.

Clusters

Clustered systems have provided a highly available and highly scalable solution since initially appearing in the 1980s in the DEC VAXcluster configuration. Clusters can combine all the components of separate machines, including CPUs, memory, and I/O

subsystems, into a single hardware entity. However, clusters are typically built by using shared disks linked to multiple “nodes” (computer systems). A high-speed interconnect between systems provides a means of exchanging data and instructions without writing to disk (see
Figure 12-3).
Each system runs its own copy of an operating system and Oracle instance. Grids, described later in this chapter, are typically made up of a few very large clusters.

CPU with L1 Cache

CPU with L1 Cache

L2 Cache

L2 Cache

Memory

Memory

Network Connection

I/O

I/O

Disk Cabling

Disk

Disk

Figure 12-3. Typical cluster (two systems shown)

Clusters

|

295

Oracle’s support for clusters dates back to the VAXcluster. Oracle provided a sophisticated locking model so that the multiple nodes could access the shared data on the disks. Clusters require such a locking model because each machine in the cluster must be aware of the data locks held by other, physically separate machines in the cluster.

Today, that Oracle solution has evolved into Real Application Clusters (RAC) (replacing the Oracle Parallel Server (OPS) that was available prior to Oracle9
i
). RAC

is most frequently used for Windows, Linux, or Unix-based clusters. Oracle provides an integrated lock manager that mediates between different servers, or nodes, that seek to update data in the same block.

RAC introduced full support of Cache Fusion, where locks are maintained in memory without frequent writing to disk. Cache Fusion is different from the standard locking mechanisms that are described in
Chapter 8,
in that it applies to blocks of data, rather than rows. The mediation is necessary since two different nodes might try to access different rows in the same physical block, which is the smallest amount of data that can be used by Oracle.

Cache Fusion initially greatly increased performance for read/write operations compared to the previous OPS and later added improved performance for write/write operations in Oracle9
i
RAC. Today, Oracle supports Sockets Direct Protocol (SDP) and asynchronous I/O protocols, lighter-weight transports than those used in previous traditional TCP/IP based RAC implementations. More recent database releases further improved performance by leveraging faster interconnects such as Infiniband networks through support of Reliable Datagram Sockets (RDS). For example, Infiniband node-to-node latency is about a tenth of the latency in Gigabit Ethernet (typically about 70-80 microseconds).

Prior to Real Application Clusters, you would configure clusters to deliver higher throughput or greater availability for the system. In the high-availability scenario, if a single node fails, a secondary node attached to the shared disk can get access to the same data. Queries can run to completion without further intervention through transparent client failover. RAC provides both availability and scalability since each node in a cluster can act as a failover node for all the other nodes in the cluster.

Real Application Clusters are increasingly used in Windows and Linux environments where a single platform cannot scale adequately or as an alternative to higher-cost Unix high-end solutions. Clustered solutions might also be deployed where high availability is desired. On Windows clustered platforms, Oracle Fail Safe might be chosen as an alternative to RAC, although data is not shared by the two systems and the second system provides only standby access to this data. Because concurrent access isn’t provided, the Fail Safe solution doesn’t offer the scalability that Real Application Clusters can provide.

296

|

Chapter 12: Oracle and Hardware Architecture

In earlier editions of this book, we described a very high-end variation of clusters known as a massively parallel processing (MPP) system. Such systems were essentially a cluster in a box with nodes connected via very high-speed and proprietary networks (see
Figure 12-4).
Open-systems vendors now rarely sell such platforms since clusters of lower-cost system components (nodes) have displaced them in the broader marketplace.

Node 1

Typical MPP Switch

Node 2

CPU CPU

CPU

CPU

L2

L2

L2

L2

Memory

Memory

High-Speed Interconnects

I/O

I/O

Disk

Disk

Node 3

Node 4

CPU

CPU

CPU

CPU

L2

L2

L2

L2

Memory

Memory

I/O

I/O

High-Speed Interconnects

Disk

Disk

Figure 12-4. Massively parallel processing (MPP) system

Clusters

|

297

Non-Uniform Memory Access Systems

Non-Uniform Memory Access (NUMA) computers, introduced in the mid-1990s, provide even greater throughput than SMP by linking multiple SMP components via distributed memory, as shown in
Figure 12-5.
Like clusters, these systems provide scaling of memory and I/O subsystems in addition to CPUs. A key difference is the single operating system copy that manages the entire platform and a directory-based cache coherency scheme to keep data synchronized. Memory access between nodes is in the hundreds of microseconds, which is much faster than going to disk in clustered configurations, and only slightly less swift than local memory bus speeds in a single SMP system. Memory capacities can range into multiple terabytes.

Typical NUMA Node

CPU

CPU

CPU

CPU

Memory Bus Links

L2

L2

Memory

Memory

I/O

I/O

Disk

Disk

CPU

CPU

CPU

CPU

L2

L2

Memory

Memory

Memory Bus Link

I/O

I/O

Disk

Disk

Figure 12-5. Non-Uniform Memory Access (NUMA) configuration
298

|

Chapter 12: Oracle and Hardware Architecture

This enables NUMA to have some major advantages over cluster solutions:

• Parallel versions of applications don’t need to be developed or certified to run on these machines (though additional performance gains may be realized when such applications can be tuned for NUMA).

• Management is much simpler on NUMA systems than on clusters because there is only one copy of the operating system to manage and only one database instance is typically deployed.

Today, the Hewlett Packard Superdome is an example of a NUMA system with demonstrated scalability in production databases that scale into dozens of terabytes of data. Since this platform behaves like, and is managed the same as, SMP systems, NUMA and SMP systems have similar tradeoffs (although NUMA systems tend to be higher priced).

Grid Computing

The “
g
” in Oracle’s database nomenclature since Oracle Database 10
g
signifies the company’s focus on enabling grid computing.
Grids
are simply pools of computers that provide needed resources for applications on an as-needed basis. The goal is to provide computing resources that transparently scale to the user community, much as an electrical utility company can deliver power to meet peak demand by accessing energy from other power providers’ plants via a power grid. Computing grids enable this dynamic provisioning of CPU and data resources (shown in
Figure 12-6).
The Oracle database with RAC forms the foundation for the provisioning of these resources.

Grid Users, Grid Controls

Application Servers/Blades

Database Servers/Blades

Network interconnects

Network Attached Storage

Figure 12-6. Sample grid configuration, including computer blades and cluster
Grid Computing

|

299

Oracle Database 10
g
introduced several important features that enable the delivery of resources when needed via a grid:

Dynamic Service Provisioning

This feature automatically allocates and reallocates server resources based on configuration and failover rules. Service requests are automatically routed to the server with the least load. If a server fails, the surviving services are automatically reallocated to the available servers.

Web services

Web services are also an inherent part of the grid landscape, because applications running on the grid want to use the same type of transparent access to components (or services) that users have to applications. Database web services provide support for queries, messaging, and DML, can access Java and PL/SQL, and can provide full XML support.

Rolling upgrades

A rolling upgrade allows you to bring down some of the nodes in the grid, upgrade their software, and then bring them back online as part of the grid. You can then repeat this procedure with the other nodes. The end result is that you can achieve a complete upgrade of your Oracle database software without having to bring down the database.

Automatic Storage Management

The ASM system also enables management of large numbers of nodes by implementing automatic data rebalancing across disks, as well as easy addition of new disks to the overall pool of storage.

Enterprise Manager Grid Control

Enterprise Manager Grid Control manages a grid infrastructure from a central location, including RAC databases, storage, Oracle Application Servers/Fusion Middleware, and network services.

Oracle Database 11
g
includes multitier service management and improvements to ASM including support for rolling upgrades, automatic bad block detection and repair, and fast mirror resynchronization. The Automatic Database Diagnostics Monitor (ADDM) for RAC determines the most globally significant performance issues for a multi-instance database, including the global cache interconnect, lock manager congestion, global resource contention (such as I/O), high-load SQL across the instances, and skew in instance response times. Hot database patching can now be applied to any Oracle database, including RAC implementations.

Disk and Storage Technology

The discussion of hardware architectures and performance in this chapter so far has centered on ways of increasing performance by increasing available system resources such as CPUs, memory, and I/O subsystems and on the parallelism that can take
300

|

Chapter 12: Oracle and Hardware Architecture

advantage of these resources. An important way to increase hardware performance is to tune for I/O, which includes spreading data across disks and providing an adequate number of access paths to the data. As a rule of thumb, your I/O should deliver 1 GB per second of throughput for every 4 CPUs with a minimum of 2 GB per second delivered. Since disk access has the greatest latency, another focus of I/O tuning is keeping data retrieved from disk in memory.

Oracle refers to proper configurations—those that feature proper I/O (especially spindles that provide access paths to storage), memory, and CPUs—as
balanced configurations
. As noted previously, Oracle Database 10
g
and later releases now include ASM for disk, greatly simplifying day-to-day management for storage. However, working with hardware vendors in order to get proper storage configurations has sometimes proven difficult, especially for data warehousing. Also, as disk capacities have grown, improvements in access times have not kept pace. This has led Oracle to develop reference configurations with several key hardware platform and storage vendors to help provide more accurate initial sizing through an Information Appliance Initiative.