Slide Deck Here

Transcription

Slide Deck Here

Independent Electricity System Operator
Rapid Migration of Big Data - Oracle DB using IBM
Enterprise Storage Tools
© 2012 IBM Corporation
Presented by:
Sajid Rizvi, Oracle Database Consultant /
IT Architect
IBM GBS
Customer Representative:
Noel Sabandal, Architecture and
Quality Assurance Lead
IESO
Contributions
Mark Bunn, Lead IT ArchitectIESO account - IBM GBS
2
2
IESO – Rapid Migration of Big Data
Business Context
3
Application (MDM/R) processes at a Glance
Smart Meter Entity (SME) - IESO
On March 28, 2007, the Independent Electricity System Operator (IESO) was
designated as the SME by Ontario Regulation 393/07 under the Ontario
Electricity Act. Responsible for the operation of a central meter data repository
the Meter Data Management and Repository (MDM/R)
 Smart meters installed across Ontario
:
4.77M
 Active SDP/meters enrolled in MDM/R
:
4.48M
 Processing intervals and register reads daily over :
107M
 Projected upon enrolment of all SDP/meters
:
120M
 Delivering Billing responses daily
:
133K
Shared Service for 71 Local Distribution Companies (LDCs)
and 9 LDC Authorized Agents
4
Operational Challenges
5
 CPU utilization reaches 100%
 No more CPU can be added due to
hardware limitation
 SAN storage utilization reaching
maximum capacity
 No more SAN can be added to the
existing infrastructure
 RMAN Backup execution time taking
over 8 hours for full Backup
 RMAN run time increases with data
growth hitting application performance.
 Additional test environment required
 Unable to add SAN and Server to the
current infrastructure
 Test Environment refresh from
Production takes 18 hours using RMAN
Restore causing delays in testing
activities.
 Fast Refresh option unavailable on
current infrastructure
 Disaster Recovery RPO and RTO not
meeting business requirement
 DR needs to be un-squished and CPU
memory borrowed.
Infrastructure Upgrade Objectives
 Rapid Migration of Production and all other testing environments to new
Infrastructure
 Provision production size test environment without production down time
 Improve Performance with upgraded Power system and Storage
 Instant Recovery in case of failure
 Improve Backup process reducing load on Production
 Meet Disaster Recovery RPO and RTO.
6
Technical Context
7
Migration Challenges
 45 TB of Production and DR Oracle Database environments to be migrated
from Old to New infrastructure during one weekend outage.
 Long testing cycles for Functional, Performance and User acceptance tests
requiring multiple cut over from Production.
 DR environment to be tested as Independent primary Production first followed
by conversion to DR environment during Parallel run.
 A parallel run of one month to be performed on old and new production
 Two Production like test environments to be cut over before Production.
 Multiple cut over required to run latest input files to compare old and new
production environments.
 Multiple iterations of a test cycle to be done using Oracle Flash Copy
8
Migrating 45 TB Storage (Oracle Databases) within a 12 hour
maintenance window.
 Design and build 9 new environments on the new infrastructure
 Initial creation/restore of large Oracle database from tape RMAN backup took 14 hours
(not including the logistics of tape movement)
 How is it possible for us to synchronize this enormous OLTP Database fast enough to
allow a seamless transition to the new infrastructure?
 Need to synchronize the database at minimum twice for parallel testing and for
production cutover
 Need to be able to repeat the synchronization at short notice to allow for rapid
recovery of testing an cutover testing or cutover
9
Existing Environment Challenges
10
Upgrade and Migration Bendfits
 SAN Storage reaching its maximum capacity and
Database size continues to grow
 Mix of Solid State & IBM DS8870 System Storage with
Easy Tier feature to automatically optimize performance
 Continuous performance degradation with increased
volume. RMAN backup conflicting with application
processing
 Gold Tier Storage added with Flash drive using Easy Tier
and IBM Flash Memory.
 CPU and Memory capacity maxed out on frame and no
new CPU power can be added
 Increased CPU and Memory with new P7 Frames and
DS8870 SVC Storage
 Additional test environments required for Performance
and volume testing
 New Production size environment for Performance test
added to the frame.
 RMAN Backup execution time taking over 8 hours
affecting application performance
 New Backup server added to off load RMAN backup
from Production using IBM Flash copy & Oracle RMAN
 Removed impact to Production completely
 Test Environment refresh from Production takes 18
hours using RMAN Restore.
 IBM FlashCopy with Oracle RMAN Restore to reduce
database Restore window to 4 hours
 Application performance slowness. Unable to process
expected volumes within allotted window.
 With upgraded state of the art hardware and Oracle
changes, the application performance increased
 Enhanced Disaster Recover capability and other
services to address LDCs’ evolving requirements
 New Disaster Recovery Environment with RTO of 6
hours compare to 24 hours in old environment
11
11
12
12
13
14
15
16
Flash Copy
17
IBM Flash Copy for Oracle Database
 FlashCopy runs at block level and across all LUNS
 Consistency Groups required for Flash copy Operation (Data+Index & FRA)
 Works well with file system, ASM over multiple LUNs or a mixture of both ASM and file
system
– DATA consistency group contains data and index files
– FRA consistency group contains Redo logs, Archivelogs and Flashback logs
 FlashCopy must be performed in two stages, Data Consistency followed by FRA.
18
19
IESO Big Data Migration
Storage Tier
20
Storage Tiers
TIER
21
Description
Proposed Solution
TIER 0
Ultra High Performance. Meet QoS for High End
Mission Critical applications
< 2ms Avg Response Time
> 1000 Avg IOPS/LUN
DS8870 with SSD
TIER 1
High Performance and/or Availability. Drive up
utilization of high-ended storage subsystems and still
maintain performance Q0S objectives
For low capacity requirements smaller less powerful
devices may meet tier definition
120 Avg IOPS/LUN
DS8870 with 300GB 15,000 rpm
Disk drive set in
RAID5/RAID10 arrays.
TIER 2
Medium Performance and/or Availability. Revenue
generating applications. Meet QoS for non-mission
critical Applications
For low capacity requirements smaller less powerful
devices may meet tier definition
80 Avg IOPS/LUN
Not Used.
TIER 3
Low Performance and/or Availability. Non-Mission
critical applications. Archiving.
< 10sec Avg Response Time
50 Avg IOPS/LUN
DS8870 with 3TB 7.2K Disk drive
set in RAID5.
TIER4
Long term retention, backup.
Client data recovery from Tape
TS35
Oracle Backup comparison before and after Migration
PROCESS
ENVIRONMENT
FREQUENCY
EXECUTION TIME
Weekly
10 hours
Moved to BKP
Moved to BKP
New Backup server
Daily
5 hours
New Backup
Daily
20 Minutes
Old Production
RMAN Full Backup
New Production
Flash Copy Backup
22
22
Infrastructure Upgrade Objectives
 Rapid Migration of Production and all other testing environments to new
Infrastructure
 Provision production size test environment without production down time
 Improve Performance with upgraded Power system and Storage
 Instant Recovery in case of failure
 Improve Backup process reducing load on Production
 Meet Disaster Recovery RPO and RTO.
23
Call to Action
IBM Announces New DS8880
• To get more Info on DS8880, click here.
www.ibm.com/DS8000
• To learn more about this migration or to see the brand
new DS8880 just announced this week, please stop by
and visit IBM at
Booth # 1301 at Oracle Open World 2015.
You can meet our presenters and see the new
hardware system.
24
Thanks to
 Vishal Verma
 Khurram Shiraz
 Jack Gao
25
Contacts
Sajid Rizvi – srizvi@ca.ibm.com
Mark Bunn - markbunn@ca.ibm.com
Noel Sabandal - noel.sabandal@smi-ieso.ca
26
Thank You
27

Slide Deck Here

Transcription

Similar documents

Dianne Fodell Charlotte Informatics Talent Panel

A3RMT - IBM Joint Brochure - A3 Remote Monitoring Technologies

Eloxite Corporation

Services for better business – and for better world.

Ponani Gopalakrishnan. Ponani Gopalakrishnan.

ProBTP and Counter Fraud

Hard Cards - IBM

View Presentation

Toshiba T1100 Plus (1)

Challenger 2/3 Level 0 Control