Lessons Learned 2 - Infusive Solutions

HOW TO AVOID A
DATABASE HEART ATTACK
3 SIMPLE LESSONS IN TECHNICAL
EXCELLENCE LEARNED FROM
OPEN HEART SURGERY
Written By: Ayman El-Ghazali
MCSE, MCSA, MCT, MCITP, MCTS
Edited and Designed By: Infusive Solutions
INFUSIVE SOLUTIONS
ABOUT THIS RESOURCE
Greetings from Infusive Solutions.
For a bit of context, our firm is a midtown Manhattan based IT staffing firm specializing in the
Microsoft technology stack. That means we’ve refined our focus on finding jobs for SQL Server
DBAs/developers and the entire range of Windows infrastructure and development professionals in
the Greater New York area.
But, while job placement is our core competency, we simultaneously take pride in helping to educate
our community whether they’re in process with us or not.
With that in mind, one of the top pieces of advice we provide is to develop deep understanding of
the business in which a DBA, developer, engineer or support professional works.
But, rather than explaining this principle ourselves, we thought it would be more powerful to hear it
from the source.
On that note, we partnered with our friend Ayman El-Ghazali - a SQL Server DBA and developer at
the American College of Cardiology in Washington D.C. who moonloghts as proprietor of
thesqlpro.com - to explain the value he enjoyed in his career by observing and analyzing the surgical practice.
The lessons are meant to be quick and punchy and we hope you feel energized to do excellent work,
no matter your industry, after completing the seven short pages.
Enjoy!
Ben Weiss
Digital Marketing Strategist, Infusive Solutions
Contact Ayman El-Ghazali
Contact Infusive Solutions:
@TheSqlPro
www.thesqlpro.com
ben@infusiveny.com
646.205.9902
www.infusivesolutions.com
@InfusiveInc
Facebook.com/InfusiveInc
LinkedIn.com/company/infusive-solutions-inc
INFUSIVE SOLUTIONS
Introduction
I recently had the opportunity to observe
triple bypass heart surgery (technically
called Coronary Artery Bypass Grafting or
CABG), a process in which three coronary
arteries are bypassed due to blockage.
As a SQL developer and DBA at the American College of Cardiology in Washington
D.C., I felt like this would be an excellent
opportunity to learn about the data I’m
responsible for collecting and protecting. It
was an insightful learning experience and
although the lessons I extrapolated from
this experience aren’t terribly technical,
they’re still something every DBA and
developer (or anyone who works in IT for
that matter) might reap value from.
So let me break down some of the lessons I
learned, relate them to how things work in
IT and how we can use these strategies to
make our jobs more fulfilling.
Lesson 1: Team
Dynamics in the
Operating Room
One of the most important observations I
made while watching the surgery was the
operating team’s dynamics.
Most of the team members had never
worked together, but were nonetheless
communicating effectively and working
with great synergy.
Editor’s Note: Best practice
alert! Supplementing your
technical acumen with
business knowledge gives you
greater appreciation for the
‘why’ behind your technical
work, providing greater
purpose and simultaneously
making you more difficult to
replace!
The team was also comprised of medical
professionals with diverse backgrounds.
Thus, rather than having a room filled with
ten surgeons with the same training, there
was only one surgeon, several nurses and a
variety of specialized technicians. One of
these professionals was a Perfusion
Technician, who had no medical/nursing
school certification, yet still played an
important role by making sure a machine
had automated the breathing and blood
pumping for the patient.
1 I Lessons Learned
There are many lessons to be learned from
this multidisciplinary, specialized approach,
which is hugely beneficial to not only
getting something done, but getting it
done in quality fashion.
INFUSIVE SOLUTIONS
PROLIFIC SUPERHERO TEAMS LIKE THE AVENGERS AND THE X-MEN WERE INCREDIBLY
PRODUCTIVE THWARTERS OF EVIL, COMBING A GAMUT OF SPECIALTY POWERS FOR A
VIRTUALLY UNSTOPPABLE FINAL PRODUCT. IT TEAMS THAT SIMILARLY WEAVE THEIR
INDIVIDUAL SKILLS INTO A POWERFUL PATCHWORK WILL POSSESS THE SAME ENERGY AND
SUPPORT TO FUEL NEXT-LEVEL OUTCOMES.
Applying Team
Dynamics to IT
One common problem I see in the
workplace is technical professionals want
to do their individual part and not
collaborate with others … those
developers who don’t care how Business
Analysts (BAs) collect requirements and
those BAs who aren’t concerned with
how the code works.
One need not be an expert on
everything happening in the IT
department, but there needs to be
overlap so the final product is the result
of collaboration from a cohesive team
that embraces a common goal.
Relating to the surgery analogy, while
the surgeon focused on opening the
patient’s chest, the Perfusionist
monitored the patient’s breathing and a
nurse extracted a vein from the patient’s
leg for grafting; all individual activities
that were occurring in unison to achieve
a strategic final product – saving the
patient’s life.
2 I Lessons Learned
This is very much in line with the Agile
approach to project management.
Far too often we work in sequence when it
is not necessary. If the doctor waited for
the nurse to finish her part (which in no
way hindered the doctor’s work) then the
process would have taken far longer and
potentially been less effective.
Bottom Line: With the complexities and
pace of the modern IT landscape, technical
professionals need to be more nimble and
aware than ever. As a result, open
communication and cooperation between
technical teams are at a premium.
In the healthcare industry, many patient
metric attributes can change at any given
time. In response, we work closely with the
business line to make our code more
flexible, enabling us to test a metric,
change it and test again while getting
feedback continuously from the other end.
INFUSIVE SOLUTIONS
When projects are completed using the
agile methodology, expectations and
outcomes are easier to manage since the
business line was involved from start to
end and working in parallel with the
developers, business analysts, QAs and
project manager.
While observing the surgery, I was considering the idea of health monitoring. With
so much knowledge about the common
causes for heart attacks, there are many
tests that can be used to monitor heart
health and ascertain if you’re at risk before
a problem becomes uncontrollable.
There are also certain lifestyle choices that
one can make to maintain a healthy heart
like regular exercise, a balanced diet and
not smoking.
Nonetheless, other factors that lead to
heart problems are out of our control such
as our genetic makeup, prior health problems and age.
This points to the fact that whether analyzing the operating room or the data center,
it’s crucial to catch the problems you can
before they occur and be prepared with
actionable solutions when an
uncontrollable problem surfaces.
Lesson 2: Monitoring
and Disaster
Recovery in the
Operating Room
Observation in the operating room: No
matter how careful one may be, there is
always the probability of something going
wrong.
With that in mind, let’s discuss lessons to
be learned about database monitoring and
disaster recovery.
3 I Lessons Learned
Applying Monitoring
and Disaster
Recovery to IT
Understanding your disaster recovery
options: In today’s IT landscape, we are
plagued with a litany of problems that
could lead to a “database heart attack”
including poor security, failing hardware,
lazy backup and much more.
INFUSIVE SOLUTIONS
Nonetheless, it is not uncommon to see
signs of forthcoming failure and either
ignore them to move on to another, cooler
project or address them incorrectly (for
example, putting a Band-Aid on a failing
system by throwing RAM at it rather than
fixing the foundational problem with new
architectural design).
But, in the face of such worst disaster
recovery practices, it’s important to understand all the proper solutions at your
disposal.
Of course, this is one of the more expensive forms of high-availability options, the
next being AlwaysOn Availability Groups. I
like to think of this solution as a hybrid
between clustering and database mirroring with the added advantage of allowing
groups of databases to have their data
synchronized to other servers in near
real-time and failed over if necessary. There
are many other options available and the
best DBAs will understand them all and
evaluate which is most appropriate for
their organization.
For example, when doctors realized that
the patient had a significant problem, they
were immediately prepared with what they
considered the best possible procedure
even though they considered other
options such as a full heart transplant.
Similarly, as a DBA, it falls under your job
description – not the developers’ – to
understand the disaster recovery tools that
could reasonably be deployed. I have been
told in the past that our disaster recovery
response is to restore all backups. However,
this does not provide high availability on
servers since recovery time could take
hours, days or even weeks.
Luckily, the team at Microsoft has made
sure we have many different options for
proactively addressing the varieties of
harm that could befall our systems.
Full server failover cluster has been significantly improved with SQL 2012, allowing
for multi-subnet clustering so that servers
can be separated geographically at different sites. As a result, if you lose power at
your data center locally, your remote data
center hundreds of miles away can pick up
the slack.
4 I Lessons Learned
Proactive Monitoring: On the other side of
disaster recovery is proactively monitoring
the health of your database servers. One
issue that plagues the DBA community is
running out of space, because it is very
expensive to get good disk arrays that are
both RAID and on a SAN.
However, regardless of your specific storage technology, there are many tools
available for monitoring disk space across
all servers as well as internal alerting
systems within the SQL Server ecosystem
that can also help you avert possible
disaster.
INFUSIVE SOLUTIONS
Combining these tools with regular maintenance such as transaction log backups,
consistency checks, performance tuning
and index maintenance can ensure that all
your databases are healthy and properly
functioning.
Know When Hardware Needs To Be
Replaced: A last point that I wanted to
touch on is thinking outside the box when
you feel your system is at its end. As I
mentioned, a great DBA knows all the
options to monitor and save a database
and also when it’s time to replace old
hardware - especially when warranties are
going to expire.
Server migrations may be time consuming
to plan and execute, but they can be
excellent for both personal and organizational development. For example, once the
difficulties of implementation have past, a
firm and its employees are left with better
and more useful hardware that enable the
data team to learn new strategies, fix old
bugs, better identify new potential problems and build team dynamics in the
process.
The Bottom Line: The key here is all about
proactive preparation. Just as heart problems can be mitigated when the patient
understands his/her controllable and
uncontrollable risk factors as well as the
myriad of care options available, so to
database issues can be averted when the
DBA is prepared for action prior to a disaster’s arrival.
KEY TAKE HOME POINTS
FROM LESSON 2
1. UNDERSTAND YOUR
OPTIONS
2. PROACTIVIVITY
3. REPLACE HARDWARE WHEN
NECESSARY
Lesson 3: Surpassing
the Call of Duty in the
Operating Room
As IT professionals, it may be fairly easy to
“get by.” Nonetheless, it’s far more powerful
to consistently step up to new challenges
and push forward past the requirements of
a job title. Consequently, being a developer, programmer, DBA, QA Tester or otherwise is not an invitation to ignore the
business side of things – even when you
successfully tackle your to do list.
Rather, as IT practitioners, we should strive
to understand and seek purpose in what
we are doing.
We also need to be innovative and find
new solutions to existing problems as well
as increase/refine our skills and learn new
technologies when possible.
5 I Lessons Learned
INFUSIVE SOLUTIONS
DOING SOMETHING FOR 20
YEARS DOESN’T MEAN YOU
HAVE 20 YEARS EXPERIENCE IF
YOU DID THE SAME THINGS
OVER AND OVER
A career is a lifelong learning process, not a
process of doing the same thing over and
over and claiming it as gained experience.
One of my good colleagues once told me,
“doing something for twenty years doesn’t
mean you have twenty years of experience
if you did the same thing 200 times.”
As you can see, this entire experience –
which took place outside the data center for me was a learning experience beyond
the job title, which helps me perform
better at my work because I understand
more about the data for which I am
responsible. With that in mind, let’s extract
one final lesson on rising above the call of
duty.
The cardiovascular innovations that have
made bypass surgery possible are incredible … the result of relentless drive and
passion from medical practitioners who
rose above their core job description to
change the way we help those afflicted
with heart disease.
6 I Lessons Learned
Continuing this trend, I observed several
members of the medical team going above
and beyond the call of duty. On one end of
the spectrum, we had a nurse act as our
“guide,” taking time out of her daily responsibilities to share her knowledge of the
procedure and how our work in the data
center affects operations at the American
College of Cardiology.
On the other end of the spectrum, a
semi-retired cardiologist – an experienced
doctor who certainly didn’t need to attach
himself to the stress of invasive surgery assisted with the procedure, helping both
the nurses and surgeons complete their
individual duties. He wasn’t there because
he needed to be, but because he saw an
opportunity to deploy his passion and
experience for the betterment of others.
INFUSIVE SOLUTIONS
Applying Surpassing the
Call of Duty to IT
At a previous job with Johnson Matthey - a
specialty chemicals company – IT employees used to take regular trips to the floor of
our manufacturing facility to understand
how our data collection and reporting
affected the day-to-day. I learned so much
by capitalizing on this opportunity to
explore the work being done outside of my
department.
For example, it was incredibly illuminating
to observe the differing degrees of quality
demanded by various company clients.
While some just wanted to pass legal
inspection, others took great care to
ensure both quality and aesthetics were
addressed at the highest levels.
Consequently, I was better informed as to
why certain data was requested in detail by
some clients and ignored by others as well
as why some of my reports had to be so
much more precise than others. At the end
of the day, I was afforded valuable knowledge as to how and why the company did
things by stepping outside of my cube
(literally) and exploring an area in which I
had no previous experience.
As an IT practitioner, technologies
frequently change and there are many
integrated components that can enhance
your relevance, skills and appeal if you’re
willing to explore and learn things that
aren’t an integral part of your job description.
7 I Lessons Learned
For example, I remember once being
scared of building OLAP cubes with SSAS.
Nonetheless, I picked up training material
and was soon playing around with cubes at
home before bedtime.
A few months later, I presented my OLAP
cube as an R&D project to my CIO and
other senior management members who
were highly impressed. I couldn’t have
done it without going outside of my
comfort zone, learning a new technology,
spending hours understanding the business and what metrics are important for
our users to see.
Bottom line: Just like the nurse and
semi-retired cardiologist, don’t let the
baseline definition of your job title act as a
fence around you. Reach out to other
departments to learn about the breadth of
your impact.
Watch webinars, attend seminars, share
your opinions through a blog, tinker with
new technologies in your spare time,
develop side projects, lend a hand to those
without your expertise and you’re liable to
find yourself with the best career positioning possible.
And that, my esteemed colleagues, is good
for the heart.
About the Author: Ayman
El-Ghazali is a passionate SQL
Server DBA/Developer and
Business Intelligence Developer. He has
worked with the SQL Server product since
2006, holds a MCSE Data Platform, MCSA SQL
2012 and MCITP SQL 2008 DBA certifications
and is also a Microsoft Certified Trainer (MCT).
He has been active with the virtual chapters of
PASS, participating in online training and
giving presentations. You can read more of his
work at thesqlpro.com.