FDNY Columbia NYFD Property Saved Indicator Report[1]

http://www.sipa.columbia.edu/academics/workshops/documents/NYFDPropertySavedI
ndicatorReport.doc.
2009 Columbia University Capstone
Project:
FDNY Property Saved Indicator
Clarisse Bleicher
Tashi Choden
Iva Kleinova
Lesli Proffitt Nordstrom
Gabor Veress
Joann Baney, Professor
TABLE OF CONTENTS
EXECUTIVE SUMMARY...........................................................................................................2
INTRODUCTION.........................................................................................................................3
SECTION 1: THE STARTING POINT......................................................................................5
The New Indicator had to be Credible, User-Friendly and Low Cost.................................5
We Built on Recommendations by Last Year’s FDNY-Columbia Project.........................5
The Indicator Measures the Added Value of the FDNY.....................................................7
SECTION 2: BUILDING THE INDICATOR............................................................................9
The Logic of the Indicator is Derived from Field Intuition.................................................9
Question 1 – What is the risk that a fire would have spread further?..................................9
Question 2 – How many square feet would have burned if the fire had spread?..............10
Question 3 – How much would it have cost to rebuild this potentially damaged area?....11
SECTION 3: IMPACT AND IMPLEMENTATION...............................................................13
The Indicator Meets Our Criteria for Success…….……..................................................13
The Results Withstand Analytical Scrutiny.......................................................................13
The Indicator Should Evolve, but Communicative Purpose Kept………...…..................14
FDNY Saved $3.1 Billion in Property Last Year……......................................................15
FDNY Needs to Integrate the Indicator into NYFIRS……...............................................16
CONCLUSION............................................................................................................................17
2
APPENDICES..............................................................................................................................18
EXECUTIVE SUMMARY
Recommendation
Our task was to develop a property saved indicator for the New York City Fire Department
(FDNY). We suggest implementing a statistical method that is based upon the current data
collection of the FDNY, so it can be easily added to the current reporting system. We estimate
that the FDNY saves around $3.1 billion of property in New York City annually.
Background
In order to become a nationally recognized benchmark, the indicator had to be credible, userfriendly and low cost. This set of criteria drove the development and evaluation of our model.
We started from the four alternatives recommended by last year’s project and pursued an inhouse, disaggregated data model based on a fire spread probability methodology.
Analysis
Property saved is calculated as a product of number of square feet burned and the cost of
rebuilding this area. We developed an intuitive risk-based concept rooted in field expertise and
guided by three main questions:
(1) What is the risk that a fire would have spread further?
(2) How many square feet would have burned if the fire had spread?
(3) How much would it have cost to rebuild this potentially damaged area?
We derive the answers to the first two questions from data already collected and reported in the
Fire Department’s database, NYFIRS. The number of square feet saved is founded on the
probability distribution of fire spread and on the average proportion of units burned. The most
important factors influencing this model are building type and floor of origin. For the third
question, we use the January 2009 building valuation data provided by the International Code
Council (ICC). The method we produced assesses property saved on a fire-by-fire basis,
allowing for different levels of aggregation. This method estimates that the FDNY $3.1 billion of
property in 2008.
Immediate Action
In the report, we suggest a few key elements of the implementation plan. Included are
recommendations for internal communication about the indicator, an implementation timeline,
and a proposal for the integration of the indicator by the vendor.
3
INTRODUCTION
The Fire Department of the City of New York (FDNY) performs a wide range of responsibilities
in executing its mission to protect life and property in the city. These include fighting fires to
save life and minimize property damage, providing pre-hospital emergency medical service,
preparing for quick responses to acts of terrorism, investigating causes and origins of fires,
enforcing NYC public safety codes, and conducting fire safety and public health presentations
and events.1
The fire department delivers these services through the efforts of more than 11,400 fire officers
and firefighters under the command of the Chief of Department; over 2,800 emergency medical
technicians, paramedics and supervisors at the Bureau of Emergency Medical Service (EMS);
and about 1,200 civilian employees. Administered by the Fire Commissioner, the FDNY today
protects more than 8 million New Yorkers in an area of 320 square miles.2
The public widely respects the FDNY for its bravery and efficiency in responding to numerous
emergencies in the city. However, the current set of indicators employed by the department
comprising response times and fatalities do not project explicitly the added value in terms of how
much property is saved by the actions of the fire department each year.
To make this more tangible with regards to property saved, our workshop was assigned the task
of developing an indicator that reflects a more accurate assessment of the FDNY’s fulfillment of
its responsibility to protect property. In consultation with the department, we decided that a
positive indicator of ‘saved’ rather than ‘lost’ property should be formulated. Over the course of
the project (January – May 2009), we designed an approach to calculate the magnitude of
property saved from structural fires. Given that no such indicator has existed so far, with fire
departments across the country generally reporting unreliable “guesstimated” statistics of
property lost, this new indicator of the FDNY is a significant breakthrough and holds enormous
potential to set a national benchmark.
Prior to our project, a previous Columbia University Capstone group in spring 2008 explored the
feasibility of three new performance indicators for the Fire Department’s Performance
Management Initiative that had been launched in 2007.3 Their work resulted in the development
of the lives saved/rescued and EMS customer service indicators, which are now in the final
stages of implementation by the FDNY. The third indicator for property lost/saved was not
formulated, although substantial research was conducted to highlight drawbacks of various
1
FDNY Annual Report 2007: Building for a 21st Century New York.
2
http://www.nyc.gov/html/fdny/html/history/fire_service.shtml
3
Report of the Spring 2008 Capstone Workshop on FDNY Performance Management System.
4
approaches used by other fire departments in this area. We utilized much of this information to
refine our own options in building a credible, user friendly and low-cost approach to measuring
property saved.
Ultimately, we designed an approach based on an intuitive, risk-based concept of property saved
as expressed by firefighting professionals, taking into consideration the specialized skills and
expertise that they bring to the field. Using real data on structural fires from the FDNY database,
we derived a probabilistic model that estimates the size of saved property in square feet by taking
into account the risk of fire spread and proportion of surface burned. Multiplying this size of
saved property by the dollar per square foot value gives the total value of property saved in
dollars.
Using this formula, we were able to estimate that the FDNY saved $3.1 billion in property in
2008, accounting only for building fires and damage caused by fire. Conservative on several
counts as detailed in the main report, the property saved estimate is still significantly higher than
the fire department’s annual operating budget of $1.53 million. Highlighting the value added of
the FDNY in this clear and tangible manner can give the department a better negotiation and
communication position with other agencies and the public.
In the rest of our report and appendices, we explain in detail the processes and logic behind this
approach to calculating property saved, provide critical analysis of the results as well as impact
assessments, and make suggestions for the implementation of the indicator.
5
SECTION 1: THE STARTING POINT
The New Indicator Had to be Credible, User-Friendly and Low Cost
At the outset of the project, three main criteria guided the development of a performance
measure relating to property for the FDNY. To become a nationally recognized benchmark for
positive and credible external and internal communication regarding the work of the fire
department, the property saved indicator had to be credible, user-friendly and low cost.
Any new model must be easily defendable and sensitive to public opinion. To meet the
credibility requirement, we emphasized that the indicator should be consistent and rely on both
well-respected external data sources and the FDNY’s own database, NYFIRS. Additionally, the
data and information used in the indicator should be complete and comparable across boroughs
and other city divisions. Finally, the indicator should be protected from outside influences or
shifts that could render the comparative aspect of the indicator obsolete.
The user-friendliness and low cost requirement drove the structure of the model. In difficult
economic times, a premium is placed on keeping cost down for municipal services. The FDNY
cannot implement a property saved indicator in a short amount of time if such implementation is
burdened by training and other costs. To maximize the chance that the FDNY effectively adopts
the indicator developed by this project, the model must be user-friendly and keep additional
resources required to a minimum.
For example, at the field level the data collection needs to be simple and realistic, not adding to
the already thorough reporting found in NYFIRS. At the compilation level the tool needs to be as
automated as possible; calculations and updates to the system have to happen without individual
chiefs’ intervention. Finally, we sought a model that would be generally intuitive and therefore
accepted by the employees of the FDNY.
Considering our mandates, we also made some important initial choices about the method we
pursued. First, we decided to only look at structural fires, as they offered the most complete data
and are responsible for a huge loss to New York City annually in terms of property damaged by
fire. Second, to emphasize the positive communication aspects of the project we decided to focus
on the property that was saved, rather than lost. This provided a unique challenge as no other fire
department is currently providing such a statistic.
We Built on Recommendations by Last Year’s FDNY-Columbia Project
Evaluating last year’s cooperation between Columbia University and the FDNY helped us to
quickly define the above mentioned cornerstones of our approach: credibility, user-friendliness
and cost-efficiency. Four alternatives that could be pursued in the development of the property
saved indicator emerged from the beginning.
For clarity we structured the main recommendations of last year’s project into a matrix. On the
one hand, the FDNY could develop its own calculation of property saved or adopt an external
one. This choice is reflected on the vertical axis of figure 1. On the other hand, the agency could
use more aggregated or more disaggregated data sources. This choice is reflected on the
6
horizontal axis of figure 1. One obvious drawback of the previous project was its insistence on
property lost. Nevertheless, we still chose to use the same analysis to decide on the overall
methodology for property saved.
Option 1: Statistical Agent Data Method
The statistical agent data method would estimate property lost through reports on compensation
paid out by insurance companies. This data can be obtained from collaboration with New York
State Insurance Department (NYSID) or the Insurance Services Office (ISO). This option is
based on external sources and the expertise of insurance companies. Employing this option
would require no extra data collected from the field and there would be minimal labor costs
associated with maintaining and reporting using this method. There is a lag-time in getting
updated data between 1½ to 2 years. Furthermore, because the data does not include the
uninsured it makes the model very risky in terms of its credibility. Ultimately, the
incompleteness of the data led us away from the idea of using statistical agent data.
Option 2: Insurance Waiver Method
A waiver system is also a product of a
partnership with the insurance
industry. It too is based on the
expertise of the insurance companies,
but can be broken down and reported
on a case-by-case basis. The external,
disaggregated option
allows the FDNY to
look at actual payout
amounts broken down
into incidents. This is
positive because the
Fire Department would be able to
analyze individual fires and reaggregate them according to the agency’s reporting needs. A crucial aspect of this method is that
it would require homeowners and renters to sign a standardized waiver allowing the FDNY to
have access to their compensation information.
Again, the drawback of this methodology is its exclusion of the uninsured and the insured that
choose not to comply with the waiver system. Also, we would have to deal with nonstandardized data requiring the creation of a position at the FDNY to manage and reconcile the
data to ensure its usefulness for comparison and aggregation purposes. Finally, this model is very
labor intensive, requiring a heavy workload of following up with the multiple insurance
companies on the various cases to gain accurate data.
7
Option 3: In-House/Disaggregated Method
This method uses external property value data ($ per sq. ft) to generate in-house statistics on
property loss. There is no estimation of damaged contents, which is positive in that assigning
value to contents is a subjective process that can create vastly different results based on who is
analyzing their worth. The parameters of an in-house model can be based on external sources and
experts. At the same time, it maintains a degree of flexibility due to the immediate availability of
data taken from the field via NYFIRS. With the possibility of multiple levels of aggregation, e.g.
via zip code, this method also requires little updating since it is based on internal data that is
already being updated and used widely by the agency.
An alternative method that can be seen as a sub-option in the category of in-house/disaggregated
option also uses NYFIRS data. Last year’s project considered that this method was considered
would be immediately available as it was based on internal data and parameters. The approach
would use NYFIRS data to report property lost or saved statistics, though without assigning a
monetary value. Property saved could be reported in percentages rather than in a dollar value.
Additionally, information on indirect losses and saves based on the displacement of families and
businesses data could be obtained. This method is low cost and largely based on data collection
that is already being done in the field for NYFIRS. The communication value of
percentages is arguably weaker than reporting property saved in a dollar amount.
It could be difficult for the public or other government agencies to understand the
value of the efforts of the FDNY if they are not given a very tangible and easily
understood statistic.
Option 4: In-House/Aggregated Method
The in-house and aggregated data method was not reported in the 2008 project but, is our natural
addition to complement the project’s recommendation options. This method generates in-house
statistics of saved property using average property value data at the zip code level from external
sources, combined with the probability of fire spread. Dollar values would be reported only on
an aggregated and not case-by-case level. One aspect of this approach that makes it more
difficult and nuanced is the task of estimating the likelihood of fire spreading from the origin and
beyond.
For the next step in creating a model, we took the options and, comparing them to the mission
statement with the core components of credibility, user-friendliness and cost efficiency, the
feasibility and attractiveness of the options became clearer. This evaluation left option 3 as the
clear choice to pursue in meeting the demands of developing the property-saved indicator though
over time there have been aspects of option 4 that have been incorporated into the model. In the
end, the method we pursued in an in-house, disaggregated model that considers fire spread
probability as a major component of our formula.
Incorporating an Added-Value Perspective
As the FDNY is a pioneer in defining the type of property indicator, we had to initially define
what this indicator aimed to measure. This step was critical in setting the direction
for building the formula.
8
Did the indicator aim at measuring what would have happened if firefighters had come later than
the average response time? While this idea was tempting, we found several limits to this way of
thinking. First, this is not specific to a fire, as the FDNY needs to arrive fast for most of its
interventions, including medical emergencies. Second, it does not measure the very particular
core skills and expertise that truly make the difference when the FDNY goes to a fire.
A data analysis (see Appendix B) showed that response time is actually only one variable among
many others, and that its impact is limited by the fact that the time when the alarm was given
varies a lot more. Only considering response time as a sole added-value of the FDNY is therefore
too narrow. Last by not least, it is focusing on a mean (arriving fast), not the end of the
intervention (extinguishing the fire), and response time already belongs to measured statistics the
department has at its disposal.
Why wasn’t insurance data used in the final model?
What then truly defines the
Insurance data is not an accurate estimate of lost/saved
added-value of a Fire
Department in terms of property property in New York City because up to 40% of renters
have insurance. Using insurance data would mean excluding
protection? To answer this
those cases of fire where an uninsured tenant or owner is
question, we had to consider
what firefighters bring compared involved and therefore greatly underestimating the value
to mere civilians who would try added to the city by the efforts of the FDNY.
to extinguish a fire themselves.
First, firefighters are brave. They
are trained to think on the spot and under deep duress. Second, firefighters are experts and
professionals of fire extinction. They have accumulated experience on strategies to fight against
a fire, and they have a deep knowledge on what usually works given certain fire characteristics.
Third, firefighters are physically trained to handle the stress of fighting fires and they are in
overall excellent physical condition. Last, they use the best equipment available to carry out their
duties.
The indicator therefore wishes to capture the added-value of all these talents (specialized
expertise, bravery, physical condition and equipment), which is a much broader perspective than
only measuring the impact of response time.
9
SECTION 2: BUILDING THE INDICATOR
The Logic of the Indicator is Derived from Field Intuition
Central to our approach of estimating the amount of property saved by the Fire Department is
capitalizing on the intuition that comes from the field.4 In other words, our method is, at its core,
informed by the knowledge and experiences of FDNY professionals, who work each day to save
lives and property from fire and other emergencies in New York City.
Deputy Chief Daniel F. Donoghue from Division 3 in Manhattan once said to the team, “When a
bed has burned, I want to say we saved the room. When the room has burned, we saved the floor.
When two apartments have burned, we saved the building. And so on.” This view is practical
and realistic as it implies the need for certain parameters to be established in order to arrive at an
estimate of how much property has been saved by the efforts of the FDNY.
This intuitive risk-based concept permeates our overall formula for property saved and ultimately
delivers a dollar amount of property saved for any given fire.
Our approach to estimating these variables requires three questions to be answered. To determine
the size of property saved in square feet, we ask:
(1) What is the risk that a fire would have spread further?
(2) How many square feet would have burned if the fire had spread?
To obtain the dollar value per square foot we ask the third question:
(3) How much would it have cost to rebuild this potentially damaged area?
The method and tools to answer these questions include: (1) a table of risk of fire spread that is
estimated from the FDNY database, NYFIRS; (2) the building dimensions (surface, number of
floors) collected and reported in NYFIRS, multiplied by an estimated proportion of burned
surface if fire had spread; and (3) a table of rebuilding costs per square-foot based on external,
expert valuation. These methods and tools are explained in further detail in the following sub-
4
Since we limit our analysis to structural (building) fires, property saved does not include other types of property
nor does it include property that has been saved from non-fire damage. In total, approximately 4000 fires per year
are included in the analysis.
1
0
sections and a detailed step by step statistical method of reading the final numbers is described in
the appendix.
Question 1 – What is the risk that a fire would have spread further?
To determine property at risk or the risk that a fire would have spread further than its point of
ignition, we develop a table listing the probability of fire spread at various levels. This table is
derived from 2008 NYFIRS data that takes into account whether fire spread is confined to object
of origin, floor of origin, room of origin, building of origin, or spread beyond the building for
each incident. This is further estimated according to building classes 1 to 4, i.e. fireproof, fireprotected, non-fireproof and wood frame structures. The following table summarizes the risk of
fire spread.
Table 1: Risk of Fire Spread
The probability of fire spread for different types of building as presented in this table reflects
patterns observed in the field by fire personnel. It shows a clear link between building class and
fire risk, namely that non-fireproof and wood frame structures are more at risk than fireproof and
fire protected structures.
For instance, if we take a fire in a building classified as Class 3 (non-fireproof structure), we
would expect that if an object is on fire, there’s a 77.1% chance that it will spread to the room; if
a room is on fire, there’s a 45.9% chance that it will spread to the floor; if a floor is on fire, then
there’s a 52.7% probability that it will spread to the building; and if the building is on fire,
there’s a 13.4% probability that it will spread to other buildings.
Question 2 – How many square feet would have burned if the fire had spread?
To answer how many square feet would have burned in the event that fire had spread, we take
into account the building dimensions involved (surface, number of floors) for which data is
already collected and reported in NYFIRS. We then multiply the dimensions (sq-ft) with an
estimated proportion of burned surface if fire had spread from one level of ignition to the next.
The assumption we make here is that if fire is confined to the room, then 100% of the room will
in fact burn; however, if the fire spreads beyond the room of origin, we can no longer assume
that the entire floor or building will always burn. This means that an average fire confined to the
floor of origin, for example, will only spread to approximately two thirds of the floor, rather than
the entire floor. These percentages are also estimated on real NYFIRS data and detailed tables
1
1
per building type, level of fire spread and, in some cases, floor of origin, can be found in
Appendix 1.
Therefore, depending on how far the fire spreads, we evaluate that number of square feet burned:
= 100sq-ft x 100% if fire spreads to the room
= (width) x (length of main floor) x (average proportion of floor burned if fire spreads to the
floor)
= (width) x (length of main floor) x (# of floors) x (average proportion of building burned if fire
spreads to the building)
= (width) x (length of main floor) x (# of floors) x (average proportion of building burned if fire
spreads beyond)
Question 3 – How much would it have cost
to rebuild this potentially damaged area?
In order to arrive at a dollar value for
property saved, we use a table of rebuilding
costs that lists average construction costs per
square foot according to building type and
occupancy. This table is based on the January
2009 building valuation data provided by the
International Code Council (ICC) and
updated every six months.5
Why wasn’t market value used to assign a
dollar value to property saved?
An update of the research findings of market
values, especially in light of the recent housing
slump, showed how unreliable and volatile
market values can be. Moreover, these
dollar/square foot valuations are vastly different
between boroughs. Using market value as a
means to gain the dollar amount would make
comparing the property saved per borough
impossible. By focusing on rebuilding costs,
which are reported as a national average by the
International Code Council (ICC), we neutralize
the effect of the constant shifts in the housing
market and focus on a more stable estimate that
can be compared across borough.
The ICC codes strengthen the credibility of
the formula because they are used at the
state, national and international levels and
are developed by experts. Since New York
City is one of the most expensive real estate
regions in the country, there is no danger of
using inflated values since the ICC reports
are national averages. If anything, New
York’s costs of rebuilding, especially in
some wealthy parts of Manhattan, will
exceed the ICC estimates. However, we prefer to err on the safe side. Furthermore, because the
dollar values are uniform within the divisions, comparison across boroughs is easily done. This
also ensures that an engine company in the Upper East Side, for example, is not getting
advantage by virtue of their location over an engine company in the Bronx. The inherent fairness
5
http://www.iccsafe.org/cs/techservices/pdf/BVD200902.pdf
1
2
of the statistic therefore minimizes its misuse against divisions or engine companies that are
based in less wealthy boroughs.
ICC classification of building types and occupancy listed in Appendix 3 is similar to the codes
used in NYFIRS. Some adjustments have been made to the original table to make it completely
compatible with NYFIRS for use in our calculation of property saved. These adjustments have
been made in consultation with the FDNY’s Bureau of Fire Prevention, and with reference to
literature explaining ICC construction types and building occupancy and use.6
While another option for dollar value for property was to take the market price per square foot,
we decided against this for a number of compelling reasons. The real estate market is volatile,
especially in NYC, with property values fluctuating in short periods of time. There is also a great
deal of variation in market value across neighborhoods, within and across boroughs.7
For instance, the average price of building property in Manhattan in January 2009 (aggregating
all kinds of property) was $1,190/sq-ft, whereas in the Bronx the average price was $198/sq-ft,
and in Brooklyn $285/sq-ft.8 Such factors make it undesirable to rely on market value in
calculating a dollar amount for property saved, as it does not provide a dependable basis of
comparability across boroughs.
6
International Code Council (ICC): 2006 ICC International Building Code (IBC), 2006.
7
The Furman Center for Real Estate and Urban Policy: “Trends in New York City Housing Price Appreciation” in
State of New York City’s Housing & Neighborhoods, 2008.
8
Source: http://www.trulia.com/real_estate/New_York-New_York/market-trends/
1
3
SECTION 3: IMPACT AND IMPLEMENTATION
The Indicator Meets Our Criteria for Success
Before we can proceed, we must check that the formula matches the criteria for success that we
set in section 1. These criteria were credibility, user-friendliness and low cost. This formula is
derived from the 2008 NYFIRS data that was made available to us from the FDNY. Though the
model is broken into building types, nearly all possible variables are taken into consideration by
virtue of the fact that they are reported in NYFIRS. Exceptions are wind speed and other
unmeasured factors that are difficult to determine at the time of the fire on the ground.
The formula is highly credible because it is based on actual structural fire data. New York City,
its high-density population and high-rises present special challenges to a fire department and
there no data or estimates that could model the situation firefighters in this city face other than
what is entered into NYFIRS. This data is credible, reported further to the state and considered
reliable. Any errors made in the system can be fixed or cleaned. Over time, data errors will
minimize with increased use of the NYFIRS reporting system and strengthen our model.
In terms of user-friendliness, the method of determining property saved requires no additional
data be collected in the field. The only people who will have to work with this output are the
statistician for the department and those Chiefs that use the data to communicate with the public
or other agencies. The difficult equations are built directly into NYFIRS to minimize
mathematical errors and reduce the time that FDNY officials will have to spend to get the dollar
value of property saved to virtually zero.
The ultimate costs of implementing this program are negligible. While there may be some costs
for implementation associated with an already scheduled major upgrade of NYFIRS, the
biannual costs to update the system are minimal. The ICC tables for rebuilding costs are updated
biannually and can be changed easily without requiring a time-consuming or costly larger
upgrade. Only major events affecting the probabilities, such as a change to policy regarding
building collapses, increasing cases of arson or a major change in building structures and
classifications, may merit a major update in the future.
The Results Withstand Analytical Scrutiny
Understanding the model in terms of how it defines the added value of the FDNY’s actions
against fire is vital. The more likely a building is prone to fire spread, e.g. a non-fireproof
structure is much more likely to burn than a fireproof building, the more value is placed on the
actions of the department. Thinking logically, it is easier for the Fire Department to put out an
oven fire in a fireproof (class 1) building with the first on scene methods of using fire
extinguishers to contain the fire. In a the class 3 building that is non-fireproof, more exhaustive
techniques and manpower must be used to contain the same kind of fire and thus, more property
is at risk.
1
4
A fire contained to the floor of origin saves more property than a small fire contained at the
origin because at that critical point more property is at risk. To draw an analogy, an emergency
room doctor treating a case of the common flu does help to keep the patient alive, but one would
hardly credit the physician with saving a life for each fever and runny nose she treats. Yet, when
a doctor observes and gives a diagnosis of a life-threatening disease she is much more
responsible for saving the person’s life. This is because the doctor has skills beyond the average
person when it comes to finding and treating disease. However, if the doctor only discovers the
disease beyond the critical point when something could be done to save the person, the doctor
could no longer have a significant impact on saving the person’s life. At best, she can make the
patient suffer less and prevent other patients from being infected.
This analogy parallels the particular skill set of firefighters. Their full abilities are not utilized in
small fires and thus the value they add to the situation is smaller compared to what they add in a
situation where the fire is larger. The value of their expertise and training, however, is reduced
once the fire spreads beyond a point where their impact is lessened. If a building is fully
engulfed, the Fire Department can only work to save nearby buildings, but will have to watch the
building of origin succumb.
The Indicator Should Evolve, but Communicative Purpose Kept
The property saved indicator as currently designed has a number of limitations that must be
taken into consideration, given the aim of the indicator, the available data and the scope. It is first
and foremost, an indicator that is designed to measure the added-value of the FDNY for a
communication purpose, not as a management tool. A lot of variables are actually out of control
for the firefighters (e.g. when the call came in, if it was arson, etc.). Given the limited number of
fires every year for each engine company, it is more relevant to use the indicator at an aggregated
level. Therefore, we believe using the property saved indicator for management purposes would
most certainly lead to unfair analysis and judgments.
Emphasizing that the indicator is used for communication purposes will be paramount to its
success in the field. If there is even a small fear that the indicator is used for management
purposes, it is possible that Chiefs filling out NYFIRS reports might have incentives to subvert
the system and improve the numbers. If they suspect that the indicator will serve to evaluate their
own or their engine’s performance, their incentives to tweak the system in their favor might be
higher. In either case, we recommend that the FDNY does not widely publicize how the formula
is calculated.
Moreover, given that this is a first step toward assessing property saved rather than property
lost—something that has not yet been done anywhere else in the country or internationally for
that matter—it was important to start somewhere but, at the same time not overextend our
approach. Instead of attempting to incorporate every possible variable, we limited the parameters
within reasonable means to ensure that we had a statistically meaningful model that can be
successfully implemented.
1
5
In particular, some interesting variables (e.g. whether sprinklers were present or not) could not
be integrated in the model because the collected variables were not reliable enough, and large
numbers of errors were found. Moving forward, significant improvements of the model could be
reached if the data reporting on these critical variables could be audited and standardized.
In an effort to refrain from inflating the dollar value of property saved by the FDNY in order to
withstand the credibility test from the perspective of various actors, we have been conservative
in our approach to calculating the final product. By choosing to employ rebuilding costs per
square foot based on national averages provided by the ICC in our calculation, the projected
amount of property saved is likely to be an underestimation of what the Fire Department could
save in dollar terms if another valuation method was used.
Additionally, the scope of the measurement had to be limited to one that was practical and
doable in the time available to us within the confines of a school semester. As such, the measure
of the property saved indictor for the FDNY has some necessary caveats as highlighted below.
For reasons mentioned already, our approach is limited to evaluating only structural (building)
fires. Other fire incidents such as brush fires are not taken into account. This means that the
amount of property saved by the FDNY from all fires will actually be much larger than what our
current indicator captures. For instance in 2008, the FDNY responded to a total 473,335 fire
incidents of which only 26,862 or 5.7% were structural fires.9 We, however, believe that most
property value is taken into account. Furthermore, since the indicator is designed as a
communication tool, property saved in structural fires will be most interesting to the public.
Similarly, actual property saved by the FDNY each year can safely be assumed to be much larger
than projected by this indicator since we only take into account fire damage (burned property).
Damage to property caused by smoke, water and other elements are not included, because it
becomes too complex to model and communicate.
FDNY Saved $3.1 Billion in Property Last Year
Starting again from the formula we can run an example to demonstrate the impact of the model.
In a class 3 (non-fireproof) structure, the most common in Manhattan, a fire is contained at the
room of origin of a 6-story building. The dimensions of the building are listed in NYFIRS as
being 100 x 75 feet and the coded building use is 429, meaning that it is a residential/multifamily dwelling. The fire started on the fourth floor. Using the probability tables that were built
from NYFIRS data for 2008 and 2009, we can determine the estimated dollar value of property
saved is $4.7 million.
This data can be taken a step further and aggregated to different units of the FDNY. Ultimately,
using the formula and the method on the 2008 data gives the result that through the special skills,
talent and bravery of the FDNY, the department saved $3.1 billion in property last year alone.
Having these dollar values to communicate to other government agencies and the public are
9
Ibid.
1
6
crucial in helping those outside the FDNY understand the value of the work of the firefighters in
the field.
FDNY Needs to Integrate the Indicator into NYFIRS
The property saved indicator requires a more vigorous upgrade of NYFIRS by the vendor than
the life saved indicator developed by the Columbia team in 2008. Additionally, strong internal
communication is needed to ensure understanding and buy-in within the FDNY, the government
offices with which they cooperate and the general public once the figures are publicized.
The acceleration of implementing this indicator will be possible because relatively few actors are
directly involved and since it does not affect data collection from the field. Furthermore, a
review by a working group in the coming months cannot involve too many actors and will be
facilitated by our team's detailed description of formulas and preparation of caveats/next steps.
Since a working group assigned with the task of preparing for implementation will likely be set
up by June 2009, our team is only providing a few key elements of the implementation plan,
including, a recommended timeline for implementation, recommendations regarding the
integration of the indicator by the vendor and recommendation for internal communication.
Because the property saved indicator requires more integration work than the life saved
indicator, we have created a calculation module included on a CD with all the deliverables
prepared by the team. There should also be an allowance for an automatic update of the table of
rebuilding costs two times a year. Yet, this upgrade is very minor as the formula is already
developed by the team. The parameters have been given and there is an example of the
calculation available in excel format for the vendor and the FDNY working group to use.
Therefore, there should be no additional costs other than the opportunity cost for
implementation.
The FDNY should discuss of the integration with the vendor as soon as the working group enters
phase 1 to ensure that the integration is complete by 2010. To prepare for the Integration by the
vendor, our deliverables include a step-by-step description of formulas and the aforementioned
example of calculation in an excel format. We are also willing to set up a meeting with the
vendor after the end of the project (until the end of June), as needed to communicate the
technical aspects of the formula we have developed.
1
7
CONCLUSION
The property saved indicator is the first of its kind in the country, as well as internationally,
making the FDNY a pioneer in assessing and reporting ‘saved’ as opposed to ‘lost’ property.
Anchored on intution from the field and backed by real, objective data from NYFIRS, this
indicator embodies the risk of fire spread and likelihood of property burned based on actual
patterns in building fires; the use of expert, external valution of property to arrive at a dollar
value for property saved further adds to its credibilty.
In terms of its implementation, a key initial step will be to ensure understanding and buy-in of
the approach within the FDNY. This calls for strong internal communication for which we have
several recommendations. The first is to develop detailed knowledge and know-how of those
personnel who will be working directly with the data to produce the indicator.
The second recommendation is to develop high-level and practice-oriented knowledge and
communication on the system of performance management as a whole, for those who will be the
end-users of the indicator for communicaton (and other) purposes. The third recommendation is
to create understanding of key principles and develop buy-in of others in the field who
eventually benefit from the indicator, but are not directly involved in its employment—
particularly in the early stages.
With regard to the indicator itself, it should be kept in mind that this is only a first step towards
measuring the true extent of property saved by the FDNY. In the long run, it is possible to
expand the scope of this measure. There is potential to do this without too much difficulty since
the fundamental approach has already been defined. The incentive to following through on this
potential in the future is compelling since much more of what the FDNY does in terms of
protecting property from various kinds of damage apart from structural fires can be projected.
1
8
This in turn can add to the bargaining position of the department vis-à-vis budgetary issues and
public communication.
As has been demonstrated for the year of 2008, the FDNY already saves a substantial $3.1
billion in property, even without accounting for non-fire damages and non-building properties.
We can reasonably expect that this figure will increase if the fire department commits to
increasing the scope of this indicator over time. This would naturally be in the best interest of the
department as public awarness of the real worth of the actions of “New York’s Bravest” will
grow tremendously.
APPENDICES
Appendix 1: Detailed Guide to Calculating the Property Saved Indicator
A1: Calculating risk coefficients
Risk coefficients are calculated from proportions of fires confined to the different levels of
firespread (confined to the object of origin, confined to the room of origin, confined to the floor
of origin, confined to the building of origin, and spread beyond the building of origin). Because
of different inherent characteristics of building types, the coefficients are estimated for four
categories of building type: 1 – Fireproof Building, 2 – Fireprotected Building, 3 – Non-fireproof
1
9
buildings, 4 – Woodframe buildings. These characteristics are summarized in Table 1. Building
types “Metal Structures” and “Heavy Timber Structures” are not estimated in the Model and
will, therefore, always receive 0 square feet of saved property.10
Table 2 shows simple proportions of fire confinement, estimating the percentage of fires in each
building type category in 2008.
Table 3 calculates the probability that the fire spreads at least to object, room, floor, building and
beyond building. The probabilities in Table 3 are derived directly from Table 2 by summing all
the fire confinement proportions above the level we are calculating. For example, in a fireproof
building, the probability that a fire spread at least to the floor of origin (16.4%) is the sum of the
10
We did not estimate these building types because of the very small number of fires that happens annually in these
buildings. It is not statistically correct to estimate any averages on only approximately 9 fires in Metal Structures
and 13 fires in Heavy Timber Structures. Fortunately, since there are so few fires, not including these two building
types will not skew the overall property saved for New York City very much.
2
0
proportions that the fire was confined to the floor of origin (12.1%), to the building of origin
(3.8%) and spread beyond the building of origin (0.6%)11:
P(fire spreads at least to floor) = P(fire confined to floor) + P(fire confined to building) + P(fire
spread beyond building of origin).
Table 4 estimates the risk coefficients used in the model. It shows the probabilities that a fire had
spread to a certain level, given that it had already spread to a previous level. Each probability is
derived directly from Table 3. We assume that a fire always travels from object of origin to
room to floor to building and beyond. Any given probability is calculated as:
P(level | level-1) = P (fire had spread at least to level)/ P(fire had spread at least to level -1)
For example, the probability that the fire spreads to the floor, given that it has already spread to a
room for a fireproof building is calculated as:
P(floor given room) = P(fire had spread at least to floor)/P (fire had spread at least to
room)
P(floor given room) = 16.4%/67.8% = 24.2%
11
Small differences are due to rounding errors.
2
1
A2. Calculating Building Dimensions
Building dimensions
In order to understand property saved in square feet, we need to know the dimensions of the
different levels of fire spread (room, floor, building, and other buildings).12 Most of these
dimensions are calculated directly from NYFIRS data; we do, however, make several
assumptions.
(1) The size of a room is 100 square feet. This is a conservative assumption, because most
rooms are larger. We do not recommend the FDNY to start collecting the data for room
sizes, because it would be very time and energy consuming. By taking this conservative
assumption, we underestimate the overall impact number. By comparison, increasing the
size of the room to 200 square feet, the overall 2008 impact grows only by 0.15%. 13 We
can see that there is not a huge difference.
(2) Buildings neighboring the building of origin are of the same size as the building of
origin. Like with the room size, FDNY does not currently report any statistics about the
size or building type of the neighboring buildings. The assumption that these buildings
will be the same size will not always hold true. We maintain that most neighborhoods
are comprised of similar sizes and classes of buildings. It will inevitably happen that a
highrise on fire will stand next to a 4 floor walk up building, therefore slightly skewing
the property saved for this specific building. Fortunately, the percentage of building
burned when the fire spreads to neighboring buildings is defined as an average
percentage of the original building that has burned in 2008 (see next sections for details).
Because this percentage is below 100% for fireproof, fireprotected and non-fireproof
buildings, we do not have to worry about the size of the other buildings around. The only
12
We do not need to take the size of an object into account, because anytime there is fire, by definition, the smallest
area that is saved is a room. An object on fire will always burn and will never be saved.
13
The growth is from $3,126,429,070 to $3,131,113,762, an absolute change of $4,684,692.
2
2
time when this percentage is above 100% is for woodframe buildings. However,
woodframe buildings are usually not located next to highrises. Therefore, the impact of
this assumption should be minimal.
Keeping these assumptions in mind, we calculate the dimensions in the following way:
Percentages of building dimensions taken into account
The next important step is to understand how much of these total room, floor, building and other
building areas actually burn in a given fire. For example, when a fire is confined to the floor of
origin, we cannot assume that this means that the entire floor has burned down. Maybe only half
of the floor has burned. Similarly, when the fire is confined to the building of origin, we cannot
assume that the entire building has burned down. Likely, two or three floors have burned.
Lastly, fire can spread to other buildings even when the whole original building has not yet
burned down.
Therefore, we need to estimate the average portion of the room, floor, building and other
buildings (the last one expressed as a percentage of the original burning building) that would
have burned. By not estimating that the entire floor has burned when a fire is confined to the
floor of origin, we make a realistic and conservative assumption. Assuming that the entire floor
burns every time the fire is confined to the floor of origin would mean an overestimation of the
parameter. For example, assuming that the entire floor always burns whenever the fire is
confined to the floor adds approximately $223 million dollars to the overall estimate of property
saved in 2008, holding the other percentage constant with reference to table 5. This change
represents a 7.15% increase in overall property saved.
The correct percentages of the area of the room, floor, building and other buildings that would
burn are, once again, estimated from NYFIRS. We make only one assumption. Because fire
spreads extremely quickly, engulfing a room in 2-3 minutes, we assume that the entire room is
always at risk, wherever the fire is confined to the object of origin. This is exactly in line with
the statement “when an object is on fire, I want to say I saved the room.”
2
3
The percentages of the area of the floor, building and beyond are calculated from NYFIRS. We
use the column “units damaged by flame” and assume that each of these units is equal to a 100
square feet large room. Then for each fire, we calculated the percentage of the floor that this
damaged area represents and the percentage of the building that this damaged area represents.
Whenever the percentage of the floor damaged when a fire was confined to the floor of origin is
greater in 100%, we assume that the data for this specific fire is unreliable. In such cases, we
assume that the entire floor was damaged by flame. We run a similar analysis for building:
whenever the fire is reported as confined to the building of origin but we find that more than
100% of the builing was damaged, we treat this as a data problem and assume that this means
that the entire building was damaged by fire.
We chose not to calculate these area percentages on a fire-per-fire basis in the final estimation of
impact because of these often large inconsistencies in the data. Rather, we chose to calculate the
average percentage of floor, building and other buildings burned for each building type. In
addition, in the case of fireproof buildings, we separate between the effect of higher and lower
floors. Table 5 represents the final calculations of the percentages of room, floor, building and
beyond building per building type and higher/lower floor.
The formula for calculating percentages of floor, building and beyond is the following, for any
given fire14:
To get the numbers in Table 6, one has to calculate the average percentage of a given spread
level, a given building and, if applicable also a given floor (higher/lower). Due to data
inconsistencies, some of the averages we received for specific fires were more than 100%. This
would mean, for example, that even though the fire was confined to the floor, more than 100% of
the floor has burned. In such cases, we assumed that specific fire has spread to the entire level
(floor or building). Having more than 100% in the case of fire spread to other buildings does not
matter, because, by definition, it makes sense that the fire would spread to more than 100% of
the building.
14
Size of room is always 100 sq ft.
2
4
Table 6 demonstrates the final average percentages of a given level burned for different building
types.
You will noticed that Table 6 is color coded. While all averages in the table are calculated using
one of the three formulas on page 5, lack of data promted us to approximate some of these
number of greater categories than the cellular combination of building type/fire spread. The
color codes inform about which method was used to average the percentages out.
The first column, with lightest gray cells indicates our estimate for the percentage of the room
that is damaged, whenever a fire is confined to the room of origin. These cells were not
calculated but the percentage was assumed to be 100%.
For the cells marked in green, we had enough observations (more than 40) to calculate individual
percentages for that specific category. For example, for fireproof buildings, approximately 25%
of the floor will burn, on average, when the fire is confined to the floor of origin. The averages
are made on a specific building type and on specific fire spread.
For the cells marked in darker shades of gray, we did not have enough observations to calculate
that particular percentage. We used an average of the fire spread level across all building types.
Therefore, for a fire protected building, we estimate that approximately 66% of the floor will
burn when the fire is confined to the floor of origin. This 66% is approximately the weighted
average of 25%, 75% and 95% (the percentages of floor burned in the other building types).
For the cells marked in medium shades of gray, we also did not have enough observations to
estimate that particular percentage. We used the same percentage that was calculated for a nonfireproof building, 74%. In this case, using the same methodology as in the previous paragraph
would have overestimated the percentages. While this number would, in reality, be probably
lower, we would have to make a wild judgment on its magnitude.
2
5
A3. Calculating property saved in square feet
We have seen that property saved is a probabilistic model. We now need to combine the
probabilities of fire spread with the correct percentages of the room, floor and building
dimensions.
The tree above demonstrates the logic of the fire spread probabilities and makes it easier to
undestand the calculation of property saved for the different situations of fire containment.
Calculating property saved when fire is confined to the object of origin
When the fire is confined to the object of origin, we use the entire tree.
Property saved (room) = a (1-b) * size of room * 100%15
Property saved (floor) = ab (1-c) * size of floor * average percentage of floor that burns
Property saved (building) = abc (1-d) * size of building * average percentage of building that
burns
Property saved (beyond building) = abcd * size of building * average percentage of building that
burns16
15
Remember, we always assume that the entire room will burn.
2
6
Total property saved when fire was confined to the object of origin = Property saved
(room) + Property saved (floor) + Property saved (building) + Property saved (beyond
building)
Calculating property saved when fire is confined to the room of origin
When the fire is confined to the room of origin, we use only a portion of the tree. We start at the
branch called “room,” not “object.”
Property saved (floor) = b (1-c) * size of floor * average percentage of floor that burns
Property saved (building) = bc (1-d) * size of building * average percentage of building that
burns
Property saved (beyond building) = bcd * size of building * average percentage of building that
burns3
Total property saved when fire was confined to the room of origin = Property saved (floor)
+ Property saved (building) + Property saved (beyond building)
Calculating property saved when fire is confined to the floor of origin
When the fire is confined to the floor of origin, we use only a portion of the tree. We start at the
branch called “floor,” not “object” or “room.”
Property saved (building) = c (1-d) * size of building * average percentage of building that burns
Property saved (beyond building) = cd * size of building * average percentage of building that
burns3
Total property saved when fire was confined to the room of origin = Property saved
(building) + Property saved (beyond building)
Calculating property saved when fire is confined to the building of origin
When the fire is confined to the building of origin, we use only a portion of the tree. We start at
the branch called “floor,” not “object,”“room,” or “floor.”
Property saved (beyond building) = d * size of building * average percentage of building that
burns3
16
Remember, we assume that other buildings are of the same size as the original building. This percentage will be
different than the percentage than the percentage of the building that burns. According to the table above, this
percentage could be either 74% or 125%, depending on the building type.
2
7
Total property saved when fire was confined to the room of origin = Property saved
(beyond building)
When the fire spreads beyond the building of origin, the model estimates, by default, that the
FDNY has not saved any property in this case.
Calculating total property saved in square feet
The total property saved for each fire is calculated as the sum of the properties saved at the
different levels. For example, the total property saved for a fire, which was confined to the object
of origin is:
Total property saved =
(Property saved (room) = a (1-b) * size of room * 100%) +
(Property saved (floor) = ab (1-c) * size of floor * average percentage of floor that burns) +
(Property saved (building) = abc (1-d) * size of building * average percentage of building that
burns) +
(Property saved (beyond building) = abcd * size of building * average percentage of building
that burns)
B1. Matching rebuilding costs with property saved in square feet
The model now estimates property saved in square feet. This amount has to be matched with a
dollar amount to reach the total value of property saved for each fire.
As outlined in the main report, we use a table provided twice a year by the International Codes
Council (ICC), which specifies the rebuilding costs for each type of building type and use. We
modified this table to fit with the codes used by the FDNY. The table included in the this section
serves just for illustration. Full table is included in Apendix 3 and in the impact calculating excel
file included with the project’s deliverables.
2
8
Each of the codes is matched with a dollar rebuilding cost, which differs for each building type.
For example, you can see that a fireproof hospital (code 321) has the highest rebuilding cost
($268 per square foot), whereas as wooframe utility building (code 807) has the lowest
rebuilding cost ($41 per square foot).
The final matching is very simple. NYFIRS reports building use codes for each fire. Therefore,
the computer just needs to match the correct combination of building type and building use to get
a rebuilding cost per square foot.
C1. Calculating the value of property saved
The rebuilding cost (for example, $268 for a hospital) is multiplied by the number of saved
square feet to receive the final property saved in dollars for the given fire.
Total value of property saved = Total property saved (result from A3) * Rebuilding cost per
square foot (result from B1)
C2. Aggregating data
2
9
Two different types of aggregated data can be reported: property saved in square feet and
property saved in dollars. For communication reasons, it is most useful to calculate the latter.
However, there will occassionally be data problems, for example when the reported size of floor
is too large or too small. In this case, the calculated number of square feet can serve a good
warning flag to check, which individual fire cases may have problems.
C3. Things to bear in mind
It is important to remember that property saved reported and judged on a fire-per-fire basis has
some limitations. One, already addressed above, is that we make an assumption tha neighboring
buildings are the same size as the building of origin. Some fires will, therefore, slightly
overestimate property saved, while others will underestimate the value.
The indicator of property saved, therefore, works better when the data is aggregated on an
engine, zip code, or borough level. The differences between individual fires even out most when
the indicator is aggregated on the level of New York City and used, as it was intended, mainly as
a communication tool.
Also, we have found some quite unrealistic data entries in NYFIRS, which are mainly related to
wrong entries of number of floors, size of floor and story of origin into NYFIRS. Wrong entries
of these specific columns can greatly overestimate or underestimate the property saved indicator.
For example, an entry in NYFIRS flagged a floor size at 25 million square feet. Many entries in
NYFIRS reported that the building of origin at over 100 stories. The tallest building in New
York City, the Empire State Building, has 101 stories. To remove these outliers, we suggest the
system automatically keeps the maximum size of floor reported at 40,000 square feet (95th
percentile of the 2008 data). Similarly, we suggest that the number of floors is limited by a
similar process.
On a related note, the FDNY should not widely publicize how the formula is calculated. If
individual chiefs filling out the NYFIRS report know that especially the number of floors and
size of floor can influence the level of property saved, they might have incentives to exaggerate
these values. No red flags will be raised if they add 3 floors to a building (because no one will
know) but this subtle subversion of the system could end up increasing the overall property
saved above its natural, real level.
We also believe that the chiefs will have lower incentives to subvert the system if they do not
have any reason to fear that the indicator will be used for anything but communication purposes.
If they suspect that the indicator will serve to evaluate their own or their engine’s performance,
their incentives to tweak the system in their favor might be higher.
3
0
Appendix 2: Statistical Analysis
Overview
In order to come up with a model to develop the property saved indicator, we decided to
use the data available from in-house data collection, i.e. the NYFIRS database system. As
previouly mentioned in the report as well as in Appendix 1, we estimated the risk of fire
spread, measured by probabilities; and also the extent of fire damage given the fires,
measured in square footage terms.
The risk of fire spread was based upon statistical calculations. First, we identified the
variables that would explain the variations in firespread. Second, we tried different
specifications of the matehematical models. Third, based upon the results, we determined
what were the key factors influencing fire spread. Fourth, we calculated the probability of
the firespread, given the different values of the selected variables.
Key variables
Based on field interviews and the data available at NYFIRS, we used the following key
variables to determine the risk of fire:
We use fire spread in our model as the key dependent17 variable:
•
Fire spread: This is coded in NYFIRS as where the fire had been confined ~
1 - Confined to object of origin, 2 - Confined to room of origin, 3 - Confined to
floor of origin, 4 - Confined to building of origin, 5 - Beyond building of origin.
The following variables were tested as independent variables:
•
Response time: This is calculated as the difference between alarm time and
arrival time. Both used minutes as metrics.
•
Building type: Coded as ~ 1 - Fireproof Structure, 2 - Fire Protected Structure, 3
- Non-Fireproof Structure, 4 - Wood Frame Structure, 5 - Metal Structure, 6 Heavy Timber Structure. Given the very few occurences for the latter two
categories, we eliminated them from our statistical analysis.
•
Borough: Coded as ~ 1 – Manhattan, 2 – Bronx, 3 - Staten Island, 4 – Brooklyn,
5 – Queens. However, in order to use these for statistical purposes, we created
17
In statistics, we use different models (i.e. mathematical equations) to draw inference between variables.
By understanding the variations in the so called independent variables, we want to explain the variations in
the dependent variable.
31
bivariate, “dummy”18 out of them, using Manhattan as the base catogory, and four
different variables for the other boroughs. If there was any difference in the
outcome, then we would see this difference directly.
•
Residential or not: NYFIRS has a column titled “Property Use”; we transformed
this to a simpler variable, saying, whether it was residential or not. Again this is a
dummy variable.
•
Day or night: Based upon the alarm time information, we established another
dummy variable, expecting a difference between firespreads during nights and
days. We coded “day=1” if the alarm was between 6AM and 11PM, and “day=0
(night)” between 11PM and 6 AM.
•
Sprinklers installed or not: Coded as 1, if there were any sprinklers installed, 0 if
no sprinklers installed.
•
Fire was suspicious or not: Recoded from the “Fire Cause” column in NYFIRS,
to a dummy variable, as whether the fire was under investigation (suspicious = 1),
or not (= 0).
•
Higher floor of origin: Recoded from “Story of Origin” in NYFIRS to a dummy
variable. If the fire started at the basement or ground floor, it was coded as 0, if on
a higher floor, it was coded 1.
Step 1: Selection of the variables associated with fire spread – Descriptive statistics
We used NYFIRS data from three consecutive years: 2007, 2008, and 2009 (to date), but
ran the analysis separately. Since we found that the 2007 data was highly incomplete
(many missing values, or non-credible outliers); and considering 2009 is still running and
not yet over, we used the 2008 data for the main analysis. Nevertheless, we checked the
main results on other years too, and concluded that the key findings do not vary
significantly from year-to-year.
χ2 tests (chi-square tests)
First, we ran individual χ2 tests on the variables. This type of test is to find out if there is
any significant19 variation in the independent variables by different levels of fire spread,
given that these variables are categorical.
18
A bivariate or dummy variable has only two values: 0 or 1 – meaning something has a value, or not. We
use this when our variable is categorical, which means it doesn’t make sense to perform mathematical
operations on them, like summing or multiplying; in this way, they are different from continuous variables.
If the category has multiple alternatives, like locations, we choose a base category, and the other categories
will be used to understand the difference between the given category and the base. So, in this case, for
example, if there is an important difference in fire spread between, say, Staten Island and Manhattan.
19
In statistics, a result is called statistically significant if it is unlikely to have occurred by chance. There
are different tests, which are used to support the idea, based upon the nature of the data. The P value in the
table shows the probability, calculated by these tests, of this likeliness.
32
The results:
Variable
χ2 value
P value
Use it in the model?
Building type
306.5533
0.000
yes
Borough
138.9242
0.000
yes
Residential
82.3700
0.000
yes
Day
31.5484
0.000
yes
Sprinklers
10.9415
0.027
yes
Suspicious
669.7122
0.000
yes
Higher floor
35.3223
0.000
yes
Regression
As response time is a continuous variable, we used a simple regression20 to determine
whether there is enough variation of the fire spread given the variation in response time,
and whether this is statistically significant.
Variable
coefficient
Standard
t-value
error
Response time
0.0058257 0.0109748 0.53306
(in minutes)
P-value
Use it in the model?
0.596
no
Step 2: Model specification
The model specification required answering two questions:
•
What type of model shall we use to determine the risks?
20
Regression models draw inference between variables by estimating a linear equation between the
variables. In practice, the linear coefficient of the dependent variable can be explained as: A difference in
values of fire spread given a 1 minute difference in the response time.
33
•
Which variables are important both from the statistical, and from the practical
perspective?
During the analysis, we eventually came up with answers for both.
Bivariate logit
First we tried a simple logistic regression21, to determine if there was a significant
difference in probabilities of fire spread given our independent variables. We ran four
models for the different levels, saying, fire spread = 1, if the fire had spread at least to
that given level (say floor of origin), and 0 if it had not.
The result was strongly significant, but was not that easily interpretable, so after
consulting with our Statistics Professor, Alan Yang, we used an ordered logit instead.
Ordered logit
By using an ordered logit, we could clearly see the overall probability distributions of the
different fire spread levels, and also the statistical significance of our independent
variables. This was also useful, because we could change the specification of the model,
by using interactions22 among the independent variables, or to transform them. This
turned out to be really important, as most of our independent variables turned out to be
insignificant.
Variable
Significant? Relevant? Use it in the model?
Building type highly
yes
yes
Borough
mixed results unclear
no
Residential
highly
yes
unclear
Day
no
unclear
no
Sprinklers
highly
yes
unclear
Suspicious
highly
unclear
no
21
We use logistic regression, or logit models to analyze events with a discrete set of outcomes. (Discrete
means here, that the variable has only a limited set of values – as opposed to continuous, which can have
infinite number of values). A bivariate logit model has two discrete outcomes: something either happening,
or not happening – and the model provides probabilities of these two events. An ordered logit can have
only one outcome, but from an ordered set.
22
By using interaction terms between different variables, we want to draw conclusions on whether the
linear coefficient changes significantly in the different subgroups. In practice this would mean, is there is a
significant difference between fire spread in Brooklyn during nights and Queens during days?
34
Higher floor
moderately
no
no
The STATA output:
Ordered logistic regression
Log likelihood =
-4678.775
Number of obs
LR chi2(41)
Prob > chi2
Pseudo R2
=
=
=
=
3748
1076.26
0.0000
0.1032
-----------------------------------------------------------------------------firespread |
Coef.
Std. Err.
z
P>|z|
[95% Conf. Interval]
-------------+---------------------------------------------------------------buildingtype |
.2820817
.1494332
1.89
0.059
-.010802
.5749654
residential |
.8459157
.3188627
2.65
0.008
.2209562
1.470875
day | -.1314419
.2187722
-0.60
0.548
-.5602275
.2973436
sprinklers |
1.079978
.6021014
1.79
0.073
-.1001193
2.260075
higherstory | -.3662107
.3536932
-1.04
0.300
-1.059437
.3270153
suspicious |
1.920488
.2911296
6.60
0.000
1.349884
2.491091
bronx |
.4606154
.4131907
1.11
0.265
-.3492234
1.270454
brooklyn |
1.221639
.3351499
3.65
0.000
.5647568
1.87852
statenisland |
.7119336
.5292169
1.35
0.179
-.3253125
1.74918
responset~es |
.0358011
.0199523
1.79
0.073
-.0033047
.0749069
buildingty~x |
.1115338
.1078272
1.03
0.301
-.0998036
.3228711
buildingty~n | -.0011485
.1059457
-0.01
0.991
-.2087982
.2065012
buildingt~ns |
.2621871
.1137432
2.31
0.021
.0392545
.4851196
buildingty~d | -.0397401
.1691387
-0.23
0.814
-.3712459
.2917656
buildingty~l | -.1635894
.0968964
-1.69
0.091
-.3535029
.0263242
buildingt~rs | -.4343168
.1852071
-2.35
0.019
-.7973161
-.0713176
buildingt~ry |
.0581471
.1396177
0.42
0.677
-.2154986
.3317928
buildingt~ay | -.1790228
.0681008
-2.63
0.009
-.3124979
-.0455478
residenti~ay |
.5299831
.1824698
2.90
0.004
.1723488
.8876174
residenti~rs |
.1049555
.3460317
0.30
0.762
-.5732542
.7831653
residenti~ry | -.0737138
.3075441
-0.24
0.811
-.6764892
.5290616
residentia~x | -.5749691
.2930347
-1.96
0.050
-1.149307
-.0006316
residentia~n |
-.546986
.2855083
-1.92
0.055
-1.106572
.0125999
residenti~ns | -.3510475
.3123941
-1.12
0.261
-.9633287
.2612336
residentia~d | -.4333913
.4715306
-0.92
0.358
-1.357574
.4907917
sprinklers~y |
-.313754
.5499141
-0.57
0.568
-1.391566
.7640577
sprinklers~x |
.3806697
.488408
0.78
0.436
-.5765925
1.337932
sprinklers~n |
.8491002
.4598633
1.85
0.065
-.0522153
1.750416
sprinklers~s | -.1413974
.6469013
-0.22
0.827
-1.409301
1.126506
sprinklers~d | -1.443521
1.393919
-1.04
0.300
-4.175553
1.288511
higherstor~x |
.2620305
.3130652
0.84
0.403
-.351566
.8756271
higherstor~s |
.523918
.3603347
1.45
0.146
-.182325
1.230161
suspicious~x | -.1968888
.2240431
-0.88
0.380
-.6360052
.2422275
suspicious~n | -.3590266
.218382
-1.64
0.100
-.7870476
.0689943
suspiciou~ns | -.4706316
.2380287
-1.98
0.048
-.9371592
-.004104
suspicious~e |
.4421572
.0746938
5.92
0.000
.2957601
.5885544
suspicious~d | -.2490664
.3443997
-0.72
0.470
-.9240775
.4259446
suspiciou~ay |
-.260656
.1468171
-1.78
0.076
-.5484123
.0271002
suspicious~l | -.9308068
.1848728
-5.03
0.000
-1.293151
-.5684627
suspiciou~rs | -.5740283
.4300243
-1.33
0.182
-1.41686
.2688038
suspiciou~ry |
.373752
.2683009
1.39
0.164
-.1521081
.899612
-------------+----------------------------------------------------------------
35
/cut1 |
1.078957
.3516349
.3897648
1.768148
/cut2 |
3.354915
.3554898
2.658168
4.051662
/cut3 |
4.307259
.3572242
3.607113
5.007406
/cut4 |
6.457829
.3688887
5.734821
7.180838
------------------------------------------------------------------------------
36
Step 3 : The final selection of variables
The following are the reasons behind dropping certain variables and using those found to
be statistically more suitable in our final model:
•
Building type: Based on our results, we could see a clear difference of outcomes
given the variances in building types. If we render the probability distribution of
fire spread given the building types, we get both statistically and practically
different distributions. Therefore, we decided to use this variable.
•
Borough: From the table above we can conclude that Brooklyn is significantly
different from Manhattan, but our conclusions are not that strong for other
boroughs. Our results even get weaker, if we take into account the interactions
with other variables. (As the variations among the boroughs are most likely the
result of variation of building type and function.) Therefore, decided not to use
this variable.
•
Residential: Although residential function seems to be really significant, we
concluded this to be a selection problem i.e. as there is a much higher proportion
of residential buildings in New York City than non-residential ones, the
occurrance of fires is also greater. We suggest further investigation, as our results
were not conclusive enough – but for this current analysis, we decided not to use
this variable.
•
Day: Although it is possible that in some circumstances (depending heavily on
building type and function) it can be important, our results could not support this
hypotheseis fully. Therefore, we did not incorporate this variable in our model.
•
Sprinklers: Although installment of sprinklres is significant, this again can be a
selection problem. Only 3% of buldings, which had fires, had sprinklres installed,
and the fires were most likely to have been confined in the room or floor of
origin. Though this is important from the operational perspective, for our analysis
it was not sufficient information to estimate risks.
37
•
Suspicious: Though the results showed high significance and this could seem
plausible, operationally they are confusing. Based on the data, it seems that the
larger the fire is, the higher the probability of an investigation to happen. As such,
it was not conclusive whether the difference between fires is really a result of
suspicious activity or not, so we did not include this variable. Furthermore, there
was a very small number of confirmed arsons every year, so this data would have
been based on a small sample and would not make a large difference on the
overall property saved in New York.
•
Higher floor: The results are not not really significant, thus we couldn’t conclude
that risk would really vary given the fire had started in higher or lower floors –
everything else is equal. However, in the final calculations, we found clear
difference in the average square foot damaged for non-fireproof and wood
structure buildings, based on the storey of origin. As such, we took into account
the higher floor of origin at the potential damaged part.
Step 4 : The final selection of methodology
Back to cross tables
Based upon our results, we concluded that the single most important factor of fire spread
levels is actually building types. However, given that it is a single variable, and actually
not really a continous one (but rather one with some discrete choices), we decided to drop
the ordered logit and the predicted value. Therefore, we come back to the real life
distribution of fire spread (which we modelled with the ordered logit).
38
39
Appendix 3: Rebuilding costs table - explanation of adjustments
Table 1 below is the original Building Valuation Data (BVD) for January 2009 provided
by the International Code Council (ICC), which gives square foot construction costs by
building type and building occupancy/use. It is updated every six months and is available
from the Code Council Website: www.iccsafe.org/cs/techservices
40
Table 2: Rebuilding Costs Table for NYFIRS (based on January 2009 ICC Building Valuation Data)
Building Class (Construction Type)
NYFIRS Building Use Code
ICC Building Use
Code
1–
2 - Fire- 3 - Non- 4 - Wood 6 - Heavy
Fireproof protected fireproof frame timber
(1A, 1B) (2A, 2B) (3A, 3B) (5A, 5B)
(4)
180, 181, 182, 183, 185, 186 A-1 Assembly, theater 194.82
A-2
Assembly,
160, 161, 162
nightclubs, restaurants,
bars, banquet halls
157.60
A-3
Assembly,
131
churches
188.37
100, 110, 111, 112, 113, 120,
121, 122, 130, 131, 134, 140,
141, 142, 143, 144, 150, 151, A-3 Assembly, general,
152, 154, 155, 170, 171, 173, community
halls,
174
158.75
libraries, museums
182.66
164.91
149.01
172.52
148.23
134.73
121.75
140.49
176.20
158.44
142.54
166.06
146.08
128.01
112.41
136.44
114, 115, 116, 123, 124, 129 A-4 Assembly, arenas 184.01
171.35
153.61
137.71
161.70
557, 564, 569, 579, 592, 593,
596, 599, 241, 342
B Business
158.20
146.79
127.86
111.84
137.63
200, 210, 211, 213, 215, 254,
255, 256
E Educational
173.28
161.87
144.53
128.07
153.03
86.17
74.15
60.7
81.04
148.53
133.95
120.52
146.81
256.81
239.63
223.51
247.66
175.24
159.17
143.05
166.08
170.85
152.69
136.17
161.69
600, 610, 614, 615, 629, 631,
635, 639, 640, 642, 644, 645,
647, 648, 655, 659, 669, 679, F - Factory and
700, 981, 984
industrial
94.94
I-1
Institutional,
459, 300, 322, 332
supervised environment 158.55
I-2
Institutional,
321, 323, 331, 340, 341, 343 hospitals
268.23
I-2 Institutional, nursing
311
homes
186.65
I-3
Institutional,
361, 363, 365
182.26
restrained
41
254, 255, 256,
I-4 Institutional,
care facilities
158.55
148.53
133.95
120.52
146.81
116.99
107.37
94.52
81.54
99.88
R-1 Residential, hotels 160.67
R-2
Residential,
multiple family
134.20
R-3 Residential, oneand two-family
128.17
R-4
Residential,
care/assisted living
158.55
150.65
135.83
122.40
148.68
124.19
109.49
96.06
122.34
121.64
114.11
105.14
118.02
148.53
133.95
120.52
146.81
78.99
67.155
53.705
73.86
59.905
50.925
40.61
55.08
500, 511, 519, 529, 539, 549,
M Mercantile
559, 571, 580, 581
439, 449
429, 460, 462, 464
400, 419
day
459
800, 839, 880, 882, 888, 891,
899, 898, 965
S Storage
87.76
807, 808, 816, 819, 849, 881,
881
U Utility, miscellaneous 67.215
Table 1: ICC Building Valuation Data
42
Based on this ICC table, a table of rebuilding costs to be used in calculating the dollar
value of property saved has been constructed. This new table (Table 2) is provided below
with details explaining the specific modifications carried out to make it compatible with
FDNY/NYFIRS building codes and occupancy classification.
43
Columns:
1. ICC construction types 1A, 1B, 2A, 2B, 3A, 3B, 4, 5A, 5B are combined under
corresponding FDNY/NYFIRS building class 1 to 6 (class 5 metal-structure
buildings does not correspond with any of the ICC construction types and has
hence not been included).
2. The sq-ft construction costs listed are average values of each combined pair i.e.:
o Class 1 fireproof building = average value of ICC type 1A & 1B
o Class 2 fire protected building = average value of ICC type 2A & 2B
o Class 3 non-fireproof building = average value of ICC type 3A & 3B
o Class 4 wood frame building = average value of ICC type 5A & 5B
3. Exceptions:
o Class 6 heavy timber building corresponds directly with ICC type 4 and this single
value is used as it is i.e. value of ICC type 4 only
o ICC construction types 3B & 5B are not permitted for building occupancy I-2
Institutional (hospitals and nursing homes); costs for this particular occupancy
type are therefore given only for types 3A & 5A; these values are used as they are
for I-2 occupancy
Rows:
1. Some of the building occupancy classifications of the ICC have also been
combined to complement NYFIRS coding, and their average values have been
taken as follows:
A1, Assembly = A-1, theater with stage + A-1, theater without stage
o A2, Assembly = A-2, nightclubs + A-2, restaurants, bars, banquet halls
o F, Factory and industrial = F-1, moderate hazard + F-2, low hazard
o S, Storage = S1, moderate hazard + S2, low hazard
o
2. Note:
o
Building use/occupancy groups left out from original ICC table are Group H,
High Hazard H1, H2, H3, H4, H5; they are generally not compatible with
property uses listed in NYFIRS codes (there are a few potential codes but they fit
better with Group F - Factory and Industrial, and have been grouped under that
category).
o
All property use codes from NYFIRS have not been listed in the table. The
following codes have been left out because they do not seem to match with any of
44
the ICC classifications and/or, they are not relevant to our focus on building fires.
These codes are: 900, 919, 921, 922, 926, 931, 936, 937, 938, 940, 941, 946, 951,
952, 960, 961, 962, 963, 972, 973, 974, 982, 983
45
Appendix 4: Sensitivity Analysis
46
47
Appendix 5: Experts Contacted
Organization
Name
Title
Contact Information
Fire Department of James J. Manahan Jr. Deputy Assistant
New York (FDNY)
Chief, Chief of
Planning and
Strategy
(718) 999-7072
FDNY, Bureau of
Fire Prevention
Thomas Jensen
Chief of Fire
Prevention
jensent@fdny.nyc.gov
FDNY, Bureau of
Fire Prevention
Thomas J. Pigott
Battalion Chief,
Technology
Management
(718) 999-1510
FDNY, New York
Fire Incident
Reporting System
(NYFIRS)
Patrick Kilgallen
Captain, Project
Manager
(718) 999-0230
FDNY, Division 3
Daniel F. Donoghue Deputy Chief
manahaj@fdny.nyc.gov
pigottt@fdny.nyc.gov
kilgalp@fdny.nyc.gov
(212) 570-4220
donoghd@fdny.nyc.gov
FDNY, Bureau of
Fire Investigation
Robert Byrnes
Chief Fire Marshall (718) 999-2117
National Fire
Marty Ahrens
Protection
Association (NFPA),
MA
Manager, Fire
Analysis and
Research Division
(617) 984-7463
National Institute of Kevin McGrattan
Standard and
Technology (NIST),
MD
Fire Research
Division
(301) 975-2712
mahrens@nfpa.org
kevin.mcgrattan@nist.gov
48