Over the Top: Process Safety Lessons Related to
Transcription
Over the Top: Process Safety Lessons Related to
HAZARDS AP 2015 Title: Over the Top: Process Safety Lessons Related to Liquid Level in Process Vessels and Tanks Ref: 51 Author: Alan Munn CEnv CEng FIChemE Theme: Lesson learnt from past incidents/accidents and human factors Key Words: Process Safety, Level Measurement, Oil and Gas, Management of Change, Human Factors, Vessels, Tanks Contact details: MMI Engineering Sdn Bhd B-3A-01, Block B East; PJ8, No.23 Jalan Barat, Seksyen 8, 46050, Petaling Jaya. Malaysia. Tel: E-mail: +60 (0) 3 7494 0533 amunn@mmiengineering.com Abstract The basic principles of level measurement have been known for many years and yet in the Oil and Gas industry there is often a problem with level instrumentation that does not work as originally intended. In some refineries operators feel that they cannot rely on some of the instruments which are often ignored or operated on manual and sometimes in the case of an alarm or trip, bypassed. There have been many incidents where poor or faulty instruments, or the lack of understanding of how they work, have been a significant contributing cause, including the major accidents at BP Texas City and Buncefield UK. This paper discusses some of the reasons why level instruments misreport the true level or their output is misunderstood and how this can lead to overfilling a vessel or tank. It also discusses some common design and installation errors as well as dispelling several myths relating to level measurement. Normal operation is considered together with abnormal conditions such as at start-up and high turndown and how in some situations the plant design makes operation above the safe upper level during start up likely and in some cases certain. Management of Change issues associated with the re-use of vessels and tanks in different services or modified plant operation together with some Human Factors issues are covered. Besides the Texas City and Buncefield incidents, several examples from the authors’ personal experience will be discussed. These are examples that the Author has been involved with during incident investigations, audits and HAZOPs over many years. Over the Top: Process Safety Lessons Related to Liquid Level in Process Vessels and Tanks Introduction There have been many incidents where poor or faulty instruments, or the lack of understanding of how they work and what they actually report, have been a significant contributing cause, including the major accidents at BP Texas City and Buncefield UK. Despite this background, many oil refineries have several processes where the operators feel that they cannot rely on some of the instruments which are then ignored or operated on manual; and sometimes in the case of an alarm or trip are bypassed. In some cases these are safety critical instruments. This paper discusses some of the reasons why level instruments misread or their output is misunderstood and how this can lead to overfilling a vessel or tank. It also discusses some common design and installation errors as well as dispelling several myths relating to level measurement. Management of Change issues associated with the re-use of vessels and tanks in different services or modified plant operation together with some Human Factors issues are also covered. Besides the Texas City and Buncefield incidents, several examples from the authors’ personal experience are discussed. These are examples that the Author has been involved with during incident investigations, audits and HAZOPs over many years. Human Factors As is often the case with many process safety related problems, Human Factors is a major concern. In many cases Operators routinely operate with instruments in a failed state or even bypassed. These are sometimes safety critical instruments. In some cases this situation has been in place for many years, often several attempts have been made to fix the instrument without success and the Operators have then given up trying to get the problem resolved. This is a classic example of ‘Normalisation of Deviance’, the routine acceptance of a high risk because nothing “bad” has happened through operating this way in the past. Inadequate knowledge of how level instruments work, what they actually report, and understanding of the different installation arrangements is another common problem. In most cases, ‘level’ instruments don’t directly measure the level in the vessel, but instead measure the ‘level’ in a separate stand-pipe or bypass line. The level in this will often be different from the level in the vessel itself for various reasons as discussed later. In fact most instruments don’t measure ‘level’ at all, but measure some other parameter such as differential pressure between two points. Failure to understand this by both Designers and Operators is very common, leading to numerous problems. Confirmation bias or seeing what one is expecting to see is another human factors issue. When a level instrument fails it often gives a false reading well within the 0-100% instrument range, whereas the actual level in the vessel is too high or too low. In many cases this false reading confirms what the Operator is expecting to see, so the Operator does not realise the error and allows the upset condition to develop further. In many respects this is a result of how operators are trained; not to question readings or look for more than one indication to confirm their situation. A vital characteristic for all people involved in designing or operating a process plant is a ‘questioning’ or challenging mind-set. A lack of this ‘questioning’ mind-set has often led to problems on the plant, with level instrument problems being just one example. This failure to challenge the design, operating or maintenance practice, allows a poorly designed or installed level instrument to be in operation in the field often with potentially serious consequences. Another common example of this is the over-reliance on vendors or suppliers of instruments. Vendors are experts in the details and operation of their equipment, but they do not understand how the particular process works in which their equipment is to be installed. The Plant or Process Engineer should be working with the vendor to ensure that the instrument and installation details will perform the required operation. The ‘questioning’ or challenging mind-set helps with this. Page 2 of 11 Over the Top: Process Safety Lessons Related to Liquid Level in Process Vessels and Tanks DP Cells DP Cells are the main work-horse of level measurement, especially in the oil and gas industry. DP Cells infer the level by measuring a pressure difference between 2 points. They have to be calibrated for a particular fluid density or SG. Many level instrument problems are the result of the DP cell being calibrated for a fluid with a different density to that which is being measured. DP = ρgh Where DP = Differential Pressure (Pa) ρ = Density (kg/m3) g = Acceleration due to gravity (9.81 m/s2) h = Height of liquid column (m) As an approximation, water has an SG of 1.0 (density = 1000 kg/m3), so 1m water = 10kPa. If the taps are 1m apart and the instrument is calibrated for water then the instrument output = 10kPa when 100% full. If the fluid is changed to a gasoline blend stock with SG = 0.70, then the DP = 7.0 kPa when 100% full and the instrument output = 7kPa or only 70%. An additional complication is that a fluids density varies with temperature, so even if an instrument is calibrated for the correct fluid, it may still read incorrectly if the temperature is different. Instruments must be calibrated for the particular fluid density at the correct temperature. Direct or Indirect Measurement, Stand-Pipes, Bridles and Bypass Lines In many cases, vessels are fitted with Stand-Pipes, Bridles or Bypass Lines and the level instrument is attached to these rather than directly to the vessel. For various reasons, the level in the Stand-Pipe, Bridle or Bypass Line may be different to that in the vessel leading to an additional cause of error. This level may be different to the vessel level because: • • • • The fluid (and therefore its density) is a different composition. The fluids temperature is different (e.g. the vessel is insulated but the bridle is not). The fluid in the main vessel contains vapour bubbles but in the bridle these have separated out. The fluid in the bridle (or main vessel) contains an extra phase (e.g. a hydrocarbon layer) but the fluid in the main vessel (or bridle) does not. • The fluid in the main vessel foams but not the fluid in the bridle. • One or both of the tapping points is plugged. All of these above cases can result in false level indication, potentially leading to the Operator or the level control system to respond incorrectly leading to either a high or a low level. Even so-called “direct measurement” technologies such as Radar can give false readings if they are installed in a stand-pipe or bypass line external to the main vessel. Plugged tapping points Many services are subject to fouling. Level instrument take-offs or tapping points plug up, leading to false readings. In most refinery services such as distillation columns and overhead drums this leads to a false high reading if either of the high or low tapping points plug. If the top leg plugs, vapour condenses causing a vacuum and drawing the liquid up, if the bottom leg plugs, vapour condenses filling the top section up. However this is not always the case, and in some situations the level may stick or even fall. Flushing or purging of the take-offs has been practiced for many years to assist with keeping the nozzles clear, but with varying degrees of success. This adds additional complexity to the system and adds fluid with a different density into the mix, complicating the DP calculation further and can be expensive in terms of operating costs. To keep a 4” nozzle clear requires a significant quantity of liquid or gas flow. Many sites Page 3 of 11 Over the Top: Process Safety Lessons Related to Liquid Level in Process Vessels and Tanks routinely blow-out or rod the take-offs, but this potentially exposes the instrument or maintenance technicians to a high risk situation. In many cases this is seen as a routine activity, sometimes but not always managed under a work permit, but often with a complacent attitude; another example of ‘Normalisation of Deviance’, the routine acceptance of a high risk. If the cause of the plugging problem cannot be eliminated, then automatic rodding systems are available that clear the nozzles on a timed schedule. Besides the reliability and potential safety benefits, these can be cost effective if one takes into account the re-processing cost of the purge or flushing medium and reduced labour costs. Case Studies The following are examples of situations where level instruments have failed. In each case reference is made to the human factors concerns, density and calibration issues and installation arrangements as discussed above: Case Study 1 - Buncefield Gasoline Tank Level No discussion on safety related level problems would be complete without referring to the Buncefield incident. The key level related aspects of this incident were that the Level gauge stuck in position, resulting in no change in output and no alarms for a considerable period and the independent high level switch failing to operate. The servo level gauge had stuck some 14 times in the preceding 3 months. This had been tolerated by the Management and Operators even though the Operators relied on the alarms to control the filling process. There was general confusion over the function of the user-set, high and high-high level alarms on the tank gauging system and there was no analysis of the need for frequent repairs and a poor fault reporting and escalation system. Besides the lack of understanding of how the system should be used, this is another example of ‘Normalisation of Deviance’, the routine acceptance of a high risk. In a similar way the failure of the independent high level switch was partially due to a lack of understanding of the post testing commissioning requirements but more importantly this instrument was not seen as safety critical. One can argue that the potential consequences of over-filling had not been identified so it was not tagged as a safety critical service, but any trip system should be seen as important and should be managed appropriately - another example of ‘Normalisation of Deviance’. Case Study 2 – Texas City Raffinate Splitter Bottoms Level Texas City is another classic example of safety related level problems. Besides deliberately overfilling the bottom of the Raffinate Splitter, which was done with good intentions, the key level related aspects of this incident were the failure to understand how the level instrument worked and interpreting its output. The high level switch on the blowdown drum also failed to operate, although even if it had, there was probably not enough time to diagnose the problem and take corrective action before the drum over flowed. A previous section describes how DP cells work; their output is dependent on the density or SG of the fluid that they are measuring. Using the following approximations for simplicity: 1m of Raffinate feed (ambient temp) = 7.0kPa (SG = 0.7) 1m of Heavy Raffinate (ambient temp) = 8.0kPa (SG = 0.8) o 1m of Raffinate feed (200 C) = 6.0kPa (SG = 0.6) o 1m of Heavy Raffinate (200 C) = 7.0kPa (SG = 0.7) The instrument was calibrated for Heavy Raffinate (Tower bottoms) not feed, SG = 0.8 versus SG = 0.7. Therefore the output was 0.7/0.8 = 87.5% of the expected output Page 4 of 11 Over the Top: Process Safety Lessons Related to Liquid Level in Process Vessels and Tanks When the bottom of the tower is full and the level is above the top nozzle, the instrument will not read greater than 100% (or in this case 87.5%). Figure 1 – Texas City Raffinate Splitter: Simplified bottoms arrangement showing effect of density difference Instruments also have to be calibrated for the correct temperature. During the initial fill, the SG was higher (cold); as the tower heated up the SG decreased (hotter) and the operators saw the level coming down, even though it was actually above 100%. This is exactly as one would expect based on the operation of a DP cell. This is an example of lack of understanding of how level instruments work; specifically DP cells that measure differential pressure between two points and how these are affected by changes in fluid density or SG. This would have been compounded by confirmation bias or seeing what one is expecting to see. The Operator expected to see the level coming down and this was what appeared to be happening albeit very slowly. Page 5 of 11 Over the Top: Process Safety Lessons Related to Liquid Level in Process Vessels and Tanks Case Study 3 - HF Alkylation Unit: Main Fractionator Bottoms Level o Alkylate is a gasoline blend stock. The column bottoms operates at about 215 C and the nozzles used to regularly plug up with iron fluoride deposits leading to a false high level. The Controller (or the Operator) would increase the bottoms flow leading to a loss of level in the tower, the LSL alarm and trip failed to operate and the bottoms pumps would cavitate, leading to a trip of the fired reboiler and possible pump seal damage or failure and potential loss of containment. This is an old case study, nowadays the low level switch which tripped the bottoms pump would probably be classed as safety critical and be designed with an independent level signal, routinely tested and SIL rated etc. The solution was to install an extra set of nozzles on the tower (expensive) with a separate level indicator and an independent controller and trips. The intent was also to trend both instruments against each other so that when one failed it would quickly be seen and could be cleared. The controller could be configured for either level instrument. In addition daily flushing and rodding was required as a continuous purge was not possible. Nowadays an automatic rodding system could be installed to keep the nozzles clear (with monel lined nozzles and taking care not to remove all the iron fluoride protective layer). Figure 2 – HF Alkylation Unit: Main Fractionator Bottoms These modifications had disappointing results, as what happened was that over several months, the routine flushing and rodding became less frequent as it was not seen as necessary since there was a spare instrument so they waited until one plugged up before clearing it. Eventually, even when one set was plugged, it was not seen as a priority to fix it as the second set worked, so it would take days or even weeks to get fixed. The end result was that loss of level still regularly occurred! This is another example of ‘Normalisation of Deviance’, the routine acceptance of a high risk. Page 6 of 11 Over the Top: Process Safety Lessons Related to Liquid Level in Process Vessels and Tanks Case Study 4 - HF Alkylation Unit: Depropanizer Bottoms Level The depropanizer bottoms level instrument was unreliable occasionally causing problems with the tower operation and pressure upsets. Generally these were not safety concerns but caused operational problems and loss of isobutane. It was also possible to discharge liquid via the PSV located below the bottom tray. The depropanizer column is fitted with a stab-in steam reboiler and as a result the bottoms section contained boiling isobutane and was very turbulent. The problem here is what do we mean by the level of a boiling liquid? Imagine trying to measure the level in a boiling kettle with all the steam bubbles being formed. We were trying to measure something that doesn’t actually exist. Measuring the ‘level’ was necessary to control the bottoms flow and prevent liquid at the PSV inlet, flooding in the column or vapour break-through downstream. The PSV was at the bottom of the tower because of HF in the overheads. The solution to this situation was to add an internal baffle and measure an ‘inferred level’ or even not to measure the ‘level’ at all, but to measure the DP between the pump suction and below the bottom tray although there was much resistance to this second proposal. This is an example of how the designers, engineers and the operators failed to understand what was actually occurring inside the column. There was also significant resistance to the concept of not measuring the ‘level‘at all since the belief that there was a defined level in the column remained. As an aside – this is why measuring the ‘level’ in boilers is often difficult or impossible. Figure 3 – Depropanizer bottoms arrangement Page 7 of 11 Over the Top: Process Safety Lessons Related to Liquid Level in Process Vessels and Tanks Case Study 5 - Low Level trip on Pump Suction Line A SIL rated low level trip had been installed to prevent damage to the pump seal and potential seal failure and loss of containment. This was not part of the original design for the plant but a modification. The plant had also been debottlenecked so that the capacity was significantly higher than the original design. The drum o diameter was about 1.2m and contained sour diesel like material at about 30–40 C. The pump would trip at high rates even though there was a liquid level in the drum as could be seen in the sight glass and on the level controller. As the plant was always running at high rates, the pump kept tripping and so the operators had disabled it. Therefore, if there was a true low level the system was unprotected. Figure 4 – Low Level trip on pump suction line In this case the problem was due to the location of the level take offs, the bottom take off being in the suction line near to the pump. As there were no available nozzles on the drum when the trip was installed, the Project Engineer had used an available connection on the suction line. The suction line was undersized and the resulting pressure drop in the line was significant even though the pump NPSH requirements were satisfied. The level switch was a float type and because of the pressure drop in the line, the level in the float chamber was lower than in the drum especially at higher rates, thus causing the float switch to activate. Page 8 of 11 Over the Top: Process Safety Lessons Related to Liquid Level in Process Vessels and Tanks Case Study 6 - Guided Wave Radar on Batch Reactor A batch reactor system was fitted with a float type level indicator in an internal slotted stilling pole. Some of the chemicals used in the reaction were quite viscous and so the float used to get stuck occasionally. The operating sequence was as follows; the reactor was initially filled with chemical A and then topped up with several other chemicals (B, C etc.), all with different densities. The reaction takes place resulting in volume and temperature changes. Clearly any instrument dependant on the density of the liquid would not work. The vendor suggested installing a Guided Wave Radar (GWR) and this was accepted by the Process Engineer as a GWR is a direct measurement technique rather than an inferred type. A HAZOP was performed on the modification during which it was discovered that the GWR was to be installed in a ‘bypass line’ rather than in the original stilling pole as expected. As a result the instrument would still have been inaccurate due to density variations as discussed previously. This is a common arrangement for GWRs; all the vendors supply this as an option. There are some benefits, but it is not direct measurement of the liquid level in the vessel. In this case the problem was due to lack of communication between the Process Engineer, the Project Engineer and the Vendor. The Process Engineer assumed that the GWR would be installed in the existing stilling well and left it to the Project Engineer and Vendor to get on with the job. The Vendor was aware of the problem with the stilling well plugging up so decided to supply the bypass line option to enable it to be cleaned. Figure 5 – Guided Wave Radar on Batch Reactor Page 9 of 11 Over the Top: Process Safety Lessons Related to Liquid Level in Process Vessels and Tanks Case Study 7 - HF Alkylation Unit Emergency Acid Dump Drum The final case study involves foaming and multiple phases, both of which are responsible for numerous level measurement problems. This was on a Re-HAZOP of a retrofit project that had been in operation for many years. The emergency dump system is designed to empty the unit of HF in a major fire or loss of containment event thus minimising the amount of HF released. The drum was sized to take the full inventory of the HF reactor system including alkylate (gasoline) and unreacted butane, about 40 Tonnes in total. Transfer time was about 8-10 minutes and any vapours would be discharged via a caustic scrubber to the flare. There were 2 problems with the system; foaming and there was no way to measure the hydrocarbon phase in the drum. Foaming is a common problem in the Oil and Gas industry. Foam has a variable density, less than the liquid that is being measured. So the indicated level was lower than the height of the foam which could result in foam being routed to the Caustic Scrubber. As the foam contained lots of HF this could cause a violent reaction in the scrubber and/or spend the caustic allowing HF breakthrough to the flare system. The level instrument only had 2 take-offs from the vessel, so the bottom connection always filled with HF and because of the different densities, the indicated level was always lower than the actual level in the drum. There was no way of measuring each layer separately; to do so, requires multiple connections to the vessel. The operators had been complaining for years that they had no way of measuring the hydrocarbon content in the drum, which made it very difficult to transfer material back to the unit in a controlled manner. The problem was that the material separated into 3 layers: • • • Foam Hydrocarbon HF The ideal solution was to install an internal GDR system that was able to detect the interfaces, however cost considerations led to multiple take offs with magnetic float indicators being installed. Figure 6 – Multiple layers in Alkylation Unit Emergency Acid Dump Drum Page 10 of 11 Over the Top: Process Safety Lessons Related to Liquid Level in Process Vessels and Tanks HAZOPs and Process Hazards Analysis (PHA) In a HAZOP or PHA we identify “causes” of hazardous events; these are either equipment failures or human failures. Level problems can be both. Although there is not time in a HAZOP for a detailed analysis of any instruments, it is important to ensure that the level instruments we are relying on for control or as safeguards actually work. Listen to the Operators who will be aware of any problems, get the Process Engineer to check instrument logs and sometimes a physical check on the plant may be necessary. Often the P&IDs incorrectly shows the instrument piping details. Another common omission in HAZOP/PHAs is to not consider all possible causes for process vessel “level” deviations. The workshop team will often assume all possible causes of high, low or no/zero level are covered by the previous HAZOP/PHA guideword discussions, e.g. flow, pressure and temperature deviations. However, as described in the case studies in this paper, there are many other causes of level deviations including foaming, calibration error, instrument technology, change in composition of feed fluid, interface levels, direct or indirect measurement etc. The process engineer should highlight relevant level issues during the HAZOP/PHA. Finally, don’t expect the Instrument Engineer, Vendor or Process Design Engineer to understand how a level instrument works in your situation. Often no one in the HAZOP team does! This should be the Process Engineer’s role. SIL Assessments Just because an instrument has independent signals/tappings and is backed up by alternative/diverse level technologies, to prevent common cause failure; and is SIL rated/certified and regularly tested etc., does not necessarily mean that the complete loop (wet-end to wet-end) meets the SIL requirement. The level in the instrument or standpipe may well be different to the level in the vessel and this can lead to significant errors which in some cases can defeat the alarm or trip systems. In addition, nozzles can and often do plug up, leading to a false level indication which can go unnoticed for long periods potentially leading to failure of the alarm or trip system. Conclusions Failure of level instruments and incorrect level measurement has caused many accidents, some of them large leading to fatalities, environmental damage and significant financial loss to owner companies and the public. The key to avoiding level related incidents is to understand how the instruments work in the particular situation; don’t assume that “someone else” will do this, which is often not the case. Don’t assume the design is correct; sometimes there are errors, especially when the project is on a fast track schedule. This is why it is so important for any design to go through a thorough design review and having client process engineers and operators on the design review and HAZOP teams to properly challenge the basis of design and safeguarding philosophy. Adopt a questioning or challenging mind-set. Operators should be taught to question instruments readings and look for independent verification, especially during upset conditions. A good Process and Process Safety Engineer (including HAZOP/PHA facilitators) must understand the measurement system and technology so they can understand and correctly interpret the reported results. The case studies in this paper are presented in the hope that some of the lessons learnt will be incorporated into future designs and operating practices and thus reduce the risk of future incidents. Page 11 of 11