Introduction
Smoke plumes contribute significantly to the air pollution in cities posing serious health risks including heart disease, lung cancer, and asthma. The New York City Department of Health recently estimated that up to 2,700 premature deaths a year could be attributed to fine particulate matter and ozone in the air \citep{health2013,Mills_2008}. In 2005, it was estimated that approximately 10,000 buildings in the city burned number 4 and 6 heating oils, which emit more air polluting Particulate Matter (PM) 2.5, Sulphur Dioxide (SO2) and nickel than alternative fuels \citet{health2013}. In 2007, NYC launched a sustainability program, titled PlaNYC, which aims to bring significant emission reductions, with a goal of 30%, by 2030 \citep{sustainability2013}. According to \citet{york2010}, NYC buildings account for 75% of all of greenhouse gas emissions (including CO2), meaning that in order to enact the necessary change, building energy usage needs to be addressed and understood. The wider implications of this study could impact many different city agencies and departments such as those overseeing energy, environment, health, transport, buildings, and housing.
The traditional methods for detection of plumes rely on extensive groups of connected sensors (sensor networks) that provide a local measure of air quality by detecting the presence of particles in the air. \citep{Brink_2013} This would require a dense and extensive network in order to gain a comprehensive view of the entire city, requiring permission from building owners to use their buildings, and a team to maintain the network. Additionally, all nodes need network connectivity to support data transfer, making the process cumbersome and costly. This project's objective is to make the spatio-temporal tagging of the plumes a real-time and viable process using image data.
Automatic image-based smoke detection models from the literature span a variety of different methods, many using hand-engineered features (e.g. threshold setting). \citet{demirel2007} take a statistical approach, using color models to detect both regions with smoke and those with fire which are constructed using hand-engineered color features such as setting thresholds for the color range. \citet{Yuan_2008} attempts to improve the false alarm rate of video-based smoke detection algorithms by incorporating the orientation of the smoke’s motion, helping remove the disturbance of other moving objects. \citet{Gubbi_2009,Ko_2013} use visual codebook style representations to detect the presence of smoke, employing support vector machines (SVMs) and random forest classifiers, respectively.
Recognizing that the existing literature was primarily rule-based models and hand-engineered features, and the potential of Convolutional Neural Networks (CNNs) given their demonstrated success in image classification, \citealt{Karpathy_2014}. \citet{Frizzi_2016} trained CNNs to detect fire and smoke in still images. Convolutional neural networks (CNNs) have emerged as the state-of-the-art image classification algorithm due to its efficient architecture that takes advantage of the stationarity and locality of patterns found in images and videos. Unlike other machine learning methods, Convolutional Neural Networks do not rely on engineered features, but rather extract the features most relevant for classification automatically based on a labeled training set. CNNs perform well for the goal of image classification; however what about the more detailed question of where a specific object is within an image. This may be the case for a plume (or multiple plumes) within an image. To correctly identify where the main objects in the images, Faster R-CNN is a specific form of CNN that includes a region proposal network which hypothesizes object locations via bounding boxes, and in a more efficient way than its predecessors, R-CNN and Fast R-CNN \citep{Ren_2017}.
This project aims to create a method for detecting and recording plumes of pollution in NYC using images gathered from the Urban Observatory at New York University’s Center for Urban Science and Progress (
CUSP-UO). The CUSP-UO studies the complex interactions between the physical, natural, and human components of the city as a coherent, definable system with the goal of enhancing public well-being, city operations, and future urban plans. CUSP-UO continuously images the Manhattan skyline at 0.1 Hz, for use in image based detection which is synoptic, persistent and non-intrusive. The daytime images can be used to detect and characterize plumes from buildings in the scene
\citep{swurtelec2015}. The project also aims to identify various statistics such as the origin, count and frequency of the plumes. This will be performed by constructing a training dataset which will be used to train Faster R-CNN models for plume location. The locations can then be mapped to geographic coordinates to identify a given source building.