Barcelona, Spain).– International researchers, with the participation of the Universitat Oberta de Catalunya (UOC), in Spain, have developed a computer vision system based on ‘deep learning’ that takes advantage of a database of data from more than 1.7 million photographs to automatically detect natural disasters through images posted on social networks.
“We have shown that the automatic detection of incidents on networks such as Twitter is feasible, and this can greatly help humanitarian aid organizations,” said UOC professor Ágata Lapedriza, a specialist in artificial intelligence who has participated in the work, led by by the Massachusetts Institute of Technology (MIT).
The research has applied computer vision tools that, once trained with 1.7 million photographs, have proven effective in analyzing, filter and detect real catastrophes, such as floods, tornadoes or forest fires, which, on the other hand, are becoming more frequent and devastating as a result of climate change.
“As there are still no tools to predict where and when this type of incident will take place, articulating a quick and effective response from the emergency services and international cooperation is essential to save lives,” Lapedriza stressed.
“Fortunately,” he added, “technology can play a very important role in these situations. Social media posts can be used as a low latency data source to understand the progression and consequences of a disaster.”
In the research, which is published in the journal ‘Transactions on Pattern Analysis and Machine Intelligence’, Lapedriza has contributed to defining the taxonomy of incidents, create the database to train the ‘deep learning’ models and run the experiments that validated this technology.
The researchers established a list with 43 categories of natural incidents (avalanches, sandstorms, earthquakes, volcanic eruptions, droughts…) and different types of accidents with some degree of human intervention (aircraft, construction, etc.).
This list, together with a typology of 49 places, allowed labeling the photographs with which the system was prepared, which contains 1,787,154 labeled images, to subsequently train the detection model.
The tags allowed the system to be trained to eliminate false positives and understand, for example, that a photograph of a bonfire does not correspond to a fire, even though it shares similar visual characteristics.
With this base made, the team trained a model to detect incidents “following the multitasking learning paradigm and using a convolutional neural network,” according to Lapedriza.
Once the deep learning model was trained to detect incidents in images, the team put it to the test in several experiments.using a huge volume of images downloaded from social networks such as Flickr and Twitter.
“Within these images, our model detected those that corresponded to incidents and matched specific incidents for which there was a record, such as the 2015 earthquakes in Nepal and Chile,” explained Lapedriza.
The authors have demonstrated with real data the potential of using this artificial intelligence tool to obtain information from social networks about natural disasters and events that require humanitarian aid.
“This opens the doors for humanitarian aid organizations to find out what is happening more efficiently and to improve the management of humanitarian aid when necessary,” added the specialist, who is now considering taking advantage of the same images of catastrophes to quantify the severity of incidents automatically, or even more effectively track how they evolve over time.
