Crowdsourcing of Weather Data on Mobile App and Deep Learning
Crowdsourcing of Weather Data on Mobile App and Deep Learning
Lior Perez
99th AMS annual meeting Crowdsourcing on Meteo-France mobile app
■ Context: ― fewer resources devoted to human observation
■ Crowdsourcing can help: ― To get a high density of human observations ― To get information on impacts of weather events
■ Dedicated observation app: NO ― Too difficult to get a large audience
■ Add a crowdsourcing module in our general public app ― Benefit from a 1M visitors per day audience Keep it simple
■ We wanted maximum participation rate
Keep it simple!
A challenge for our culture of weather experts...
In first version:
■ Only immediate observation ■ Only for geolocalized users ■ No quantitative observation
■ Very few details in each observation Feedback and gamification to increase user engagement Success in quantity and quality
■ 10k to 40k observations every day
■ Approx 1000 obs / h disseminated on all the French territory ■ Good quality, very few outliers ■ Large increase of observations rate in severe weather (13 observations of a tornado at 1am) Outliers filtering
■ Methods investigated: ― Obvious outliers removal ► For instance: Hail + Fog + Sun + Strong wind ― Anomaly detection using the multivariate gaussian distribution, to detect unreliable users
■ Conclusions: ― Most unreliable users don’t come back ― Returning users are generally reliable ― Fake observations < 1% What are we doing with the data? Internal visualization interface for forecasters Subjective product validation
■ Product : distinction of hydrometeors
No precipitation / No snow lying on the ground
No precipitation / Snow lying on the ground
No precipitation / Ground invisible (night or clouds)
Drizzle
Rain
Drizzle over frozen ground
Rain over frozen ground
Freezing drizzle
Freezing rain
Rain and snow mixed
Slushy snow
Wet snow
Dry snow
Slushy snow lying on the ground
Wet snow lying on the ground
Dry snow lying on the ground
Ice pellets
Small hail
Medium hail
Large hail
Validation of the distinction between snow and rain New feature: Observation with picture Enabling users to post pictures
■ New feature: observation with picture Issue
We need real time moderation
OK Not OK Image classification: a problem solved by Deep Learning
■ Dogs vs. cats
Dog Cat Image classification: a problem solved by Deep Learning
■ ImageNet: Database of 14 million hand-annotated images
■ ImageNet Challenge: Image classification models, better than human performance Transfer Learning
1) Use an image classification model that has been trained on 1.2 million of images from ImageNet ― Inception v3
2) Re-train it: specialize it on our two classes ― Class 1: OK, it’s related to weather ― Class 2: Not OK, it’s not related to weather
It’s an easy an quick process! Prepare a training dataset: images in two folders (OK / Not OK)
OK Not OK Pictures on the app in production since November
■ No incident, the automatic moderation system has worked
Accepted Rejected
0 Crowdsourcing of weather data: conclusion
■ Excellent user feedback ― Already used ► By forecasters ► For subjective validation of products ― Good public participation level
■ Perspectives ― Use of Deep Learning image classification to identify the type of weather on pictures ― Enable advanced users to make more detailed observations