Research

My research focuses on the intersection of computer vision and Earth observation. Computer vision enables computers to “see” and therefore to perform autonomous image analysis methods on large amounts of data. Earth observation data feature unique traits that make them an interesting test bed for computer vision methods: virtually unlimited amounts of multi-modal (different sensors) time-series (observations at different times) data are available for virtually any location on Earth at almost no cost. The combination of these two fields enable me to develop efficient Deep Learning methods for computer vision problems with applications to Earth observation and beyond.

Some approaches that I am currently interested in include self-supervised learning in combination with data fusion methods. Furthermore, I am interested in weakly supervised learning methods and applications to real-world problems.

Some of my recent research results are listed in the following:

• Traffic Noise Estimation from Satellite Imagery with Deep Learning | 2022-06

Applied Research Earth Observation Deep Learning Segmentation Traffic Noise Sentinel-2

Road traffic noise is a global issue that can lead to severe health effects. Despite the ubiquity of traffic noise, its quantification or estimation is complicated and detailed road traffic maps are only available for select countries or areas. We investigate whether it is possible to train a segmentation model to esimate road traffic noise from satellite imagery.

• Contrastive Self-Supervised Learning for Multi-modal Earth Observation Data | 2022-06

Fundamental Research Earth Observation Deep Learning Self-supervised Learnig Data Fusion Transformers Classification Segmentation

Self-supervised learning provides a powerful means to pretrain models based on un-labeled data. Un-labeled Earth observation data are abundant: this circumstance combined with the availability of multi modal data makes Earth observation a perfect playground for self-supervised learning. Our early results are very promising...

• Estimating Power Plant Greenhouse Gas Emissions from Satellite Imagery | 2021-12

Applied Research Earth Observation Deep Learning Multi-task Learning Power Plants CO2 Sentinel-2

Tracking Greenhouse Gas emissions will be increasingly important in the future, as power generation from fossil fuels is supposed to fade out. Independent tools to monitor power plants are required for this task. We present a method that uses freely available satellite imagery to estimate power generation and CO2 emission rates on a global scale.

• Estimation of Surface Level NO2 from Remote Sensing Data | 2021-11

Applied Research Earth Observation Deep Learning Air Pollution NO2 Sentinel-2

Air pollution is a major health issue and often also contributes to climate change. Measuring air pollution is costly and therefore only available in some countries. We investigated whether freely available Earth observation data can be utilized to estimate air pollution on the surface level.

• Commercial Vehicle Traffic Detection from Satellite Imagery with Deep Learning | 2021-11

Applied Research Earth Observation Deep Learning Traffic Sentinel-2

Can we identify trucks from space and estimate truck traffic rates anywhere on the planet? Yes, we can!

In my previous research in astronomy, I investigated the physical properties of asteroids and comets and built open-source scientific software.

For a more complete list of research projects, please consult the archive.

Team

Research is a team effort. Our team is part of the Artificial Intelligence and Machine Learning chair, headed by Prof. Dr. Damian Borth, who is also involved in many of our projects. Our success relies heavily on the work of our students; the following students are or were (co-)supervised by me:

Current Students

• Joelle Hanna*, PhD in Computer Science, since 2021, official supervisor: Damian Borth.

• Linus Scheibenreif*, PhD in Computer Science, since 2021, official supervisor: Damian Borth.

• Yosef Asefaw, Bachelor in Business Administration, Deep clustering of traffic camera imagery.
• Aron Baeriswyl, Master in Quantitative Economics/Finance, Improving Road Noise Estimation with Contrastive Self-Supervised Learning.
• Yannick Cadonau, Bachelor in Business Adminiistration, Setting realistic expectations for solar power production forecasts utilizing weather predictions.
• Lorenz Dreyer, Master in General Management, Characterization of Open-pit Mining Areas from Satellite Imagery.
• Flurin Jacomet, Bachelor in Economics, Identifying Active Mining Operations.

Former Students

• Robin Sutter, Time Series Forecasting of Photovoltaic Energy with Physics-guided Deep Learning Ensembles, Master in Business Innovation, 2022.
• Leonardo Eicher, Traffic Noise Estimation from Remote Imaging Data with Deep Learning, Bachelor in Business Administration, 2022; results were published at IGARSS 2022 (open access: unisg).
• Moritz Blattner, Detection and Monitoring of Commercial Vehicles from Satellite Images using Deep Learning, Master in Business Innovation, 2021; results were published at the Tackling Climate Change with Machine Learning workshop at ICML 2021.
• Samuel Navarro-Meza, Rapid response characterization of Near-Earth Asteroids using RATIR, Universidad Nacional Autónoma de México/Northern Arizona University, PhD, 2021; co-mentored with Mauricio Reyes-Ruiz and David Trilling; results were published in The Astronomical Journal.
• Nathan Smith, Thermophysical Modeling of Phobos, Northern Arizona University Physics Master student, 2017; co-mentored with Christopher Edwards and David Trilling; results were published at the Ninth International Conference on Mars 2019.
• Louis Dan Avner, The Flagstaff Robotic Survey Telescope, Northern Arizona University Physics Master student, 2017; co-mentored with David Trilling and Ted Dunham.
• Cassandra Lejoly, Near-Infrared and Optical Observations of Centaurs and Trans-Neptunian Objects, Northern Arizona University Physics Master student, 2016; thesis.