Weather isn't just sun and rain!
In this blog, we present how we handle massive amounts of complex weather data in our data management software.
We show you how we can deal with thousands of weather stations producing a wide variety of actual and forecasted metrics.
Weather data
Weather data information is valuable in various sectors and industries such as: energy and utilities, transportation and logistics, emergency management, tourism, insurance, construction, retail, research, and many more.
It enables informed decision-making and risk management based on current and predicted weather conditions.
Weather data types
To analyse the climate changes but also to make use of the various approaches for alternative energy sources it is important to store weather data properly. And we will see in the following there are tons of weather data to store - from whichever direction the use case is coming from. Some of the common types of the variety of weather data that are stored are the following:
- Temperature:
This includes current temperature, maximum and minimum temperature, and temperature variations over time.
- Precipitation:
Information about rainfall, snowfall, sleet, and other forms of precipitation, including intensity, accumulation, and duration.
- Humidity:
The amount of moisture present in the air, often expressed as relative humidity.
- Wind:
Wind speed, direction, gusts, and patterns.
- Atmospheric Pressure:
Air pressure at a given location, often measured in millibars or inches of mercury.
- Visibility:
The extent to which objects are visible in the atmosphere, affected by factors such as fog, mist, or haze.
- Cloud Cover:
The amount or percentage of the sky covered by clouds.
- Sunrise and Sunset Times:
The time when the sun rises and sets at a particular location.
- Weather Conditions:
Descriptions of the overall weather conditions, such as sunny, cloudy, rainy, snowy, etc.
- Weather Forecasts:
Predictions of future weather conditions based on various meteorological models and data.
Additionally, weather data can include historical weather records, climate averages, severe weather alerts, air quality information, UV index, and more. These data are collected from weather stations, satellites, radar systems, and other meteorological instruments.
Weather data in ODSL
Example: DWD Data (Deutscher Wetter Dienst - German Weather Service)
As an example we store weather data from DWD and use the weather stations in Hamburg, a city in north Germany, in the following examples.
Storing Geospatial Data
Storing and visualization of the geo data for the weather stations in Hamburg using the ODSL Web-Portal:
Storing the information of weather stations
For the different weather stations in various locations we store the information in objects and their properties. The following screenshot gives a view how it looks like in the ODSL Web-Portal.
For an entire overview, here is a list of properties for one weather station in Hamburg (city) that we are actually storing in Mongo database:
Wind related:
- Wind direction
- Absolute error wind direction
- Absolute error wind speed 10m above surface
- Wind speed
- Maximum wind gust within the last n = {1, 3, 12} hour
- Probability: Occurrence of gusts >= n = {25kn, 40kn, 55kn} within the last n = {6, 12} hours
- Probability: Occurrence of thunderstorms within the last n = {1, 6, 12, 24} hours
- Probability: Occurrence of drizzle within the last n = {1, 6, 12} hours
Cloud cover related:
- Total cloud cover
- Cloud cover below 500 ft.
- Effective cloud cover
- High cloud cover (>7 km)
- Low cloud cover (lower than 2 km)
- Cloud cover low and mid level clouds below 7000 m
- Midlevel cloud cover (2-7 km)
- Potential evapotranspiration within the last 24 hours
- Probability for fog within the last n = {1, 6, 12, 24} hours
Pressure related:
- Surface pressure
- Absolute error surface pressure
Dew-point related
- Dew-point 2m above surface
- Absolute error dew-point 2m above surface
Precipitation related:
- Duration of precipitation within the last hour
- Probability of precipitation > n = {0.1mm, 0.2mm, 0.3mm, 0.5mm, 0.7mm, 1.0mm, 2.0mm, 3.0mm, 5.0mm, 10.0mm, 15mm, 25mm} during the last hour
- Probability of precipitation > n = {0.0mm, 0.2mm, 1.0mm, 5.0mm} during the last m = {6, 12, 24} hours
- Total precipitation during the last n = {1, 3, 6, 12, 24} hours consistent with significant weather
- Total liquid precipitation during the last hour consistent with significant weather
- Probability: Occurrence of {convective, stratiform, solid, liquid} precipitation within the last n = {1, 6, 12} hours
- Probability: Occurrence of freezing rain within the last n = {1, 6, 12} hours
- Snow-Rain-Equivalent during the last n = {1, 3} hours
Sunshine related:
- Relative sunshine duration within the last 24 hours
- Sunshine duration during the last n = {1, 3} hours
- Yesterdays total sunshine duration
- Probability: relative sunshine duration > n = {0%, 30%, 60%} within 24 hours
- Global Irradiance
- Global irradiance within the last hour
Temperature related:
- Temperature 5cm above surface
- Minimum surface temperature at 5cm within the last 12 hours
- Mean temperature during the last 24 hours
- Minimum temperature - within the last 12 hours
- Temperature 2m above surface
- Maximum temperature - within the last 12 hours
- Absolute error temperature 2m above surface
Visibility related:
- Visibility
- Probability: Visibility below 1000m
Significant weather related:
- Optional significant weather (highest priority) during the last n = {1, 3, 6, 12, 24} hours
- Significant Weather
- Significant Weather of the last 3 hours
- Past weather during the last 6 hours
The data can be used for visualization, further analysis as an input for ODSL smart curves, which means input for simple or complex calculations, such as weather derivatives. In the following we visualize the information for wind speed and cloud cover.
Wind Speed
Looking at the wind speed data stored as a forward curve with hourly tenors - here for 22 March 2023.
Cloud Cover
Looking at the cloud cover data stored as a forward curve with hourly tenors - here for 22 March 2023.
Conclusion
Storing weather data needs a proper data management tool and approach as there are vast amounts of various data types to be stored. This data can be used for different use cases such as data analysis, weather derivatives, alternative energy sources management and many more.
Next steps
Do you want to see this in action and see how you can benefit from OpenDataDSL?
Fill out the form below, we will contact you to arrange a personally tailored demo.
How about a demo?
Our team is here to find the right solution for you, contact us to see this in action.
Fill out your details below and somebody will be in contact with you very shortly.
More information or free trial?
Tell us about your project, and we can let you know how we can help.
Contact us at info@opendatadsl.com