Dr. Julia Dark is Senior Software Engineer at TileDB working on geospatial applications.
The Geostationary Operational Environment Satellite-R (GOES-R) series provides continuous satellite imagery of the Earth’s eastern hemisphere. GOES-R series datasets are made available through multiple cloud service providers via NOAA’s Big Data Program. The datasets include Level 1b and Level 2 satellite data split into directories of NetCDF files stored for consecutive time periods. This talk will show how to use TileDB Embedded, an open-source universal storage engine, to combine data from multiple GOES-R products into a single easily-accessible dataset.
In this talk, I will show how to ingest data from the GOES-R Advance Baseline Imager (ABI) and Geostationary Lightning Mapper (GLM) into cloud-ready storage using TileDB Embedded. I will discuss the pros and cons of keeping the original NetCDF data model, and show how to combine datasets that consist of both dense and sparse arrays. With the arrays stored in TileDB Embedded, I will show how to efficiently slice weather data, locally and remotely on cloud object storage; how to use data versioning to time-travel across any changes to an array; and give an overview of some of the open-source tools that integrate directly with TileDB Embedded.