Low Cost and High Available Container-Based OGC Web Services for hosting OneGeology and INDE SDIs - The CPRM's experience on move to FOSS4G
2021-09-29, 16:00–16:30, Ushuaia

This work aims to present the experience acquired by CPRM (Geological Survey of Brazil), in the last two years, with the implementation of FOSS4G (OGC Servers, Spatial Databases) in Docker Swarm Clusters. The developed infrastructure can be implemented in low-cost physical servers or virtual machines, and has a simple administration.


The Geological Survey of Brazil (CPRM) is now under a digital transformation process. One of the pillars of this process involves speed, scalability, security and availability of data produced by the researchers. Furthermore, CPRM is creating a favorable environment for the adoption of new paradigms of software architecture, focused in distributed computing. In recent months, through the Department of Institutional Information and the divisions of Geoprocessing and IT Engineering, developed a container-based clustered environment, to host FOSS4G applications. The aim of this work is to present the state of the art of brand-new architecture to host OGC-based data and metadata services, whose standards were established by INDE (Official Brazilian SDI). In addition, CPRM also participates in OneGeology, which is an initiative to integrate global geological maps, whose software architecture, like most INDE and partner institutions, is based either on FOSS4G – mainly Geoserver, Geonetwork and PostgreSQL/PostGIS. The cluster now is hosted in RNP’s Data Center, Brasilia. In it’s internal network, there is a pool of 8 servers (4 manager and 4 worker nodes), with 8 cores and 8 GB of RAM with Linux OS (RHEL7) and Docker Engine 19.03, with orchestration with Swarm, whose is docker-native. In the DMZ network, two extra servers, with modest hardware requirements, were configured with HAProxy and Keepalived, listening to each other simultaneously. These servers have the function of receiving and encrypting public requests and redirects to the Docker nodes. The network structure was defined in 4 environments composed of independent networks: the first, public, with access to the internet controlled by a firewall; the second, restricted, with controlled access with connection manager and virtual IP service with failover and failback; the third, internal, which receives connections and directs to servers (Ingress Network); and the fourth, exclusive, among the component containers of each service. The Traefik Edge Router is an exclusive stack if services that redirect all incoming connections from HAProxy to the applications hosted on cluster. The persistent data volumes (upload area, database namespaces) are allocated on NFS server, with a more controlled backup routine – with future plans to move to GlusterFS. The administration of Docker Swarm is done in Portainer (Web based Docker Admin) and the image repository, in SUSE Portus. To deploy INDE and OneGeology services, a set of standard images was developed, based on the official Docker images, whose settings have been customized for this purpose. Images of Geoserver 2.17 and Geonetwork 3.10, were developed from compatible Apache Tomcat and OpenJDK. In addition, images of PostgreSQL 11/PostGIS 2.5 and reverse proxy (Nginx), not to expose Tomcat's non-standard HTTP ports. Each of the services can be scaled individually, with the creation of replicas that uses data from the shared areas. Finally, this architecture were performed with FOSS, and that physical, or virtual servers, should be replaceable, within the perspective of public institutions, the implementation of a decentralized architecture, and can be applied in hardware of low cost.


Authors and Affiliations

Carlos Eduardo Mota (1)
Alvaro Gomes Sobral Barcellos (1)
Marcelo Pacheco Zoel (2)
Marcio Vinicius Queiroz (2)

(1) Geological Survey of Brazil: Geoprocessing Division
(2) Geological Survey of Brazil: IT Infrastructure Division

Track

Transition to FOSS4G

Topic

Government and Institutions

Level

2 - Basic. General basic knowledge is required.

Language of the Presentation

English

Geologist, PhD with +10 years of experience in FOSS4G, especially Python, PostGIS, MapServer and GeoServer.