From OLAP to OLTP: How Copel Simplified Reverse ETL Using BigQuery's

Google Cloud announced the general availability of BigQuery exports to AlloyDB on November 14, 2026, enabling teams to export analytical datasets into transactional PostgreSQL instances using declarative SQL. According to Google Cloud Blog's report, this native integration removes reverse ETL middleware, Cloud Storage staging buckets, and multi-step pipeline orchestration. Direct parallel connections now link analytical clusters with operational databases.

This technical analysis examines native export architecture, benchmark metrics, and system improvements demonstrated by Brazilian electric utility Copel during its telemetry migration. Systems architects, database administrators, and data engineers managing hybrid analytical and transactional cloud workloads can use these findings to evaluate reverse ETL migration strategies.

01Legacy telemetry pipelines and staging buckets

Copel manages telemetry streams from millions of smart electricity meters, consolidating readings into a BigQuery analytical Gold layer. Operational systems, including customer support dashboards, mobile applications, and billing engines, require this enriched dataset inside AlloyDB for PostgreSQL. To synchronize both database engines, Copel previously relied on complex DAG orchestration within Google Cloud Managed Service for Apache Airflow.

Under the legacy architecture, BigQuery queries extracted data and dumped fragmented CSV files into Google Cloud Storage buckets. Airflow then executed SQL commands to create temporary tables in AlloyDB, preventing write locking and table corruption during ingestion. Sequential gCloud scripts subsequently parsed those CSV files to load data across AlloyDB clusters and shards.

Finally, an explicit SQL transaction block dropped production tables and renamed temporary tables to production status. Managing this multi-step architecture required continuous pipeline monitoring, intermediate file staging governance, and complex service account IAM impersonation across multiple cloud services. The process introduced operational latency and recurring maintenance bottlenecks for data engineers.

02Declarative exports across parallel database connections

The general availability release replaces intermediate file manipulation with BigQuery's native EXPORT DATA statement targeting AlloyDB instances. Analytical export jobs communicate directly with AlloyDB over parallel connections running across Google's private network backbone. Database teams configure the connection identifier, instance URI, and target table name directly inside declarative SQL statements without intermediary scripts.

Copel's implementation extracts operational records from smart meter datasets while serializing structured fields into PostgreSQL JSONB on the fly. By wrapping nested metrics inside BigQuery's TO_JSON function, the export engine maps source records directly to AlloyDB table columns. The resulting values arrive fully indexed and queryable for downstream transactional services upon completion.

Direct export communication eliminates Cloud Storage staging files and removes custom gCloud loading commands from the pipeline. The underlying engine coordinates parallel writes across database shards without temporary tables or manual renaming transactions. Data engineering teams maintain schema validation and column transformations directly within the declarative export query.

03Benchmark throughput across two hundred million records

Copel executed two production benchmarks to evaluate export throughput, data scalability, and cluster hardware stability under operational conditions. Engineers compared an incremental export against a full-table transfer using an AlloyDB N2 instance configured with 16 virtual CPUs and 128 gigabytes of RAM. Both benchmark runs recorded row throughput, total dataset size, and sustained processor utilization.

The incremental benchmark exported 18,145,589 rows representing 2.32 gigabytes of logical data, achieving an ingest throughput of approximately 3.1 million rows per minute. The full-table benchmark transferred 207,484,577 rows containing 188.87 gigabytes of logical telemetry data. During this full-scale workload, the pipeline sustained an export throughput of approximately 4.2 million rows per minute.

Hardware telemetry showed processor utilization on the destination AlloyDB cluster remained stable at approximately 85 percent throughout both benchmark evaluations. Sustaining 4.2 million rows per minute on complex JSONB fields highlighted deep optimizations between BigQuery's distributed processing engine and AlloyDB's storage layer. Neither workload triggered memory degradation, database timeouts, or connection failures during execution.

04Infrastructure cost cuts and simplified airflow orchestration

Adopting declarative exports enabled Copel to delete dozens of custom shell scripts, error-handling routines, and staging maintenance procedures. The company simplified its Airflow DAG down to a single BigQueryInsertJobOperator task executing the export SQL statement. Airflow now oversees the transfer as a standard BigQuery query job, drastically reducing operational complexity for data platform teams.

Bypassing Google Cloud Storage eliminated intermediate object storage fees alongside cumulative read and write API operation expenses. Data security increased because records move across Google's internal network without ever landing in readable, unencrypted CSV files on disk. Isolating pipeline execution within a single SQL statement streamlined system auditing, access control, and operational troubleshooting.

Solutions and cloud architect Jelson A. Cordeiro and BigQuery product manager Nick Orlove noted that eliminating staging buffers redirects engineering hours toward high-value analytics.

Removing staging buckets also eliminated complex cross-service IAM impersonation chains between Airflow, Cloud Storage, and AlloyDB instances. The resulting architecture maintains distributed write speed while preserving transactional data integrity for critical billing operations.

05Direct reverse exports across cloud operational engines

The general availability of AlloyDB exports expands Google's built-in reverse ETL capabilities across its managed database portfolio. Operational services increasingly require analytical data synchronization without third-party integration appliances or fragile custom connectors. In addition to AlloyDB, BigQuery provides native export integrations for Cloud Bigtable, Cloud Spanner, and Pub/Sub messaging topics.

Engineering teams running real-time operational applications can initiate direct exports using standard Google Cloud IAM connection objects. Because destination tables accept rows through optimized parallel channels, database administrators avoid building custom connection pools or manual retry logic. Teams can schedule recurring sync jobs through standard Airflow operators, Cloud Composer, or native BigQuery scheduled queries.

As telemetry data volumes expand, eliminating staging hops between analytical warehouses and transactional databases ensures reliable data freshness for downstream systems. Organizations maintaining multi-stage CSV pipelines can evaluate declarative SQL exports against existing AlloyDB clusters. Direct database exports deliver enterprise throughput while minimizing infrastructure overhead across modern cloud data architectures.