Production MLOps at GS Caltex with Amazon SageMaker Canvas, part 1

South Korean energy and chemical firm GS Caltex has integrated Amazon SageMaker Canvas with Amazon SageMaker Pipelines to run no-code machine learning models across industrial operations. According to AWS Machine Learning Blog's report, the company automated the handoff between plant process engineers and machine learning teams. This automation reduced model deployment timelines from approximately one week to a few hours while establishing recurring batch inference on 30-minute intervals.

This report outlines how GS Caltex bridged visual model training and enterprise pipeline engineering without writing custom inference services from scratch. It details the operational division of labor, architectural components, preprocessing exports, input schema alignments, and model governance mechanisms. The architecture serves enterprise machine learning engineers, data platform architects, and operations specialists aiming to operationalize citizen data science assets at scale.

01Process engineer models in production pipelines

GS Caltex structured its operational machine learning lifecycle around two specialized engineering disciplines. Process engineers leverage domain expertise regarding plant variables and operational data to build and validate predictive models in SageMaker Canvas without code. Meanwhile, machine learning engineers translate those validated assets into executable SageMaker Pipelines jobs, overseeing model configurations, execution history, and production endpoints.

Transitioning graphical assets into production workflows historically generated significant manual engineering friction. Technical staff previously spent days configuring production data paths, aligning model settings, and bridging distinct data preparation and inference routines. By standardizing this handoff into an automated repeatable pipeline, GS Caltex preserved domain-driven modeling while eliminating manual deployment bottlenecks.

The production architecture divides responsibilities across two operational phases. The first phase manages the packaging, validation, and registration of newly trained model versions. The second operational phase executes automated batch inference jobs every 30 minutes, drastically cutting infrastructure spend compared to maintaining dedicated 24/7 real-time endpoints.

02EventBridge schedules and S3 data paths

The core runtime architecture relies on Amazon EventBridge to initiate scheduled inference workflows at regular 30-minute intervals. Once triggered, SageMaker Pipelines retrieves incoming tabular records from Amazon Simple Storage Service (Amazon S3) for ingestion. The workflow coordinates preprocessing steps, executes SageMaker Batch Transform, and delivers final prediction files back into dedicated S3 storage directories.

To ensure complete tracking and operational resilience, machine learning engineers inspect step execution status and review logs within Amazon CloudWatch. The pipeline manages failure notifications and records a comprehensive history of historical pipeline runs. This centralized monitoring ensures operational teams detect data inconsistencies or processing stalls without intervening in running plant processes.

Deploying this architecture requires an environment equipped with Python, Boto3, and the SageMaker Python SDK 2.x release. Teams must establish an Amazon EventBridge service role permitted to trigger pipelines, alongside an execution role with S3 and SageMaker permissions. The reference implementation demonstrates this orchestration using a sample loan classification model.

03SageMaker Processing and flow file exports

Data preparation in SageMaker Canvas uses Data Wrangler to generate a .flow file detailing transformation rules and source queries. To productionize these steps, engineers export data to Amazon S3 via SageMaker Processing, disabling the Auto job configuration option. This manual setting exposes the container image URI, S3 flow file path, role permissions, and designated output nodes.

Engineers must verify specific metadata fields, such as internal_metadata.dw_job.container_uri, directly within the Amazon SageMaker AI console instead of assuming default presence. Pinning the exact container image ensures consistent transformations across different environments, especially when deploying workloads across multiple AWS Regions. Output node naming must strictly match the exported identifier format of node_id dot output_name.

Production execution mandates preserving learned parameters, including numerical scaling and categorical encoding values established during training, rather than refitting them on fresh inference data. Furthermore, because the initial preparation step routes data to a static S3 path referenced inside the flow file, teams must prevent overlapping pipeline runs or inject execution-specific paths into copied flow specifications.

04Target column placeholders and feature alignment

A fundamental mismatch between training datasets and production inference datasets centers on the target prediction column. Because inference tables lack the target outcome, flows containing transformations that anticipate this column fail during automated execution. GS Caltex resolved this structural conflict by implementing a sequenced five-step data handling chain across the pipeline.

The execution sequence follows PrepareInput, PreprocessNewData, DropTargetColumn, BatchPredict, and AssembleResults in strict order. The initial PrepareInput step checks incoming inference files and injects a synthetic placeholder target column matching the training data type and index position. After Data Wrangler transformations run inside PreprocessNewData, DropTargetColumn discards the synthetic column before sending features to BatchPredict.

This placeholder technique functions effectively only when transformations do not derive features from, filter by, or calculate target encodings from the target column. If flows incorporate target-dependent logic, engineers must isolate and refactor the inference transformations independently. Teams validate the final output to ensure column counts, data types, and feature orders mirror training specifications exactly.

05Model Registry approvals and deployment cycles

Before any trained model serves production workloads, it must pass through Amazon SageMaker Model Registry for formal governance. Process engineers register completed Canvas models alongside their evaluation metrics, parameter configurations, and artifact locations. Machine learning engineers review these artifacts and validation statistics to confirm performance benchmarks before granting official production approval.

Once approved within Model Registry, the model artifact binds to the scheduled SageMaker Pipelines workflow for automated execution. The deployment mechanism isolates intermediate and finalized operational outputs under unique execution directories formatted as runs with an execution identifier. This path partitioning safeguards against data overwrites across successive 30-minute transform runs.

By standardizing asset handoffs and formalizing approval checkpoints, GS Caltex established an enterprise-grade bridge between visual citizen modeling and industrial deployment. The operational model allows non-programming chemical and plant engineers to build high-value predictive tools while maintaining rigorous cloud infrastructure governance. Model deployment overhead dropped from an entire business week to a few hours.