Pinterest and Intel Take Open-Source Profiling to Internet Scale

A few sentences added claims the source does not make, and the last section plus the deck are outside the length limits.Pinterest has partnered with Intel to extend gProfiler, an open-source continuous profiler that provides production performance visibility across an internet-scale fleet of infrastructure, according to InfoQ's report by Craig Risi on November 10, 2026. The work adds dynamic profiling, hardware-level telemetry, wall-time profiling, and performance-counter flame graphs, and it hardens the profiler for large production fleets. Pinterest contributed those capabilities back to the open-source project rather than building another proprietary profiling platform.

This piece is for platform and service engineers who run Java, Python, C++, Go, and Node.js in production and already use separate profilers per language. It explains how gProfiler orchestrates those tools on a host, how dormant agents take remote instructions, and what Intel PerfSpect and wall-time profiling add beside CPU samples. It also follows profiles from the host agent through S3 and SQS into ClickHouse for gProfiler Performance Studio, and sets that design beside Parca, Grafana Pyroscope, Datadog, and Intel VTune.

01Hundreds of thousands of instances

Pinterest says it operates hundreds of thousands of instances supporting 640 million monthly active users, with services written in Java, Python, C++, Go, and Node.js. At that scale, even a small efficiency improvement can have significant infrastructure and cost implications. The challenge was that engineers were using separate tools such as perf, async-profiler, and py-spy, producing fragmented profiles that were difficult to correlate across languages and machines.

Rather than building another proprietary profiling platform, Pinterest worked with Intel to extend gProfiler and contribute those capabilities back to the open-source project. The collaboration adds dynamic profiling, hardware-level telemetry, wall-time profiling, and performance-counter flame graphs, while also hardening the system for large production fleets.

02Unified flame graphs on one host

Application execution on these services crosses multiple layers. A Java application may spend time inside the JVM, native libraries, or the Linux kernel, while a Python service can involve interpreter, native, and kernel activity. Instead of requiring engineers to select and manually correlate separate profilers, gProfiler combines those views.

The agent discovers processes on each host, starts the appropriate profilers, and merges their stack traces into the unified flame graph. It uploads the resulting profile from the host, and the Java, Python, C++, Go, and Node.js services need no application code changes.

03Remote triggers through heartbeats

The new capability allows engineers to select services, hosts, or clusters and trigger profiling remotely through the UI or API. Agents normally remain dormant and receive profiling instructions through periodic heartbeats. Profiling frequency, duration, and the profilers used can change without redeploying the agent.

Dormant agents normally consume less than 1% CPU and 100 MB of memory while they wait for the next heartbeat. That arrangement turns profiling into an operational capability rather than something that requires engineers to gain shell access to individual machines.

04PerfSpect counters and wall time

Integration with Intel PerfSpect allows gProfiler to collect hardware-level information such as cache behavior, memory bandwidth, and CPU stalls. Pinterest also added PMC flame graphs, which correlate functions with hardware performance events such as cache misses and branch mispredictions. Those PMC graphs are the performance-counter flame graphs from the collaboration.

Another addition is wall-time profiling. CPU profiling shows where processor time is being spent, but a service can still be slow while consuming relatively little CPU because threads are waiting on I/O, locks, or remote procedure calls. Wall-time profiling makes those waiting periods visible.

05Studio, ClickHouse, and VTune

The gProfiler architecture separates collection from analysis. Agents run on individual hosts, while gProfiler Performance Studio aggregates and visualizes the resulting profiles. Profiles are stored in S3, passed through SQS for asynchronous indexing, and stored in ClickHouse for analytical queries, while a React frontend provides flame-graph exploration and aggregated data supports comparisons across hosts, services, and time periods.

Pinterest and Intel hardened the platform for production before deploying it across the fleet, addressing agent resource consumption, disk usage, and multi-terabyte ingestion. Parca and Grafana Pyroscope use eBPF and related sampling technologies to provide fleet-wide profiling, while commercial platforms such as Datadog offer continuous profiling alongside broader observability capabilities. gProfiler instead emphasizes system-wide, multi-runtime profiling and open-source extensibility, orchestrating perf and async-profiler rather than replacing them, and adding hardware-level profiling through Intel's tooling.

Intel's VTune provides much deeper processor and application performance analysis, but it was designed primarily as a performance-engineering tool rather than a cloud-native fleet profiler. Pinterest specifically notes that this difference influenced the decision to build on gProfiler rather than adapt VTune for its environment. At Pinterest's scale the cost of waiting to reproduce problems or manually profile machines is too high, so continuous profiling provides a baseline, dynamic profiling investigates anomalies, and runtime, system, and hardware data become another observability layer, unlike small-scale profiling enabled when a problem appears.