javascript —

Pinterest and Intel Take Open-Source Profiling to Internet Scale

Pinterest claims it operates hundreds of thousands of instances supporting 640 million monthly active users, with services written in Java, Python, C++, Go, and Node.js.

Pinterest and Intel Take Open-Source Profiling to Internet Scale

At that scale, even a small efficiency improvement can have major infrastructure and cost implications. The challenge was that engineers were using separate tools such as perf, async-profiler, and py-spy, producing fragmented profiles that were difficult to correlate across languages and machines.

Rather than building another proprietary profiling platform, Pinterest worked with Intel to extend gProfiler and contribute those capabilities back to the open-source project. The collaboration adds dynamic profiling, hardware-level telemetry, wall-time profiling, and performance-counter flame graphs, while also hardening the system for large production fleets. gProfiler acts as an orchestration layer over several existing profilers.

It combines system-wide perf data with runtime-specific tools such as async-profiler for Java and eBPF-based profiling for Python, producing a unified flame graph across the processes running on a host. This is important for modern services where application execution crosses multiple layers.

Earlier coverage

Reported by infoq.com.

javascriptcloud

Related coverage

Elsewhere in this edition