Gimel: PayPal’s Analytics Data Processing Platform

By

It is no secret that the volume, velocity and variety of data is exploding. As a result, the case to move workloads to the Hadoop big data ecosystem is becoming easier by the day. With Hadoop and the big data world come the additional complexities of managing the semantics of reading from and writing to various data stores. SQL has not been the lingua franca of the analytics world so far. In this world, you need to deal with a lot of choices – compute engines like Map Reduce, Spark, Flink, Hive, etc. and data stores like HDFS/Hive, Kafka, Elasticsearch,… Read more

Learnings from Using a Reactive Platform – Akka/Squbs

By

Introduction Simple and reliable APIs are important for winning customer trust in all industries. This is especially true when the API is a customer facing endpoint that accepts payment instructions. Add to that the ability to submit 15,000 payments in a single request and reliability becomes critical. Let’s talk a bit about Payouts – a PayPal product through which users can submit mass payment requests. It’s an asynchronous interaction – a request with multiple payment instructions returns a token. The caller can then use the token to poll for the payment outcomes or listen for webhooks. As you can imagine,… Read more

Creating a PayPal / Credit Card Payment within an Android Application

By

In this tutorial we’re going to learn how to set up the PayPal Android SDK to process a simple payment via either a PayPal payment or a credit card purchase. At the end of this example, you should have a simple button in an application that, when clicked, will forward the user to PayPal to confirm a set payment, then return the user back to the application and log the confirmation of payment. The complete application code for this example is available in the PayPal Developer Github Repository. Let’s get started. The first step is to obtain and add the… Read more

API Style Guide and Patterns: The Evolutionary Journey Continues

By

APIs as Products Around four years ago, we began the journey at PayPal to transform all of PayPal’s core capabilities into a platform of discoverable, well-encapsulated, reusable API-driven products. We have used a customer-oriented approach to go from a very monolithic and siloed architecture to a much more loosely coupled set of over 150 services with well designed, modern APIs. We framed this transformational exercise as an organizational change initiative, with the goal of bringing a fundamental shift in how we design and build APIs. We made it a priority to identify and serve all key “customer” constituencies – developers… Read more

Autoscaling applications @ PayPal

By

PayPal’s infrastructure has grown tremendously over the past few years hosting multitudes of applications serving billions of transactions every year. Our strong belief in decoupling the infrastructure and the app development layer enables us to independently evolve quicker. With more than 2500 internal applications and more than 100k instances across our entire infrastructure, we are committed in ensuring high availability of our applications and optimal resource utilization. In our previous post, we discussed in detail about Nyx, an automated remediation system for our infrastructure. The underlying modular architecture of Nyx is designed to listen to a spectrum of signals from… Read more

Democratizing Experimentation Data for Product Innovations

By

This blog is organized as follows: Introduction to A/B Testing Next Generation PayPal Experimentation Platform Next Generation PayPal Experimentation Analytics Platform Hypothesis Testing Null Hypothesis Z-Test Z-Score Two-Sample Z-Test P-Value P-Value of a Two-Tailed Z-Test Druid Druid Post Aggregators zscore2sample pvalue2tailedztest Implementation of Druid Post Aggregator for Two-Sample Z-test Druid Query Evaluation of Post Aggregators A Druid Query Example using Zscore and Pvalue Post Aggregators Next Step in furthering Druid Performance of Post Aggregators Concluding Remarks Acknowledgements Introduction to A/B Testing A/B Testing (also known as experimentation or bucket testing), enables the product teams to gain more insights and understandings of PayPal users,… Read more

Releasing squbs 0.9.0

By

squbs (pronounced “sqewbs” and rhymes with “cubes”) has already made past headlines (details) for its great performance and scalability. squbs 0.9 is the biggest update to squbs yet. It paves the road towards squbs 1.0. The updates in squbs 0.9 covers the following areas: Full migration from Spray to Akka HTTP Support for HTTP end-to-end streaming and back-pressure with the new low-level FlowDefinition API for service definitions Ultimate resiliency Lowest-possible latency Brand-new streaming HTTP client With integrated client configuration and circuit breaker Java API as first class citizens (besides the solid, powerful Scala API) enabling end-to-end Java use cases Rich set… Read more

Beam Me Up – Profiling a Beam-over-Spark Application

By

As we move forward with adopting Apache Beam for some of our streaming needs, our Beam applications need to be tested for stability. Such tests are aimed at ensuring performance does not degrade over time, and applications are capable of maintaining desired performance characteristics (e.g., latency) as they run over long periods of time. When we ran a Beam-over-Spark application (Beam 0.7.0-SNAPSHOT ; Spark 1.6.2) for a period of several hours, the batch processing time was increasing unexpectedly (e.g., regardless of traffic seasonality). In this post we share the steps and methods we used to diagnose the performance degradation we witnessed in our application’s (batch) processing time, a diagnosis which ultimately led… Read more

PayPal bttn for Commerce

By

In my first rotation of PayPal’s Technology Leadership Program (TLP), I was fortunate enough to work on our Western Europe region out of our Paris office. The team there wanted to tap into the Internet of Things (IoT) market and with PayPal’s strategic movement from just a button on a website to existing across all contexts – including the offline world — it was clear that a physical button that integrates with our Braintree APIs was something worth investigating. After some investigation, we found bt.tn, a start-up that has an innovative approach to buttons and is based out of Helsinki, Finland.… Read more