Revolutionizing Catalog Management: Overcoming Performance, Reliability, and Data Inconsistency Challenges with Tech-Forward Architecture for Rappi

Company Name

Rappi

Location

LATAM

Industry

Travel Tech Solutions

99.99%

Increased Data Consistency

98% Improved Catalog Service Reliability

97%

Uptime of Rappi's Catalog Service

450MM requests per day

Client

Rappi is a leading on-demand delivery platform that operates in nine countries, providing services across sectors ranging from food, groceries, and eCommerce to pharmacy, travel, and payment.

The only native multi-vertical provider in Latin America, Rappi has established itself as the company which provides the best customer experience in each vertical, offering solutions that outperform competitors and exceed customer expectations. One of Rappi’s vertical is Consumer Packaged Goods (CPG). In order to optimize this vertical, Rappi developed Сatalog - cutting-edge app which is specifically designed to store, update and publish the latest product information, ensuring that customers always have access to the most comprehensive and up-to-date selection of products within Rappi’s UserApp.

Challenge

Two years ago, Rappi managed its product catalog using third-party software. However, with the rapid growth of Rappi's UserApp, the need for scalability and flexibility became evident. Without the capability to keep up with Rappi’s evolving business, the third-party software no longer met the company’s needs. Rappi set out to find another platform partner to help them deliver an even better app experience.

In order to overcome this limitation, Rappi developed new in-house architecture that centralized their product information source. However, as Rappi's ecosystem grew, various internal services attempted, less successfully, to integrate with the new product catalog. The new architecture needed re-engineering to keep up with business growth. This led to three main improvement challenges for the catalog:

Performance and Scalability:
The current architecture struggled to keep up with the increasing number of products and categories, thus increasing the volume of data that needed to be updated in real time. Scaling resources did not provide improvements, resulting in slower loading times and frequent crashes..

Reliability:
The architecture was regularly coupled with single points of failure, leading to frequent downtimes without recovery procedures. This made it difficult to maintain reliability, with the potential of leading to issues such as data loss, incorrect product information, and unreliable updates.

Data inconsistency:
The complex data ingest process resulted in data loss during downtimes, leading to severe data inconsistency. There was a risk of duplicate data, inconsistent information, and conflicting updates. This could lead to confusion for customers and internal teams, causing errors and inconsistencies in the product information.

Solution

Rappi partnered with DataArt to re-engineer their catalog and address the above-mentioned challenges, improving the architecture to support continued flourishing of their business.

The solution involved three main phases:

Modernizing internal data processing: To solve data ingestion problems and enhance consistency, DataArt implemented a new distributed data processing pipeline.
Bespoke software development to reduce bugs: To solve reliability and consistency issues, DataArt developed new nanoservices with fewer single points of failure, gradually decoupling and replacing the existing ones.
Innovating new features: After stabilizing the new architecture, DataArt shifted focus to queued bugs and the development of new features. Our additions addressed technical debt, improving staff productivity by reducing the number of development teams from three to one.

Business Benefits

Increased data consistency to 99.99%
Ensured 97% uptime for Rappi’s Catalog service
Improved Catalog service reliability to 98%
Enhanced system to be able to support up to 450MM requests per day
Improved availability and data consistency for business continuity
Resized the software infrastructure for maximum cost efficiency
Integration opportunities with various data consumers and sources

Redefining Rappi’s Catalog with Tech-Forward Architecture: Highlights

Layers
- The Product Manager Software: Responsible for generating and maintaining general instances of products.
- The Inventory Layer: Used information from the Product Manager to create specific instances of products.
- The Database Layer: Queued and replicated the information to the UserApp.
Actors
- API Clients: Internal Rappi clients who needed to query and update products.
- Integration Engine: Internal clients or retailers/stores that needed to bulk-edit product information.
APIs
- Prices API: On-demand price control.
- Overrides API: Application of business logics to specific products.
- Audit API: Monitor and activity log to find inconsistencies and problems.

Tech Stack

Data store:

AWS DocDB

Messaging system:

Apache Kafka

Databases:

MySQL/PostgreSQL/JSON/influxdb

AWS services:

S3 and SQS

Cache:

Redis

Programming languages:

Node and GO

Please provide your contact details, and we will get back to you promptly.

First name*

Last name*

Email*

Job title

Company*

Phone

What would you like to discuss?*

How did you learn about DataArt?

What is your role?

I agree to receive DataArt's monthly marketing newsletter.

I have read and agree to the Privacy Policy*