Solution
The DataArt team delivered a cloud-native, AI-enabled royalties processing and management pipeline built for high-volume ingestion, automated matching, and continuous growth. The pipeline also supports traceability, reprocessing, and human-in-the-loop exception handling to meet audit and operational requirements.
Key capabilities include:
Ingestion, Processing, and Recipient Management
- Automated ingestion and normalization of royalty statements from Digital Service Providers (DSPs) and music collection societies.
- Rules-driven allocation of royalties to songwriters and other recipients by royalty type, territory, and contract terms.
- Centralized management of registrations and agreements for publishers, societies, artists, and works to maintain a unified network of eligible payment recipients.
AI-Driven Matching and Enrichment
- The system enriches incoming records with payment and context attributes and persists the results into core processing systems.
- Machine learning models automate matching of sound recordings to musical works, reducing manual review effort.
Automation, Scalability, and Cloud Engineering
- Event-driven services enable horizontal scaling to handle high-throughput royalty processing and reprocessing.
- AWS Lambda and Step Functions support orchestration, while Kubernetes and Kafka support scalable processing and streaming.
- Amazon SageMaker supports ML model development and deployment for matching and enrichment use cases.
- A purpose-built user interface allows analysts to review exceptions and confirm edge cases.
Outcomes
- AI-Powered Matching at Scale: Machine learning automates 80%+ of eligible recording-to-work matching cases, reducing manual effort and accelerating downstream royalty allocation.
- Enhanced Processing Efficiency: 90%+ of royalty processing now runs in the cloud, and 6B+ royalty lines have been processed through the new cloud-based pipeline. Kubernetes and Kafka-based processing and streaming improved throughput and reliability.
- Scalability and Business Agility: The modernized platform supports onboarding new catalogs and clients without planned downtime and scales as data volumes and customer demand grow.
- Reduced Operational Bottlenecks: Automation and cloud workflows minimize manual intervention, freeing internal teams to focus on exceptions and higher-value work.
- Human Oversight: Analyst review tools enable confirmation of edge cases and provide feedback loops to improve rules and models over time.
Technology
AWS
(Lambda, Step Functions, SageMaker, EMR)
Kubernetes
Kafka
Oracle Database
Python
