Agility Relies Upon Trust

As a data re-monetization company focusing on real time, opt-in internet data, LRG draws information from a vast network of sources. Before they can offer such data as leads to their clients, they engage in a multi-step process:

• compile the data into one location
• avoid purchasing duplicate data
• validate and clean purchased data for regulatory compliance
• ensure they do not sell duplicate data to clients.

While data-pulling systems exist for static databases, no such system existed for LRG’s unique focus on real time, opt-in data and the complex network of sources they rely upon. Before partnering with Smilefish and AWS, completing the above process required a labor-intensive regimen of compiling data to spreadsheets and validating, cleaning, and hunting down duplicate data by multiple data processors. This regimen resulted in lengthy turnaround times, risk of human error with compliance and duplicate data, and increased overhead for LRG. Ultimately, because data trust required a herculean effort, data agility suffered, and this negatively impacted LRG’s bottom line

Leveraging AWS to Implement a Single Source of Truth Data Fabric Solution

To overcome these challenges, LRG partnered with Smilefish and AWS to establish a single source of truth data catalog that would manage compliance and governance, visualization, ETL and customizable API integrations, as well reporting and self-service analytics. Smilefish built LRG’s custom data fabric solution to handle large amounts of data in preparation for AI and machine learning and leveraged AWS cloud technologies to provide a centralized, scalable, and secure data integration platform. To accomplish this required multiple phases and close integration with AWS services:

Phase 1: Data Migration

In Phase 1 Smilefish migrated all LRG data to AWS without disruption to their business by:

• Integrating Smilefish data fabric governance with AWS IAM Security
• Customizing ReactJS admin applications and deploying in S3 with CloudFront
• Managing backend services with Lambda and EC2
• Cataloging all onsite data inventory in RDS
• Extracting Salesforce data to CSV and cataloging in RDS
• Uploading the raw leads datafiles to S3 buckets and analyzing inventory with AWS Athena to ensure data consistency, compliance, and quality
• Uploading the raw leads datafiles to S3 buckets and analyzing inventory with AWS Athena to ensure data consistency, compliance, and quality
• Combining Smilefish data analytics and reporting with Tableau for self-service analytics.

Phase 2: Customize the Data Fabric Solution

Phase 2 involved customizing the Smilefish data fabric to meet LRG’s data marketplace requirements by:

• Integrating the data search, data acquisition, data integration, and data and fulfillment process with Salesforce, Stripe, and PayPal
• Automating the data lifecycle by integrating with Salesforce, Pipe Drive, Stripe, and PayPal
• Integrating the ETL process with the API gateway and Lambda, as well as SNA for email and text
• Architecting the ETL process to utilize a microservices approach via Lambda and SQS
• Implementing notifications via SNS and SES
• Monitoring via Gatekeeper and CloudWatch.

Phase 3: Analysis and Deployment

Phase 3 shifted the focus to business analytics, data aging, data utilization, and monitoring sales targets by integrating the consumer data from on-premises to cloud with AWS Glue and Athena.

A Data Revolution

After implementing the Smilefish data fabric solution with AWS technologies, LRG experienced significant improvements in data trust and agility, as well as business outcomes, including:

• Reduced order processing time from 3 days to 3 minutes, resulting in faster delivery of data files to customers and higher customer satisfaction
• Reduced fraud rate by 80%
• Instant delivery of unsold products from LRG's warehouse, reducing inventory costs and increasing revenue
• Implementation of an intuitive client management solution that improved customer retention and increased revenue by ensuring customers only buy unique leads
• Increased data processing efficiency, allowing one data processor to handle three times the orders
• Developed a custom invoicing solution that saves five hours per week of manually filing documents and invoices, thereby enabling LRG to focus on other areas of their business.

Conclusion:

By managing their data culture with the help of AWS technologies and Smilefish’s single source of truth data fabric solution, LRG achieved significant improvements in data trust and agility, as well as business outcomes. These include faster order processing, reduced inventory costs, increased revenue, improved customer retention, and increased efficiency. The solution provided LRG with a centralized, scalable, and secure data integration and analytics platform that enabled them to monetize their data with confidence.

About Smilefish:

Smilefish is a machine learning and data science company located in Newport Beach, California. They offer software and data management solutions across a broad range of industries. To learn more, visit https://www.smilefish.com/.

About Leads Research Group:

Leads Research Group is a lead generation, targeted data, and direct marketing company located in Huntington Beach, California. They offer high caliber marketing strategies and turn-key campaigns at competitive rates. To learn more, visit https://lrg.media/.