Helping Boost Resiliency, Reliability for Acceleration Partners’ Data Platform

When dealing with big data and multiple affiliates, streamlining processes and systems is key to success. For affiliate marketing services company Acceleration Partners, this was vitally important as they manage affiliate marketing campaigns across a variety of platforms for a wide array of brands.

The company's account managers work directly with publishers —online sites and stores of all sizes—to manage and measure the results of these campaigns. Acceleration Partners’ leadership had a vision to build a centralized platform, APVision, that would differentiate them in the affiliate marketing space. They enlisted the help of SPR to enable reporting for the platform, build big data ingestion pipelines, plug into multiple affiliate networks, and collect data daily that could be used for internal and customer-facing reports.

As a tech-enabled organization, Acceleration Partners needed their technology to function seamlessly, and they needed a team of experts to support their initiatives as they transitioned to building an internal IT team, including appointing a CTO. 

With a strong understanding of the goal of the project, SPR reviewed the backend of the software and identified a few challenges. 

In order for account managers to track impressions and ad data to evaluate the effectiveness of affiliate campaigns, the platform needed to pull information from multiple sources. The different affiliate marketing data providers all provided data through different methods and in different formats. Some providers also lacked documentation or had issues with their own APIs. It was difficult to track down issues in the code when integrations and the ETL process would fail.

Since the platform received data from a high volume of sources, data standardization and platform resiliency were essential. The team needed to be sensitive to loads and throttling in the ETL process.

The holiday shopping season is a critical point in the year for AP, and they needed all integrations to be in place and functioning well to take advantage of this revenue-generating time of year.

To address these challenges, SPR focused on building a stronger data foundation, an ETL pipeline to connect to multiple networks and standardize the data. SPR studied the code and architecture to intuit where problems were occurring then replicate the issues and debug as necessary.

SPR helped the client upgrade to newer vendor APIs and dedicated time to ensure individual integrations were running optimally.  We added new integrations to their ETL pipeline and were able to clean up and lighten the load issues. We also ensured that data was processing both more quickly and more cleanly. Through this process, the team created a resilient and reliable data pipeline and increased data quality by standardizing the data.

The cloud-native application allows Acceleration Partners’ publisher development team, which recruits new affiliates, to manage those affiliates in a centralized database and then report on their performance from the main data warehouse, replacing a spreadsheet the team had been using previously.

Through SPR’s deep data expertise and understanding of integrations and data pipelines, we helped build a stable foundation of data integrity that relies on best practices data processes. This foundation has since empowered Acceleration Partners to build more functionality on top of what SPR created and enables them to focus on bringing new features to their users.

Acceleration Partners selected Cyrus Clemensen as CTO of the IT department while SPR was mid-project. Clemensen supported the work SPR had completed and provided guidance to the SPR team. “Acceleration Partners found a great partner in SPR,” said Clemensen. “They had the expertise in architecture development and design we needed, were great collaborators, and stepped in to improve a system that they didn’t initially build, which can be a challenge. I’m impressed by their ability to identify issues and build us a big data engine and reporting system that is one of our most valuable assets at AP.”

Technologies used:

• SQS
• Lambdas
• S3
• Python Glue
• Redshift warehouse
• RDS
• Amplify
• ECS