NordHero helped Firstbeat Technologies multiply the speed of big-data analytics and dramatically reduce costs by leveraging serverless cloud capabilities on AWS.

The Hero

Firstbeat is the leading provider of physiological analytics and services for well-being and sports. Firstbeat’s innovative technology transforms heartbeat data into personalized information on stress, recovery and exercise. Over two decades, Firstbeat has analyzed over 100 billion heartbeats and helped over 10,000 organizations and thousands of sports teams worldwide to reach their health and performance potential with science-backed confidence. Today, Firstbeat’s services are offered in more than 70 countries.

The Challenge

Firstbeat provides accurate data and insights for professionals and individuals for better health and performance. Over the years, Firstbeat has collected very extensive databases in employee well-being and professional sports. The cornerstone of Firstbeat has always been science-based facts and very high reliability, including the consistency of the analytics results. When developing and updating analytic algorithms, it is crucial to run analytics tests with large datasets and multiple versions of the same algorithm to see how the new algorithm performs compared to the previous ones.

Firstbeat’s production platforms already utilize cloud services quite extensively. Still, the capacity of workloads running in an on-premise environment limited the data analytics team’s speed of innovation.

“Running a new version of an analytics algorithm with a large dataset could take up to 4-5 days with on-premise servers, not even to mention running multiple versions of the same algorithm. That didn’t fulfill our target for the analytics development process and the speed of innovation.”, says Tero Myllymäki, Head of Physiology Research at Firstbeat Technologies.

The Work

The idea of high serverless parallelism

NordHero was invited to the Firstbeat analytics team to plan the cloud migration of their analytics workloads, as NordHero’s cloud architects had worked very closely with Firstbeat’s development teams building their business platforms on the AWS cloud.

Instead of a traditional lift-and-shift approach, the joint team came up pretty quickly with an idea of a new big-data analytics process that would rely on a small serverless footprint and massive scalability when needed.

The team planned a new processing pipeline for large-scale analytics using batch processing to distribute and ramp up the analytics services and provide the data sets for each processor. Each processing task would run in a container, and the pipeline would finally combine the results.

“At that time, no one knew exactly whether it would work, how fast it would be with our massive data set, and what the running costs would be. Still, the idea was so compelling that we decided to give it a try.”, Tero Myllymäki continues.

Building the PoC for new analytics architecture

The joint team between Firstbeat and NordHero was assigned to build the proof-of-concept architecture on AWS that would allow massive parallelization of the analysis tasks.

The team came up with a PoC stack that utilized:

• AWS Glue to pre-process, transform, and post-process the data in a format that can be easily analyzed and queried
• AWS Batch to manage the queue for big-data analytics with high parallelism
• AWS ECS serverless with Fargate tasks to parallel run the in-house analytical engine
• AWS Step Functions and AWS Lambda to organize aspects of the processing pipeline
• Amazon Athena to run analytical queries on both the raw and different versions of the analyzed data
• AWS QuickSight to visualize the data
• Amazon S3 to store both incoming and analyzed data

With the PoC solution, Firstbeat could run the analytics in the AWS cloud significantly faster than on their on-premise environment and only pay for the resources when actively used.

Leveraging the architecture for all Firstbeat business lines

Convinced by the PoC results, Firstbeat decided to move forward with the plan and scale the platform with production data sets and all Firstbeat business lines. Firstbeat’s analytics team adopted the new architecture fast and started building production data pipelines and analytics algorithms on top of the architecture.

The Outcome

Firstbeat was positively surprised by the productivity leap enabled by the new platform.

“While connecting the new analytics capabilities to the production data sets, we started to see the exponential difference it would mean for our work processes. With the new approach, we could run large-scale analytics jobs in the cloud in less than one hour, and running the job on hundreds of tasks in parallel would cost only a few euros per run. That’s a different game compared to the past with 4-5 days per job and costing hundreds of euros per month even if we didn’t run any jobs.”, summarizes Tero Myllymäki, smiling.

The power of cloud-based analytics has made it possible to speed up the analytics development process significantly. Instead of running only one job per week, Firstbeat can now run multiple jobs per day - even run the same job with numerous different parameters during the same day, and very quickly visualize the results and demonstrate the impacts on a large scale.

“This change has a remarkable impact on our speed of innovation. And what’s more important, it has had a positive impact on work satisfaction within the analytics team. The team feels the work is more rewarding as there’s a much faster feedback loop than previously, and because you can now produce more value with less time.”, Myllymäki concludes.