Debt Financing Fintech

In order to maintain this competitive advantage, they needed to migrate their data to Snowflake, a data warehouse, to help maintain accurate and efficient ways of reporting on the business. The previous analytical layer was hosted on Microsoft SQL Server; however, as the fintech grew and more data analysts were hired to perform analysis, the datasets and queries to produce data analytics became unwieldy. Not only did the Microsoft Analyst SQL Server begin to show latency and other shortcomings, but the data assets producing key performance metrics were being generated away from a central source, increasing silos and inconsistencies. Data was being managed by numerous people, in a variety of formats, creating distinct subject matter experts across data assets that were often producing duplicative metrics. 

This situation presented an opportunity to not only move to a more efficient platform, but to bring all siloed data assets and owners into a consistent format using one source of truth. The decision was made to migrate from the Microsoft Analyst SQL Server to Snowflake as their data warehouse for analytics. 

To complete a migration of this size and importance to the business within a reasonable timeframe, the company looked to Ippon as a partner to bring 3 additional data engineers onto the team to accelerate the project. 

The primary objective was to perform a “lift and shift” migration of 300+ critical data assets to Snowflake. The data assets not only came from the legacy Microsoft Analyst SQL Server, but from personal notebooks and sandbox environments as well. Data assets that were a part of the migration included: datasets, python notebooks, email reports, Tableau reports, and other sources. 

The team was responsible for understanding and recreating each data asset by working with different subject matter experts across 10+ business domains. The data team had an aggressive deadline to complete all major migration efforts by the end of the year, so that the Microsoft Analyst SQL Server could be fully decommissioned by the end of the following month. 

By November, the team had completed about 85% of the migration; they were on track to hit their year-end deadline for completion. As the migration efforts were coming to a close, it would become increasingly important to communicate the remaining work and impacts to the business to ensure the appropriate support was available to get it over the finish line. To help with this communication and support, the client decided to hire an Ippon Product Manager to the data engineering team. The Product Manager would act as a hybrid ScrumMaster/ Project Manager to help track the remaining data assets to be migrated, act as a liaison between the business and the data engineers to push for all final approvals, and consistently communicate the status of the overall migration project. 

Constant communication in the final weeks of the migration project helped bring transparency to the business and allowed them to understand the impacts or expectations from their roles. It also gave stakeholders the opportunity to highlight any last minute requests or considerations before the project was marked as complete. Weekly updates were sent out to all stakeholders and technology partners throughout the final month of the migration to show progress, remaining dependencies, and changes in behavior required from the business for next steps. The data team hit their target completion date for the migration to Snowflake and was able to begin some of the decommissioning activities in parallel. 

The month of January consisted of tying up any loose ends from the migration, educating stakeholders on best practices in the new data environment, and pushing those stakeholders for final validation to get them comfortable before the legacy Microsoft Analyst SQL Server was decommissioned. Weekly updates continued to be sent throughout January with a focus on the decommissioning activities that were taking place throughout the month. Giving a preview of the actions being taken in the current week, as well as what was planned for the following weeks, gave the business plenty of time to voice any concerns and be aware of any impacts to their processes. 

On January 28th, the Microsoft Analyst SQL Server was decommissioned. With the added data expertise and guidance from our Ippon consultants, along with a strong partnership with the client, the data team performed a successful data migration to Snowflake within 1.5 years. It was a seamless transition to fully operating in Snowflake and effortlessly allowed the team to begin planning their 2022 priorities for the new platform. 

The Ippon team is continuing to partner with this client in the first half of the year to help them drive their agenda forward for managing data in Snowflake. The team is focused on refactoring data models and teaching best practices for data optimization while encouraging data governance across the fintech’s business.