The client needed a modern enterprise-level solution providing timely, accurate, and predictable access to external (market data) and internal data. The goal was to enable portfolio managers to make more informed, data-driven decisions and give data scientists the ability to build and train machine learning models. The customer collaborates with numerous internal teams focused on market research and sources data from various vendors. Its existing technology—used prior to Ippon’s modernization efforts — became outdated and lacked flexibility. Moreover, there was a multitude of market data solutions creating silos within the firm. Consequently, the company required a refreshed approach to data ingestion and a modern approach to data democratization.

After evaluating its use case and performance requirements, Ippon established a new data platform from scratch in partnership with Databricks. The pipelines within the client’s AWS environment, seamlessly integrate on-premises databases and various vendor AWS accounts. Our team incorporated diverse source types and configured data transformation jobs to run in near real-time or on schedules specified by the client.

Ippon built an S3-based data lake and enabled near-real-time integrations with external vendors using Kafka. Data pipeline orchestration was managed via a custom framework deployed on Kubernetes (EKS), while data transformation workflows leverage Databricks DLT and Spark jobs to ensure efficient, scalable data processing.

By adopting Databricks, the client gained the flexibility to leverage multiple tools—such as PySpark, Spark, SQL, and even Excel—allowing teams to use the approach that best fit their needs. This modernization effort freed the client from the constraints of on-premises SQL servers and provided a more dynamic, scalable platform for data processing and analytics.

Ippon has modernized a data platform for a global asset management firm. The company operates primarily as a multi-strategy hedge fund and alternative investments industry. This organization pursues a broad range of investment strategies across different asset classes — most notably in long/short equity, macro, systematic, and quantitative approaches.