On an individual basis, each portfolio is limited by a human capacity problem – they can only monitor a limited number of stocks effectively, maxing out between 65-100 for a mature PM.

From a scale perspective, PMs may expand their portfolio coverage by leveraging support from 1 or more Research Assistants, however, training is intensive (up to 4 years for high-performing talent), and the role is typically temporary as RAs work to develop themselves into full portfolio managers, yielding high churn.

So then, can an AI-enabled digital twin replicate the investment strategy and stock modeling at or above a Research Assistant to better expand or scale the capacity of a Portfolio Manager that can support monitoring 200 or more stocks continuously?

Ippon divided this challenge into several ways:

1. Business Validation - How can we validate the business idea quickly?
2. AI Validation - How can we validate that an AI-generated solution meets or exceeds existing alternatives? And authentically without hallucinations
3. Feasibility Validation - Can we develop the POC in a way that validates extensibility and scale?

Ippon selected a new product innovation team of four, leveraging a skilled Product Manager with vertical experience, plus an AI-trained development team including an Enterprise Architect, ML Specialist, and Data Architect.

To approach the business challenge, Ippon hypothesized a custom Proof-of-Concept solution. The build would integrate with an advanced LLM and be tested against a single portfolio manager on a set of 10 stocks. The engagement capped at 6 weeks to show accurate surfacing insights comparable to a Research Assistant.

Simultaneously, the client tested the broader Build vs Buy decision using a bake-off comparison of Ippon against other custom builds and fine-tuned small model solutions. Ippon’s resulting build was a top-2 performer in accuracy, on equivalent footing with a fit-for-purpose (COTS) model designed for market insights, with an edge on retaining IP.

Regarding the AI efficacy challenge, Ippon started with the end Portfolio Manager in mind, developing a scorecard from the portfolio manager-provided summaries based on earnings calls with original transcripts on which those calls were based. A total of 132 summaries covering 66 companies across a variety of financial quarters for the 2023 / 2024 earnings season were provided. The scorecard was developed and tested on a new subset of 10 additional stocks.

The technical solution utilized a bespoke RAG chain architecture for data ingestion and processing, with single-shot learning using Hugging Face libraries. Ippon upped the ante by testing multiple language models (e.g., Cohere, Mistral-Large, GPT-4.0) for accuracy and performance. The technical achievements in short order, ultimately delivered, included:

• Flexible open architecture built on open-sourced model tools, deployed on Azure, that is also highly scalable to handle hundreds of transcripts simultaneously
• Immediate ability to scale and test additional portfolio managers
• Evidence-based recommendations for best LLM, learning strategy, and future state architecture
• A framework for evaluating and grading generative AI accuracy
• Accompanying cost assessment for planned architecture at scale to evaluate investment vs value

With any new product innovation, the first and most critical question to answer is whether the business idea has merit or warrants investment right now. With respect to AI, the technology is so new and evolving rapidly that the product-led approach Ippon provided in evaluating merit and value and demonstrating architecture, technologies, and scoring was tremendously beneficial in enabling the client to make decisions about moving forward.

Our client was a pioneering hedge fund and portfolio manager (PM) incubator, providing aspiring portfolio managers with support, training, and seed money to build a portfolio within their areas of expertise and specific style. Each PM monitors different segments and uses a unique investment approach to achieve financial returns.