Scaling AI Application Development at BlackRock: Building Custom Knowledge Apps for Investment Operations
BlackRock’s engineering team compressed AI application development from 8 months to days by building a framework that puts prompt engineering and model strategy selection directly into domain experts’ hands.
The Challenge: Complex AI Apps for Investment Operations
BlackRock’s investment operations teams process massive amounts of financial data daily. These teams—the backbone of investment activities—need sophisticated internal tools for document extraction, workflow automation, Q&A systems, and agentic processes.
Building these AI applications traditionally took 3-8 months per use case. The new issue operations team exemplifies this challenge: when companies go public or stocks split, the team must extract data from prospectuses and term sheets, then set up securities in internal systems before portfolio managers can trade them.
Three Core Bottlenecks in AI Development
Prompt Engineering Complexity
Financial documents require extensive domain knowledge. Simple prompts quickly expand to three-paragraph descriptions of complex instruments. Teams struggled with prompt versioning, comparison, and performance evaluation across different document types.
LLM Strategy Selection
Different financial instruments demand different approaches:
- Simple corporate bonds work with in-context learning
- Complex documents spanning thousands of pages exceed token limits
- High-volume overnight analysis requires GPU clusters
- Real-time processing needs burstable infrastructure
Teams spent months testing combinations of RAG approaches, chain-of-thought reasoning, and various model providers.
Deployment Infrastructure
AI applications introduce new deployment challenges beyond traditional access control and distribution:
- GPU cluster allocation for intensive analysis
- Burstable clusters for periodic processing
- Cost control across different model providers
- Integration with downstream compliance systems
The Solution: Sandbox and App Factory Framework
BlackRock built two integrated components to accelerate development:
Sandbox: Domain Expert Playground
The sandbox gives investment operations experts direct control over AI configuration without engineering bottlenecks. Users can:
- Create extraction templates with field dependencies and validation rules
- Configure multiple quality control checks
- Define inter-field relationships (callable bonds require call dates and prices)
- Test different LLM strategies against document sets
- Compare extraction results across model configurations
App Factory: Automated Deployment
The app factory operates as a cloud-native operator that transforms sandbox configurations into production applications. It handles:
- Infrastructure provisioning based on use case requirements
- Integration with downstream compliance systems
- User access controls and data security
- Cost optimization across model providers
Implementation Architecture
The framework separates concerns across four layers:
Data Platform: Ingests and processes financial documents with tagging, labeling, and embedding Orchestration Layer: Transforms data through configurable pipelines Sandbox: Enables domain experts to build and test extraction templates App Factory: Deploys configurations as production applications
Domain experts iterate rapidly in the sandbox, then pass validated configurations to the app factory for automated deployment.
Key Technical Decisions
Human-in-the-Loop Design
BlackRock prioritized human oversight over full automation. Financial regulations require “four eyes” verification, making agentic systems unsuitable for production deployment despite their appeal.
Modular Component Strategy
The framework provides reusable components for prompt management, model strategy selection, and validation rules. Teams mix and match these components rather than building custom solutions.
Multi-Model Provider Support
Different use cases require different model capabilities. The framework abstracts provider selection, allowing teams to switch between models based on document complexity, processing volume, and cost constraints.
Results and Lessons Learned
BlackRock reduced AI application development time from months to days by removing engineering bottlenecks from domain expert workflows.
Critical Success Factors
Invest in Domain Expert Training: Financial document analysis requires deep prompt engineering skills. Generic AI training doesn’t suffice for complex financial instruments.
Educate on LLM Strategy Selection: Teams need frameworks for choosing between RAG, chain-of-thought, and hybrid approaches based on document characteristics and processing requirements.
Evaluate ROI Rigorously: AI applications can cost more than off-the-shelf solutions. Calculate infrastructure, development, and maintenance costs against business value.
Next Steps
Organizations building enterprise AI applications should prioritize frameworks that empower domain experts while maintaining engineering oversight. The sandbox-to-factory pattern works particularly well for document-heavy industries requiring regulatory compliance and human verification.