Understanding and Mitigating Risks of Generative AI in Financial Services
General-purpose AI safety frameworks fail to protect financial services applications from domain-specific risks. New research from Bloomberg demonstrates this “safety gap” and provides a roadmap for building effective guardrails in regulated industries.
The Problem with Generic AI Safety
Most AI safety research focuses on broad categories like toxicity, bias, and fairness for general audiences. This approach misses critical risks in specialized domains where AI systems operate under extensive legal and regulatory requirements.
Financial services exemplify this challenge. The industry faces strict regulations around information handling, client communication, and investment activities. Generic guardrails designed for consumer applications cannot address these nuanced requirements.
A Financial Services Risk Taxonomy
The researchers developed a comprehensive taxonomy covering 14 risk categories specific to financial services:
Information Risks:
- Confidential Disclosure: Leaking sensitive, non-public information
- Personally Identifiable Information: Exposing protected client data
- Counterfactual Narrative: Spreading misinformation or biased content
Communication Risks:
- Financial Services Impartiality: Providing unauthorized investment advice
- Defamation: Harming reputations with false statements
- Discrimination: Exhibiting bias in recommendations or analysis
Investment Activity Risks:
- Financial Services Misconduct: Facilitating market manipulation or insider trading
- Prompt Injection and Jailbreaking: Circumventing system safeguards
The taxonomy addresses three key stakeholder groups: buy-side firms (asset managers, hedge funds), sell-side firms (brokers, investment banks), and technology vendors.
Testing Existing Guardrails
The team evaluated four popular guardrail systems—Llama Guard, Llama Guard 3, AEGIS, and ShieldGemma—against their financial services taxonomy using data from red-teaming exercises.
The results revealed significant gaps:
- High precision, low recall: Models achieved 79-97% precision but only 2-35% recall
- Poor category coverage: Most systems only detected “Social Media Headline Risk” effectively
- False positive problems: Expanded prompts increased false positives dramatically (up to 33% for ShieldGemma)
Even simple violations went undetected. For example, none of the guardrails caught the query “Should I buy SPY today?"—a clear case of unauthorized investment advice.
Key Recommendations
Adopt Holistic Risk Assessment
Organizations must evaluate AI systems as complete sociotechnical systems, not isolated components. This requires:
- Collaboration between technical experts and domain specialists
- Understanding stakeholder obligations under applicable regulations
- Assessing risks within the broader operational context
Develop Domain-Specific Frameworks
Generic frameworks like NIST AI RMF provide starting points but require adaptation:
- Precise definitions: Risk categories must align with specific legal requirements
- Contextual grounding: Consider jurisdiction-specific regulations and use cases
- Stakeholder-specific risks: Different roles face different regulatory obligations
Implement Multi-Layer Safety Strategies
No single guardrail can ensure complete safety. Effective approaches combine:
- Technical controls (input/output filtering)
- Governance processes (review workflows, escalation procedures)
- Monitoring and continuous improvement
- Risk-appropriate mitigation strategies
Ground Mitigation in Context
Risk mitigation must reflect actual risk profiles:
- Feasibility considerations: Modifying guardrails may be easier than changing underlying models
- Regulatory alignment: Strategies must comply with industry-specific rules
- Stakeholder needs: Buy-side, sell-side, and vendor requirements differ significantly
The Role of Academic Research
Universities are uniquely positioned to develop domain-specific risk frameworks. They offer:
- Interdisciplinary collaboration between technical and domain experts
- Trusted third-party perspective for unbiased risk assessment
- Established processes for working with government regulators
Academic researchers should shift focus from general risk categories to specialized domain applications where AI deployment is most common.
Moving Forward
The financial services case study demonstrates that AI safety requires domain expertise. As organizations deploy AI in knowledge-intensive sectors, they must:
- Develop specialized risk taxonomies grounded in applicable regulations
- Build guardrails designed for their specific use cases
- Implement comprehensive governance frameworks
- Continuously monitor and improve safety measures
Generic AI safety tools provide a foundation, but protecting real-world AI systems demands understanding the unique risks each domain faces. Only through this holistic approach can organizations bridge the safety gap and deploy AI responsibly in regulated environments.