Standard Bank deploys automated AI risk checks, with humans keeping final approval
AWS and the bank report two-hour risk reports across six assessments, compared with a manual two-week review. The system gathers evidence—not regulatory certification.
Listen to this story
The audio brief
Story brief
3 key pointsStandard Bank has put an AWS-built assessment workflow into production to screen generative-AI applications before launch, using a model accessed through Amazon Bedrock to compare documentation, code, and live-test results against more than 300 policy and standards questions. It generates traceable risk scores and remediation materials, but reviewers verify or amend findings and the bank’s risk committee retains approval; the process does not certify regulatory compliance. Scheduled reassessments and code-change-triggered reruns remain planned, not active continuous monitoring.
- 01
AWS and Standard Bank report that six assessments generated risk reports in about two hours, versus a traditionally manual two-week review; this does not include final approval.
- 02
The companies report a 14% improvement in use-case risk scores after remediation.
- 03
The score combines an adjustable severity level with a likelihood measure based on the share of required evidence the model cannot find.
Standard Bank Group is using AI to prepare the evidence needed to approve other AI applications. In a November 20 case study, AWS and the bank said their automated assessment generated comprehensive risk reports in about two hours across six assessments, compared with a traditionally manual two-week review. The risk committee still makes the approval decision.
Built with AWS Professional Services, the assessment is running in the bank’s production environment to evaluate generative AI applications before they go live. It feeds existing model-risk and change-risk processes rather than replacing them. The speed comparison is a result reported by the bank and its technology supplier, not a claim that final approval takes two hours.
The process starts with roughly 40 intake questions covering risks such as incomplete technical documentation and insufficient human oversight. A pre-assessment agent scans an application’s code repository and documentation, then drafts answers with links to relevant files. Delivery teams review the answers, correct mistakes and supply evidence the agent missed.
- System documentation explaining how the application was designed, built and evaluated.
- Policy documents defining responsibilities and procedures, including incident handling.
- Source code and system instructions showing how controls are implemented.
- Live calls to the application showing how it responds in practice.
Live testing combines questions supplied by delivery teams with attempts to bypass safeguards and off-topic questions the application should decline. The pipeline also repeats and rewords prompts to check whether answers remain consistent in meaning. When teams provide expected answers, it measures how closely responses match them. Every response is saved for reviewers to inspect.
Everything runs within Standard Bank’s AWS account, where documents, code and test results are stored. AWS Step Functions coordinates the work, and a foundation model accessed through Amazon Bedrock evaluates the evidence. This is “LLM-as-a-judge”: a language model checking material against defined criteria, rather than a person reading every artifact from scratch.
AWS and Standard Bank report a 14% improvement in use-case risk scores after remediation.
The judge answers more than 300 yes-or-no questions tied to external AI standards, frameworks and the bank’s own policies. The likelihood score reflects the share of required evidence the model cannot find. Combined with a severity level that the risk team can adjust, it produces a rating from Very low to Critical.
Each missing piece of evidence links to a proposed mitigation and the relevant standard or policy clause. A fixed, inspectable formula converts answers into scores. Evaluations run multiple times to check consistency, with conflicting findings flagged in the review interface. The system stores each judgment alongside its supporting evidence so reviewers and auditors can trace the result.
Outputs include an executive presentation, a mitigation checklist, a technical report and a dashboard tracking scores over time. Before an approval decision, reviewers confirm findings and can amend answers with a business justification. The assessment does not certify regulatory compliance; Standard Bank remains responsible for those determinations.
A question factory generates additional checks from internal policy documents. The bank’s risk team has used it to translate its own rules into questions applied to every use case. When a policy changes, subsequent assessments can apply the updated checks with minimal manual rework, according to the case study.
For delivery teams, the assessment is optional during proof-of-concept work but mandatory at the build stage. It currently evaluates applications before launch. Scheduled reassessment, and reruns triggered by code or configuration changes, are planned once live workloads are connected—not described as an already operating continuous-monitoring capability.
Sources
- aws.amazon.comHow Standard Bank Group automates AI risk assessment for generative AI | Amazon Web Services
Reader comments
Newest comments first. Replies stay oldest first.