Sanctions Screening Effectiveness – Benchmarking

A significant offering of our AML Assurance business is the sanctions screening effectiveness testing that we provide, and a key aspect of that is the benchmark testing that we include.

Benchmarking is where, in our screening test pack we include a standard set of test cases covering numerous different test scenarios which enable us to assess the effectiveness of the screening system, and crucially to compare your own results against results collated from other similar organisations over many years.

In recent months we have been actively undertaking benchmarking for different organisations both in the UK and globally, which typify the different areas where benchmarking proves valuable.

1. Financial Institutions – Production Screening Systems – Regular Assurance

Executing a comprehensive test file against a system configuration that is used for daily sanctions screening provides an assessment of the effectiveness of screening, but absolute test results tell us very little. If a set of test scenarios score a test result of say 70% what does that mean, is that a good or poor test score? Benchmarking compares this score with results for similar organisations and thereby we can determine how well the result compares with industry standard test results, providing assurance over how effective the screening system is.

2. Financial Institutions – Screening Systems Upgrades and Replacements – Tuning/Comparison

Benchmarking is a very useful testing tool when undertaking system upgrades, perhaps implementing new software versions, or new system replacements. The benchmarking approach can be applied to compare the results from different system configuration settings enabling the configuration to be tuned to achieve the best results, or for comparing the results from two or more different systems. Not only does this technique enable the best/most appropriate configuration or system solution to be determined, it also allows the process to be quantified and to provide a rationale for any subsequent decision making of which system or system configuration settings are recommended to be implemented.

3. Screening System Vendor – Software Development

Here we work directly with screening system vendors, where the benchmarking we provide enables the vendor to assess how well their screening system performs against benchmark comparisons, and, similar to the configuration tuning described previously, when the vendor is undertaking software development work we can use the benchmarking facility to assess the effect on screening performance of any software changes.

This article has been a brief introduction to SQA AML Assurance’s sanctions screening effectiveness work, focused on benchmarking and highlighting where our benchmarking technique is particularly useful. Subsequent articles will delve into this topic in more detail, but in the meanwhile, if you would like to know more about our work, then please contact us.