A control existing doesn’t show it works; it working doesn’t show it addresses the right risk. That’s why internal auditors, ITGC assessors and SOC 2 auditors test two questions separately: is the control designed correctly, and did it operate as designed throughout the period? In the field the two are often confused; people say “the procedure is written, so the control exists” and the test ends there. Below we show how we carry out design and operating effectiveness tests, when each of the four basic test methods is enough, and a sample test working paper for an access control.
Design and operating effectiveness
Design effectiveness asks whether the control, when applied as defined, can bring the related risk down to an acceptable level. The elements examined: is the control tied to the right risk, does its frequency fit the speed of the risk, does the person performing it have sufficient authority and knowledge, is the data the control uses as input complete and accurate, is it defined what happens when an exception is found?
Operating effectiveness asks whether the control was applied as designed, consistently and by authorized people throughout the audit period. A single successful application doesn’t prove operating effectiveness. It must be shown that the control worked in all the samples selected from the period.
In IT control and compliance audits the order doesn’t change: design is tested first. If the design is deficient, the operating test is wasted effort, because a badly designed control working without error doesn’t reduce risk. Example: a control that closes leavers’ accounts every three months can be applied perfectly every quarter. But if a leaver’s account can stay open for three months, the problem is in the design, not the operation.
Comparison of test methods
ISO 19011:2018 and financial audit standards define similar methods for collecting evidence. The four basic methods used in control testing are compared below.
| Method | What is done? | Level of assurance | Typical use |
|---|---|---|---|
| Inquiry | Interview with the control owner | Low; insufficient on its own | Understanding the control, planning the test |
| Observation | Watching the control being performed | Medium; covers only that moment | Physical controls, point-in-time operations |
| Review | Examining records, approvals and reports | Medium–high | Approvals, review records |
| Reperformance | The auditor runs the control again themselves | High | Reconciliations, automated calculations |
The methods are combined. A typical combination: inquiry to learn how the control works, inspection to verify evidence in the selected samples, reperformance on a critical sample. Observation is a valuable but narrow method. A control owner who knows they’re being watched may perform the control differently; also, observation says nothing about the rest of the period.
In inspection, the source of the evidence also sets the level of assurance. A report pulled directly from the system is more reliable than a manually compiled Excel table; this distinction is explained in detail in the screenshot vs system report comparison.
Design testing with a walkthrough
A walkthrough means following a single transaction from start to finish. The aim is to see with your own eyes that the control exists and is applied as described in the document. Steps:
- Read the control definition: Are the purpose, risk, owner, frequency, input, output and exception handling written down?
- Start the process with the control owner: Ask them to describe every step from the moment the control is triggered to its outcome.
- Follow a transaction: Pick a real example (for example the access request of an employee who started last month) and see the evidence of each step in the system.
- Question the reliability of the input: Where does the list or report the control uses come from, and is it complete?
- Assess whether it addresses the risk: If this control works as designed, does it prevent the related risk or detect it in time?
The fourth step is the one most often skipped. If the user list reviewed was prepared by hand, an account not on the list is never reviewed. If the input is incomplete, so is the control.
Operating testing across the period
The operating test uses sampling to show that a control whose design has been approved worked throughout the audit period. Defining the population, verifying its completeness and setting the sample size according to control frequency are explained in the sample selection in internal audit guide. After the samples are selected, predefined test attributes are checked for each one. Test attributes are derived from the control definition and applied the same way to every sample: “is there an approval”, “is the approval before the transaction”, “is the approver authorized”, and so on.
Mistakes you may come across in operating testing:
- Letting the auditee choose the samples.
- Setting test attributes after the samples have been reviewed.
- Testing a control that changed during the period as a single control. If the control changed mid-period, it must be tested as two separate periods.
- Brushing an exception aside as “one-off”. No conclusion is written until the cause of the exception has been investigated; how to word the finding is explained in the writing audit findings guide.
Sample test working paper
The working paper below was prepared for a quarterly user access review control. Values and record numbers are examples.
CONTROL : ACC-04 Quarterly access review
RISK : R-12 Unauthorized access to a critical application
DEFINITION : ERP role assignments are reviewed by the relevant
department manager within 15 working days after
each quarter end; permissions requested for removal
are removed by IT within 5 working days.
TYPE / FREQ : Manual, detective / Quarterly
PERIOD : 01.01.2025 – 31.12.2025 POPULATION: 4 reviews
DESIGN TEST (walkthrough, Q3 review)
Input list pulled from ERP via system report? Yes (report + parameters)
Reviewer is a manager who knows the roles? Yes
Removal request and its execution traceable? Yes (ticket)
Result: Design effective
OPERATING TEST (all 4 quarters)
Test attributes: A) completed on time B) all users on the list
C) authorized approval D) removals within 5 working days
Q1: A✓ B✓ C✓ D✓
Q2: A✓ B✓ C✓ D✗ (2 removals done in 9 working days)
Q3: A✓ B✓ C✓ D✓
Q4: A✓ B✓ C✓ D✓
Reperformance: Q4 list pulled from ERP again, user count matched.
RESULT : 1 exception. Cause: leave period in the IT team, no queue tracking.
Reported as finding AF-2025-07.
PREPARED BY : [name] DATE: [date] REVIEWED BY: [name]
A good working paper is detailed enough that another auditor could reach the same conclusion with the same samples. The file names or repository links of the evidence are added to the working paper. Writing “reviewed, OK” and moving on isn’t a test. Which evidence is expected in an access review control is explained separately in the access review evidence guide.
Testing differences between automated and manual controls
| Dimension | Manual control | Automated control |
|---|---|---|
| Source of error | Human; forgetting, carelessness | Configuration; a wrong or changed setting |
| Sample | Several samples depending on frequency | Usually verifying the configuration once |
| Main test method | Inspection, reperformance | Configuration review, test transaction |
| Additional controls it relies on | The control owner’s competence | Change management and privileged access |
Accepting an automated control on a one-off test depends on showing the configuration didn’t change during the period. If the Active Directory password policy is being tested, the setting is reviewed once (the output of Get-ADDefaultDomainPasswordPolicy, and Get-ADFineGrainedPasswordPolicy -Filter * for fine-grained password policies). Then the privileged accounts that could change this setting during the period, and the change records, are checked. A part that can be skipped is fine-grained policies: the default policy looks solid, while a looser PSO tied to a privileged group never crosses anyone’s mind. If the change management control isn’t effective, broader testing is needed for the automated control; the configuration is re-verified at several points in the period.
Semi-automated controls, meaning controls where a person reviews a report produced by the system, are tested in two parts: that the report is complete and accurate (the automated part), and that the review was done throughout the period (the manual part). These hybrid controls are a frequent source of exceptions in practice. The report’s accuracy is assumed, not tested.
To assess the design and operating effectiveness of the control set independently, our IT security audit and control effectiveness work can be used, and to set up the access, change and operations controls themselves, our IT general controls (ITGC) work. To see access and configuration controls working in the live environment, define the scope of a technical cybersecurity audit.
Frequently asked questions
If the design test fails, is the operating test done?
Usually not. If the control isn’t designed to address the risk, it working without error throughout the period doesn’t reduce the risk either. The finding is reported as a design deficiency; after the control is redesigned, the operating test is done in the next period.
Can a control be tested by inquiry alone?
No. Inquiry is necessary to understand how the control works but doesn’t give assurance on its own; what a person describes may differ from what’s actually done. Always support inquiry with observation, inspection or reperformance.
Why is a single sample considered enough for automated controls?
An automated control works the same way every time with the same configuration. Once the configuration is verified and it can be shown that it didn’t change during the period, it can be concluded that the control worked consistently. That’s why testing automated controls relies on the effectiveness of change management and privileged access controls.
Is a walkthrough the same as a design test?
A walkthrough is the most common method of design testing, but it isn’t a design test on its own. A walkthrough follows a transaction from start to finish to show the control exists and is applied as described; a design test additionally assesses whether the control actually reduces the related risk.