In a company with 400 change records, 120 joiners and 90 leavers in a year, it’s neither possible nor necessary for the internal auditor to look at all of them. Sampling is the way to reach a defensible conclusion about the whole population by reviewing a limited number of records. Set up wrongly, the audit report gives false assurance. ISO 19011:2018 addresses sampling in management system audits with two approaches, judgmental and statistical. Below we explain step by step how to define the population in ISMS and ITGC internal audits, how to select samples, how many samples to take and what to do when an exception turns up.
The purpose and limits of sampling
The purpose is to obtain reasonable assurance that a control worked as designed throughout the audit period. Accept two limits from the start:
- Reasonable assurance isn’t absolute assurance. No exceptions in the sample doesn’t prove there are no exceptions in the population. The report language is built accordingly: “No exceptions were identified in the 25 samples reviewed.”
- Sampling isn’t suitable for every control. There’s no sample for a control that runs once a year (for example the annual management review); the single event is reviewed. For an automated control (for example a password complexity policy), verifying the configuration once and testing the change controls that protect it is usually enough.
Sampling is a separate topic from how the selected records are tested. How design and operating effectiveness are tested with the selected samples is covered in the control effectiveness testing guide.
Defining the population and its completeness
A sample can’t be more accurate than the list it was drawn from. A carefully selected sample from an incomplete list produces a wrong conclusion. When defining the population, these four elements must be written down:
- Control event: What are we counting? “Approved change” or “change released to production”? They’re different populations; the second definition also catches unapproved changes. The first can’t.
- Period: The start and end date of the audit period.
- Scope: Which systems, locations or units are included?
- Source: From which system, with which report and with which parameters was the list pulled?
How is completeness verified?
Using the list the auditee sent by email directly is the weakest link in the audit. To verify completeness:
- Have the list pulled from the source system in front of the auditor, or ask for a screenshot of the query parameters.
- Reconcile with an independent source: If the leaver list comes from the HR system, compare it with accounts disabled in Active Directory during the period. If the change list comes from the ticketing system, pick a few changes from the server or network device configuration history and check whether they’re on the list (reverse testing).
- Check for gaps or duplicates in record numbers.
- Verify that boundary records at the start and end of the period are included.
In the field, reverse testing is most productive on network devices. A change that appears in the configuration archive on a switch or firewall, or in configuration change records in syslog, but has no counterpart in the ticketing system shows work that bypassed the control. “Small” fixes made at night usually turn up here.
Random, systematic and judgmental sampling
| Method | How is it selected? | Strength | Limitation |
|---|---|---|---|
| Random | Every record has an equal chance; reproducible random numbers | Unbiased, defensible | May miss high-risk records |
| Systematic | Interval k = N ÷ n; every k-th record from a random start | Covers the period evenly | Misleads if the list has a periodic pattern |
| Judgmental | The auditor selects based on risk | Focuses on high-risk areas | Weak for generalizing to the whole population |
Judgmental sampling is common in ISMS internal audits and ISO 19011 accepts it; but the reason for the selection must be written down. The most defensible way is to combine the two methods: for example, selecting 20 change records at random and 5 judgmentally (weekend changes, emergency changes, privileged account changes). The random part gives you the right to generalize. The judgmental part hunts for risk.
Reproducible random selection
So that random selection can be reproduced by someone else, use a fixed seed value and write it in the working paper. Selecting from a CSV with PowerShell:
$pop = Import-Csv .\changes_2026.csv
$pop.Count # write the population count in the working paper
$pop | Get-Random -Count 25 -SetSeed 20261110 |
Sort-Object RecordNo |
Export-Csv .\sample_changes.csv -NoTypeInformation -Encoding UTF8
The same with Python:
import csv, random
with open("changes_2026.csv", encoding="utf-8") as f:
pop = list(csv.DictReader(f))
random.seed(20261110)
sample = random.sample(pop, 25)
print(len(pop), [r["RecordNo"] for r in sample])
With the same seed and the same input file, the same tool returns the same samples every time it runs. Different tools (PowerShell and Python) don’t produce the same result with the same seed. Also record which tool and which version was used (for example the output of $PSVersionTable.PSVersion or python --version); the behavior of the random number generator can change between versions. Adding the hash of the input file (for example Get-FileHash or sha256sum output) to the working paper shows the population hasn’t changed since. Without a hash, you can’t prove the file didn’t change.
Sample size by control frequency
Calculating statistical sample sizes is rarely needed in management system internal audits. Common practice is to use a predefined guidance table based on how often the control operates. The values below are example values compiled from ranges often used in IT control audits; they should be approved in the organization’s audit methodology before use.
| Control frequency | Occurrences per year | Sample size (example) |
|---|---|---|
| Yearly | 1 | 1 |
| Quarterly | 4 | 2 |
| Monthly | 12 | 2–5 |
| Weekly | 52 | 5–15 |
| Daily | about 250–365 | 20–40 |
| Several times a day / event-based | hundreds or more | 25–60 |
Which end of the range to choose depends on the risk assessment. The upper end is preferred when: the control protects a high-risk area, an exception was found at the previous audit, the control is manual and complex, or the person running the control changed during the period. If the control design changed during the period (for example the approval flow moved from email to the ticketing system), the old and new designs are treated as two separate populations and samples are selected from each.
In small populations (for example 8 leavers in the period), if the number in the table exceeds the population, all records are reviewed. This isn’t sampling but a full review, and should be named as such in the working paper.
What to do when an exception is found
When a deviation turns up in the sample, the first reflex shouldn’t be to set that record aside as an “explainable case” and pick a new one in its place. That’s distorting the result. The recommended order:
- Confirm the exception: Is it really a control failure, or is the evidence elsewhere? Give the auditee reasonable time to present evidence.
- Understand the cause: Is it a one-off human error or a systemic gap? If there’s a systemic cause (for example the approval step never being run for emergency changes), extending the sample doesn’t change the result; the control isn’t effective.
- Consider extending the sample: If the exception looks isolated, additional samples are selected. The number of additional samples should be defined in advance in the audit methodology; some methodologies require as many additional samples as the original sample size. If exceptions also turn up in the additional samples, the conclusion is that the control isn’t effective.
- Target similar records: If the exception is tied to a specific person, system or period, select additional judgmental samples from that subset.
- Write the finding: The number of exceptions, the total sample and the population size must be stated clearly in the finding text. For the structure of the finding, see the writing audit findings guide.
Documenting the sample
A well-documented sample lets another auditor select the same samples and reach the same conclusion. The working paper should contain these fields:
- The control tested and its reference (Annex A or ITGC number)
- Population definition, period, source system and report parameters
- Population size and how completeness was verified
- Selection method, seed value or justification for judgmental selection
- Sample size and, if it deviates from the guidance table, the justification
- Test result and evidence reference for each sample
- Exceptions, the extension decision and the conclusion
Mistakes you may come across
- Taking the population from a table the auditee sent by email: Because the source and parameters aren’t known, completeness can’t be defended.
- Letting the auditee choose the samples: The request “show us a few examples” results in the neatest records being presented. Every time.
- Selecting from only part of the period: If all samples were selected from the last two months, the conclusion is valid only for those two months.
- Shrinking the sample size afterwards: If fewer samples than planned were reviewed because time ran out, this and the resulting limitation must be written in the working paper.
Planning sampling decisions at audit program level is covered in the ISO 27001 internal audit program guide, and the evidence structure of a frequently sampled control such as access review in the access review evidence guide. Carrying out internal audits, including sample design, is within the scope of our ISO 27001 internal audit and ITGC audit readiness services. The primary source for audit principles is the ISO 19011:2018 standard.
Frequently asked questions
Is statistical sampling mandatory in internal audit?
No. ISO 19011:2018 recognizes both judgmental and statistical sampling; judgmental sampling is common in management system internal audits. The selection method and its justification must be documented, and the result reported in language consistent with the limits of the sample.
How many samples are needed for automated controls?
Because an automated control works the same way every time with the same configuration, verifying the configuration once is usually enough. However, this conclusion depends on the change and access controls protecting that configuration also being effective.
If a sample shows an exception, can I replace that sample?
No. Replacing a sample containing an exception with a new one distorts the result. The exception is confirmed, its cause investigated and, if needed, the sample extended; the exception stays in the working paper in every case.
What should be done if the population is small?
If the sample size in the guidance table is close to or larger than the population, all records are reviewed. In the working paper, this is stated as a full review, not sampling.