CIA Part 2 Sampling and Data Analysis

Begin with the objective and population

Sampling is a way to examine less than an entire population while addressing a defined audit objective. Before selection, define what is being tested, the relevant period, the unit of selection, and the source population. If those pieces are vague, the result may be difficult to interpret.

Validate population completeness and accuracy. Reconcile counts or totals to an independent source, review filters, confirm dates and status fields, and consider whether deleted, manual, or off-system items exist. A population supplied by the process owner may be a starting point, not a validated sampling frame.

Example: a report lists 800 new vendor records, but master-file logs show 835 creations and updates. Determine why the populations differ and which one contains relevant changes before drawing a sample. Testing 30 entries from the smaller report cannot address omitted records.

Choose a selection approach

Statistical sampling uses probability-based selection and defined methods to support estimates within stated confidence and precision assumptions. Nonstatistical sampling uses auditor judgment and can be suitable for testing, but it does not automatically support a statistical estimate.

Random selection can reduce selection bias when the population is complete and each unit has a known chance of selection. Systematic selection can be efficient if periodic patterns do not distort it. Judgmental selection can target high-risk, unusual, or material items. Discovery sampling may be designed to find rare deviations under specific assumptions. Choose based on the objective and risk.

Targeted selection is especially useful when the auditor wants to examine high-dollar transactions, new vendors, late approvals, or transactions near thresholds. But targeted selection answers a targeted question. If 5 of 20 selected high-risk transactions fail, do not state that 25% of all transactions fail unless the design supports that inference.

A sample size should reflect the risk, population, control frequency, expected deviation, desired assurance, and method. More items are not always better if the population is wrong or the test is poorly designed. The auditor should document why the approach is appropriate.

Define attributes and evaluate deviations

Before testing, specify the attribute that constitutes a pass or exception. For an approval control, attributes might include the approver’s authority, date before payment, correct transaction, and evidence of review. If the test is defined only as “approval present,” a late or unauthorized approval might be mistakenly treated as effective.

When an exception appears, verify it and investigate context. Was evidence stored in another system? Did a documented exception apply? Is the transaction a duplicate or reversal? Did a compensating control detect the issue? The auditor should not remove an exception simply because management offers an explanation; corroborate the explanation.

After evaluating deviations, decide whether to expand testing, change the risk assessment, or report a finding. One isolated error may warrant further work depending on severity and risk. A repeated pattern may indicate systemic weakness. Document the basis for the decision and any remaining uncertainty.

Data analysis is a method, not a conclusion

Data analysis can identify patterns, outliers, duplicate records, sequence gaps, unusual timing, concentrations, or relationships. The result points to questions for audit; it does not always explain cause. A spike in late approvals could reflect a real control lapse, a changed workflow, or an incorrect date field.

Check data quality before interpreting results. Confirm source, extraction date, filters, field meaning, missing values, duplicates, currency, time zone, and reconciliation totals. If the data exclude reversals or include test records, adjust the analysis or explain the limitation.

Use the right analytical tool. Trend analysis compares values across time; variance analysis compares actual with budget or expectation; ratio analysis evaluates relationships; benchmarking compares performance with a selected reference; workflow analysis examines paths and handoffs. Each technique has assumptions and can mislead if context changes.

Worked example: duplicate payments

An audit extracts two years of accounts-payable transactions and finds pairs with the same vendor, amount, and invoice date. The auditor first validates the vendor identifiers, reversal treatment, currency, and population totals. Then the auditor examines potential matches against original invoices, credit memos, payment dates, and adjustment records.

Some pairs are legitimate split payments and some are reversals. A smaller group shows two completed payments with no credit or separate invoice. The auditor verifies authorization and checks whether a detective duplicate-invoice control flagged them. The analysis identifies candidate exceptions; document inspection and corroboration establish which represent duplicate payments.

If the auditor selected all flagged pairs, the results describe those pairs and control response. They do not establish the error rate among all payments because unflagged duplicates may exist and the flagged set is not a random sample. A separate design may be needed to estimate prevalence or evaluate the control more broadly.

Using continuous monitoring and tools

Computer-assisted audit techniques can extract data, test rules, compare files, or monitor indicators. The auditor should understand the query logic, access rights, change controls, and output validation. A script that selects transactions over a threshold can be useful only if the threshold, currency, date, and exception logic are correct.

Continuous monitoring can identify issues quickly, but an alert is not a validated finding. Establish who reviews alerts, how false positives are handled, whether thresholds are current, and whether actions are documented. Audit may evaluate the monitoring control but should not become the operational owner of business alerts.

When technical skills are needed, involve a competent specialist. The engagement team remains responsible for understanding how the analysis supports the objective and for reviewing the interpretation.

How to report sample and analytics results

State the population, period, selection method, attributes, and result clearly. Distinguish an observed exception count from a projected rate. Explain material limitations and whether additional work was performed.

For analytics, describe the indicator and follow-up, not just the chart. “The analysis identified 46 payments with similar vendor and amount attributes; inspection confirmed 7 duplicate payments” is clearer than “analytics found duplicate payments” if only some matches were confirmed.

A report should not bury limitations in a footnote. If population data were incomplete or one location was excluded, explain what assurance the work can and cannot provide.

Common sampling traps

A sample from a biased list cannot fix the bias. A large sample does not guarantee representativeness. A high-risk targeted sample should not be presented as a population estimate. An exception should not be extrapolated without the method to support projection.

Candidates should also avoid expanding samples reflexively. First validate the exception, assess risk, understand whether an exception path exists, and decide what additional evidence would answer the objective. Expanding the sample is one possible response, not a universal first step.

Exam decision sequence

Use this sequence: define objective and assertion; validate population; choose a selection method; define attributes; test and corroborate exceptions; evaluate the inference; decide on further work; communicate results and limits.

When a question describes an incomplete extract, population validation comes before sampling. When it describes a targeted selection, report only what the selection supports. When an analysis identifies an outlier, investigate the cause before labeling it an error or fraud.

A sample design example

Assume an auditor tests whether monthly reconciliations were reviewed across 12 months and 30 locations. The objective is to assess consistent operation, not merely whether review is possible. The auditor should define the unit (location-month), identify the full population, decide whether to cover every month or select units, specify review attributes, and consider concentration of risk.

If the auditor selects only year-end reconciliations because they are convenient, seasonal gaps may remain. A stratified selection across locations and months can better address variation, depending on the question and method. Document how the choice supports the intended conclusion.

Interpreting an unusual trend

A chart shows a sharp increase in manual journal entries during the final week of each quarter. The pattern is a risk indicator. The auditor should identify preparers, approvers, accounts, timing, and supporting documentation, and compare the pattern with legitimate close procedures.

If the entries are authorized and consistently supported, the trend may reflect a business cycle. If one user posts entries without review or entries reverse shortly after quarter-end, follow up and assess control operation. The analytical pattern helps focus testing; it does not by itself establish misstatement or misconduct.

Documenting analysis reproducibly

Record the source data, extraction method, fields, filters, transformation, and assumptions so another auditor can reproduce the analysis. Validate a selection back to source records and reconcile totals to an independent control. Preserve code or query versions where applicable under the organization’s workpaper standards.

A screenshot of a chart is not enough to show how data were processed. Clear documentation allows a reviewer to distinguish a genuine population pattern from a query defect or changed business definition.

If the auditor tests every item in a small population, that is a census rather than a sample. A census removes selection sampling error for that defined population, but it does not eliminate data errors, misinterpretation, or limitations in the period and scope.

This distinction is useful when reporting results.

Suppose all 12 monthly control logs are tested and two months have missing evidence. The auditor can state that two of the 12 logs lacked support, subject to verifying the population. Whether that result means the control failed for two months depends on the control criteria and available alternative evidence. It does not alone establish that every transaction in those months was wrong.