Measuring the mail: how to ensure statistical accuracy for postal sector operations and regulations

A team of Analysys Mason consultants has been applying its expertise in the postal sector for a number of years now, drawing on more than 25 years of the company's experience in the wider communications industry. In this article, we focus on applying statistical analysis techniques to a critical issue for postal sector operations and their regulatory oversight.

Statistical analysis: real mail sampling, volume measurements and mail characteristics

This is an important matter in the industry for three reasons.

• The volume of items in the pipeline may not be absolutely known. Mechanised sortation goes a long way to 'counting' items, but there will always be some, or a lot, of manual handling, bulkier packet sortation, machine rejects and so on. Statistical measures (such as tray and bag fills and container counts) are helpful to know how many items are passing through the pipeline, and to reconcile different volume measurements at different stages (for example, outward sortation, inward sortation and delivery).
• Quality of service is an important measure in the postal pipeline. Mail users, operator(s) and the industry regulator want to know whether transit times are reaching the (high) percentage on-time delivery targets. Statistical sampling of mail transit time is needed to measure this performance indicator. There is plenty of guidance for this approach – for example, in the form of European Committee for Standardization (CEN) standards – but the implementation of the guidance is critical.
• The calculation of revenue-derived mail volumes is an essential element in many service costing and regulatory accounting situations. Revenue-derived and operational volume measurements are used in different ways by operators and regulators, and can also be compared or reconciled with each other. However, revenue-derived calculations can be subject to uncertainty, which stems from the underlying mail statistics applied.

Revenue-derived mail volumes were the subject of our recent study for a European regulator

The incumbent mail company's revenue is subject to a statutory audit, so is 'known' with certainty. Dividing this revenue, categorised accordingly, by average unit prices leads to a measurement of volumes, known as 'revenue-derived traffic'. However, the average unit prices are strongly dependent on the assumed profile of mail items – that is, the combination of: letters, flats, packets and parcels; large and small items; heavy and light items; and other possible price-distinguishing criteria. Costs are also allocated to products on the basis of weights and other measured handling characteristics. Operators and regulators need to have confidence that the underlying statistics of the profile of mail are accurate if audited accounts or regulatory accounting information includes calculations based on revenue-derived traffic and characteristic-based cost allocations.

Analysys Mason has completed a range of tasks relating to statistical analysis in the postal sector, including:

• investigating the real mail sampling carried out by the incumbent operator
• reviewing the design, plans, procedures and implementation of the sampling
• scrutinising the accuracy and conformity of the data collected
• conducting parallel reviews of operational volume measurements to identify possible reconciliation activities and causes of divergence
• applying both simple and complex statistical calculations to reveal some critical issues with the underlying statistics, and their use within the regulatory accounts.

Figure 1 shows an example of frequency distribution. The first shows how many times an induction of a given number of items was received for sample measurement – this distribution shows no systematic characteristics. However, the second frequency distribution of how many items were sampled from the induction clearly shows a systematic effect, and a statistical accuracy problem that required investigation and correction.

Figure 1: Frequency distributions observed in a set of mail inductions and the corresponding set of mail sampling [Source: Analysys Mason, 2013]

For some reason, sample mail measurement appeared to concentrate on multiples of five items. This finding could not be reconciled with the requirement for randomisation as set out in the sampling instructions and governing principles that ultimately lead into the audited regulatory accounts. As a result of our work in this instance, changes were applied to the procedures used for mail sampling, and the regulator also gained a much greater understanding of the accuracy present in the regulatory accounts.

Contact Ian Streule to discuss our work in the postal sector in more detail.