The Parliamentary Commissioner for the Environment, Simon Upton, has repeated his call for a full independent review of Overseer before any widespread adoption of it as a regulatory tool.
And he says we also need better understanding of what nutrient models can and can’t do to assist in building a picture and better communication of what is happening to water quality.
Upton highlighted several gaps and faults in this information to a recent Environmental Defence Society conference.
The PCE is analysing Overseer as a tool for measuring water pollution from agricultural sources. Upton told the conference he is not yet in a position to preview findings on his Overseer report. But the need for better understanding of nutrient transport, models and communication were among aspects which so far stand out to him in his findings.
“Overseer provides useful information about nutrient loss from a range of land uses. However it stops at the root zone. The model is silent about what happens when those nutrients leave a slim 60cm sliver at the land’s surface.
“We know that nutrients translocate. For instance, nitrogen follows a variety of different flow paths deep into groundwater travelling laterally through the soil or travelling via the surface,” he says.
Upton outlined many factors influencing how nutrients travel including the ability of soil to attenuate nutrient. These differences would ideally be taken into account when managing land use and water quality in regional plans, he says.
“That’s if you knew about them. From our discussions with councils, CRIs and universities we understand there are still big gaps in our understanding of nutrient transport across and through a catchment,” he says.
“For example, GNS has highlighted, we understand, the properties of about 40% of the aquifers in New Zealand. Soil databases are also patchy in scale, age and quality. Estimated coverage of NZ is 30% (estimates of the more detailed new layer) although it is, in fairness, 61% of productive land.
“There are numerous data sets available that can assist our understanding of nutrient transport across catchments. The inventory of the current databases is impressive and represents a major investment by taxpayers over decades. But they are maintained by a variety of different organisations.
“They come in various states of comprehensiveness and vintage.
“Take just one example for one of those datasets – hydrogeology. Zooming in behind one of those data sets shows there is no nationally consistent classification of aquifers although GNS has made a couple of attempts.
“But these data sets are not always accessible, or evenly accessible, or have not been joined up in the most useful way to feed into regional council plan thinking.
“If we could make better use of mobilising that, of leveraging that, we could better understand nutrient transport in three dimensions. We have made a very good start.”
Upton also says we need to do a better job communicating whatever information we have about water quality status.
“We could be doing more to assist the community and policymakers [to] understand whether a lake, river or estuary is healthy.”
But he says of indicators that we have more of the least valuable indicators and the least number of the most valuable indicators.
The Macro Invertebrate Community index which uses direct evidence of small animals like snails, worms and crustaceans to indicate stream health is the best evidence and one of the least used.
He says this is a reflection of data availability which in turn is probably a reflection of how difficult or costly it is to gain.
“As a result, direct indicators of stream health are not always available and even when they are the data is analysed and displayed in different ways by different organisations. All this leaves a concerned member of the community none the wiser about what is actually going on.”
He gave as an example the different information on the lower end of the Mataura River, near Gore, carried on the Land, Air, Water Aotearoa website to the data set maintained by Stats NZ and the Ministry for the Environment.
“So we are not really sure what is going on at this site.”
If you can measure it, you can monitor it
Simon Upton says while the fate of nitrogen beyond the root zone is well understood at a general level, the diffuse nature of nutrient losses from land and the large spatial scale of catchments make actual measurements of nitrogen loss at a catchment scale impossible.
In the absence of suitable direct measurements, many models of varying levels of complexity have been developed. Because of the costs involved in sampling, models usually rely on calibration based on a small number of sites.
But even where predictions are made by a calibrated model they can be accompanied by large uncertainties.
“Take, for example, the movement of nitrogen through groundwater. The model might assume that attenuation of nitrogen is uniform across a sub-catchment. However we know from studies that land can be highly heterogeneous. Differences even metres apart can cause attenuation to differ significantly.
“Take, for example, research by Massey University showing that nitrogen attenuation varied from 30% to 70% in over 15 sub-catchments in the Tararua groundwater management zone. This means the amount of nitrogen attenuated from leaching from the root zone – that’s what Overseers looks after – and reaching the river varies, between 30% to 70%. That is significant by any standard. Dealing with uncertainty is a formidable technical challenge.
“We will never be able to eliminate uncertainty when modelling complex biological physical systems.”
What decisionmakers and modellers need to understand are the uncertainties underlying the model.
“I found that those uncertainties that significantly influence model outcomes, and communicating their importance, is key to using models in decisionmaking processes.”
He says this doesn’t just apply to catchment models.
“In the case of Overseer, a formal uncertainty analysis has never been carried out. There have been informal analyses but some of these are old.”
A 25-30% model uncertainty is often quoted, but it is derived from just a look at the nitrogen sub-model which is one of several sub-models and is 17 years old,” Upton explains.
“This means people using Overseer to support decisionmaking may be lacking some important information about what the model can and can’t tell them in a policy or regulatory context.
“It represents a major investment and it is very impressive but this uncertainty analysis I think most scientists would say is actually a standard bit of the operating procedure.”