Shifting The Burden: Using a Questionnaire and Panel Review to Ensure that Ecosystem Services are Taken into Account in Project Appraisal

Takeo Kuwabara and Griffin Smith, 2018
Construction of high-density towers and luxury villas on the first island of the Forest City megadevelopment project.

The Problem: A Gap Between Ecosystem Services Theory and Practice
Natural ecosystems are being lost to development in fast-growing countries around the world. While land development may often be a desirable objective, conversion from one land use to another rarely takes account of the myriad benefits that natural ecosystems, directly or indirectly, provide to humans. This is because markets often fail to capture the value of these benefits monetarily. If there were an easy way to compute the market value of ecosystem services, and local regulations required that their loss be taken into account, the disincentives to development that requires large-scale conversion of natural ecosystems would likely be substantially higher.

While the measurement and valuation of ecosystem services at global, national and landscape scales have been the focus of considerable research, how to best apply the ecosystem service lens to project appraisal at the local level remains unclear. Most studies describing real-world application of local ecosystem services assessment are primarily accounts by the authors to describe what they tried.1 Thus, we have a growing number of novel theoretical and methodological proposals, but no generally agreed upon set of practices.2 Practitioners need a set of legally defensible methods that can be applied in many different contexts at modest cost.3

It is important that researchers continue efforts to refine best practice methodologies, which have the potential to improve outcomes and generate learning where practitioners can apply them. However, we contend that, due to many common constraints, it will rarely be feasible for most of the methodologies proposed in the literature to be implemented at the project scale, especially in developing countries. Key constraints include a pervasive lack of reliable and fine-scale data,4-6 the short time frame available for conducting studies between when projects are proposed and when regulatory approvals are granted;7,8 the lack of study funding available to most stakeholders and local governments,3,9 and a lack of the interdisciplinary expertise required to implement the complex ecosystem service analyses called for in the literature.10

Only highly simplified methodologies have any chance of being adopted. A number of relatively simple digital modelling and mapping tools have been developed over recent years (e.g. InVEST,11 ARIES12). However, as with all models, the outputs that these tools produce are constrained by the quality of the inputs that go in; a lack of high resolution data often prevents them from producing meaningful or accurate results at the project scale.13

The barriers to thoughtful application of ecosystem service assessment methodologies are particularly high in the private sector. There is scope to test best practice methodologies in large-scale, publicly funded research studies. Yet because national environmental impact assessment (EIA) regulations rarely mandate the inclusion of ecosystem services analysis, or prescribe methodologies for doing so, private developers (facing strong imperatives to generate quick results so that development can proceed) are unlikely to include such considerations in their project impact studies. When they do, their analyses are likely to be highly simplified, cursory efforts.

This reality creates the potential for project proponents to either disregard ecosystem services in their impact assessment reports or to integrate inaccurate or even intentionally misleading information. Thus, the burden often falls on local stakeholders who depend on ecosystem services to question the proponent’s findings and attempt to influence the outcome of the appraisal process – a significant challenge for stakeholders with limited resources and political power.

Instead of seeking ever greater technical ‘accuracy’ in methodologies for forecasting the size and value of ecosystem service impacts, we think it is important to acknowledge the constraints in practice, and offer pragmatic advice for how to make ecosystem service analysis most useful in this context. More importantly, we want to empower concerned stakeholders to raise provocative questions about the ecosystem service benefits at risk if land conversion is allowed to proceed, and to enhance their ability to hold developers and decision-making authorities accountable for these losses in a public manner.

The Solution: An Ecosystem Services Questionnaire and Panel Review Procedure
We believe there is a simple approach that policy makers can introduce to ensure that, at a minimum, public deliberation takes place about the likely ecosystem service impacts of large-scale projects. Importantly, it would shift the burden for demonstrating the significance of these likely impacts so that responsibility does not fall entirely on those who stand to lose.

We propose that local, state, or national authorities require proponents of projects that exceed a defined threshold of probable impacts or that involve certain activities (e.g. land reclamation) to respond to a standardized questionnaire describing the impacts their projects are likely to have on ecosystem services.

The questionnaire would ask the developer to define the key ecosystem services relevant to their project and describe their plan to reduce or avoid the negative impacts their project is likely to have on these services. The developer would also have to explain how they determined the value of these services, how they intend to ensure that the services will be provided in some other way, and how their proposed project might be designed to protect or enhance important ecosystem services in the area.

The relevant authority would be required to appoint an independent panel to review developer responses to such a questionnaire on behalf of the city or territorial government. The panel would include technical experts such as ecologists, engineers, economists, along with planning agency and other staff and designees of elected officials like the mayor or city council.

Crucially, the relevant public agency would make developer responses to the questionnaire publicly available for review, arrange for a period of public comment, and require a response from the developer. In some cases, it might provide concerned stakeholders with resources to assist them in scrutinizing the project proponent’s response. Civil society organizations might provide a list of questions that concerned stakeholders could ask during any discussion of the questionnaire responses. The relevant agency could hold an informal public hearing to discuss the proponent’s response to the questionnaire as well as public reactions. An agency would not be required to have specific legal authority to hold such an informal public discussion.

The most important benefit our approach provides is to shift the burden of assessing a project’s likely impacts on ecosystem services from those who stand to lose to those who stand to benefit the most. It would force project proponents to acknowledge that ecosystems offer important benefits to local populations. Groups with concerns would not have to prove that they can forecast benefits and costs with precision. Rather, they would be able to expect the developer or sponsoring agencies to account for how they have estimated ecosystem services likely to be lost, and how they intend to make sure that the public receives these same benefits in other ways.

We argue that requiring a project proponent to show that they have considered the effects of their development on ecosystem services, if only briefly, is more important than the precision of the methods used to carry out such analyses. Much of the project review literature focuses on the importance of using ecosystem services information to inform decisions, but there are steps that need to be taken before decision-makers can use such information effectively.14 By forcing all sides to engage with the idea of ecosystem services, our proposed process will heighten public awareness and prod more strategic thinking among private developers, decision-makers, and public stakeholders.

The questionnaire, public comment and panel review we are proposing have the additional benefit of opening up a proponent’s methodological approach and assumptions to scrutiny. Many non-objective judgements must be made prior to initiating any ecosystem service valuation (these include setting the scope of the study area, choosing the services to be included, deciding the time frame for the assessment, etc.).15Any findings will be extremely sensitive to how these parameters are set. With increasing experience in assessing responses to the questionnaire requirement we are suggesting, decision-making agencies and concerned publics should become more adept at assessing the validity of project proponent’s technical choices.

Additionally, our approach need not be established through a change in the law, simplifying the administrative process for any city or state to introduce the questionnaire/panel idea as a pilot initiative.

In the following section, we consider how our proposed approach might have altered outcomes for stakeholders, using a controversial real estate development project on reclaimed land off the coast of Malaysia.

An Illustration: The Case of Forest City in Johor, Malaysia
This case illustrates how ecosystem services have been handled by private developers in a country where environmental regulations and review processes are enforced, but ecosystem services have been given scant attention. We first describe what occurred, before imagining a different outcome if our proposed questionnaire and panel review procedure were in place.16

The country’s southern tip in the state of Johor was designated as a special economic zone in 2006. The Iskandar Development Region, as it was named, was intended to take advantage of Johor’s immediate adjacency to prosperous Singapore to attract foreign investment. Since the zone was established, there has been a rise of foreign-owned real estate investment along the Johor coast.

In January 2014, Chinese developer Country Garden began a 2,000 hectare land reclamation project, Forest City in the straits between Malaysia and Singapore. The vast project appeared to have been developed independently of, other regional planning efforts; Forest City did not feature on any maps created by the Iskandar Regional Development Agency (IRDA).
Environmental concerns
The Forest City site is directly atop the largest seagrass bed in peninsular Malaysia and adjacent to a large mangrove area designated as a Ramsar Wetland of International Importance in 2003. A 2011 IRDA shoreline management plan describes the project site as explicitly preserved for resources management in order to “protect marine flora and fauna, especially [the] seagrass bed”.17

Country Garden began reclamation work without having completed an EIA as required under Malaysian federal law for projects of this size, because the proponents proposed to divide the project into smaller pieces – each just under the scale that would trigger a mandatory EIA. Neither local fishermen nor the Singaporean government were aware of this major reclamation project until barges began depositing large amounts of sand in the ocean. In May 2014, Singapore submitted an official enquiry to the Malaysian federal government, triggering the Malaysian Department of Environment (DOE) to issue a stop work order to Country Garden and require that it complete a detailed EIA (DEIA).
Country Garden hired a local consultant to produce the DEIA report, which was approved by the DOE in January 2015 after a costly year-long halt to construction and extensive project redesign. Country Garden scaled down the project to 1,600 hectares and altered the plan from a single island to four smaller islands surrounding the seagrass bed, now designated a “preserve” (Fig. 1).18

Samuel Barnard and Lawrence Susskind
Figure 1. Map of planned Forest City landforms and surrounding area.


The Malaysian DOE’s Handbook of Environmental Impact Assessment Guidelines includes identifying the environmental costs and benefits of a project to the community as one of five objectives of an EIA. The DOE’s 2008 ‘Guidelines on the Economic Valuation of the Environmental Impacts of Prescribed Activities19 suggests a framework to use in calculating the monetary value of project impacts on the environment that cannot be eliminated through mitigation measures. The EIA practitioner retains the discretion to choose their methods, including which impacts they consider significant enough to monetize.

Chapter 16 of the Forest City DEIA focuses on the economic value of the project’s likely impacts on ‘environmental services’. The consultant hired by Country Garden focused on mudflats lost to reclamation, the impact on the seagrass area, and the heightened travel costs for local fisherman whose fishing grounds would be destroyed (not technically an environmental impact). They chose not to estimate the impact the project might have on water quality on the grounds that Country Garden planned to install silt curtains intended to limit adverse effects to negligible levels. The consultant did not consider the impact of the project on ecosystem services provided by adjacent mangrove forests.

The consultant used a benefit transfer approach to estimate the monetary value of the environmental services included in the analysis. For each service, the consultant determined monetary value based on the results of studies conducted elsewhere, at varying times and scales. The DEIA does not include a bibliography that would permit crosschecking of the referenced studies.

The DEIA estimated the total annual value of environmental impacts at RM10.26 million, or US$3.14 million.20 This equates to a net present value over 50 years of RM115.98 million (US$35.37 million) at an 8% discount rate or RM193.21 million (US$59.09 million) at a 4% discount rate. The reduction in fishing revenues caused by permanent loss of mudflats account for almost 90% of these impact figures.

The DEIA does not include an interpretation of the relative scale of the estimated monetary value associated with Forest City’s likely impacts. The results of the economic valuation exercise are presented in a half-page of text at the end of the 74-page DEIA Executive Summary. However, the Summary notes that a “huge commitment is needed by the Proponent [Country Garden] in implementing all mitigation measures proposed so that this development will be beneficial not only to the Proponent, but also to the local communities and the State of Johor.”21
How was the economic valuation used?
It is not clear that the economic valuation of Forest City’s projected impacts on ecosystem services influenced subsequent environmental management deliberations or decisions to compensate affected stakeholders. Country Garden was compelled by the Johor State government to provide a compensation fund for local fishermen of RM104 million.

Malaysian DEIA guidelines require consultation with affected stakeholders. In a public meeting to present the DEIA, Country Garden received unexpectedly vociferous pushback from local stakeholders, who were not satisfied with the assessment of Forest City’s projected environmental and economic impacts or the process by which it was approved.21
Imagining an alternative process
The process undertaken in the Forest City case generated suboptimal results for most parties involved. The developer faced a very expensive delay, was required to redesign its project and forced to compensate fishermen it had harmed, lost trust in the eyes of local stakeholders, and suffered hits to its local and international reputation. Villagers’ livelihoods were severely damaged by the disruption to the fishery on which they depended. The state of Johor and the regional development authority lost credibility by not initially requiring a full-blown EIA and allowing a flawed project to proceed. The national government lost credibility with its neighbor, Singapore. How might an alternative approach have altered the process and the outcome? In this section, we imagine hypothetical implementation of our proposed ecosystem services questionnaire and panel response.

The questionnaire Imagine that, as part of an established procedure, IRDA had required Country Garden, to complete in parallel a questionnaire regarding their project’s likely impact on ecosystem services. Box 1 lists some simple questions that might have been included.


In the case of Forest City, we imagine that Country Garden’s submission would be similar to the economic analysis of environmental impacts included in the DEIA it was ultimately required to prepare.

The panel In the case of Forest City, IRDA would convene a panel of five experts to review the questionnaire response, such as a representative of the Johor DOE office; a local hydrological engineer; a coastal ecologist; an IRDA staff member; and an economist. The role of the panel would have been to review the project proponent’s responses, ask clarifying questions and request additional detail. The panel would convene after the proponent submitted its DEIA, but before the DOE made a decision about whether to approve or reject the project. It would have a month to review the questionnaire response, during which time it might have consulted with additional technical experts and NGOs.

After a month of deliberation and investigation, the panel would prepare a public letter to IRDA summarizing its review of the adequacy of the proponent’s responses. The panel’s judgement would not be legally binding, but it would have brought substantial publicity and enormous pressure to bear on the DOE as it made its decision regarding the adequacy of the DEIA. The letter might have asked about Country Garden’s lack of consideration of Forest City’s impacts on ecosystem services provided by the surrounding mangrove forest. It might have asked whether the proponent had conducted a sensitivity analysis regarding the effectiveness of proposed mitigation measures such as silt curtains. It might also have decided that certain studies used to calculate ecosystem service values were too far-removed physically or temporally to generate credible figures.

We can imagine a scenario in which some stakeholders were not content with the consultant’s technical approach to preparing its ecosystem services analysis. After the panel’s decision was made public, we assume there would be numerous letters to the editor, social media reaction and even a meeting organized by advocacy groups regarding Country Garden’s response to the panel’s concerns.

Outcomes We assume that the Panel would have attended the advocacy meeting. The dialogue that ensued might have had (1) an educational impact — helping everyone better understand the issues at stake; (2) a political impact — rousing various groups to band together to make recommendations to Country Garden and the relevant regulatory and political bodies insisting on further changes in the plan; (3) a financial impact — some of the sources of Country Garden’s project funding might have raised concerns in light of the Panel’s criticism of the project; and (4) regulatory impact — other regulatory agencies that also had to provide approvals might have been swayed by the Panel’s thorough report, and decided to hold Country Garden to a higher set of standards.

This process might also have reshaped the way EIA practitioners in Malaysia interpreted prevailing EIA requirements. If a reknowned consultant used by Country Garden wanted to be selected by other similarly situated clients, they might push Country Garden to respond to the Panel’s concerns.

This is a hypothetical imagining to clarify what it would take to implement our proposal. Critics might be concerned about the added time and cost burden associated with an additional development review process and the proposed process’ lack of legal teeth in regulating appraisal outcomes. We respond to those concerns here and reflect on the potential benefits of our proposed approach for encouraging what we call “deliberative environmental management.”
Additional time and cost burden While the process we have suggested would represent an additional requirement for obtaining project approval, we believe the added time and financial costs involved are small, especially in light of the impact and longevity of a project of this magnitude, which will last for many decades, or even generations. Modest investments of time and money can actually save on long-term costs. They can certainly help to sustain the reputation of the development entities involved.
Lack of legal teeth Although the panel convened to review the developer’s questionnaire response would not have legal authority to approve or reject the proposed project, it would shed light on the developer’s assumptions and forecasts, as well as their approach to mitigating negative effects (and maximizing beneficial effects) for local residents. The political will required to use the informal (i.e. non-binding) process we are proposing will depend on the fortitude of the political leadership in the country and the region.
The benefits of deliberative environmental management By requiring an interdisciplinary panel review and creating an opportunity for focused public deliberation, the process we propose seeks to introduce a level of engagement into what is otherwise likely to be almost exclusively top-down decision-making. It responds to calls for the creation of ‘platforms for decision making that marry the interpretative (social) complexity of sustainable resource management and the analytical (scientific) needs of informed – evidence based – decision making.’22 This analytic-deliberative approach also acknowledges that social values are central to the way scientific results are interpreted, and that it is not credible to pretend otherwise.23 Moreover, increasing opportunities for affected stakeholders to participate in local discussions and hold decision-makers accountable, increases the prospect that the results will be “owned” by all parties involved.23

The authors gratefully acknowledge the financial support provided by the Sam Tak Lee Real Estate Entrepreneurship Lab at MIT.


    1. Laurans, Y, Rankovic, A, Billé, R, Pirard, R & Mermet, L. Use of ecosystem services economic valuation for decision making: questioning a literature blindspot. Journal of Environmental Management 119, 208–19 (2013).
    2. McComb, G, Lantz, V, Nash, K & Rittmaster, R. International valuation databases: overview, methods and operational issues. Ecological Economics 60, 461–72 (2006).
    3. Baker, J, Sheate, WR, Phillips, P & Eales, R. Ecosystem services in environmental assessment – help or hindrance? Environmental Impact Assessment Review 40, 3–13 (2013).
    4. Bagstad, KJ, Semmens, DJ, Waage, S & Winthrop, R. A comparative assessment of decision-support tools for ecosystem services quantification and valuation. Ecosystem Services 5, 27–39 (2013).
    5. Rosenberger, R, & Phipps, T in Environmental Value Transfer: Issues and Methods (Navrud, S & Ready R, eds), Ch. 3 (Springer Netherlands, 2007).
    6. Troy, A & Wilson, MA. Mapping ecosystem services: practical challenges and opportunities in linking GIS and value transfer. Ecological Economics 60, 435–49 (2006).
    7. Brown, G & Fagerholm, N. Empirical PPGIS/PGIS mapping of ecosystem services: a review and evaluation. Ecosystem Services 13, 119–33 (2015).
    8. Johnston, RJ, Rosenberger, RS, Rolf, J & Brouwer, R. Benefit Transfer of Environmental and Resource Values 1-582 (Springer, New York City, 2015).
    9. Johnston, RJ & Rosenberger, RS. Methods, trends and controversies in contemporary benefit transfer. Journal of Economic Surveys 24, 479–510 (210).
    10. Ruckelshaus, M, McKenzie, E, Tallis, H, Guerry, A, Daily, G, Kareiva, P, Polasky, S et al. Notes from the field: lessons learned from using ecosystem service approaches to inform real-world decisions. Ecological Economics 115, 11–21 (2013).
    11. InVEST [online]
    12. ARIES (ARtificial Intelligence for Ecosystem Services) [online]
    13. Vorstius, A & Spary, C. A comparison of ecosystem service mapping tools for their potential to support planning and decision-making on a local scale. Ecosystem Services 15, 75-83 (2015).
    14. Mckenzie, E, Posner, S, Tillmann, P, Bernhardt, JR, Howard, K & Rosenthal, A. Understanding the use of ecosystem service knowledge in decision making: lessons from international experiences of spatial planning. Environment and Planning C: Government and Policy 32, 320–40 (2014).
    15. Spangenberg, JH & Settele, J. Precisely incorrect? Monetising the value of ecosystem services. Ecological Complexity 7, 327–37 (2010).
    16. The details of the Forest City case were compiled in the course of preparing a Massachusetts Institute of Technology digital case study on the project. Most of the information detailed in this section is contained and sourced in the case study: Susskind, L, Kuwabara, T, Williams, M, Mukahhal, A, Allen, N et al. Forest City: a case study of a real estate megaproject in Asia. MIT STL Real Estate Entrepreneurship Lab [online], The site should be accessed using Google Chrome.
    17. IRDA. Shoreline Management Plan Blueprint for Iskandar Malaysia 21 (Iskandar Regional Development Authority, Johor Bahru, Malaysia, 2011).
    18. Renderings based on the Forest City masterplan by Sasaki Associates are available at:
    19. DOE. Guidelines on the Economic Valuation of the Environmental Impacts for EIA Projects (Ministry of Natural Resources and Environment, Putrajaya, Malaysia, 2008).
    20. 2014 figures. Converted at average exchange rate for 2014 (RM3.27 per US$)
    21. Dr. Nik & Associates. The Proposed Forest City Island Reclamation and Mixed Development, Johor: Detailed Environmental Impact Assessment (Dr. Nik & Associates, Kuala Lumpur, Malaysia, 2014) Appendices B.6 and C.
    22. Fish, R. Environmental decision making and an ecosystems approach: some challenges from the perspective of social science. Progress in Physical Geography 35, 671–80 (2011).
    23. Slootweg, R. Ecosystem services in SEA: are we missing the point of a simple concept? Impact Assessment and Project Appraisal 34, 79–86 (2015).