Bioproduct Approval Regulation: An Analysis of Front-line Governance Complexity
Lisa F. Clark and Peter W.B. Phillips
VALGEN, University of Saskatchewan
The behaviors and seemingly disconnected exchanges between actors operating within subcomponents of hierarchical decision-making systems can contribute to unanticipated broader-system effects. The concept of complexity is a useful way of better understanding the influential nature of these connections on decision-making outcomes. This article presents the findings on subsystem complexity in bioeconomy governance from research undertaken by members of the VALGEN Regulation and Governance team. It demonstrates how applying Social Networking Analysis (SNA) and kurtosis analysis to regulatory frameworks can be used to uncover complexity within multilevel governance. SNA reveals how informal interactions in the decision-making process can impact the regulatory process. Kurtosis analysis shows how inputs into regulatory frameworks are not evenly reflected in the outputs. The article discusses the results of these methodologies applied to approvals in Canada of plants with novel traits and argues that appropriate qualitative and quantitative data sources are important to understanding complexity within the governance structures of the bioeconomy.
Key words: bioproducts, complexity, governance, kurtosis, regulation, SNA.Introduction
Decisions emerging from regulatory systems for innovative technologies carrying a degree of unknown, future risk are often claimed to be based on scientific evidence generated through reproducible and repeatable sets of rules and standard operating procedures. Basing risk-assessment procedures on scientific calculations of the probability of risk of harm at certain levels of exposure is argued to produce rational, optimal outcomes because this method of measuring risk is assumed to be norm and value free. This assumption is increasingly challenged by the integration of different types of evidence into deliberations regarding the management of innovative technologies that carry a degree of uncertainty regarding unknown, future risks (Sundlof, 2000). It is widely argued that normative considerations influence decision-making regarding the safety and risks to humans and the environment associated with bioproducts and crops across jurisdictions (Brunk, Haworth, & Lee, 1995; Jasanoff, 2005; Mills, 2002; Tait & Levidow, 1992). Normative understandings of acceptable levels of exposure to risk are not typically captured in decision-making flow charts, yet they have serious implications for how decisions are made. The intersection of formal rules of decision-making and varying interpretations of risk within decision-making systems can contribute to suboptimal outcomes, unpredictability, and stakeholder misperceptions of how the system works.
Though it is difficult to pinpoint the cause of unpredicted outcomes emerging from multi-level, multi-actor decision-making systems, Simon’s (1962) The Architecture of Complexity provides an important conceptual starting point to help unpack hierarchical decision-making systems found within governance structures. By analyzing the interactions and behaviors of individuals or departments operating within a given system, Simon argues that convoluted and often invisible exchanges or transactions occurring within subcomponents of decision-making systems can be revealed. Revealing these unseen information transactions can shed light on factors that impact the behavior of the system that may not be apparent in official decision-making procedures. However, while Simon’s concept of complexity and his description of ‘decomposable systems’ are seen to potentially be very useful ways of understanding how inefficiencies might be generated, social science scholars continue to struggle with ways of operationalizing complexity and determining whether the inefficiencies observed in output patterns of decision-making systems can be explained by subsystem complexity.
This article demonstrates two ways of operationalizing the concepts of decomposability and complexity to uncover subsystem interactions within governance structures for bioproducts and crops: Social Networking Analysis (SNA) and kurtosis analysis. Both reveal ways of mapping out patterns of interactions within regulatory systems and identifying what information is necessary to conduct further analysis. These empirical tests show the importance of detailed information about subsystem interactions and system inputs and outputs to attain results that carry a significant level of confidence. This article begins by discussing Simon’s ideas of complexity and decomposability and how they can be used in studies of governance. It then explains SNA and kurtosis analysis and how they are used by social scientists to investigate complexity within governance structures. Drawing from the illustrative case of the Canadian regulatory system for plants with novel traits (PNT) approvals, the article discusses how the VALGEN Regulation and Governance team1 used qualitative and quantitative data for SNA and kurtosis analysis. Though we can conclude that these empirical tools are useful means of investigating complexity within multilevel governance structures that are part of the bioeconomy, the primacy of detailed information regarding subsystem processes and interactions is made apparent.
The Theory of Complexity, Decomposability, and Governance
Herbert Simon was one of the first social scientists to apply the scientific idea of complexity to the social world. Though he published in the 1960s on this topic, only in the last decade has Simon’s idea of complexity been revisited and used as a framing mechanism to explain the interaction between structured rules and actors’ informal behaviors in decision-making systems. As neo-institutionalism, network theory, economic behavioralism, and the governance thesis have become dominant approaches in the social sciences, Simon’s conceptualizations of complexity and decomposability have gained new traction (Anderson, 1999).
Complexity describes the interactions or behaviors of actors within a system of rules (Funtowicz & Ravetz, 1994; Simon, 1962). It draws attention to how interactions between actors may influence the behavior of a system over time and may alter outcomes emerging from a system that deviate from predicted ones. Complexity, like positive feedback loops, can originate as a small change that gains momentum and builds up within the system and begins to change behaviors or modify the system. A positive feedback loop is a small change in the system that creates an increase in the magnitude of change and is fed back into the system at another point, which causes change. A positive feedback loop is often used to describe a situation when a small event within a system can, over time, have big impacts on a larger system but is not easily identifiable by breaking down a system into its components. Herd mentality is often given as an example of a positive feedback loop that changes the behavior of the system. Complexity has the potential to influence the outcomes of the system, but may be difficult to trace back to a single interaction, or series of interactions, within a system.
A decomposable system is defined as a system that has subcomponents with a degree of independence from one another and from the system as a whole. This understanding of the architecture of a system helps to explain how the interactions of subcomponents within systems can contribute to unanticipated outcomes. Instead of focusing on the behaviors of individual components, Simon (1962, p. 474) explores the interactions between the subcomponents as sources of complexity generation and argues that in order to understand the behavior of a system, it must be decomposed. A system should be broken down into its elements and the relationships between those elements analyzed to locate points where subcomponents may act in certain ways that do not necessarily conform to the behaviors or decisions of other subcomponents in a system in which they share the common objectives of the system. It is at these points within the system where complexity, through sometimes seemingly insignificant actions by subsystem components, can be generated and fed back through a system via positive feedback loops.
Social scientists have found many uses for complexity and decomposability in understanding decision-making systems by explaining how change is generated within regulatory and governance systems (Baumgartner et al., 2009; Hoppe, 2011). Governance itself produces novel issues of complexity because it is a horizontal approach to decision-making that includes public and private actors and the fluid dynamic between legitimacy and liability. Governance defines the blurred lines between formal decision-making structures and the informal behaviors that can influence how decisions are made within regulatory structures (Giddens, 1990; Majone, 1995; Stoker, 1998). Black (2002, p. 4) defines complexity found within governance structures as “…a decentred understanding of regulation [which] emphasizes both causal complexity and the complexity of interactions between actors and society.” Thinking of the generation of complexity as a consequence of decision-making within governance structures can clarify unanticipated policy outcomes. Viewing decision-making as a complex system populated by feedback loops draws attention to how informal and isolated actions within the system can, under the right circumstances, influence the processes of policy making. Kooiman’s work on governance builds on Simon’s claims and develops the decomposability approach to understanding governance structures. This approach illuminates neglected or overemphasized relations within decision-making structures that may mistakenly be identified as causes of particular outcomes in governance structures (Kooiman, 2003, p. 59). In this way, as discussed in the following section, Simon’s ideas of complexity and decomposable systems have much in common with a network approach to understanding decision-making and behaviors within a system of governance (see Amaral & Ottino, 2004).
New Methods for Examining Complexity in Governance
Our research has adapted and adopted two recently developed methods—SNA and kurtosis—to assess the impact of complexity on governance of the bioproducts space in Canada.
Social Networking Analysis
Network theory and SNA are used in a wide array of the social sciences to investigate the social power derived from interconnectedness (Borgatti, Mehra, Brass, & Labianca, 2009; Castells, 2000) and actor interactions within governance frameworks. SNA measures the structure and patterns of connectedness of individual nodes, or actors, and highlights the social dynamics of power that inhabit networks of interactions. It is a research paradigm, as opposed to a theory (Marin & Wellman, 2010). Like complexity theory, SNA focuses on the linkages between actors in addition to the actors’ own attributes. Node-level measures help to identify actors within a network that are either leaders or exert influence over the behavior of other actors or networks.
Ryan (2008, p. 61) identifies three measures that are the strongest indicators of importance and influence cultivated from the relationships between actors within a network: 1) the total degree of centrality, 2) ‘betweenness’ centrality, and 3) the eigenvector measure of centrality. The total degree of centrality identifies which actor is the hub and has the greatest access to ideas and thoughts of others in the network. The metric indicates which actor the network revolves around and who is central to holding the network together. Those with a high rank on this metric are more connected to others in the same network. Betweenness centrality is the identification of an actor as a broker or gatekeeper that can connect or disconnect groups within the network. These brokers are situated within the network to connect otherwise non-linked groups and to shorten the link between the groups. The eigenvector measure of centrality determines how linked-in an actor is with multiple actors with multiple connections. This actor occurs on many of the shortest paths between links with other actors. Together, these metrics help to uncover the importance of linkages within networks and are able to identify actors that may not appear to be central to decision-making or have access to power, but exert influence through their connections to those who hold authority and influence decision-making (Borgatti & Foster, 2003).
In multi-level governance structures housing a variety of stakeholders, SNA can be particularly useful in understanding the importance of connectedness to stakeholders’ influence within decision-making structures. It can draw attention to stakeholders within decision-making structures who may not have official access to power and formal avenues of decision-making, but may still influence decision-making nonetheless. In this way, SNA can help to explain why certain outcomes emerge from decision-making structures inhabited by networks of stakeholders that deviate from those predicted by decision-trees and official spheres of decision-making power. Governance networks can be found in a number of policy areas, such as environmental policy and regulatory frameworks for innovative technologies. Both policy areas require the cooperation of multiple stakeholders—namely industry, civil society, and government working together, exerting influence through their connections to each other and not necessarily through their individual attributes or authoritative positions within decision-making structures. SNA is a useful way of understanding outcomes from networks of decision-making with multiple stakeholders and multiple levels of interaction. It shows how stakeholders navigate rules and structures by laying out the actors, their attributes, who they interact with and how often. It thus provides an empirical way of mapping out interactions and determining what kind of information is necessary to explore whether complexity exists within a decision-making system.
Kurtosis analysis is a statistical test that measures the distribution of a random variable.2 Kurtosis analysis measures the ‘peakedness’ of the probability distribution of cases. It essentially refers to the extent to which a distribution of cases is flatter (more spread out from the mean) or more peaked (more than the expected number of cases crowding the mean) than normally found on a distribution curve. Kurtosis is the best single measure available for comparative distributional analysis of random variables (Robinson, 2011). The value of a kurtosis test indicates whether a distribution curve is normal, leptokurtic, or platykurtic. Statistical programs such as STATA or SAS use the value of 3 to represent a normal distribution when conducting the test for kurtosis. If the value is greater than the normal kurtosis value (>3), it is considered leptokurtic. If the value is less than the normal kurtosis value (<3), it is considered platykurtic.
Leptokurtosis is associated with distributions that are simultaneously peaked and have fat tails in comparison with a normal distribution curve (the ‘bell curve’). Leptokurtic values demonstrate a higher-than-expected number of cases in the outer edges of the distribution curve in comparison with the normal distribution curve. This implies that cases are crowded at certain points deviating from a normal distribution curve. Platykurtosis is often associated with case distributions that are simultaneously ‘less peaked’ and have ‘thinner tails’ than normal distributions. The distribution of cases is much flatter than a normal distribution curve. Leptokurtosis and platykurtosis indicate a distribution of cases that deviate from the normal distribution curve. Using this metric can reveal instances where a variable such as a system output deviates from expected distribution patterns based on the distribution of inputs into the system. By decomposing the system, inputs, outputs, and interactions within the system can be identified and can be analyzed for patterns. The mismatch between expected input and output patterns (or lack of a pattern in either inputs and outputs) may be an indication of complexity.
Kurtosis analysis has been used to investigate the behavior of stock market returns (Bai, Russell, & Tiao, 2001; Brounen, Porras Prado, & Stevenson, 2008; Kim & White, 2003) as well as in studies of budgetary processes in a number of countries (Baumgartner, Foucault, & François, 2006; Robinson, 2011). One of the most illustrative cases of how kurtosis analysis has been used to uncover complexity within governance frameworks is Jones and Baumgartner’s 2005 study of US policy making. Jones and Baumgartner’s study of 50 years of US policy-making uses kurtosis to determine whether outcomes (policy) occurred at a predictable, incremental rate or if there were periods where outcomes did not follow a predicted pattern of distribution. The focus of their study was to better understand how policymakers manage the stream of information from various sources, how decision-making processes function, and how issues are prioritized within decision-making systems. Overall, they examine how the decision-making system solves—or fails to solve—problems as opposed to focusing on how individual preferences or attributes are realized through policy action. Jones and Baumgartner found through their analysis of information inflows and agenda setting that policy outcomes over a 50-year period were huddled and bunched up during certain periods of time, which did not correspond to the distributional patterns of policy inputs. This implies that the average number of outcomes during the specified period of time did not change at a predicted, incremental rate. There were more unexplained outliers from the normal distribution curve than predicted. This is what they call a ‘punctuated equilibrium,’ referring to instances where outputs reflect a few moderate and some extreme decisions, which Baumgartner et al. (2009, p. 607) classify as ‘catch up adjustments.’ The ‘stickiness’ of institutional cultures, as well as vested interests and the bounded rationality of individual decision-makers, can collectively, but in varying ways, have significant implications on the distribution of policy outcomes emerging from the policy-making environment over time. Despite the sifting of information and issue prioritization that occurs at the front-end of the decision-making system, the distribution of policy outcomes over time did not adhere to a normal distribution pattern.
What Jones and Baumgartner show by using kurtosis tests to demonstrate the non-incremental distribution of policy outputs is that there are multiple, non-linear interactions and exchanges of information occurring within the decision-making system that influence how and when policies come to fruition and what shape they take. These interactions are not always observable and cannot always be detected based on knowing and understanding the rules of the system or on the individual attributes of actors. Complexity exists within complicated decision-making systems and performing kurtosis analysis helps to explain why, in some cases, policy outcomes deviate from predicted patterns of distribution.
The ideas of complexity and decomposability are useful ways of explaining how informal interactions within decision-making systems can impact outcomes. Both SNA and kurtosis analysis provide empirical ways of examining decision-making systems to determine where to look to investigate if complexity is present. The richness, integrity, and availability of the data are paramount to the usefulness of these empirical tests. They hold significant promise for scholars seeking to identify where complexity could be present within a decision-making system and what may contribute to its existence.
The Case of Canadian System of Approvals for Plants with Novel Traits
Uncovering complexity within decision-making systems for innovative technologies like bioproducts and crops presents unique challenges to scholars because of the role science, uncertainty, and industry play in policy decisions. The regulatory decision-making structures for bioproducts and crops in North America are heavily based on pre-existing risk-assessment procedures developed to scientifically assess the safety of more established technologies such as petrochemicals. Since bioproducts and crops use technologies that carry a degree of uncertainty pertaining to future risk, there is an information gap in these methods of assessment. Risk-assessment procedures cannot assess unknown risks, but biosafety protocols can rigorously evaluate the known risks associated with the bioproduct so that it can be commercialized for stakeholders who want to benefit from its innovations. Multiple stakeholders engage in ongoing exchanges of information and work together within decision-making structures to meet biosafety standards to commercialize bioproducts. These factors add layers of complication to decision-making structures for bioproducts and crops, as interactions are not necessarily linear or outlined in detail in formal decision-making procedures. The VALGEN Regulation and Governance research team chose to look at the PNT approvals system in Canada to test whether SNA and kurtosis analysis can be useful ways of identifying complexity within decision-making structures for bioproducts and crops. This section discusses how we gathered and evaluated qualitative and quantitative data sources to perform SNA and kurtosis analysis.
Important sources of qualitative data regarding decision-making systems are policies, policy frameworks, and the role of actors within the system. The Canadian regulatory system for the unconfined environmental release of PNTs can be characterized as a complex decision-making system that has formalized operating procedures populated by agencies sharing jurisdiction over PNT approvals. One of the core principles of the regulatory system is that a ‘novel’ plant (all plants that undergo some form of modification that have not previously been used in Canada could be considered novel) must be put through a rigorous set of risk-assessment and biosafety procedures before it is declared ‘safe’ for unconfined environmental release and then can be commercialized. Novelty is not exclusive to genetically modified plants, but all plants that undergo some form of modification that has not previously been used in Canada are considered novel (e.g., mutagenesis, cell fusion, and traditional breeding). Twenty-two of the approved PNTs are non-living modified organisms (using conventional breeding, or mutagenesis, but not using what the Cartagena Protocol on Biosafety [Secretariat of the Convention on Biological Diversity, 2000] defines as ‘modern biotechnology’). The PNT moves through the regulatory system by meeting certain biosafety protocols. Biosafety testing is based on scientific information and appropriate data relative to the environmental risk of the PNT compared to its counterparts established in the Canadian environment. Testing must show that the PNT, once released, is not able to comingle with native species of a similar genotype (gene flow) or become a weed (weediness), must have its pest potential tested, must not be toxic to humans and/or animals, and must not pose a threat to biodiversity (Canadian Food Inspection Agency [CFIA], 2009). Figure 1 is a simplified illustration of the regulatory process for PNTs.
Figure 1. Regulatory pathways for PNTs.
Sources: Compiled by authors from Bean (2011) and CFIA-PBO (2008).
Figure 1 is an idealized version of the chain of decision-making. It shows that each specified agency (CFIA, Health Canada [HC], and the Pest Management Regulation Agency [PMRA]) has a role in the decision-making process. It also shows at what point in the decision-making process each agency holds jurisdiction. Each stage of the regulatory system is dictated by regulations set out in Directives (DIR). The series of Directives dictate the requirements a pending PNT must meet in order to be approved for unconfined environmental release without explicitly discussing the method used to meet those requirements. The Directives are vital parts of Canada’s product-based safety evaluation system for PNTs. Figure 1 shows that there are three possible further sets of regulatory guidelines that PNTs going through the approval process must follow, depending upon their intended end use, what type of PNT is being approved, and if it is to be approved for human consumption.
Figure 1 outlines the major stages in the approval process, but there are nuances that are part of the regulatory system not captured in the simplified diagram. It does not indicate, for instance, at what stage the proponent interacts with the agencies, or how the agencies interact with one another, or typically how long it takes for a PNT to move between stages. To fill in the information gaps, the VALGEN research team constructed a list of questions about the decision-making system used in interviews with several regulators in the Canadian regulatory system for PNTs from the Plant Biosafety Office (PBO), CFIA, and Agriculture and Agri-Food Canada (AAFC). The interviewer asked questions regarding how the regulatory process works (in terms of the stages of the process) and how long each stage typically takes. The interviewer also asked questions about how the agencies interact with each other and with the proponents throughout the regulatory process. After analyzing the responses from regulators, the VALGEN research team used Figure 1 as a starting point to further decompose the regulatory system; the team then pieced together publicly available flow charts and information regarding interactions in PNT Decision Documents to create a more detailed decision-making flowchart (Figure 2).3 The flowchart was then sent back to regulators for feedback. Regulators in the PBO then evaluated the decision-making pathway the VALGEN team constructed, and suggested corrections regarding interactions and stages where appropriate. Figure 2 shows the detailed decision-making flow chart constructed by the VALGEN research team incorporating the regulator’s editorial comments.
Figure 2. Regulatory pathway for PNTs (including optional steps).
Source: Compiled by authors.
This version of how the regulatory system works is much more detailed than Figure 1. The shading of text boxes indicates that they are optional—meaning they do not necessarily occur during every PNT approval process and are not required to achieve approval in all cases. The dotted lines linking the shaded text boxes indicate the optional interactions that occur between stages of the approval process and actors. Figure 2 shows that the Plant and Biotechnology Risk Assessment Unit (PBRA), a subagency of the CFIA’s Science Branch in the evaluation of PNTs, engages with the PBO in the evaluation of information submitted by the proponent, yet it is not referred to in other released information from the CFIA. The important role the ‘pre-submission meeting’ plays in initiating the approval process is another stage that was not evident from the flowchart in Figure 1. We learned of this stage in the regulatory process from the interviews with regulators, as it is not included in publicly available descriptions of the approval process. The pre-submission meeting is when proponents meet with regulators before they enter their product into the PNT approval process to discuss informational requirements and other details of the regulatory process. The attendance of the regulators at the pre-submission meeting varies based on the bioproduct in question. Though the pre-submission meeting is considered an optional step, almost all of the proponents engage in this stage with regulators. Figure 2 highlights the interactions between agencies in addition to segments of process through feedback loops. Detailing the processes reveals the complexity of interactions among agencies and where each agency fits into the decision-making framework. It also provides some indication of how agencies engage with the policies and within the process. The ‘optional’ pathways of interaction create a level of difficulty in understanding how the decision-making system works.
Two of the most complete quantitative data sources regarding the PNT approval process in Canada are field trials and PNT approvals. In both cases, the data is publicly available and clearly communicated on the CFIA’s website. Before a PNT reaches the unconfined environmental release stage, data from field trials in confined plots of land must be collected and analyzed. Field trial data can be taken from field trials in Canada or from other jurisdictions if the data can be proven to be valid for the Canadian environment. Since 1988, 37 different crops with a variety of traits (predominantly herbicide tolerance) have been field tested in Canada including wheat, tomato, squash, sunflower, potato, lentil, flax, corn, soybean, and alfalfa. Not all confined field trials have resulted in PNT approvals. There are many reasons why field trialed PNTs are not considered in the approval stage. In some cases, they are deemed economically unviable; CFIA poses questions to the proponent that cannot be answered during the application process regarding weediness or outcrossing, and some PNTs are simply never commercialized. Most companies take 5 to 10 candidates through the first few years of trials before deciding which to commercialize. PNTs that do not perform during field trials as desired are abandoned and the application for approval is halted. There are no publicly available data on how many PNTs enter the approval process that do not proceed to the final approval stage, or at what stage applications are dropped. Figure 3 shows the number of field trials (in hundreds) and the number of PNT approvals by year. There have been 8,803 confined field trials in Canada from 1988 to 2010 evaluated for environmental release, livestock feed safety, and food safety approval. 2009 had the highest number of field trials (817).
Figure 3. Number of PNT field trials (in hundreds) and approvals (1988-2010).
Source: Compiled by authors using CFIA (2011) data.
As seen in Figure 3, there is an increase in field trials from 1988 until 1994, then a brief dip. The number of field trials then increases above 1994 levels in 2009. The distribution of PNT approvals between 1994 and 2010 also exhibits a lack of pattern in distribution over time. Approvals gradually increased for three years until they peaked at 13 in 1997, at which point the number of approvals per year stayed consistent from 1998 to 2010. We would expect to see some type of pattern of field trials over time, as well as a pattern of approvals. Between 1995 and 2010, the average time between the year of first field trial data considered in the approval and the approval year for all PNTs is 3.7 years. With that in mind, what we would predict the distribution table to reveal is an approximate 4-year time lag between field trials and approvals. There does not appear to be an identifiable pattern of distribution of field trials or approval cases over time in Figure 3. The year with the highest number of approvals (1997) proceeds the year with the highest number of field trials (2009) by more than a decade.
There may be many reasons why field trials and approvals vary from year to year. For example, policy change at the national level could have an impact on inputs and outputs. The passing of Part V of the Canada Seeds Regulations in 1996 pertaining to the release of seed (which required proponents to provide the CFIA with more information), or the EU moratorium on GMOs established in 1999 and lasting until 2003, could have influenced the pattern of field trials. But for the purpose of this article, we are not attempting to speculate on how externalities affected the number of yearly field trials or approvals in Canada. We are more interested in mapping out the data that are available regarding elements of the regulatory system, and identifying whether the data reflects predicted patterns of distribution over the years. We expected to see some trend or pattern over time—a gradual increase or a gradual decrease as the approval system matures and stakeholders become more familiar with how it works.
A Social Network Analysis of the PNT Approval Process
By compiling data from publicly available decision-making flow charts and PNT Decision Documents (represented in Figure 2), we were able to construct a matrix of agencies involved in the regulatory system and how they interact with each other. We were able to break down the activities of departments and agencies into how subagencies are involved in the decision-making process. Figure 4 is a SNA diagram of maximum interactions (including optional steps) of actors in Canadian regulatory framework for PNTs created using Organizational Risk Analysis (ORA) software.4 Figure 4 includes the interactions necessary to have a PNT approved as a novel food.
Figure 4. Maximum interactions of actors in PNT approvals.
Source: Calculated by authors using ORA software.
The web of interactions shown in Figure 4 is the maximum number of interactions among agencies involved in regulating PNTs in Canada. This includes all necessary steps, optional steps, and Directive changes expressed in Figure 2. This SNA is an idealized visualization of interactions between actors, as interactions are dependent on the specific PNT being approved and whether it is seeking approval as a novel food. Many of the interactions (indicated by the lines between nodes) are bidirectional, meaning communications and information are initiated and exchanged from both directions. There are two linkages, however, that are not bidirectional. The link between the Novel Foods Section (NFS) and the Food Rulings Committee, and the Food Rulings Committee and the proponent (the applicant seeking approval for the PNT) are not bidirectional linkages. Based on the reading of Directives and evaluating the information about the regulatory system on the CFIA website, it would be expected that the PBO has the highest rank of total degree of centrality because the PBO (housed under the CFIA) is generally understood to be the central agency involved in designing the regulation and the approvals of PNTs. Though it may appear from Figure 4 that the NFS is the most connected to other actors in the network, this does not necessarily mean it is the most powerful.
From the available node-level measures produced by ORA, degree of centrality, betweenness, and the eigenvector reveal some interesting information about actors in the PNT approval network, which is displayed in Table 1. The arrows indicate which agency/department’s value is above the mean for the entire network. For all three measures, the NFS has the highest value. This indicates that the NFS is very interactive within the network. The PBO has the second-highest rank in terms of degree of centrality and eigenvector, but has a below-average value for the betweenness metric, which indicates the probability of the shortest path between two actors. Though the PBO may be interconnected within the network, its links are not as short as those between the NFS and those of the proponent. The eigenvector value indicates that the NFS has the most influence within the network. It has many linkages to higher ranked actors within the network, which contributes to its first-place ranking for this metric.
Table 1. Centrality measures for PNT approval network.
In all three of these metrics, the NFS ranks the highest. The NFS is housed under Health Canada and works in conjunction with the Bureau of Chemistry Safety and the Bureau of Nutrition Sciences to assess the safety of novel foods. It takes part in assessments and testing of novel foods and also develops policy regarding safety assessments of novel foods. Examples of genetically modified novel foods approved in Canada are corn and soybean. A PNT that cannot be used as a food or as a feed would not have to engage with the NFS. SNA outputs of the maximum interactions in the approval of a PNT show that some actors within the system who are not officially recognized as power brokers may have influence on decision-making. The regulatory system of approval for PNTs in Canada contains a very small network of actors. The lack of availability of qualitative information to attach to the actors or the links between them must be taken into consideration when interpreting the centrality measures results. Network hubs may hold different kinds of access to information within networks. Decision-making pathways are not always transparent from publicly available information or formalized decision-making charts. What the SNA of the Canadian system of approval for PNTs shows is that while informal interactions in the decision-making process are not always identifiable from official decision-making flow charts, they have the potential to impact the regulatory process. More information regarding the interactions between regulators and proponents during the approval process (in terms of when the interactions happen) would add an important layer of reliability to the outputs from the ORA software. This SNA provides a useful demonstration of how the empirical tool can be used in other regulatory contexts and what types of data are necessary to conduct an SNA of a regulatory system for bioproducts and crops.
The PNT regulatory case is a small network that has large information holes in terms of interactions. There was quite a bit of information that could not be extracted during the interview process, as regulators are not at liberty to discuss the content of the interactions with proponents due to the confidentiality of business information, which made data gathering difficult. It is important, however, to capture as many interactions as possible in the regulatory system so as to maximize transparency and comprehension of the regulatory system for stakeholders. It is very difficult to modify a regulatory system for innovative technologies that requires flexibility to navigate the regulatory challenges ahead without adequate information; what is not identified or recorded cannot be improved upon. Thus, results in this case must be interpreted with caution. The results from SNA reveal that the more connections an actor has does not necessarily indicate that it is more powerful. Powerful actors, then, do not always have to be connected with the most actors in the network.
Kurtosis Analysis of the PNT Approval Process
The test for kurtosis is used to measure the distribution of outcomes to diagnose how inputs are filtered through decision-making systems. In the case of the regulatory system for PNTs, field-trial data was used as the input. Publicly available field-trial data includes crop type, year field trial was planted, province where field trial was located, the traits of the plant being field-trialed, and the proponent. There are many challenges with using field-trial data as the input into the regulatory system. Field trial data is not an ideal data source, since multiple field trials take place simultaneously and not all field trials produce data used in risk assessment. Further, not all crops that are field-trialed become approved PNTs, and field-trial data from outside of Canada is sometimes used in risk assessments. Some field trials are abandoned during the approval process, but information regarding the crops that get pulled out of the regulatory process is not publicly available. There are other variables that would have been more appropriate to use as an input for kurtosis analysis, i.e., the pre-submission meetings. Unfortunately, details regarding the pre-submission meetings are not publicly available, thus the reason for relying on field-trial data. For the outputs, we chose to use the PNT approvals since these are the last stage of the regulatory process. The public data available regarding PNT approvals includes crop type; traits; proponent; date of approval; and CFIA, Health Canada, and Environmental Canada’s individual approval decisions. This provides enough information to input into a statistical program and run the kurtosis test.
A useful way of graphically representing the outputs from kurtosis analysis is the non-parametric kernel density estimation (KDE). The KDE is use for visual comparison with the theoretical probability density function. It describes the relative likelihood for probability distribution of a random variable at any given point. The KDE is used in this case as an exploratory tool to fit empirical probable density for both the field trials data (taken from Figure 3). The results below are superimposed with a standard normal density curve based on the mean and variance of field trial and approval data. This shows a general picture of how far the empirical distribution deviates from the normal distribution curve. Figure 5 is the graphical representation of kurtosis analysis on the field-trial data as expressed by KDE.
Figure 5. Average proportion of total field trials, 1988-2010.
Source: Calculated by authors using STATA software.
The kurtosis analysis of total field trials reveals that the actual density of yearly distributions of field trials (blue line) into the system closely adheres to the estimated density of yearly distributions of field trials (red line). The total number of field trials over time (1988 to 2010) has a kurtosis value of 2.1. This value is lower than the normal kurtosis value of 3, but it indicates that despite being slightly platykurtic, the distribution of cases is relatively normal. This indicates that the number of field trials were distributed from 1988 to 2010 in Canada occurred at a relatively incremental rate, which increased over time as expected. Despite how the distribution of the number of field trials appears in Figure 3, the inputs into the system have a relatively normal distribution.
If the distribution of outcomes (PNT approvals) from a decision-making system reflects the normal distribution observed among inputs, we would expect to see the distribution of outputs from the regulatory system for PNTs to adhere to the normal distribution curve. Figure 6 is the graphical representation of kurtosis analysis on approvals data as expressed by KDE. The red line in Figure 6 represents the estimated density of yearly distributions of PNT approvals. The blue line is the actual density of yearly distributions of PNT approvals. The blue line indicates a clustering (peak) of approvals at 4.3 in some years, and lower than expected rates of approval in others at the 11 point. The kurtosis value for the gross total (an aggregate of all crop types by year) PNT approvals is 4.8. The distribution curve is leptokurtic, implying that the distribution has a high peak, weak shoulders and longer tails compared to a normal distribution curve. What we find instead of a gradual, evenly distributed population of cases is clustering in the distribution of outputs—a distribution curve with a high peak and more than normal cases of outliers. The distribution of approvals between 1994 and 2010 does not follow a normal distribution curve, and does not demonstrate a normal distribution as generally found in the distribution of field trials over time.
Figure 6. Average proportion of total approvals, 1995-2010.
Source: Calculated by authors using STATA software.
The kurtosis analysis reveals that the inputs (the field trials) into the regulatory system follow a predicted distribution, yet approvals are not distributed as predicted. The leptokurtic distribution curve indicates that complexity may be at work at some point in the regulatory system, as we see a punctuated equilibrium in the distribution of cases rather than incrementalism. This may indicate that at some point within the regulatory system, interactions not reflected in standard operating procedures may be occurring that are influencing the density and distribution of outcomes over time. Factors that influence outcomes that are not explicitly covered in available data makes it difficult for stakeholders outside of the system to comprehend what is happening within the system. Activities that are not easily identifiable in formalized standard operating procedures are very difficult to correct and make the decision-making system hard for stakeholders to navigate and understand. This could contribute to inefficient use of resources within the system and may contribute to a slowed pace of outputs.
As for the results of the kurtosis tests, caution must be exercised as well. Confidence in the kurtosis outputs could be increased with a larger dataset of inputs and outputs, as well as finding a more appropriate input, such as details regarding the pre-submission meetings. Yet, the confidentiality of business information restricts full information about applications that have been ceased. The small source population of approvals also adds to the caution in interpreting the kurtosis results. A larger dataset would add confidence to these results.
Formal decision-making trees for science-intensive regulatory systems have difficultly capturing the social dynamics of the system. Informal interactions and small, seemingly unimportant, changes within the system can have significant influence on outcomes. Decomposability provides a method of breaking down decision-making systems without breaking down the linkages between the subsystem components, revealing a lot of information about the inner workings of the system. Understanding these interactions can help to identify important points in the system where complexity may arise. This article has explored ways of using empirical tools to better understand how regulatory systems for bioproducts and crops work and ways of locating where complexity may be generated. There are several key findings from experimenting with empirical tools of SNA and kurtosis analysis to identify complexity within regulatory systems for bioproducts.
To have an effective regulatory system for bioproducts and to move towards reaching some consensus among multiple regulatory frameworks, we must understand how decision-making systems work at the national level and why they do not always work as stakeholders anticipate. Identifying whether complexity is present within regulatory systems can help reveal neglected or under-emphasized interactions and contribute to efforts to make the regulatory system for bioproducts more transparent. Yet while complexity is discussed and analyzed by academics in a myriad of ways, it remains difficult to operationalize and to identify within systems. Complexity lacks a universal definition or standard operationalizing methodology, which contributes to imprecision in investigating its existence and causes. Yet, the concept of complexity is useful in capturing an important aspect that inhabits the tangled web of decision-making within governance structures. Often real-time decision-making processes are not linear and it is not always easy to identify which actor is involved, or the extent of their involvement at certain points in the system.
This article shows that an effective way of getting at complexity within the ‘black box’ of governance is by decomposing decision-making systems. Decomposing systems into interrelated subcomponents helps to identify key institutions, agents, and interactions of power and influence. It also is a useful way of mapping out how policy is linked together, and how agents interact with institutions and policies. Decomposing decision-making systems is a functional way of identifying appropriate qualitative and quantitative data sources necessary to conduct further empirical tests.
The most significant conclusion of this article is the importance of information to understanding the workings of a regulatory system for bioproducts and crops. The more complete the information is, the more reliable the results from empirical tools. To conduct both SNA and kurtosis analysis, we had to construct an ideal model with maximum interactions between agents involved in the decision-making system because not enough information is available regarding the actual interactions that take place within the regulatory system. What the idealized vision of interactions, inputs, and outputs shows is that even in an idealized model, there is difficulty in identifying patterns of distributions and interactions. Interviews with regulators about their role in the regulatory process, and using a number of qualitative sources to piece together interactions and outputs from decision-making systems (in this case Decision Documents), can reveal nuances in decision-making structures not captured in publicly available representations. Interviews with applicants that have tried to navigate their way through the system would help with filling information holes. Taking a case study of a particular crop and comparing it across jurisdictions may reveal some important insights regarding the effectiveness and efficiency of decision-making systems. Overall, this article has demonstrated that both SNA and kurtosis analysis are useful in understanding how decision-making systems work and they can be applied in many contexts beyond that of Canada. Linkages that are not always reflected in flowcharts and standard operating procedures are important sources of information regarding how decisions are made around the commercialization of bioproducts and crops, as well as the actors involved in making them. Identifying these linkages is crucial to developing more effective governance structures for bioproducts and crops.
1 The Value Addition Through Genomics and GE3LS (VALGEN) research project’s Regulation and Governance team members are located at the University of Saskatchewan.
2 A ‘random variable’ means that the value of the variable is not fixed. A random variable’s value is varied due to change (i.e., a roll of the dice).
3 Each approved PNT has a publicly available Decision Document describing the approval process for the PNT. These Decision Documents are written and issued by the original applicants (proponents).
4 ORA software is freeware available through Carnegie Mellon’s Computational Analysis of Social and Organizational Systems (CASOS; http://www.casos.cs.cmu.edu/projects/ora/software.php). ORA software produces visual representations and centrality measure statistics.
Amaral, L.A.N., & Ottino, J.M. (2004). Complex networks: Augmenting the framework for the study of complex systems. European Physical Journal, B38, 147-162.
Anderson, P. (1999). Complexity theory and organization science. Organization Science, 10(3), 216-232.
Bai, X., Russell, J., & Tiao, G. (2001). Kurtosis of GARCH and stochastic volatility models with non-normal innovations (Working Paper). Chicago, IL: University of Chicago.
Baumgartner, F., Breunig, C., Green-Pedersen, C., Jones, B.D., Mortensen, P.B., Nuytemans, M., et al. (2009). Punctuated equilibrium in perspective. American Journal of Political Science, 53(3), 603-620.
Baumgartner, F., Foucault, M., & François, A. (2006, April 25-30). Punctuated equilibrium in French budgeting processes. Paper prepared for the workshop on ‘The comparative dynamics of problem framing: How science and power speak to each other,’ European Consortium for Political Research (ECPR) Joint Sessions, Nicosia, Cyprus.
Bean, J. (2011, October 19). Regulatory oversight of plants with novel traits. Paper prepared for the AgBio 3D (Ontario Genomics) conference, Guelph, Ontario.
Black, J. (2002). Critical reflections on regulation. Australian Journal of Legal Philosophy, 27, 1-36.
Borgatti S.P., & Foster P.C. (2003). The network paradigm in organizational research: A review and typology. Journal of Management, 29(6), 991-1013.
Borgatti, S.P., Mehra, A., Brass, D.J., & Labianca, G. (2009). Network analysis in the social sciences. Science, 323, 892-895.
Brounen, D., Porras Prado, M., & Stevenson, S. (2008). Kurtosis and consequences: The case of international property shares (Working Paper).
Brunk, C., Haworth, L., & Lee, B. (1995). Value assumptions in risk assessment: A case study of the Alacholr controversy. Waterloo, Ontario: Wilfred Laurier Press.
Canadian Food Inspection Agency (CFIA). (2011). PNTs—Approved confined research field trials: Terms and conditions. Ottawa, Ontario: Author. Available on the World Wide Web: http://www.inspection.gc.ca/plants/plants-with-novel-traits/approved-under-review/field-trials/eng/1313872595333/1313873672306.
CFIA. (2009). Directive 2009-09: Plants with novel traits regulated under part V of the seeds regulations: Guidelines for determining when to notify the CFIA. Appendix 2. Ottawa, Ontario: Author. Available on the World Wide Web: http://www.inspection.gc.ca/plants/plants-with-novel-traits/applicants/directive-2009-09/appendix-2/eng/1304467835325/1304467914847.
CFIA, & Plant Biosafety Office (PBO). (2008). The regulation of plants with novel traits in Canada. Ottawa, Ontario: Author. Available on the World Wide Web: http://www.inspection.gc.ca/english/plaveg/bio/pntchae.shtml.
Castells, M. (2000). The rise of the network society. New York: Wiley-Blackwell.
Funtowicz, S., & Ravetz, J.R. (1994). Emergent complex systems. Futures, 26.6, 568-582.
Giddens, A. (1990). The consequences of modernity. Cambridge: Polity Press.
Hoppe, R. (2011). The governance of problems: Puzzling, powering and participation. London: The Policy Press.
Jasanoff, S. (2005). Designs on nature: Science and democracy in Europe and the United States. Princeton, NJ: Princeton University Press.
Jones, B.D., & Baumgartner, F. (2005). The politics of attention: How governments prioritize problems. Chicago: University of Chicago.
Kim, T.H., & White, H. (2003, September). On more robust estimation of skewness and kurtosis: Simulation and application to the S&P500 Index. San Diego: University of California-San Diego, Department of Economics. Available on the World Wide Web: http://escholarship.org/uc/item/7b52v07p (accessed 5 October 2011).
Kooiman, J. (2003). Governing as governance. London: Sage.
Majone, G. (1995). The rise of the regulatory state in Europe. West European Politics, 17, 77-101.
Marin, A., & Wellman, B. (2010). Handbook of social network analysis. London: Sage.
Mills, L.N. (2002). Science and social context: The regulation of recombinant bovine growth hormone in North America. Toronto: McGill-Queen’s Press.
Robinson, R. (2013). Punctuated equilibrium and constitutional change: A quantitative analysis of precedential fluidity. Policy Studies Journal, Forthcoming.
Ryan, C.D. (2008). Evaluating performance of research networks: A socioeconomic framework for assessing funded research projects. Verlag, Germany: VDM.
Secretariat of the Convention on Biological Diversity. (2000). Cartagena Protocol on Biosafety to the Convention on Biological Diversity: Text and annexes. Montreal, Canada.
Simon, H.A. (1962). The architecture of complexity. Proceedings of the American Philosophical Society, 106(6), 467-482.
Stoker, G. (1998). Governance as a theory: Five propositions. International Social Science Journal, 155, 17-28.
Sundlof, S. (2000). The role of science in regulatory decision-making. AgBioForum, 3(2&3), 137-140. Available on the World Wide Web: http://www.agbioforum.org.
Tait, J., & Levidow, L. (1992). Proactive and reactive approaches to risk regulation: The case of biotechnology. Futures, 24, 219-231.
Contributors to the research discussed in this article are VALGEN research assistants Jaime Leonard and Nyankomo Marwa. Camille D. Ryan also provided invaluable help with the SNA component of this article. Thank you to Neil Hibbert, Bruce Doern, and John de Jong for comments on earlier drafts of this article.
Suggested citation: Clark, L.F., & Phillips, P.W.B. (2013). Bioproduct approval regulation: An analysis of front-line governance complexity. AgBioForum, 16(2), 112-125. Available on the World Wide Web: http://www.agbioforum.org.
In This ArticleFigure 1
|© 2013 AgBioForum | Design and support provided by Express Academic Services | Contact ABF: firstname.lastname@example.org|