<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<TEI xmlns="http://www.tei-c.org/ns/1.0">
  <teiHeader>
    <fileDesc>
      <titleStmt>
        <title type="main" level="a">A Natural Language Processing approach to measure expertise in the Delphi-based scenarios</title>
        <author>
          <persName n="1" ref="https://orcid.org/0000-0002-0190-6061" type="ORCID">
            <forename>Yuri</forename>
            <surname>Calleo</surname>
            <placeName type="affiliation">University College Dublin, Italy</placeName>
          </persName>
          <persName n="2" ref="https://orcid.org/0000-0002-9139-1451" type="ORCID">
            <forename>Simone</forename>
            <surname>Di Zio</surname>
            <placeName type="affiliation">University of Chieti-Pescara G. D'Annunzio, Italy</placeName>
          </persName>
          <persName n="3" ref="https://orcid.org/0000-0002-1535-1239" type="ORCID">
            <forename>Francesco</forename>
            <surname>Pilla</surname>
            <placeName type="affiliation">University College Dublin, Ireland</placeName>
          </persName>
        </author>
        <respStmt>
          <resp>This is a section of <title>ASA 2022 Data-Driven Decision Making</title>(DOI: <idno type="DOI">10.36253/979-12-215-0106-3</idno>) by </resp>
          <name>Enrico di Bella, Luigi Fabbris, Corrado Lagazio</name>
        </respStmt>
      </titleStmt>
      <publicationStmt>
        <publisher>Firenze University Press</publisher>
        <pubPlace>Firenze</pubPlace>
        <date when="2023">2023</date>
        <idno type="DOI">https://doi.org/10.36253/979-12-215-0106-3.29</idno>
        <availability>
          <p>Available for academic research purposes</p>
          <p>Open Access</p>
          <p>Copyright Author(s)</p>
          <licence source="text" target="https://creativecommons.org/licenses/by/4.0/legalcode">
            <p>Content licence CC BY 4.0</p>
          </licence>
          <licence source="metadata" target="https://creativecommons.org/publicdomain/zero/1.0/legalcode">
            <p>Metadata licence CC0 1.0</p>
          </licence>
        </availability>
      </publicationStmt>
      <sourceDesc>
        <p>This is original content, published for academic research purposes</p>
      </sourceDesc>
    </fileDesc>
    <encodingDesc>
      <appInfo>
        <application version="2.2" ident="Booksflow">
          <desc>Digital edition XML powered by Booksflow</desc>
        </application>
      </appInfo>
    </encodingDesc>
    <profileDesc>
      <abstract xml:lang="en">
        <p>The Delphi-based scenarios (DBS) development implies the assumption of different choices, through the gathering of information and the assessment of alternative resolutions (Panpatte and Takale, 2019). During the last decades, the spread of environmental hazards has increased quickly, much to request different responses in order to develop a sustainable future for humanity planning the present (McMichael and Lindgren, 2011). Since the DBS is a creative process (Nowack et al., 2011), different figures are selected to make choices, including academics, stakeholders and citizens. However, one of the main challenges remains the measurement of expertise, in fact, during the process, the experts should be assessed based on their competences in order to avoid any conflict in the final results and, eventually, weigh their answers. In recent years, some contributions adopted the self-assessments for the experts’ evaluation (Sossa et al., 2019), but many issues still remain (such as strong subjectivity and cognitive biases which produce over or underestimation). We develop a new method to estimate the expertise by using Natural Language Processing to acquire information, extracting the contributions of experts in each topic. First, starting from a draft list of selected experts, we identify the category of reference (e.g., academia, industry, local authority, citizens etc.). We build a data repository with the personal pages (URLs) of each expert to then use Python to extract from the URLs, the number of contributions related to a keyword, different for each category (e.g., publications for academics, reports and projects for stakeholders etc.). Finally, we proceed adopting a coefficient of production with a weighted sum of the results. To practically demonstrate our approach, we applied this method to a cohort of known experts, part of the “Smart control of the climate resilience” (SCORE) H2020 European project to estimate their expertise in specific areas.</p>
      </abstract>
      <textClass>
        <keywords>
          <list>
            <item>future scenarios</item>
            <item>Delphi</item>
            <item>expertise assessment</item>
          </list>
        </keywords>
      </textClass>
    </profileDesc>
  </teiHeader>
  <text>
    <body>
      <p>It is available online at https://doi.org/10.36253/979-12-215-0106-3.29<ref target="https://doi.org/10.36253/979-12-215-0106-3.29" /></p>
      <div>
        <listBibl>
          <head>References</head>
          <bibl n="112013">
            <bibl>Barroso O. J. M., &amp;amp; Cabero A. J. (2013), La utilizaci&amp;#243;n del juicio de experto para la evaluaci&amp;#243;n de TIC: el coeficiente de competencia experta. Bord&amp;#243;n. Revista de Pedagog&amp;#237;a, 65 (2), pp. 25-38.</bibl>
            <idno type="DOI">10.13042/brp.2013.65202.</idno>
          </bibl>
          <bibl n="112014">Bishop P., Hines A., Collins T. (2007), The current state of scenario development: An overview of techniques, Foresight, 9(1), pp. 5–25.</bibl>
          <bibl n="112015">Bonaccorsi A., Apreda R., Fantoni G. (2020), Expert biases in technology foresight. Why they are a problem and how to mitigate them, Technological Forecasting &amp;amp; Social Change, 151, pp. 1–17.</bibl>
          <bibl n="112016">
            <bibl>Dalkey N., &amp;amp; Helmer O. (1963), An experimental application of the Delphi method to the use of experts. Management science, 9(3), pp. 458-467.</bibl>
            <idno type="DOI">10.1287/mnsc.9.3.458.</idno>
          </bibl>
          <bibl n="112017">
            <bibl>Di Zio S., Bolzan M., Marozzi M. (2021), Classification of Delphi outputs through robust ranking and fuzzy clustering for Delphi-based scenarios, Technological Forecasting &amp;amp; Social Change, 173, pp. 1211400040.</bibl>
            <idno type="DOI">10.1016/j.techfore.2021.121140</idno>
          </bibl>
          <bibl n="112018">
            <bibl>Di Zio S., Rosas, J. D. C., &amp;amp; Lamelza, L. (2017), Real Time Spatial Delphi: Fast convergence of experts&amp;#39; opinions on the territory. Technological Forecasting and Social Change, 115, pp. 143-154.</bibl>
            <idno type="DOI">10.1016/j.techfore.2016.09.029.</idno>
          </bibl>
          <bibl n="112019">Gary J. E., &amp;amp; Heiko A. (2015), The future of foresight professionals: Results from a global Delphi study. Futures, 71, pp. 132-145.</bibl>
          <bibl n="112020">Gordon T. J. (1994). The delphi method. Futures research methodology, 2(3), pp. 1-30.</bibl>
          <bibl n="112021">
            <bibl>Gorn L., Kleemann J., &amp;amp; F&amp;#252;rst, C. (2018), Improving the Matrix-Assessment of Ecosystem Services Provision—The Case of Regional Land Use Planning under Climate Change in the Region of Halle, Germany.&amp;#160;Land,&amp;#160;7(2), pp. 76.</bibl>
            <idno type="DOI">10.3390/land7020076.</idno>
          </bibl>
          <bibl n="112022">Kosow H., Ga&amp;#223;ner R. (2008), Methods of future and scenario analysis: overview, assessment, and selection criteria, Bonn, Deutsches Institut f&amp;#252;r Entwicklungspolitik.</bibl>
          <bibl n="112023">Mullen P. (2003), Delphi: myths and reality, Journal of Health Organisation and Management, 17 (1), pp. 37–52.</bibl>
          <bibl n="112024">Nair V. G. (2014), Getting started with beautiful soup. Packt Publishing Ltd.</bibl>
          <bibl n="112025">Nowack M., Endrikat J., &amp;amp; Guenther E. (2011), Review of Delphi-based scenario studies: Quality and design considerations. Technological Forecasting and Social Change, 78(9), pp. 1603-1615.</bibl>
          <bibl n="112026">Rodr&amp;#237;guez P. O. S. (1995), Land use conflicts and planning strategies in urban fringes: a case study of western Caracas, Venezuela.</bibl>
          <bibl n="112027">
            <bibl>Rowe G., &amp;amp; Wright G. (1999), The Delphi technique as a forecasting tool: issues and analysis.&amp;#160;International journal of forecasting,&amp;#160;15(4), pp. 353-375.</bibl>
            <idno type="DOI">10.1016/S0169-2070(99)00018-7.</idno>
          </bibl>
          <bibl n="112028">Saaty T.L. (1980), The Analytic Hierarchy Process. McGraw-Hill, New York.</bibl>
          <bibl n="112029">
            <bibl>Sossa J. W. Z., Halal, W., &amp;amp; Zarta, R. H. (2019), Delphi method: analysis of rounds, stakeholder and statistical indicators.&amp;#160;Foresight.</bibl>
            <idno type="DOI">10.1108/FS-11-2018-0095.</idno>
          </bibl>
          <bibl n="112030">Varho V., Rikkonen P., &amp;amp; Rasi, S. (2016), Futures of distributed small-scale renewable energy in Finland—A Delphi study of the opportunities and obstacles up to 2025. Technological Forecasting and Social Change, 104, pp. 30-37.</bibl>
        </listBibl>
      </div>
    </body>
  </text>
</TEI>