<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<TEI xmlns="http://www.tei-c.org/ns/1.0">
  <teiHeader>
    <fileDesc>
      <titleStmt>
        <title type="main">On the Two-fold Role of Logic Constraints in Deep Learning</title>
        <author>
          <persName n="1" ref="https://orcid.org/0000-0002-6799-1043" type="ORCID">
            <forename>Gabriele</forename>
            <surname>Ciravegna</surname>
            <placeName type="affiliation">Centai Institut, Italy</placeName>
          </persName>
        </author>
      </titleStmt>
      <publicationStmt>
        <publisher>Firenze University Press</publisher>
        <pubPlace>Florence</pubPlace>
        <date when="2025">2025</date>
        <idno type="DOI">https://doi.org/10.36253/979-12-215-0680-8</idno>
        <availability>
          <p>Available for academic research purposes</p>
          <p>Open Access</p>
          <p>Copyright Author(s)</p>
          <licence source="text" target="https://creativecommons.org/licenses/by/4.0/legalcode">
            <p>Content licence CC BY 4.0</p>
          </licence>
          <licence source="metadata" target="https://creativecommons.org/publicdomain/zero/1.0/legalcode">
            <p>Metadata licence CC0 1.0</p>
          </licence>
        </availability>
      </publicationStmt>
      <seriesStmt>
        <title>Premio Tesi di Dottorato Città di Firenze</title>
      </seriesStmt>
      <sourceDesc>
        <bibl type="monograph">
          <edition n="1">Digital edition PDF</edition>
          <date>2025</date>
          <idno type="ISBN" subtype="electronic">979-12-215-0680-8</idno>
          <biblScope unit="page">126 pages</biblScope>
          <extent>0.00 MB</extent>
          <availability status="free">
            <p>This is original content, published in Open Access. It is also available to read for free online at <ref target="https://media.fupress.com/files/pdf/24/15692/44798">https://media.fupress.com/files/pdf/24/15692/44798</ref></p>
          </availability>
        </bibl>
        <bibl type="monograph">
          <edition n="2">Digital edition XML</edition>
          <date>2025</date>
          <idno type="ISBN" subtype="electronic">979-12-215-0681-5</idno>
          <availability status="free">
            <p>It is available to read for free online</p>
          </availability>
        </bibl>
        <bibl type="monograph">
          <edition n="3">Print edition</edition>
          <date>2025</date>
          <idno type="ISBN" subtype="print">979-12-215-0679-2</idno>
          <biblScope unit="page">126 pages</biblScope>
          <availability status="restricted">
            <p>It is available for online purchase at <ref target="https://books.fupress.com/isbn/9791221506808">https://books.fupress.com/isbn/9791221506808</ref></p>
          </availability>
        </bibl>
      </sourceDesc>
    </fileDesc>
    <encodingDesc>
      <appInfo>
        <application version="2.2" ident="Booksflow">
          <desc>Digital edition XML powered by Booksflow</desc>
        </application>
      </appInfo>
    </encodingDesc>
    <profileDesc>
      <creation>
        <tag>peer-reviewed</tag>
        <rs type="FUP_policy" source="https://doi.org/10.36253/fup_best_practice">Firenze University Press Best Practice in Scholarly Publishing</rs>
        <rs type="scientific_cloud" source="https://doi.org/10.36253/fup_best_practice.2">FUP Scientific Cloud for Books</rs>
        <rs type="peer_review" resp="scientific_board" source="https://books.fupress.com/scientific-board/c/153">Premio Tesi di Dottorato Città di Firenze 2023</rs>
      </creation>
      <abstract xml:lang="en">
        <p>Deep Learning (DL) is a branch of Artificial Intelligence (AI) that focuses on training deep neural networks. Thanks to their ability to process large amounts of data, these networks have achieved remarkable results across a variety of fields. Despite these successes, DL still faces several limitations that hinder its adoption in real-world scenarios. This thesis addresses three key challenges: reducing the need for supervision, defending against adversarial attacks, and explaining neural network behavior. The first two challenges are tackled through learning from constraints, which incorporates domain knowledge to guide the learning process and enhance model robustness. The third challenge, on the other hand, is addressed using learning of constraints, which helps identify and formalize logical relationships among learned tasks, thereby providing interpretable explanations of the networks’ behavior.</p>
      </abstract>
      <abstract xml:lang="it">
        <p>Deep Learning (DL) is a branch of Artificial Intelligence (AI) that focuses on training deep neural networks. Thanks to their ability to process large amounts of data, these networks have achieved remarkable results across a variety of fields. Despite these successes, DL still faces several limitations that hinder its adoption in real-world scenarios. This thesis addresses three key challenges: reducing the need for supervision, defending against adversarial attacks, and explaining neural network behavior. The first two challenges are tackled through learning from constraints, which incorporates domain knowledge to guide the learning process and enhance model robustness. The third challenge, on the other hand, is addressed using learning of constraints, which helps identify and formalize logical relationships among learned tasks, thereby providing interpretable explanations of the networks’ behavior.</p>
      </abstract>
      <textClass>
        <keywords>
          <list>
            <item>Deep Learning (DL)</item>
            <item>Logic Constraints</item>
            <item>Active Learning</item>
            <item>Adversarial Defense</item>
            <item>Logic Explanations</item>
          </list>
        </keywords>
      </textClass>
    </profileDesc>
  </teiHeader>
  <text>
    <body>
      <p>It is available online at https://doi.org/10.36253/979-12-215-0680-8<ref target="https://doi.org/10.36253/979-12-215-0680-8" /></p>
      <div>
        <listBibl>
          <head>References</head>
          <bibl n="198030">
            <bibl>Agrawal, R. (2019). Introduction to deep learning. https://medium.com/@rochak.agrawal/introduction-to-deep-learning-5ffd8b625b00. Accessed: 2021-10-18.</bibl>
            <idno type="DOI">10.1007/978-1-4842-6579-6_1</idno>
          </bibl>
          <bibl n="197976">
            <bibl>Akcay, S., Atapour-Abarghouei, A., and Breckon, T. P. (2018). Ganomaly: Semi-supervised anomaly detection via adversarial training. In Asian Conference on Computer Vision, pages 622-637. Springer.</bibl>
            <idno type="DOI">10.1007/978-3-030-20893-6_39</idno>
          </bibl>
          <bibl n="197961">
            <bibl>Alayrac, J.-B., Uesato, J., Huang, P.-S., Fawzi, A., Stanforth, R., and Kohli, P. (2019). Are labels required for improving adversarial robustness? In Neural Information Processing Systems, pages 12214-12223.</bibl>
            <idno type="DOI">10.18653/v1/d19-1419</idno>
          </bibl>
          <bibl n="198033">Alvarez-Melis, D. and Jaakkola, T. S. (2018). Towards robust interpretability with self-explaining neural networks. arXiv preprint arXiv:1806.07538.</bibl>
          <bibl n="197919">Andriushchenko, M., Croce, F., Flammarion, N., and Hein, M. (2020). Square Attack: A Query-Efficient Black-Box Adversarial Attack via Random Search. In Vedaldi, A., Bischof, H., Brox, T., and Frahm, J.-M., editors, Computer Vision - ECCV 2020, pages 484-501, Cham. Springer International Publishing.</bibl>
          <bibl n="198047">Angelino, E., Larus-Stone, N., Alabi, D., Seltzer, M., and Rudin, C. (2018). Learning certifiably optimal rule lists for categorical data.</bibl>
          <bibl n="198000">Araujo, A., Meunier, L., Pinot, R., and Negrevergne, B. (2020). Robust Neural Networks using Randomized Adversarial Training. arXiv:1903.10219 [cs, stat]. arXiv: 1903.10219.</bibl>
          <bibl n="198101">
            <bibl>Aristotle (350 B.C.). Posterior analytics.</bibl>
            <idno type="DOI">10.1093/oseo/instance.00262308</idno>
          </bibl>
          <bibl n="197990">
            <bibl>Ash, J. T., Zhang, C., Krishnamurthy, A., Langford, J., and Agarwal, A. (2019). Deep batch active learning by diverse, uncertain gradient lower bounds. arXiv preprint arXiv:1906.03671.</bibl>
            <idno type="DOI">10.1109/lra.2023.3347131/mm1</idno>
          </bibl>
          <bibl n="197916">
            <bibl>Athalye, A., Carlini, N., and Wagner, D. (2018). Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In Dy, J. and Krause, A., editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 274-283. PMLR.</bibl>
            <idno type="DOI">10.22215/etd/2024-16076</idno>
          </bibl>
          <bibl n="198066">
            <bibl>Babbar, R. and Scholkopf, B. (2018). Adversarial extreme multi-label classification. arXiv preprint arXiv:1803.01570.</bibl>
            <idno type="DOI">10.1145/3018661.3018741</idno>
          </bibl>
          <bibl n="198003">
            <bibl>Barbiero, P., Ciravegna, G., Giannini, F., Lio, P., Gori, M., and Melacci, S. (2021). Entropy-based logic explanations of neural networks. arXiv preprint arXiv:2106.06804.</bibl>
            <idno type="DOI">10.1609/aaai.v36i6.20551</idno>
          </bibl>
          <bibl n="197937">
            <bibl>Beluch, W. H., Genewein, T., Nurnberger, A., and Kohler, J. M. (2018). The power of ensembles for active learning in image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9368-9377.</bibl>
            <idno type="DOI">10.1109/cvpr.2018.00976</idno>
          </bibl>
          <bibl n="198007">
            <bibl>Bendale, A. and Boult, T. E. (2016). Towards open set deep networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1563-1572.</bibl>
            <idno type="DOI">10.1109/cvpr.2016.173</idno>
          </bibl>
          <bibl n="198024">
            <bibl>Betti, A., Gori, M., and Melacci, S. (2019). Cognitive action laws: The case of visual features. IEEE transactions on neural networks and learning systems.</bibl>
            <idno type="DOI">10.1109/tnnls.2019.2911174</idno>
          </bibl>
          <bibl n="198061">
            <bibl>Bibal, A. and Frenay, B. (2016). Interpretability of machine learning models and representations: an introduction. In ESANN.</bibl>
            <idno type="DOI">10.14428/esann/2024.es2024-6</idno>
          </bibl>
          <bibl n="197915">
            <bibl>Biggio, B., Corona, I., Maiorca, D., Nelson, B., ≈†rndiƒ&amp;#225;, N., Laskov, P., Giacinto, G., and Roli, F. (2013). Evasion attacks against machine learning at test time. In Blockeel, H., Kersting, K., Nijssen, S., and ≈Ωelezny, F., editors, Machine Learning and Knowledge Discovery in Databases (ECML PKDD), Part III, volume 8190 of LNCS, pages 387-402. Springer Berlin Heidelberg.</bibl>
            <idno type="DOI">10.1007/978-3-642-40994-3_25</idno>
          </bibl>
          <bibl n="198048">
            <bibl>Biggio, B. and Roli, F. (2018). Wild patterns: Ten years after the rise of adversarial machine learning. Pattern Recognition, 84:317-331.</bibl>
            <idno type="DOI">10.1016/j.patcog.2018.07.023</idno>
          </bibl>
          <bibl n="198071">
            <bibl>Breiman, L., Friedman, J., Stone, C. J., and Olshen, R. A. (1984). Classification and regression trees. CRC press.</bibl>
            <idno type="DOI">10.1201/9781315139470-8</idno>
          </bibl>
          <bibl n="197985">Brinker, K. (2003). Incorporating diversity in active learning with support vector machines. In Proceedings of the 20th international conference on machine learning (ICML-03), pages 59-66.</bibl>
          <bibl n="197958">
            <bibl>Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. arXiv preprint arXiv:2005.14165.</bibl>
            <idno type="DOI">10.1109/lra.2023.3347131/mm1</idno>
          </bibl>
          <bibl n="197948">Burbidge, R., Rowland, J. J., and King, R. D. (2007). Active learning for regression based on query by committee. In International conference on intelligent data engineering and automated learning, pages 209-218. Springer.</bibl>
          <bibl n="198086">Cao, X. and Tsang, I. W. (2021). Bayesian active learning by disagreements: A geometric perspective.</bibl>
          <bibl n="197953">
            <bibl>Carlini, N., Athalye, A., Papernot, N., Brendel, W., Rauber, J., Tsipras, D., Goodfellow, I., Madry, A., and Kurakin, A. (2019). On evaluating adversarial robustness. arXiv preprint arXiv:1902.06705.	10.1167/19.10.190c</bibl>
            <idno type="DOI">10.1167/19.10.190c</idno>
          </bibl>
          <bibl n="197968">Carlini, N. and Wagner, D. (2017a). Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the ACM Workshop on Artificial Intelligence and Security, pages 3-14.</bibl>
          <bibl n="198004">Carlini, N. and Wagner, D. A. (2017b). Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy, pages 39-57. IEEE Computer Society.</bibl>
          <bibl n="197983">
            <bibl>Carmon, Y., Raghunathan, A., Schmidt, L., Duchi, J. C., and Liang, P. S. (2019). Unlabeled data improves adversarial robustness. In Neural Information Processing Systems, pages 11190-11201.</bibl>
            <idno type="DOI">10.18653/v1/d19-1423</idno>
          </bibl>
          <bibl n="197920">Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., and Elhadad, N. (2015). Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 1721-1730.</bibl>
          <bibl n="198031">Carvalho, D. V., Pereira, E. M., and Cardoso, J. S. (2019). Machine learning interpretability: A survey on methods and metrics. Electronics, 8(8):832.</bibl>
          <bibl n="198083">
            <bibl>Castro, J. L. and Trillas, E. (1998). The logic of neural networks. Mathware and Soft Computing, 5:23-37.</bibl>
            <idno type="DOI">10.1007/978-1-4757-2937-5_16</idno>
          </bibl>
          <bibl n="197999">
            <bibl>Chen, X., Mottaghi, R., Liu, X., Fidler, S., Urtasun, R., and Yuille, A. (2014). Detect what you can: Detecting and representing objects using holistic models and body parts.</bibl>
            <idno type="DOI">10.1109/cvpr.2014.254</idno>
          </bibl>
          <bibl n="198046">
            <bibl>Chen, Z., Bei, Y., and Rudin, C. (2020). Concept whitening for interpretable image recognition. Nature Machine Intelligence, 2(12):772-782.</bibl>
            <idno type="DOI">10.1038/s42256-020-00265-z</idno>
          </bibl>
          <bibl n="198039">Choi, J., Elezi, I., Lee, H.-J., Farabet, C., and Alvarez, J. M. (2021). Active learning for deep object detection via probabilistic modeling.</bibl>
          <bibl n="198021">
            <bibl>Ciravegna, G., Barbiero, P., Giannini, F., Gori, M., Lio, P., Maggini, M., and Melacci, S. (2021). Logic explained networks. arXiv preprint arXiv:2108.05149.</bibl>
            <idno type="DOI">10.1016/j.artint.2022.103822</idno>
          </bibl>
          <bibl n="197914">Ciravegna, G., Giannini, F., Gori, M., Maggini, M., and Melacci, S. (2020a). Human-driven fol explanations of deep learning. In Twenty-Ninth International Joint Conference on Artificial Intelligence and Seventeenth Pacific Rim International Conference on Artificial Intelligence {IJCAI-PRICAI-20}, pages 2234-2240. International Joint Conferences on Artificial Intelligence Organization.</bibl>
          <bibl n="198017">
            <bibl>Ciravegna, G., Giannini, F., Melacci, S., Maggini, M., and Gori, M. (2020b). A constraint-based approach to learning and explanation. In AAAI, pages 3658-3665.</bibl>
            <idno type="DOI">10.1609/aaai.v34i04.5774</idno>
          </bibl>
          <bibl n="198072">
            <bibl>Cohen, W. W. (1995). Fast effective rule induction. In Machine learning proceedings 1995, pages 115-123. Elsevier.</bibl>
            <idno type="DOI">10.1016/b978-1-55860-377-6.50023-2</idno>
          </bibl>
          <bibl n="198001">Coppedge, M., Gerring, J., Knutsen, C. H., Lindberg, S. I., Teorell, J., Altman, D., Bernhard, M., Cornell, A., Fish, M. S., Gastaldi, L., et al. (2021). V-dem codebook v11.</bibl>
          <bibl n="198028">Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and brain sciences, 24(1):87-114.</bibl>
          <bibl n="197917">Croce, F. and Hein, M. (2020a). Minimally distorted adversarial examples with a fast adaptive boundary attack. In III, H. D. and Singh, A., editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 2196-2205, Virtual. PMLR.</bibl>
          <bibl n="197980">
            <bibl>Croce, F. and Hein, M. (2020b). Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International Conference on Machine Learning, pages 1-12.</bibl>
            <idno type="DOI">10.1109/satml59370.2024.00028</idno>
          </bibl>
          <bibl n="198037">
            <bibl>Das, A. and Rad, P. (2020). Opportunities and challenges in explainable artificial intelligence (xai): A survey. arXiv preprint arXiv:2006.11371.</bibl>
            <idno type="DOI">10.14744/iacapaparxiv.2020.20003</idno>
          </bibl>
          <bibl n="197924">
            <bibl>d‚&amp;#196;&amp;#244;Avila Garcez, A. S., Gori, M., Lamb, L. C., Serafini, L., Spranger, M., and Tran, S. N. (2019). Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning. Journal of Applied Logics - IfCoLog Journal, 6(4):611-632.</bibl>
            <idno type="DOI">10.1007/978-1-4471-0211-3_9</idno>
          </bibl>
          <bibl n="198076">De Raedt, L. and Kimmig, A. (2015). Probabilistic (logic) programming concepts. Machine Learning, 100(1):5-47.</bibl>
          <bibl n="197922">
            <bibl>Demontis, A., Melis, M., Pintor, M., Jagielski, M., Biggio, B., Oprea, A., Nita-Rotaru, C., and Roli, F. (2019). Why do adversarial attacks transfer? Explaining transferability of evasion and poisoning attacks. In 28th USENIX Security Symposium (USENIX Security 19). USENIX Association.</bibl>
            <idno type="DOI">10.1109/sp.2018.00057</idno>
          </bibl>
          <bibl n="197957">
            <bibl>Di Nola, A., Gerla, B., and Leustean, I. (2013). Adding real coefficients to ≈&amp;#199;ukasiewicz logic: An application to neural networks. In International Workshop on Fuzzy Logic and Applications, pages 77-85. Springer.</bibl>
            <idno type="DOI">10.1007/978-3-319-03200-9_9</idno>
          </bibl>
          <bibl n="198040">
            <bibl>Diligenti, M., Gori, M., and Sacca, C. (2017). Semantic-based regularization for learning and inference. Artificial Intelligence, 244:143-165.</bibl>
            <idno type="DOI">10.1016/j.artint.2015.08.011</idno>
          </bibl>
          <bibl n="197938">
            <bibl>Donadello, I., Serafini, L., and Garcez, A. D. (2017). Logic tensor networks for semantic image interpretation. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI‚&amp;#196;&amp;#244;17, page 1596-1602. AAAI Press.</bibl>
            <idno type="DOI">10.24963/ijcai.2017/221</idno>
          </bibl>
          <bibl n="198056">
            <bibl>Doshi-Velez, F. and Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608.</bibl>
            <idno type="DOI">10.1007/978-3-319-98131-4_1</idno>
          </bibl>
          <bibl n="197930">
            <bibl>Do≈&amp;#176;iloviƒ&amp;#225;, F. K., Brƒ&amp;#231;iƒ&amp;#225;, M., and Hlupiƒ&amp;#225;, N. (2018). Explainable artificial intelligence: A survey. In 2018 41st International convention on information and communication technology, electronics and microelectronics (MIPRO), pages 0210-0215. IEEE.</bibl>
            <idno type="DOI">10.23919/mipro.2018.8400040</idno>
          </bibl>
          <bibl n="198054">
            <bibl>Ducoffe, M. and Precioso, F. (2017). Active learning strategy for cnn combining batchwise dropout and query-by-committee. In ESANN.</bibl>
            <idno type="DOI">10.1109/icip.2010.5653635</idno>
          </bibl>
          <bibl n="198042">
            <bibl>Ducoffe, M. and Precioso, F. (2018). Adversarial active learning for deep networks: a margin based approach. arXiv preprint arXiv:1802.09841.</bibl>
            <idno type="DOI">10.22541/au.149693987.70506124</idno>
          </bibl>
          <bibl n="197950">
            <bibl>Erhan, D., Courville, A., and Bengio, Y. (2010). Understanding representations learned in deep architectures. Department dInformatique et Recherche Operationnelle, University of Montreal, QC, Canada, Tech. Rep, 1355(1).</bibl>
            <idno type="DOI">10.15376/frc.2005.2.1269</idno>
          </bibl>
          <bibl n="198099">
            <bibl>EUGDPR (2017). Gdpr. general data protection regulation.</bibl>
            <idno type="DOI">10.1211/pj.2017.20203048</idno>
          </bibl>
          <bibl n="198060">Freitas, A. A. (2014). Comprehensible classification models: a position paper. ACM SIGKDD explorations newsletter, 15(1):1-10.</bibl>
          <bibl n="198088">
            <bibl>Fu, L. (1991). Rule learning by searching on adapted nets. In AAAI, volume 91, pages 590-595.</bibl>
            <idno type="DOI">10.1109/ijcnn.1991.170488</idno>
          </bibl>
          <bibl n="198089">
            <bibl>Gal, Y., Islam, R., and Ghahramani, Z. (2017). Deep bayesian active learning with image data.</bibl>
            <idno type="DOI">10.1038/s41598-019-50587-1</idno>
          </bibl>
          <bibl n="198052">
            <bibl>Ghorbani, A., Wexler, J., Zou, J., and Kim, B. (2019). Towards automatic concept-based explanations. arXiv preprint arXiv:1902.03129.</bibl>
            <idno type="DOI">10.22541/au.149693987.70506124</idno>
          </bibl>
          <bibl n="198044">
            <bibl>Gnecco, G., Gori, M., Melacci, S., and Sanguineti, M. (2015). Foundations of support constraint machines. Neural computation, 27(2):388-480.</bibl>
            <idno type="DOI">10.1162/neco_a_00686</idno>
          </bibl>
          <bibl n="197918">
            <bibl>Goldberger, A. L., Amaral, L. A., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C.-K., and Stanley, H. E. (2000). Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals. circulation, 101(23):e215-e220.</bibl>
            <idno type="DOI">10.1161/01.cir.101.23.e215</idno>
          </bibl>
          <bibl n="198049">
            <bibl>Gong, T., Liu, B., Chu, Q., and Yu, N. (2019). Using multi-label classification to improve object detection. Neurocomputing, 370:174-185.</bibl>
            <idno type="DOI">10.1016/j.neucom.2019.08.089</idno>
          </bibl>
          <bibl n="198026">
            <bibl>Goodfellow, I., McDaniel, P., and Papernot, N. (2018). Making machine learning robust against adversarial inputs. Communications of the ACM, 61(7):56-66.</bibl>
            <idno type="DOI">10.1145/3134599</idno>
          </bibl>
          <bibl n="198018">
            <bibl>Goodfellow, I. J., Shlens, J., and Szegedy, C. (2015). Explaining and harnessing adversarial examples. In International Conference on Learning Representations.</bibl>
            <idno type="DOI">10.5220/0006123702260234</idno>
          </bibl>
          <bibl n="198050">
            <bibl>Gori, M. and Melacci, S. (2013). Constraint verification with kernel machines. IEEE Trans. Neural Networks Learn. Syst., 24(5):825-831.</bibl>
            <idno type="DOI">10.1109/tnnls.2013.2241787</idno>
          </bibl>
          <bibl n="197913">
            <bibl>Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwinska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J. P., Badia, A. P., Hermann, K. M., Zwols, Y., Ostrovski, G., Cain, A., King, H., Summerfield, C., Blunsom, P., Kavukcuoglu, K., and Hassabis, D. (2016). Hybrid computing using a neural network with dynamic external memory. Nature, 538:471-476.</bibl>
            <idno type="DOI">10.1038/nature20101</idno>
          </bibl>
          <bibl n="198008">
            <bibl>Grosse, K., Manoharan, P., Papernot, N., Backes, M., and McDaniel, P. (2017). On the (statistical) detection of adversarial examples. arXiv preprint arXiv:1702.06280.</bibl>
            <idno type="DOI">10.1007/978-3-319-66399-9_4</idno>
          </bibl>
          <bibl n="197984">
            <bibl>Guidotti, R., Monreale, A., Ruggieri, S., Pedreschi, D., Turini, F., and Giannotti, F. (2018a). Local rule-based explanations of black box decision systems. arXiv preprint arXiv:1805.10820.</bibl>
            <idno type="DOI">10.1609/aaai.v33i01.33019780</idno>
          </bibl>
          <bibl n="197986">
            <bibl>Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., and Pedreschi, D. (2018b). A survey of methods for explaining black box models. ACM computing surveys (CSUR), 51(5):93.</bibl>
            <idno type="DOI">10.1145/3236009</idno>
          </bibl>
          <bibl n="198058">Gunning, D. (2017). Explainable artificial intelligence (xai). Defense Advanced Research Projects Agency (DARPA), nd Web, 2(2).</bibl>
          <bibl n="198062">Han, J., Pei, J., and Yin, Y. (2000). Mining frequent patterns without candidate generation. ACM sigmod record, 29(2):1-12.</bibl>
          <bibl n="197923">Harmon, S. A., Sanford, T. H., Xu, S., Turkbey, E. B., Roth, H., Xu, Z., Yang, D., Myronenko, A., Anderson, V., Amalou, A., et al. (2020). Artificial intelligence for the detection of covid-19 pneumonia on chest ct using multinational datasets. Nature communications, 11(1):1-7.</bibl>
          <bibl n="198029">Hastie, T. and Tibshirani, R. (1987). Generalized additive models: some applications. Journal of the American Statistical Association, 82(398):371-386.</bibl>
          <bibl n="197991">Haussmann, E., Fenzi, M., Chitta, K., Ivanecky, J., Xu, H., Roy, D., Mittel, A., Koumchatzky, N., Farabet, C., and Alvarez, J. M. (2020). Scalable active learning for object detection.</bibl>
          <bibl n="197981">He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778.</bibl>
          <bibl n="197925">Hein, M., Andriushchenko, M., and Bitterwolf, J. (2019). Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 41-50.</bibl>
          <bibl n="197973">Hendrycks, D. and Gimpel, K. (2017). A baseline for detecting misclassified and out-of-distribution examples in neural networks. Proceedings of International Conference on Learning Representations.</bibl>
          <bibl n="197940">
            <bibl>Hendrycks, D. and Gimpel, K. (2017). Early methods for detecting adversarial images. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings. OpenReview.net.</bibl>
            <idno type="DOI">10.1088/2050-6120/aaa7bf</idno>
          </bibl>
          <bibl n="198034">Hoffmann, R., Minkin, V. I., and Carpenter, B. K. (1996). Ockham‚&amp;#196;&amp;#244;s razor and chemistry. Bulletin de la Societe chimique de France, 2(133):117-130.</bibl>
          <bibl n="198057">
            <bibl>Holte, R. C. (1993). Very simple classification rules perform well on most commonly used datasets. Machine learning, 11(1):63-90.</bibl>
            <idno type="DOI">10.1023/a:1022631118932</idno>
          </bibl>
          <bibl n="198053">
            <bibl>Houlsby, N., Huszar, F., Ghahramani, Z., and Lengyel, M. (2011). Bayesian active learning for classification and preference learning.</bibl>
            <idno type="DOI">10.1103/physreva.85.052120</idno>
          </bibl>
          <bibl n="198045">Huang, S. H. and Xing, H. (2002). Extract intelligible and concise fuzzy rules from neural networks. Fuzzy Sets and Systems, 132(2):233-243.</bibl>
          <bibl n="197947">Huysmans, J., Dejaeger, K., Mues, C., Vanthienen, J., and Baesens, B. (2011). An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models. Decision Support Systems, 51(1):141-154.</bibl>
          <bibl n="198032">
            <bibl>Ichnowski, J., Avigal, Y., Satish, V., and Goldberg, K. (2020). Deep learning can accelerate grasp-optimized motion planning. Science Robotics, 5(48).</bibl>
            <idno type="DOI">10.1126/scirobotics.abd7710</idno>
          </bibl>
          <bibl n="197939">
            <bibl>Joshi, A., Mukherjee, A., Sarkar, S., and Hegde, C. (2019). Semantic adversarial attacks: Parametric transformations that fool deep classifiers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4773-4783.</bibl>
            <idno type="DOI">10.1109/iccv.2019.00487</idno>
          </bibl>
          <bibl n="197934">
            <bibl>Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., ≈Ωidek, A., Potapenko, A., et al. (2021). Highly accurate protein structure prediction with alphafold. Nature, 596(7873):583-589.</bibl>
            <idno type="DOI">10.1038/s41586-021-03819-2</idno>
          </bibl>
          <bibl n="198027">Kasabov, N. K. (1996). Learning fuzzy rules and approximate reasoning in fuzzy neural networks and hybrid systems. Fuzzy sets and Systems, 82(2):135-149.</bibl>
          <bibl n="198023">
            <bibl>Kazhdan, D., Dimanov, B., Jamnik, M., Lio, P., and Weller, A. (2020). Now you see me (cme): Concept-based model extraction. arXiv preprint arXiv:2010.13233.</bibl>
            <idno type="DOI">10.3233/faia200380</idno>
          </bibl>
          <bibl n="198041">
            <bibl>Kim, B., Gilmer, J., Wattenberg, M., and Viegas, F. (2018a). Tcav: Relative concept importance testing with linear concept activation vectors.</bibl>
            <idno type="DOI">10.1111/j.1540-5885.2010.00784.x</idno>
          </bibl>
          <bibl n="197926">
            <bibl>Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., et al. (2018b). Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning, pages 2668-2677. PMLR.</bibl>
            <idno type="DOI">10.1109/ijcnn48605.2020.9206946</idno>
          </bibl>
          <bibl n="197929">
            <bibl>Kindermans, P.-J., Hooker, S., Adebayo, J., Alber, M., Schutt, K. T., Dahne, S., Erhan, D., and Kim, B. (2019). The (un) reliability of saliency methods. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, pages 267-280. Springer.</bibl>
            <idno type="DOI">10.1007/978-3-030-28954-6_14</idno>
          </bibl>
          <bibl n="198009">Kirsch, A., van Amersfoort, J., and Gal, Y. (2019). Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning.	10.1137/1.9781611977653.ch83</bibl>
          <bibl n="198077">
            <bibl>Klement, E. P., Mesiar, R., and Pap, E. (2013). Triangular norms, volume 8. Springer Science &amp;amp; Business Media.</bibl>
            <idno type="DOI">10.1016/b978-044451814-9/50003-3</idno>
          </bibl>
          <bibl n="198064">
            <bibl>Koh, P. W., Nguyen, T., Tang, Y. S., Mussmann, S., Pierson, E., Kim, B., and Liang, P. (2020). Concept bottleneck models.</bibl>
            <idno type="DOI">10.2139/ssrn.4835782</idno>
          </bibl>
          <bibl n="197969">
            <bibl>Koller, D., Friedman, N., D≈&amp;#230;eroski, S., Sutton, C., McCallum, A., Pfeffer, A., Abbeel, P., Wong, M.-F., Meek, C., Neville, J., et al. (2007). Introduction to statistical relational learning. MIT press.</bibl>
            <idno type="DOI">10.7551/mitpress/7432.003.0008</idno>
          </bibl>
          <bibl n="197966">
            <bibl>Krzywinski, M. and Altman, N. (2013). Error bars: the meaning of error bars is often misinterpreted, as is the statistical significance of their overlap. Nature methods, 10(10):921-923.	10.1038/nmeth.2659</bibl>
            <idno type="DOI">10.1211/pj.2017.20203048</idno>
          </bibl>
          <bibl n="198087">
            <bibl>LeCun, Y. (1998). The mnist database of handwritten digits. http://yann.lecun.com/exdb/mnist/.</bibl>
            <idno type="DOI">10.32614/cran.package.idx2r</idno>
          </bibl>
          <bibl n="197951">
            <bibl>Letham, B., Rudin, C., McCormick, T. H., Madigan, D., et al. (2015). Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model. Annals of Applied Statistics, 9(3):1350-1371.</bibl>
            <idno type="DOI">10.1214/15-aoas848</idno>
          </bibl>
          <bibl n="198069">Liu, B. (2007). Web data mining: exploring hyperlinks, contents, and usage data. Springer Science &amp;amp; Business Media.</bibl>
          <bibl n="198078">
            <bibl>Loshchilov, I. and Hutter, F. (2017). Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101.</bibl>
            <idno type="DOI">10.22541/au.149693987.70506124</idno>
          </bibl>
          <bibl n="197954">
            <bibl>Lou, Y., Caruana, R., and Gehrke, J. (2012). Intelligible models for classification and regression. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 150-158.</bibl>
            <idno type="DOI">10.1145/2339530.2339556</idno>
          </bibl>
          <bibl n="198063">
            <bibl>Lundberg, S. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. arXiv preprint arXiv:1705.07874.</bibl>
            <idno type="DOI">10.22541/au.149693987.70506124</idno>
          </bibl>
          <bibl n="198070">
            <bibl>Ma, W. J., Husain, M., and Bays, P. M. (2014). Changing concepts of working memory. Nature neuroscience, 17(3):347.</bibl>
            <idno type="DOI">10.1038/nn.3655</idno>
          </bibl>
          <bibl n="197928">Ma, X., Li, B., Wang, Y., Erfani, S. M., Wijewickrema, S., Schoenebeck, G., Houle, M. E., Song, D., and Bailey, J. (2018). Characterizing adversarial subspaces using local intrinsic dimensionality. In International Conference on Learning Representations.</bibl>
          <bibl n="197979">Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. (2018). Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations.</bibl>
          <bibl n="198092">
            <bibl>Marcus, G. (2018). Deep learning: A critical appraisal. arXiv preprint arXiv:1801.00631.</bibl>
            <idno type="DOI">10.22541/au.149693987.70506124</idno>
          </bibl>
          <bibl n="198012">
            <bibl>Marler, R. T. and Arora, J. S. (2004). Survey of multi-objective optimization methods for engineering. Structural and multidisciplinary optimization, 26(6):369-395.</bibl>
            <idno type="DOI">10.1007/s00158-003-0368-6</idno>
          </bibl>
          <bibl n="197998">
            <bibl>Marra, G., Giannini, F., Diligenti, M., and Gori, M. (2019). Lyrics: A general interface layer to integrate logic inference and deep learning. arXiv preprint arXiv:1903.07534.</bibl>
            <idno type="DOI">10.1007/978-3-030-46147-8_17</idno>
          </bibl>
          <bibl n="197975">
            <bibl>McCallumzy, A. K. and Nigamy, K. (1998). Employing em and pool-based active learning for text classification. In Proc. International Conference on Machine Learning (ICML), pages 359-367. Citeseer.</bibl>
            <idno type="DOI">10.1145/1102351.1102445</idno>
          </bibl>
          <bibl n="198075">
            <bibl>McCluskey, E. J. (1956). Minimization of boolean functions. The Bell System Technical Journal, 35(6):1417-1444.</bibl>
            <idno type="DOI">10.1002/j.1538-7305.1956.tb03835.x</idno>
          </bibl>
          <bibl n="198055">McColl, H. (1878). The calculus of equivalent statements (third paper). Proceedings of the London Mathematical Society, 1(1):16-28.</bibl>
          <bibl n="198014">McKelvey, R. D. and Zavoina, W. (1975). A statistical model for the analysis of ordinal level dependent variables. Journal of Mathematical Sociology, 4(1):103-120.</bibl>
          <bibl n="198035">
            <bibl>Melacci, S. and Belkin, M. (2011). Laplacian support vector machines trained in the primal. Journal of Machine Learning Research, 12(Mar):1149-1184.</bibl>
            <idno type="DOI">10.1007/978-3-642-40728-4_24</idno>
          </bibl>
          <bibl n="197964">
            <bibl>Melacci, S., Ciravegna, G., Sotgiu, A., Demontis, A., Biggio, B., Gori, M., and Roli, F. (2020). Domain knowledge alleviates adversarial attacks in multi-label classifiers. arXiv preprint arXiv:2006.03833.</bibl>
            <idno type="DOI">10.1109/tpami.2021.3137564</idno>
          </bibl>
          <bibl n="197932">
            <bibl>Melacci, S., Globo, A., and Rigutini, L. (2018). Enhancing modern supervised word sense disambiguation models by semantic lexical resources. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).</bibl>
            <idno type="DOI">10.18653/v1/d19-1359</idno>
          </bibl>
          <bibl n="198016">Melacci, S. and Gori, M. (2012). Unsupervised learning by minimal entropy encoding. IEEE transactions on neural networks and learning systems, 23(12):1849-1861.</bibl>
          <bibl n="197963">
            <bibl>Melacci, S., Maggini, M., and Gori, M. (2009). Semi-supervised learning with constraints for multi-view object recognition. In International Conference on Artificial Neural Networks, pages 653-662. Springer.</bibl>
            <idno type="DOI">10.1007/978-3-642-04277-5_66</idno>
          </bibl>
          <bibl n="197931">
            <bibl>Melis, M., Demontis, A., Biggio, B., Brown, G., Fumera, G., and Roli, F. (2017). Is deep learning safe for robot vision? Adversarial examples against the iCub humanoid. In ICCVW Vision in Practice on Autonomous Robots (ViPAR), pages 751-759. IEEE.</bibl>
            <idno type="DOI">10.1109/iccvw.2017.94</idno>
          </bibl>
          <bibl n="198096">
            <bibl>Mendelson, E. (2009). Introduction to mathematical logic. CRC press.</bibl>
            <idno type="DOI">10.1201/b18519</idno>
          </bibl>
          <bibl n="197959">
            <bibl>Miller, D. J., Xiang, Z., and Kesidis, G. (2020). Adversarial learning targeting deep neural network classification: A comprehensive review of defenses against attacks. Proceedings of the IEEE, 108(3):402-433.</bibl>
            <idno type="DOI">10.1109/jproc.2020.2970615</idno>
          </bibl>
          <bibl n="198025">
            <bibl>Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological review, 63:81-97.</bibl>
            <idno type="DOI">10.1037/h0043158</idno>
          </bibl>
          <bibl n="198084">
            <bibl>Minsky, M. and Papert, S. A. (2017). Perceptrons: An introduction to computational geometry. MIT press.</bibl>
            <idno type="DOI">10.7551/mitpress/11301.001.0001</idno>
          </bibl>
          <bibl n="197935">
            <bibl>Miyato, T., Maeda, S.-i., Koyama, M., and Ishii, S. (2018). Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 41(8):1979-1993.</bibl>
            <idno type="DOI">10.1109/tpami.2018.2858821</idno>
          </bibl>
          <bibl n="197988">
            <bibl>Miyato, T., Maeda, S.-i., Koyama, M., Nakae, K., and Ishii, S. (2016). Distributional smoothing with virtual adversarial training. In International Conference on Learning Representation.</bibl>
            <idno type="DOI">10.1109/tpami.2018.2858821</idno>
          </bibl>
          <bibl n="198098">
            <bibl>Molnar, C. (2020). Interpretable machine learning. Lulu. com.</bibl>
            <idno type="DOI">10.21105/joss.00786</idno>
          </bibl>
          <bibl n="197965">
            <bibl>Morgado, P. and Vasconcelos, N. (2017). Semantically consistent regularization for zero-shot recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6060-6069.</bibl>
            <idno type="DOI">10.1109/cvpr.2017.220</idno>
          </bibl>
          <bibl n="197952">
            <bibl>Mpagouli, A. and Hatzilygeroudis, I. (2007). Converting first order logic into natural language: A first level approach. In Current Trends in Informatics: 11th Panhellenic Conference on Informatics, PCI, pages 517-526.</bibl>
            <idno type="DOI">10.1007/978-3-642-22194-1_14</idno>
          </bibl>
          <bibl n="197978">
            <bibl>Najafi, A., Maeda, S.-i., Koyama, M., and Miyato, T. (2019). Robustness to adversarial perturbations in learning from incomplete data. In Neural Information Processing Systems, pages 5542-5552.</bibl>
            <idno type="DOI">10.1016/b978-0-12-824020-5.00018-1</idno>
          </bibl>
          <bibl n="197962">
            <bibl>Naseer, M. M., Khan, S. H., Khan, M. H., Shahbaz Khan, F., and Porikli, F. (2019). Cross-domain transferability of adversarial perturbations. Advances in Neural Information Processing Systems, 32:12905-12915.</bibl>
            <idno type="DOI">10.1109/iccv48922.2021.00761</idno>
          </bibl>
          <bibl n="198079">Nielsen, M. A. (2015). Neural networks and deep learning, volume 25. Determination press San Francisco, CA.</bibl>
          <bibl n="197987">
            <bibl>Papernot, N., McDaniel, P., and Goodfellow, I. (2016a). Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277.</bibl>
            <idno type="DOI">10.1145/3052973.3053009</idno>
          </bibl>
          <bibl n="197946">
            <bibl>Papernot, N., McDaniel, P., Wu, X., Jha, S., and Swami, A. (2016b). Distillation as a defense to adversarial perturbations against deep neural networks. In 2016 IEEE Symposium on Security and Privacy (SP), pages 582-597. IEEE.</bibl>
            <idno type="DOI">10.1109/sp.2016.41</idno>
          </bibl>
          <bibl n="197994">
            <bibl>Park, S., Park, J., Shin, S.-J., and Moon, I.-C. (2018). Adversarial dropout for supervised and semi-supervised learning. In Thirty-Second AAAI Conference on Artificial Intelligence.</bibl>
            <idno type="DOI">10.1609/aaai.v32i1.11634</idno>
          </bibl>
          <bibl n="197945">Pemstein, D., Marquardt, K. L., Tzelgov, E., Wang, Y.-t., Krusell, J., and Miri, F. (2018). The v-dem measurement model: latent variable analysis for cross-national and cross-temporal expert-coded data. V-Dem Working Paper, 21.</bibl>
          <bibl n="197944">
            <bibl>Pi, T., Li, X., and Zhang, Z. M. (2017). Boosted zero-shot learning with semantic correlation regularization. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pages 2599-2605.</bibl>
            <idno type="DOI">10.24963/ijcai.2017/362</idno>
          </bibl>
          <bibl n="198038">
            <bibl>Pop, R. and Fulop, P. (2018). Deep ensemble bayesian active learning : Addressing the mode collapse issue in monte carlo dropout via ensembles.</bibl>
            <idno type="DOI">10.11606/d.45.2019.tde-17032019-222659</idno>
          </bibl>
          <bibl n="198073">
            <bibl>Quine, W. V. (1952). The problem of simplifying truth functions. The American mathematical monthly, 59(8):521-531.</bibl>
            <idno type="DOI">10.1080/00029890.1952.11988183</idno>
          </bibl>
          <bibl n="198093">Quinlan, J. R. (1986). Induction of decision trees. Machine learning, 1(1):81-106.</bibl>
          <bibl n="198090">Quinlan, J. R. (2014). C4. 5: programs for machine learning. Elsevier.	10.1109/icmla.2014.116</bibl>
          <bibl n="197943">
            <bibl>Ras, G., van Gerven, M., and Haselager, P. (2018). Explanation methods in deep learning: Users, values, concerns and challenges. In Explainable and Interpretable Models in Computer Vision and Machine Learning, pages 19-36. Springer.</bibl>
            <idno type="DOI">10.1007/978-3-319-98131-4_2</idno>
          </bibl>
          <bibl n="198074">
            <bibl>Rathmanner, S. and Hutter, M. (2011). A philosophical treatise of universal induction. Entropy, 13(6):1076-1136.</bibl>
            <idno type="DOI">10.3390/e13061076</idno>
          </bibl>
          <bibl n="197933">
            <bibl>Ribeiro, M. T., Singh, S., and Guestrin, C. (2016). &amp;quot;Why should I trust you?&amp;quot; explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135-1144.</bibl>
            <idno type="DOI">10.1145/2939672.2939778</idno>
          </bibl>
          <bibl n="197996">
            <bibl>Ribeiro, M. T., Singh, S., and Guestrin, C. (2018). Anchors: High-precision model-agnostic explanations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32.</bibl>
            <idno type="DOI">10.1609/aaai.v32i1.11491</idno>
          </bibl>
          <bibl n="198094">Rivest, R. L. (1987). Learning decision lists. Machine learning, 2(3):229-246.</bibl>
          <bibl n="197982">Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., and Zhong, C. (2021). Interpretable machine learning: Fundamental principles and 10 grand challenges. arXiv preprint arXiv:2103.11251.</bibl>
          <bibl n="198091">Russell, S. J. and Norvig, P. (2016). Artificial intelligence: a modern approach. Malaysia</bibl>
          <bibl n="198068">
            <bibl>Sabour, S., Frosst, N., and Hinton, G. E. (2017). Dynamic routing between capsules. arXiv preprint arXiv:1710.09829.</bibl>
            <idno type="DOI">10.22541/au.149693987.70506124</idno>
          </bibl>
          <bibl n="197921">Saeed, M., Villarroel, M., Reisner, A. T., Clifford, G., Lehman, L.-W., Moody, G., Heldt, T., Kyaw, T. H., Moody, B., and Mark, R. G. (2011). Multiparameter intelligent monitoring in intensive care ii (mimic-ii): a public-access intensive care unit database. Critical care medicine, 39(5):952.</bibl>
          <bibl n="197971">Samangouei, P., Kabkab, M., and Chellappa, R. (2018). Defense-GAN: Protecting classifiers against adversarial attacks using generative models. In International Conference on Learning Representations.</bibl>
          <bibl n="197972">Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., and Muller, K.-R. (2020). Toward interpretable machine learning: Transparent deep neural networks and beyond. arXiv preprint arXiv:2003.07631.</bibl>
          <bibl n="198005">
            <bibl>Santoro, A., Raposo, D., Barrett, D. G. T., Malinowski, M., Pascanu, R., Battaglia, P., and Lillicrap, T. (2017). A simple neural network module for relational reasoning.</bibl>
            <idno type="DOI">10.1017/s0140525x19001365</idno>
          </bibl>
          <bibl n="197942">Sato, M. and Tsukimoto, H. (2001). Rule extraction from neural networks via decision tree induction. In IJCNN‚&amp;#196;&amp;#244;01. International Joint Conference on Neural Networks. Proceedings (Cat. No. 01CH37222), volume 3, pages 1870-1875. IEEE.</bibl>
          <bibl n="198036">
            <bibl>Schohn, G. and Cohn, D. (2000). Less is more: Active learning with support vector machines. Machine Learning-International Workshop then Conference.</bibl>
            <idno type="DOI">10.1109/mlsp.2012.6349736</idno>
          </bibl>
          <bibl n="197927">Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017). Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision, pages 618-626.</bibl>
          <bibl n="198082">
            <bibl>Sener, O. and Savarese, S. (2018). Active learning for convolutional neural networks: A core-set approach.</bibl>
            <idno type="DOI">10.1007/978-1-4842-3790-8_8</idno>
          </bibl>
          <bibl n="198006">
            <bibl>Shafahi, A., Huang, W. R., Studer, C., Feizi, S., and Goldstein, T. (2019). Are adversarial examples inevitable? In International Conference on Learning Representations.</bibl>
            <idno type="DOI">10.1109/iccv.2019.00660</idno>
          </bibl>
          <bibl n="198015">
            <bibl>Sheatsley, R., Hoak, B., Pauley, E., Beugin, Y., Weisman, M. J., and McDaniel, P. (2021). On the robustness of domain constraints. arXiv preprint arXiv:2105.08619.</bibl>
            <idno type="DOI">10.1145/3460120.3484570</idno>
          </bibl>
          <bibl n="198080">
            <bibl>Simon, H. A. (1956). Rational choice and the structure of the environment. Psychological review, 63(2):129.</bibl>
            <idno type="DOI">10.1037/h0042769</idno>
          </bibl>
          <bibl n="198102">Simon, H. A. (1957). Models of man</bibl>
          <bibl n="198067">
            <bibl>Simon, H. A. (1979). Rational decision making in business organizations. The American economic review, 69(4):493-513.</bibl>
            <idno type="DOI">10.2307/142819</idno>
          </bibl>
          <bibl n="197997">
            <bibl>Simonyan, K., Vedaldi, A., and Zisserman, A. (2013). Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034.</bibl>
            <idno type="DOI">10.5244/c.28.6</idno>
          </bibl>
          <bibl n="198059">
            <bibl>Soklakov, A. N. (2002). Occam‚&amp;#196;&amp;#244;s razor as a formal basis for a physical theory. Foundations of Physics Letters, 15(2):107-135.</bibl>
            <idno type="DOI">10.1023/a:1020994407185</idno>
          </bibl>
          <bibl n="198081">
            <bibl>Solso, R. L., MacLin, M. K., and MacLin, O. H. (2005). Cognitive psychology. Pearson Education New Zealand.</bibl>
            <idno type="DOI">10.1037/10517-057</idno>
          </bibl>
          <bibl n="198010">Song, Q., Jin, H., Huang, X., and Hu, X. (2018). Multi-label adversarial perturbations. In 2018 IEEE International Conference on Data Mining (ICDM), pages 1242-1247.</bibl>
          <bibl n="197989">
            <bibl>Sotgiu, A., Demontis, A., Melis, M., Biggio, B., Fumera, G., Feng, X., and Roli, F. (2020). Deep neural rejection against adversarial examples. EURASIP J. Information Security, 2020(5).</bibl>
            <idno type="DOI">10.1186/s13635-020-00105-y</idno>
          </bibl>
          <bibl n="198019">
            <bibl>Su, J., Vargas, D. V., and Sakurai, K. (2019). One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 23(5):828-841.</bibl>
            <idno type="DOI">10.1109/tevc.2019.2890858</idno>
          </bibl>
          <bibl n="197970">
            <bibl>Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. (2014a). Intriguing properties of neural networks. In International Conference on Learning Representations.</bibl>
            <idno type="DOI">10.1109/cvpr.2014.276</idno>
          </bibl>
          <bibl n="197955">
            <bibl>Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. (2014b). Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR 2014.</bibl>
            <idno type="DOI">10.1109/cvpr.2014.276</idno>
          </bibl>
          <bibl n="197949">
            <bibl>Tavares, A. R., Avelar, P., Flach, J. M., Nicolau, M., Lamb, L. C., and Vardi, M. (2020). Understanding boolean function learnability on deep neural networks. arXiv preprint arXiv:2009.05908.	10.22541/au.149693987.70506124</bibl>
            <idno type="DOI">10.14744/iacapaparxiv.2020.20003</idno>
          </bibl>
          <bibl n="198085">
            <bibl>Teso, S. (2019). Does symbolic knowledge prevent adversarial fooling? arXiv preprint arXiv:1912.10834.</bibl>
            <idno type="DOI">10.22541/au.149693987.70506124</idno>
          </bibl>
          <bibl n="198011">
            <bibl>Teso, S. and Kersting, K. (2019). Explanatory interactive machine learning. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pages 239-245.</bibl>
            <idno type="DOI">10.1145/3306618.3314293</idno>
          </bibl>
          <bibl n="197936">
            <bibl>Thulasidasan, S., Chennupati, G., Bilmes, J. A., Bhattacharya, T., and Michalak, S. (2019). On mixup training: Improved calibration and predictive uncertainty for deep neural networks. Advances in Neural Information Processing Systems, 32.</bibl>
            <idno type="DOI">10.2172/1525811</idno>
          </bibl>
          <bibl n="198013">
            <bibl>Tong, S. and Koller, D. (2001). Support vector machine active learning with applications to text classification. Journal of machine learning research, 2(Nov):45-66.</bibl>
            <idno type="DOI">10.3724/sp.j.1087.2012.01359</idno>
          </bibl>
          <bibl n="198051">
            <bibl>Towell, G. G. and Shavlik, J. W. (1993). Extracting refined rules from knowledge-based neural networks. Machine learning, 13(1):71-101.</bibl>
            <idno type="DOI">10.1007/bf00993103</idno>
          </bibl>
          <bibl n="198065">
            <bibl>Tsukimoto, H. (2000). Extracting rules from trained neural networks. IEEE Transactions on Neural networks, 11(2):377-389.</bibl>
            <idno type="DOI">10.1109/72.839008</idno>
          </bibl>
          <bibl n="197992">
            <bibl>Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. (2011a). The Caltech-UCSD Birds-200-2011 Dataset. Technical Report CNS-TR-2011-001, California Institute of Technology.</bibl>
            <idno type="DOI">10.1109/iccv.2011.6126539</idno>
          </bibl>
          <bibl n="197993">
            <bibl>Wah, C., Branson, S., Welinder, P., Perona, P., and Belongie, S. (2011b). The Caltech-UCSD Birds-200-2011 Dataset. Technical Report CNS-TR-2011-001, California Institute of Technology.</bibl>
            <idno type="DOI">10.1109/iccv.2011.6126539</idno>
          </bibl>
          <bibl n="198095">
            <bibl>Winston, P. H. and Horn, B. K. (1986). Lisp. Addison Wesley Pub., Reading, MA.</bibl>
            <idno type="DOI">10.1177/001698627802200110</idno>
          </bibl>
          <bibl n="198043">
            <bibl>Witten, I. H. and Frank, E. (2005). Data mining: Practical machine learning tools and techniques 2nd edition. Morgan Kaufmann, San Francisco.</bibl>
            <idno type="DOI">10.1186/1475-925x-5-51</idno>
          </bibl>
          <bibl n="197974">
            <bibl>Wu, Y., Bamman, D., and Russell, S. (2017). Adversarial training for relation extraction. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1778-1783.</bibl>
            <idno type="DOI">10.18653/v1/d17-1187</idno>
          </bibl>
          <bibl n="197960">
            <bibl>Yeh, C.-K., Kim, B., Arik, S., Li, C.-L., Pfister, T., and Ravikumar, P. (2020). On completeness-aware concept-based explanations in deep neural networks. Advances in Neural Information Processing Systems, 33.</bibl>
            <idno type="DOI">10.3233/faia210362</idno>
          </bibl>
          <bibl n="197977">
            <bibl>Ying, R., Bourgeois, D., You, J., Zitnik, M., and Leskovec, J. (2019). Gnnexplainer: Generating explanations for graph neural networks. Advances in neural information processing systems, 32:9240.</bibl>
            <idno type="DOI">10.1007/978-981-16-6054-2_5</idno>
          </bibl>
          <bibl n="198097">
            <bibl>Yoo, D. and Kweon, I. S. (2019). Learning loss for active learning.</bibl>
            <idno type="DOI">10.1109/cvpr.2019.00018</idno>
          </bibl>
          <bibl n="197967">
            <bibl>Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T., and Xiao, J. (2015). Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365.</bibl>
            <idno type="DOI">10.3390/app10144913</idno>
          </bibl>
          <bibl n="197956">
            <bibl>Yu, M. and Dredze, M. (2014). Improving lexical embeddings with semantic knowledge. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 545-550.</bibl>
            <idno type="DOI">10.3115/v1/p14-2089</idno>
          </bibl>
          <bibl n="198020">
            <bibl>Zeiler, M. D. and Fergus, R. (2014). Visualizing and understanding convolutional networks. In European conference on computer vision, pages 818-833. Springer.</bibl>
            <idno type="DOI">10.1007/978-3-319-10590-1_53</idno>
          </bibl>
          <bibl n="197995">
            <bibl>Zhai, R., Cai, T., He, D., Dan, C., He, K., Hopcroft, J., and Wang, L. (2019). Adversarially robust generalization just requires more unlabeled data. arXiv preprint arXiv:1906.00555.</bibl>
            <idno type="DOI">10.1109/lra.2023.3347131/mm1</idno>
          </bibl>
          <bibl n="198002">
            <bibl>Zhao, Z., Guo, Y., Shen, H., and Ye, J. (2020). Adaptive object detection with dual multi-label prediction. In European Conference on Computer Vision, pages 54-69. Springer.</bibl>
            <idno type="DOI">10.1007/978-3-030-58604-1_4</idno>
          </bibl>
          <bibl n="198100">Zhdanov, F. (2019). Diverse mini-batch active learning.</bibl>
          <bibl n="198022">Zhu, X. and Goldberg, A. B. (2009). Introduction to semi-supervised learning. Synthesis lectures on artificial intelligence and machine learning, 3(1):1-130.</bibl>
          <bibl n="197941">
            <bibl>Zilke, J. R., Loza Mencia, E., and Janssen, F. (2016). Deepred - rule extraction from deep neural networks. In Calders, T., Ceci, M., and Malerba, D., editors, Discovery Science, pages 457-473, Cham. Springer International Publishing.</bibl>
            <idno type="DOI">10.1007/978-3-319-46307-0_29</idno>
          </bibl>
        </listBibl>
      </div>
    </body>
  </text>
</TEI>