Publications

2024

1. Preprint

   How fair are we? From conceptualization to automated assessment of fairness definitions

   Giordano d’Aloisio, Claudio Di Sipio, Antinisca Di Marco, and Davide Di Ruscio

   arXiv preprint arXiv:2404.09919, 2024

   Abs arXiv Bib

   Fairness is a critical concept in ethics and social domains, but it is also a challenging property to engineer in software systems. With the increasing use of machine learning in software systems, researchers have been developing techniques to automatically assess the fairness of software systems. Nonetheless, a significant proportion of these techniques rely upon pre-established fairness definitions, metrics, and criteria, which may fail to encompass the wide-ranging needs and preferences of users and stakeholders. To overcome this limitation, we propose a novel approach, called MODNESS, that enables users to customize and define their fairness concepts using a dedicated modeling environment. Our approach guides the user through the definition of new fairness concepts also in emerging domains, and the specification and composition of metrics for its evaluation. Ultimately, MODNESS generates the source code to implement fair assessment based on these custom definitions. In addition, we elucidate the process we followed to collect and analyze relevant literature on fairness assessment in software engineering (SE). We compare MODNESS with the selected approaches and evaluate how they support the distinguishing features identified by our study. Our findings reveal that i) most of the current approaches do not support user-defined fairness concepts; ii) our approach can cover two additional application domains not addressed by currently available tools, i.e., mitigating bias in recommender systems for software engineering and Arduino software component recommendations; iii) MODNESS demonstrates the capability to overcome the limitations of the only two other Model-Driven Engineering-based approaches for fairness assessment.

   @article{d2024fair,
     title = {How fair are we? From conceptualization to automated assessment of fairness definitions},
     author = {d'Aloisio, Giordano and Di Sipio, Claudio and Di Marco, Antinisca and Di Ruscio, Davide},
     journal = {arXiv preprint arXiv:2404.09919},
     year = {2024},
   }

2023

1. Conference

   Data-Driven Analysis of Gender Fairness in the Software Engineering Academic Landscape

   Giordano d’Aloisio, Andrea D’Angelo, Francesca Marzi, Diana Di Marco, Giovanni Stilo, and Antinisca Di Marco

   In European Conference on Software Architecture - ECSA, 2023

   arXiv Bib

   @inproceedings{d2023data,
     title = {Data-Driven Analysis of Gender Fairness in the Software Engineering Academic Landscape},
     author = {d'Aloisio, Giordano and D'Angelo, Andrea and Marzi, Francesca and Di Marco, Diana and Stilo, Giovanni and Di Marco, Antinisca},
     booktitle = {European Conference on Software Architecture - ECSA},
     year = {2023}
   }

2. Workshop

   Towards a Prediction of Machine Learning Training Time to Support Continuous Learning Systems Development

   Francesca Marzi, Giordano d’Aloisio, Antinisca Di Marco, and Giovanni Stilo

   In International Workshop on Quality in Software Architecture - QUALIFIER, 2023

   arXiv Bib

   @inproceedings{marzi2023towards,
     title = {Towards a Prediction of Machine Learning Training Time to Support Continuous Learning Systems Development},
     author = {Marzi, Francesca and d'Aloisio, Giordano and Di Marco, Antinisca and Stilo, Giovanni},
     booktitle = {International Workshop on Quality in Software Architecture - QUALIFIER},
     year = {2023}
   }

3. Conference

   Democratizing Quality-Based Machine Learning Development through Extended Feature Models

   Giordano d’Aloisio, Antinisca Di Marco, and Giovanni Stilo

   In Fundamental Approaches to Software Engineering, 2023

   Abs Bib HTML PDF

   ML systems have become an essential tool for experts of many domains, data scientists and researchers, allowing them to find answers to many complex business questions starting from raw datasets. Nevertheless, the development of ML systems able to satisfy the stakeholders’ needs requires an appropriate amount of knowledge about the ML domain. Over the years, several solutions have been proposed to automate the development of ML systems. However, an approach taking into account the new quality concerns needed by ML systems (like fairness, interpretability, privacy, and others) is still missing.

   @inproceedings{daloisio_democratizing_2023,
     address = {Cham},
     series = {Lecture {Notes} in {Computer} {Science}},
     title = {Democratizing {Quality}-{Based} {Machine} {Learning} {Development} through {Extended} {Feature} {Models}},
     copyright = {All rights reserved},
     isbn = {978-3-031-30826-0},
     doi = {https://doi.org/10.1007/978-3-031-30826-0_5},
     language = {en},
     booktitle = {Fundamental {Approaches} to {Software} {Engineering}},
     publisher = {Springer Nature Switzerland},
     author = {d’Aloisio, Giordano and Di Marco, Antinisca and Stilo, Giovanni},
     editor = {Lambers, Leen and Uchitel, Sebastián},
     year = {2023},
     keywords = {/unread, Feature Models, Low-code development, Machine Learning System, Software Product Line, Software Quality},
     pages = {88--110},
   }

4. Journal

   Debiaser for Multiple Variables to enhance fairness in classification tasks

   Giordano d’Aloisio, Andrea D’Angelo, Antinisca Di Marco, and Giovanni Stilo

   Information Processing & Management, 2023

   Abs Bib HTML

   Nowadays assuring that search and recommendation systems are fair and do not apply discrimination among any kind of population has become of paramount importance. This is also highlighted by some of the sustainable development goals proposed by the United Nations. Those systems typically rely on machine learning algorithms that solve the classification task. Although the problem of fairness has been widely addressed in binary classification, unfortunately, the fairness of multi-class classification problem needs to be further investigated lacking well-established solutions. For the aforementioned reasons, in this paper, we present the Debiaser for Multiple Variables (DEMV), an approach able to mitigate unbalanced groups bias (i.e., bias caused by an unequal distribution of instances in the population) in both binary and multi-class classification problems with multiple sensitive variables. The proposed method is compared, under several conditions, with a set of well-established baselines using different categories of classifiers. At first we conduct a specific study to understand which is the best generation strategies and their impact on DEMV’s ability to improve fairness. Then, we evaluate our method on a heterogeneous set of datasets and we show how it overcomes the established algorithms of the literature in the multi-class classification setting and in the binary classification setting when more than two sensitive variables are involved. Finally, based on the conducted experiments, we discuss strengths and weaknesses of our method and of the other baselines.

   @article{daloisio_debiaser_2023,
     title = {Debiaser for {Multiple} {Variables} to enhance fairness in classification tasks},
     volume = {60},
     copyright = {All rights reserved},
     issn = {0306-4573},
     url = {https://www.sciencedirect.com/science/article/pii/S0306457322003272},
     doi = {https://doi.org/10.1016/j.ipm.2022.103226},
     language = {en},
     number = {2},
     urldate = {2022-12-22},
     journal = {Information Processing & Management},
     author = {d’Aloisio, Giordano and D’Angelo, Andrea and Di Marco, Antinisca and Stilo, Giovanni},
     year = {2023},
     keywords = {Machine learning, Multi-class classification, Preprocessing algorithm, Bias and Fairness, Equality},
     pages = {103226}
   }

5. Journal

   The toolkit disaster preparedness for pre-disaster planning

   Donato Di Ludovico, Chiara Capannolo, and Giordano d’Aloisio

   International Journal of Disaster Risk Reduction, 2023

   Abs Bib HTML

   The University of L’Aquila “Territori Aperti” (Open Territories) project deals with the topics of prevention and management of natural disasters and the reconstruction and development processes in the affected areas. One of its tasks is developing research on the Toolkit Disaster Preparedness (TDP) aimed at Pre-Disaster Planning. The TDP is structured in this study as a support for the construction of Recovery Strategies and Actions, and concerns the collection and analysis of good practices on post-disaster reconstruction management (Experience Sheets (ESs)), their elaboration into Disaster Preparedness Recommendation Sheets (DPRSs), and the transposition of these into the Recovery Plan. The methodology for the construction of the Recovery Plan was structured in two macro-activities. The first concerns structuring the Toolkit and the related set of sheets (ESs→DPRSs). The second concerns the transfer of the DPRSs to the Recovery Strategies, so that the recommendations and success measures of the former become the actions of the latter. The Toolkit methodology was applied to the case studies of the Abruzzo 2009 earthquake and the Central Italy 2016-17 earthquake. The next steps of the research will concern testing the methodology in the second macro-activity, i.e. the construction of the Recovery Plan, again in the territorial context of the two aforementioned areas.

   @article{di_ludovico_toolkit_2023,
     title = {The toolkit disaster preparedness for pre-disaster planning},
     volume = {96},
     copyright = {All rights reserved},
     issn = {2212-4209},
     url = {https://www.sciencedirect.com/science/article/pii/S2212420923003692},
     doi = {https://doi.org/10.1016/j.ijdrr.2023.103889},
     language = {en},
     journal = {International Journal of Disaster Risk Reduction},
     author = {Di Ludovico, Donato and Capannolo, Chiara and d'Aloisio, Giordano},
     year = {2023},
     keywords = {/unread, Disasters, Pre-disaster planning, Preparedness, Recovery, Resilience, Toolkit},
     pages = {103889}
   }

6. Workshop

   A Decision Tree to Shepherd Scientists through Data Retrievability

   Andrea Bianchi, Giordano d’Aloisio, Francesca Marzi, and Antinisca Di Marco

   In Second Workshop on Reproducibility and Replication of Research Results, 2023

   Abs Bib HTML

   Reproducibility is a crucial aspect of scientific research that involves the ability to independently replicate experimental results by analysing the same data or repeating the same experiment. Over the years, many works have been proposed to make the results of the experiments actually reproducible. However, very few address the importance of data reproducibility, defined as the ability of independent researchers to retain the same dataset used as input for experimentation. Properly addressing the problem of data reproducibility is crucial because often just providing a link to the data is not enough to make the results reproducible. In fact, also proper metadata (e.g., preprocessing instruction) must be provided to make a dataset fully reproducible. In this work, our aim is to fill this gap by proposing a decision tree to sheperd researchers through the reproducibility of their datasets. In particular, this decision tree guides researchers through identifying if the dataset is actually reproducible and if additional metadata (i.e., additional resources needed to reproduce the data) must also be provided. This decision tree will be the foundation of a future application that will automate the data reproduction process by automatically providing the necessary metadata based on the particular context (e.g., data availability, data preprocessing, and so on). It is worth noting that, in this paper, we detail the steps to make a dataset retrievable, while we will detail other crucial aspects for reproducibility (e.g., dataset documentation) in future works.

   @inproceedings{bianchi2023decision,
     title = {A Decision Tree to Shepherd Scientists through Data Retrievability},
     author = {Bianchi, Andrea and d'Aloisio, Giordano and Marzi, Francesca and Di Marco, Antinisca},
     booktitle = {Second Workshop on Reproducibility and Replication of Research Results},
     year = {2023},
     doi = {https://doi.org/10.48550/arXiv.2304.05767}
   }

2022

1. Workshop

   Enhancing Fairness in Classification Tasks with Multiple Variables: A Data- and Model-Agnostic Approach

   Giordano d’Aloisio, Giovanni Stilo, Antinisca Di Marco, and Andrea D’Angelo

   In Advances in Bias and Fairness in Information Retrieval, 2022

   Abs Bib HTML PDF

   Nowadays assuring that search and recommendation systems are fair and do not apply discrimination among any kind of population has become of paramount importance. Those systems typically rely on machine learning algorithms that solve the classification task. Although the problem of fairness has been widely addressed in binary classification, unfortunately, the fairness of multi-class classification problem needs to be further investigated lacking well-established solutions. For the aforementioned reasons, in this paper, we present the Debiaser for Multiple Variables, a novel approach able to enhance fairness in both binary and multi-class classification problems. The proposed method is compared, under several conditions, with the well-established baseline. We evaluate our method on a heterogeneous data set and prove how it overcomes the established algorithms in the multi-classification setting, while maintaining good performances in binary classification. Finally, we present some limitations and future improvements.

   @inproceedings{10.1007/978-3-031-09316-6_11,
     author = {d'Aloisio, Giordano and Stilo, Giovanni and Di Marco, Antinisca and D'Angelo, Andrea},
     editor = {Boratto, Ludovico and Faralli, Stefano and Marras, Mirko and Stilo, Giovanni},
     title = {Enhancing Fairness in Classification Tasks with Multiple Variables: A Data- and Model-Agnostic Approach},
     booktitle = {Advances in Bias and Fairness in Information Retrieval},
     year = {2022},
     publisher = {Springer International Publishing},
     address = {Cham},
     pages = {117--129},
     isbn = {978-3-031-09316-6},
     doi = {https://doi.org/10.1007/978-3-031-09316-6_11},
   }

2. Conference

   Quality-Driven Machine Learning-based Data Science Pipeline Realization: a software engineering approach

   Giordano d’Aloisio

   In 2022 IEEE/ACM 44th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), 2022

   Abs Bib HTML PDF

   The recently wide adoption of data science approaches to decision making in several application domains (such as health, business and even education) open new challenges in engineering and implementation of this systems. Considering the big picture of data science, Machine learning is the wider used technique and due to its characteristics, we believe that a better engineering methodology and tools are needed to realize innovative data-driven systems able to satisfy the emerging quality attributes (such as, debias and fariness, explainability, privacy and ethics, sustainability). This research project will explore the following three pillars: i) identify key quality attributes, formalize them in the context of data science pipelines and study their relationships; ii) define a new software engineering approach for data-science systems development that assures compliance with quality requirements; iii) implement tools that guide IT professionals and researchers in the realization of ML-based data science pipelines since the requirement engineering. Moreover, in this paper we also presents some details of the project showing how the feature models and model-driven engineering can be leveraged to realize our project.

   @inproceedings{9793779,
     author = {d’Aloisio, Giordano},
     booktitle = {2022 IEEE/ACM 44th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)},
     title = {Quality-Driven Machine Learning-based Data Science Pipeline Realization: a software engineering approach},
     year = {2022},
     volume = {},
     number = {},
     pages = {291-293},
     doi = {https://doi.org/10.1109/ICSE-Companion55297.2022.9793779},
   }

3. Preprint

   Modeling Quality and Machine Learning Pipelines through Extended Feature Models

   Giordano d’Aloisio, Antinisca Di Marco, and Giovanni Stilo

   2022

   arXiv:2207.07528 [cs]

   Abs arXiv Bib

   The recently increased complexity of Machine Learning (ML) methods, led to the necessity to lighten both the research and industry development processes. ML pipelines have become an essential tool for experts of many domains, data scientists and researchers, allowing them to easily put together several ML models to cover the full analytic process starting from raw datasets. Over the years, several solutions have been proposed to automate the building of ML pipelines, most of them focused on semantic aspects and characteristics of the input dataset. However, an approach taking into account the new quality concerns needed by ML systems (like fairness, interpretability, privacy, etc.) is still missing. In this paper, we first identify, from the literature, key quality attributes of ML systems. Further, we propose a new engineering approach for quality ML pipeline by properly extending the Feature Models meta-model. The presented approach allows to model ML pipelines, their quality requirements (on the whole pipeline and on single phases), and quality characteristics of algorithms used to implement each pipeline phase. Finally, we demonstrate the expressiveness of our model considering the classification problem.

   @misc{daloisio_modeling_2022,
     title = {Modeling {Quality} and {Machine} {Learning} {Pipelines} through {Extended} {Feature} {Models}},
     copyright = {All rights reserved},
     booktitle = {arXiv preprint},
     urldate = {2022-07-19},
     publisher = {arXiv},
     author = {d'Aloisio, Giordano and Di Marco, Antinisca and Stilo, Giovanni},
     year = {2022},
     note = {arXiv:2207.07528 [cs]},
     keywords = {Computer Science - Machine Learning, Computer Science - Software Engineering}
   }

2021

1. Workshop

   SismaDL: an ontology to represent post-disaster regulation

   Francesca Caroccia, Damiano D’Agostino, Giordano d’Aloisio, Antinisca Di Marco, and Giovanni Stilo

   In 12th Workshop on Information Logistics and Digital Transformation, 2021

   Abs Bib

   The emergency caused by a natural disaster must be tackled promptly by public institutions. In this situation, Governments enact specific laws (i.e., decrees) to handle the emergency and the reconstruction of destroyed areas. As it happened in 2009 and 2016 when the Italian Government issued several, very different, decrees to face respectively the earthquakes of L’Aquila and Centro Italia. In this work, we propose SismaDL, a LKIF based ontology, that models the laws in the domain of natural disasters. SismaDL has been used to model the aforementioned laws to build a knowledge base useful to reason about why one regulation is less effective and efficient than the other. SismaDL is the first step of a wider project whose aims are: i) compare laws in the domain of natural disaster; ii) integrate such laws in the Semantic Web; iii) evaluate the effectiveness of a post-disaster reconstruction law; iv) identify good practices to build a reference normative model of the natural disaster regulation. This project is a founding step towards the development of accurate and timely IT systems for efficient and high quality disaster management and reconstruction services to support Governments and local institutions in case of natural disasters.

   @inproceedings{caroccia_sismadl_nodate,
     title = {{SismaDL}: an ontology to represent post-disaster regulation},
     language = {en},
     author = {Caroccia, Francesca and D’Agostino, Damiano and d'Aloisio, Giordano and Marco, Antinisca Di and Stilo, Giovanni},
     pages = {14},
     file = {Caroccia et al. - SismaDL an ontology to represent post-disaster re.pdf},
     booktitle = {12th Workshop on Information Logistics and Digital Transformation},
     year = {2021}
   }