Discovery Lab (Completed)

A collaboration between the University of Amsterdam, VU University Amsterdam, and Elsevier.

Discovery Lab (Completed)

De Boelelaan 1105 + Science Park 900, 1098 XH Amsterdam

Jump to

The Discovery Lab centers its research on three core themes: intelligent scientific search and literature analysis, automated hypothesis formulation, and data interpretation tooling.

To drive these advancements, the lab leverages key technical AI components, focusing primarily on:

Information Retrieval
Knowledge Representation & Reasoning
Natural Language Processing
Machine Learning

With a commitment to explainable AI systems.

Sustainable Development Goals

About the lab

The Discovery Lab is dedicated to bridging the gap between machine intelligence research and the evolving needs of modern scientific discovery.

The lab’s mission and vision are to build the fundamental knowledge base, technological tools, and infrastructure necessary to transition to a new generation of data-driven science, where researchers are empowered and supported by intelligent tools to search literature, formulate hypotheses, and interpret complex data.

The impact of the lab lies in accelerating the current transformation of science, achieved by combining the expertise of the Vrije Universiteit Amsterdam, the University of Amsterdam, and Elsevier to pioneer methods and methods-driven technologies that support scientists in navigating and advancing scientific knowledge.

Research projects

Knowledge graph – Seeks to advance the ability to construct, use and study large-scale knowledge graphs that integrate knowledge across heterogeneous scientific content and data.

‍

Reinforcement learning – Investigates the use of reinforcement learning to work with structured multi-modal information such as user context, knowledge graphs, or text as inputs.

‍

Question answering – Focuses on challenging scenarios that require question answering models to reason, gather, and synthesize disjoint pieces of information, within context, to generate an answer.

Publications

Discovery Lab

71 entries « ‹ 1 of 2 › »

2025

Pernisch, R.; Dobriy, D.; Polleres, A.

The Massive Problem of Remote Changes in Ontology Reuse Proceedings Article

In: Companion Proceedings of the ACM on Web Conference 2025, pp. 1254–1258, Association for Computing Machinery, New York, NY, USA, 2025, ISBN: 979-8-4007-1331-6.

Abstract | Links | BibTeX

Höpner, N.; Eshuijs, L.; Alivanistos, D.; Zamprogno, G.; Tiddi, I.

Automatic Evaluation Metrics for Artificially Generated Scientific Research Proceedings Article

In: First Workshop on AI and Scientific Discovery: Directions and Opportunities, 2025.

Bakel, R.; Cochez, M.; Koopmann, P.

Towards Conceptual Clustering in EL with Simulation Graphs Proceedings Article

In: Proceedings of the 38th International Workshop on Description Logics - DL 2025, 2025.

Kamburjan, E.; Pernisch, R.; Corcho, Ó.; Chaves-Fraga, D.

On Dependencies in Knowledge Graph Construction Proceedings Article

In: Proceedings of the 6th International Workshop on Knowledge Graph Construction co-located with 22nd Extended Semantic Web Conference (ESWC 2025), Portorož, Slovenia, June 1, 2025, CEUR-WS.org, 2025.

Pelletreau-Duris, T.; van Bakel, R.; Cochez, M.

Do Graph Neural Network States Contain Graph Properties? Proceedings Article

In: Proceedings of The 19th International Conference on Neurosymbolic Learning and Reasoning, pp. 15–51, PMLR, 2025.

Abstract | Links | BibTeX

Brunink, Y.; Cochez, M.; Urbani, J.

The ART of Link Prediction with KGEs Proceedings Article

In: Proceedings of The 19th International Conference on Neurosymbolic Learning and Reasoning, pp. 519–539, PMLR, 2025.

Abstract | Links | BibTeX

2024

Adamik, M.; Pernisch, R.; Tiddi, I.; Schlobach, S.

ORKA: An Ontology for Robotic Knowledge Acquisition Proceedings Article

In: Proceedings of 24th International Conference on Knowledge Engineering and Knowledge Management (EKAW-24), Springer, 2024.

Adamik, M.; Pernisch, R.; Tiddi, I.; Schlobach, S.

Advancing Robotic Perception with Perceived-Entity Linking Proceedings Article

In: The Semantic Web - ISWC 2024, Springer, 2024.

Pernisch, R.; Poveda-Villalón, M.; Conde-Herreros, D.; Chaves-Fraga, D.; Stork, L.

When Ontologies met Knowledge Graphs: A Methodology Tale Proceedings Article

In: The Semantic Web: ESWC 2024 Satellite Events, LNCS, Hersonissos, Greece, 2024.

Mansoury, M.; Mobasher, B.; Hoof, H.

Mitigating Exposure Bias in Online Learning to Rank Recommendation: A Novel Reward Model for Cascading Bandits Proceedings Article

In: 2024, ISBN: 979-8-4007-0436-9.

Conde-Herreros, D.; Stork, L.; Pernisch, R.; Poveda-Villalón, M.; Corcho, Ó.; Chaves-Fraga, D.

Propagating Ontology Changes to Declarative Mappings in Construction of Knowledge Graphs Proceedings Article

In: Proceedings of the 5th International Workshop on Knowledge Graph Construction co-located with 21th Extended Semantic Web Conference (ESWC 2024), Hersonissos, Greece, May 27, 2024, CEUR-WS.org, 2024.

Pernisch, R.; Huijgens, H.; Schlobach, S.; Mattheij, R.; Benders, F.; Beusekom, H.; Bomhof, F.

Knowledge Graph Lifecycle Management within Hybrid Artificial Intelligence Solutions Proceedings Article

In: Software Lifecycle Management for Knowledge Graphs Workshop Co-located with the ISWC 2024, CEUR-ws.org, 2024.

Alivanistos, D.; van der Bijl, S.; Cochez, M.; van Harmelen, F.

The Effect of Knowledge Graph Schema on Classifying Future Research Suggestions Proceedings Article

In: Natural Scientific Language Processing and Research Knowledge Graphs - First International Workshop, NSLP 2024, Hersonissos, Crete, Greece, May 27, 2024, Proceedings, pp. 149–170, Springer, 2024.

Huang, J.; Oosterhuis, H.; Mansoury, M.; Hoof, H.; de Rijke, M.

Going Beyond Popularity and Positivity Bias: Correcting for Multifactorial Bias in Recommender Systems Proceedings Article

In: 2024, ISBN: 979-8-4007-0431-4.

Ren, H.; Galkin, M.; Zhu, Z.; Leskovec, J.; Cochez, M.

Neural Graph Reasoning: A Survey on Complex Logical Query Answering Journal Article

In: 2024, ISSN: 2835-8856.

Wang, R.; Rossetto, L.; Cochez, M.; Bernstein, A.

QAGCN: Answering Multi-relation Questions via Single-Step Implicit Reasoning over Knowledge Graphs Proceedings Article

In: The Semantic Web - 21st International Conference, ESWC 2024, Hersonissos, Crete, Greece, May 26-30, 2024, Proceedings, Part I, pp. 41–58, Springer, 2024.

Liu, Y.; Li, M.; Ariannezhad, M.; Mansoury, M.; Aliannejadi, M.; de Rijke, M.

Measuring Item Fairness in Next Basket Recommendation: A Reproducibility Study Proceedings Article

In: Springer Nature Switzerland, 2024.

Cucumides, T.; Daza, D.; Barcelo, P.; Cochez, M.; Geerts, F.; Reutter, J. L.; Orth, M. R.

UnRavL: A Neuro-Symbolic Framework for Answering Graph Pattern Queries in Knowledge Graphs Proceedings Article

In: The Third Learning on Graphs Conference, 2024.

2023

Daza, D.; Alivanistos, D.; Mitra, P.; Pijnenburg, T.; Cochez, M.; Groth, P.

BioBLP: a modular framework for learning on multimodal biomedical knowledge graphs Journal Article

In: vol. 14, no. 1, pp. 20, 2023, ISSN: 2041-1480.

Abstract | Links | BibTeX

Verkijk, S.; Roothaert, R.; Pernisch, R.; Schlobach, S.

Do you catch my drift? On the usage of embedding methods to measure concept shift in knowledge graphs Proceedings Article

In: Proceedings of the 12th Knowledge Capture Conference 2023, pp. 70–74, Association for Computing Machinery, New York, NY, USA, 2023, ISBN: 979-8-4007-0141-2.

Abstract | Links | BibTeX

Mansoury, M.; Duijvestijn, F.; Mourabet, I.

Potential Factors Leading to Popularity Unfairness in Recommender Systems: A User-Centered Analysis Proceedings Article

In: BNAIC/BeNeLearn Joint International Scientific Conferences on AI and Machine Learning, 2023.

Ducu, A. M.; Cochez, M.

Qualifier Recommendation for Wikidata Proceedings Article

In: Athens, Greece, 2023.

Abstract | Links | BibTeX

Heuss, M.; Cohen, D.; Mansoury, M.; de Rijke, M.; Eickhoff, C.

Predictive Uncertainty-based Bias Mitigation in Ranking Proceedings Article

In: 32nd ACM International Conference on Information and Knowledge Management, 2023, ISBN: 979-8-4007-0124-5.

Cochez, M.; Alivanistos, D.; Arakelyan, E.; Berrendorf, M.; Daza, D.; Galkin, M.; Minervini, P.; Niepert, M.; Ren, H.

Approximate Answering of Graph Queries Book Section

In: Compendium of Neurosymbolic Artificial Intelligence, vol. 369, pp. 373–386, IOS Press, 2023, ISBN: 978-1-64368-407-9.

Avlonitis, S.; Lavi, D.; Mansoury, M.; Graus, D.

Career Path Recommendations for Long-term Income Maximization: A Reinforcement Learning Approach Proceedings Article

In: RecSys in HR’23: The 3rd Workshop on Recommender Systems for Human Resources, in conjunction with the 17th ACM Conference on Recommender Systems, 2023.

Huang, J.; Oosterhuis, H.; Mansoury, M.; Hoof, H.; de Rijke, M.

Managing Multifactorial Bias in Recommender Systems Proceedings Article

In: CONSEQUENCES Workshop on Causality, Counterfactuals, and Sequential Decision-Making in conjunction with ACM RecSys 2023, 2023.

Mansoury, M.; Mobasher, B.

Fairness of Exposure in Dynamic Recommendation Proceedings Article

In: CONSEQUENCES Workshop on Causality, Counterfactuals, and Sequential Decision-Making in conjunction with ACM RecSys 2023, 2023.

Renaux, A.; Terwagne, C.; Cochez, M.; Tiddi, I.; Nowé, A.; Lenaerts, T.

A knowledge graph approach to predict and interpret disease-causing gene interactions Journal Article

In: vol. 24, no. 1, pp. 324, 2023.

Polleres, A.; Pernisch, R.; Bonifati, A.; Dell'Aglio, D.; Dobriy, D.; Dumbrava, S.; Etcheverry, L.; Ferranti, N.; Hose, K.; Jiménez-Ruiz, E.; Lissandrini, M.; Scherp, A.; Tommasini, R.; Wachs, J.

How Does Knowledge Evolve in Open Knowledge Graphs? Journal Article

In: 2023.

Abstract | Links | BibTeX

Pal, V.; Lassance, C.; Déjean, H.; Clinchant, S.

Parameter-Efficient Sparse Retrievers and Rerankers Using Adapters Proceedings Article

In: Advances in Information Retrieval, pp. 16–31, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-28238-6.

Abstract | Links | BibTeX

Chung, Kornpol; Pernisch, Romana; Schlobach, Stefan

Descriptive Comparison of Visual Ontology Change Summarisation Methods Proceedings Article

In: Pesquita, C.; Skaf-Molli, H.; Efthymiou, V.; Kirrane, S.; Ngonga, A.; Collarana, D.; Cerqueira, R.; Alam, M.; Trojahn, C.; Hertling, S. (Ed.): The Semantic Web: ESWC 2023 Satellite Events, pp. 54–58, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-43457-0 978-3-031-43458-7.

Arakelyan, E.; Minervini, P.; Daza, D.; Cochez, M.; Augenstein, I.

Adapting Neural Link Predictors for Data-Efficient Complex Query Answering Proceedings Article

In: OpenReview.net, New Orleans, 2023.

Abstract | Links | BibTeX

@inproceedings{arakelyan_adapting_2023,

title = {Adapting Neural Link Predictors for Data-Efficient Complex Query Answering},

author = {E. Arakelyan and P. Minervini and D. Daza and M. Cochez and I. Augenstein},

url = {https://openreview.net/forum?id=1G7CBp8o7L},

doi = {10.48550/arXiv.2301.12313},

year  = {2023},

date = {2023-07-11},

urldate = {2023-07-11},

publisher = {OpenReview.net},

address = {New Orleans},

abstract = {Answering complex queries on incomplete knowledge graphs is a challenging task where a model needs to answer complex logical queries in the presence of missing knowledge. Prior work in the literature has proposed to address this problem by designing architectures trained end-to-end for the complex query answering task with a reasoning process that is hard to interpret while requiring data and resource-intensive training. Other lines of research have proposed re-using simple neural link predictors to answer complex queries, reducing the amount of training data by orders of magnitude while providing interpretable answers. The neural link predictor used in such approaches is not explicitly optimised for the complex query answering task, implying that its scores are not calibrated to interact together. We propose to address these problems via CQD 

, a parameter-efficient score textbackslashemphadaptation model optimised to re-calibrate neural link prediction scores for the complex query answering task. While the neural link predictor is frozen, the adaptation component – which only increases the number of model parameters by 

 – is trained on the downstream complex query answering task. Furthermore, the calibration component enables us to support reasoning over queries that include atomic negations, which was previously impossible with link predictors. In our experiments, CQD 

 produces significantly more accurate results than current state-of-the-art methods, improving from 

 to 

 Mean Reciprocal Rank values averaged across all datasets and query types while using 

 of the available training query types. We further show that CQD 

 is data-efficient, achieving competitive results with only 

 of the complex training queries and robust in out-of-domain evaluations. Source code and datasets are available at https://github.com/EdinburghNLP/adaptive-cqd.},

keywords = {},

pubstate = {published},

tppubtype = {inproceedings}

}

Karim, M. Rezaul; Islam, T.; Shajalal, M.; Beyan, O.; Lange, C.; Cochez, M.; Rebholz-Schuhmann, D.; Decker, S.

Explainable AI for Bioinformatics: Methods, Tools and Applications Journal Article

In: vol. 24, no. 5, 2023.

Kim, T.; Cochez, M.; Lavet, V. F.; Neerincx, M.; Vossen, P.

A Machine With Human-Like Memory Systems Proceedings Article

In: arXiv preprint arXiv:2204.01611, 2023.

Scherp, A.; Richerby, D.; Blume, T.; Cochez, M.; Rau, J.

Structural Summarization of Semantic Graphs Using Quotients Journal Article

In: vol. 1, no. 1, 2023.

Abstract | BibTeX

Xiong, B.; Nayyeri, M.; Daza, D.; Cochez, M.

Reasoning beyond Triples: Recent Advances in Knowledge Graph Embeddings Proceedings Article

In: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, CIKM 2023, Birmingham, United Kingdom, October 21-25, 2023, pp. 5228–5231, ACM, 2023.

2022

Xiong, B.; Cochez, M.; Nayyeri, M.; Staab, S.

Hyperbolic Embedding Inference for Structured Multi-Label Prediction Proceedings Article

In: NeurIPS2022, 2022.

Alivanistos, D.; Santamaría, S. B.; Cochez, M.; Kalo, J. C.; Krieken, E.; Thanapalasingam, T.

Prompting as Probing: Using Language Models for Knowledge Base Construction Proceedings Article

In: Proceedings of the Semantic Web Challenge on Knowledge Base Construction from Pre-trained Language Models 2022 co-located with the 21st International Semantic Web Conference (ISWC2022), pp. 11–34, CEUR-ws.org, 2022.

Vardasbi, A.; de Rijke, M.; Dehghani, M.

Intersection of Parallels as an Early Stopping Criterion Proceedings Article

In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pp. 1965–1974, Association for Computing Machinery, New York, NY, USA, 2022, ISBN: 978-1-4503-9236-5.

Abstract | Links | BibTeX

@inproceedings{vardasbi_intersection_2022,

title = {Intersection of Parallels as an Early Stopping Criterion},

author = {A. Vardasbi and M. de Rijke and M. Dehghani},

url = {https://dl.acm.org/doi/10.1145/3511808.3557366},

doi = {10.1145/3511808.3557366},

isbn = {978-1-4503-9236-5},

year  = {2022},

date = {2022-10-17},

urldate = {2022-10-17},

booktitle = {Proceedings of the 31st ACM International Conference on Information & Knowledge Management},

pages = {1965–1974},

publisher = {Association for Computing Machinery},

address = {New York, NY, USA},

series = {CIKM '22},

abstract = {A common way to avoid overfitting in supervised learning is early stopping, where a held-out set is used for iterative evaluation during training to find a sweet spot in the number of training steps that gives maximum generalization. However, such a method requires a disjoint validation set, thus part of the labeled data from the training set is usually left out for this purpose, which is not ideal when training data is scarce. Furthermore, when the training labels are noisy, the performance of the model over a validation set may not be an accurate proxy for generalization. In this paper, we propose a method to spot an early stopping point in the training iterations of an overparameterized (NN) without the need for a validation set. We first show that in the overparameterized regime the randomly initialized weights of a linear model converge to the same direction during training. Using this result, we propose to train two parallel instances of a linear model, initialized with different random seeds, and use their intersection as a signal to detect overfitting. In order to detect intersection, we use the cosine distance between the weights of the parallel models during training iterations. Noticing that the final layer of a NN is a linear map of pre-last layer activations to output logits, we build on our criterion for linear models and propose an extension to multi-layer networks, using the new notion of counterfactual weights. We conduct experiments on two areas that early stopping has noticeable impact on preventing overfitting of a NN: (i) learning from noisy labels; and (ii) learning to rank in information retrieval. Our experiments on four widely used datasets confirm the effectiveness of our method for generalization. For a wide range of learning rates, our method, called Cosine-Distance Criterion (CDC), leads to better generalization on average than all the methods that we compare against in almost all of the tested cases.},

keywords = {},

pubstate = {published},

tppubtype = {inproceedings}

}

Pernisch, R.; Dell’Aglio, D.; Serbak, M.; Gonçalves, R. S.; Bernstein, A.

Visualising the effects of ontology changes and studying their understanding with ChImp Journal Article

In: vol. 74, pp. 100715, 2022, ISSN: 1570-8268.

Abstract | Links | BibTeX

@article{pernisch_visualising_2022,

title = {Visualising the effects of ontology changes and studying their understanding with ChImp},

author = {R. Pernisch and D. Dell’Aglio and M. Serbak and R. S. Gonçalves and A. Bernstein},

url = {https://www.sciencedirect.com/science/article/pii/S1570826822000117},

doi = {10.1016/j.websem.2022.100715},

issn = {1570-8268},

year  = {2022},

date = {2022-10-01},

urldate = {2022-10-01},

volume = {74},

pages = {100715},

abstract = {Due to the Semantic Web’s decentralised nature, ontology engineers rarely know all applications that leverage their ontology. Consequently, they are unaware of the full extent of possible consequences that changes might cause to the ontology. Our goal is to lessen the gap between ontology engineers and users by investigating ontology engineers’ understanding of ontology changes’ impact at editing time. Hence, this paper introduces the Protégé plugin ChImp which we use to reach our goal. We elicited requirements for ChImp through a questionnaire with ontology engineers. We then developed ChImp according to these requirements and it displays all changes of a given session and provides selected information on said changes and their effects. For each change, it computes a number of metrics on both the ontology and its materialisation. It displays those metrics on both the originally loaded ontology at the beginning of the editing session and the current state to help ontology engineers understand the impact of their changes. We investigated the informativeness of materialisation impact measures, the meaning of severe impact, and also the usefulness of ChImp in an online user study with 36 ontology engineers. We asked the participants to solve two ontology engineering tasks – with and without ChImp (assigned in random order) – and answer in-depth questions about the applied changes as well as the materialisation impact measures. We found that ChImp increased the participants’ understanding of change effects and that they felt better informed. Answers also suggest that the proposed measures were useful and informative. We also learned that the participants consider different outcomes of changes severe, but most would define severity based on the amount of changes to the materialisation compared to its size. The participants also acknowledged the importance of quantifying the impact of changes and that the study will affect their approach of editing ontologies.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Due to the Semantic Web’s decentralised nature, ontology engineers rarely know all applications that leverage their ontology. Consequently, they are unaware of the full extent of possible consequences that changes might cause to the ontology. Our goal is to lessen the gap between ontology engineers and users by investigating ontology engineers’ understanding of ontology changes’ impact at editing time. Hence, this paper introduces the Protégé plugin ChImp which we use to reach our goal. We elicited requirements for ChImp through a questionnaire with ontology engineers. We then developed ChImp according to these requirements and it displays all changes of a given session and provides selected information on said changes and their effects. For each change, it computes a number of metrics on both the ontology and its materialisation. It displays those metrics on both the originally loaded ontology at the beginning of the editing session and the current state to help ontology engineers understand the impact of their changes. We investigated the informativeness of materialisation impact measures, the meaning of severe impact, and also the usefulness of ChImp in an online user study with 36 ontology engineers. We asked the participants to solve two ontology engineering tasks – with and without ChImp (assigned in random order) – and answer in-depth questions about the applied changes as well as the materialisation impact measures. We found that ChImp increased the participants’ understanding of change effects and that they felt better informed. Answers also suggest that the proposed measures were useful and informative. We also learned that the participants consider different outcomes of changes severe, but most would define severity based on the amount of changes to the materialisation compared to its size. The participants also acknowledged the importance of quantifying the impact of changes and that the study will affect their approach of editing ontologies.

Harper, C. A.; Daniel, R.; Groth, P.

Question Answering with Additive Restrictive Training (QuAART): Question Answering for the Rapid Development of New Knowledge Extraction Pipelines Proceedings Article

In: Knowledge Engineering and Knowledge Management - 23rd International Conference, EKAW 2022, Bolzano, Italy, September 26-29, 2022, Proceedings, pp. 51–65, Springer, 2022.

Mansoury, M.; Mobasher, B.; Hoof, H.

Exposure-Aware Recommendation using Contextual Bandits Proceedings Article

In: 5th FAccTRec Workshop on Responsible Recommendation in conjunction with ACM RecSys 2022, 2022.

Abdollahpouri, H.; Sahebi, S.; Elahi, M.; Mansoury, M.; Loni, B.; Nazari, Z.; Dimakopoulou, M.

MORS 2022: The Second Workshop on Multi-Objective Recommender Systems Proceedings Article

In: Sixteenth ACM Conference on Recommender Systems, pp. 658–660, ACM, Seattle WA USA, 2022, ISBN: 978-1-4503-9278-5.

Daza, D.; Cochez, M.; Groth, P.

SlotGAN: Detecting Mentions in Text via Adversarial Distant Learning Proceedings Article

In: Proceedings of the Sixth Workshop on Structured Prediction for NLP, pp. 32–39, Association for Computational Linguistics, Dublin, Ireland, 2022.

Abstract | Links | BibTeX

Wang, X.; Harmelen, F.; Cochez, M.; Huang, Z.

Scientific Item Recommendation Using a Citation Network Proceedings Article

In: Knowledge Science, Engineering and Management, pp. 469–484, Springer International Publishing, Cham, 2022, ISBN: 978-3-031-10986-7.

Wang, X.; Harmelen, F.; Huang, Z.

Recommending scientific datasets using author networks in ensemble methods Journal Article

In: vol. 5, no. 2, pp. 167–193, 2022, ISSN: 2451-8484.

Abstract | Links | BibTeX

Penha, G.; Vakulenko, S.; Dusek, O.; Clark, L.; Pal, V.; Adlakha, V.

The Seventh Workshop on Search-Oriented Conversational Artificial Intelligence (SCAI'22) Proceedings Article

In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 3466–3469, Association for Computing Machinery, New York, NY, USA, 2022, ISBN: 978-1-4503-8732-3.

Abstract | Links | BibTeX

Vardasbi, A.; Sarvi, F.; de Rijke, M.

Probabilistic Permutation Graph Search: Black-Box Optimization for Fairness in Ranking Proceedings Article

In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 715–725, Association for Computing Machinery, New York, NY, USA, 2022, ISBN: 978-1-4503-8732-3.

Abstract | Links | BibTeX

@inproceedings{vardasbi_probabilistic_2022,

title = {Probabilistic Permutation Graph Search: Black-Box Optimization for Fairness in Ranking},

author = {A. Vardasbi and F. Sarvi and M. de Rijke},

url = {https://doi.org/10.1145/3477495.3532045},

doi = {10.1145/3477495.3532045},

isbn = {978-1-4503-8732-3},

year  = {2022},

date = {2022-07-07},

urldate = {2022-07-07},

booktitle = {Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval},

pages = {715–725},

publisher = {Association for Computing Machinery},

address = {New York, NY, USA},

series = {SIGIR '22},

abstract = {There are several measures for fairness in ranking, based on different underlying assumptions and perspectives. textbackslashacPL optimization with the REINFORCE algorithm can be used for optimizing black-box objective functions over permutations. In particular, it can be used for optimizing fairness measures. However, though effective for queries with a moderate number of repeating sessions, textbackslashacPL optimization has room for improvement for queries with a small number of repeating sessions. In this paper, we present a novel way of representing permutation distributions, based on the notion of permutation graphs. Similar totextasciitildetextbackslashacPL, our distribution representation, calledtextasciitildetextbackslashacPPG, can be used for black-box optimization of fairness. Different fromtextasciitildetextbackslashacPL, where pointwise logits are used as the distribution parameters, intextasciitildetextbackslashacPPG pairwise inversion probabilities together with a reference permutation construct the distribution. As such, the reference permutation can be set to the best sampled permutation regarding the objective function, makingtextasciitildetextbackslashacPPG suitable for both deterministic and stochastic rankings. Our experiments show thattextasciitildetextbackslashacPPG, while comparable totextasciitildetextbackslashacPL for larger session repetitions (i.e., stochastic ranking), improves overtextasciitildetextbackslashacPL for optimizing fairness metrics for queries with one session (i.e., deterministic ranking). Additionally, when accurate utility estimations are available, e.g., in tabular models, the performance of textbackslashacPPG in fairness optimization is significantly boosted compared to lower quality utility estimations from a learning to rank model, leading to a large performance gap with PL. Finally, the pairwise probabilities make it possible to impose pairwise constraints such as "item $d_1$ should always be ranked higher than item $d_2$.'' Such constraints can be used to simultaneously optimize the fairness metric and control another objective such as ranking performance.},

keywords = {},

pubstate = {published},

tppubtype = {inproceedings}

}

There are several measures for fairness in ranking, based on different underlying assumptions and perspectives. textbackslashacPL optimization with the REINFORCE algorithm can be used for optimizing black-box objective functions over permutations. In particular, it can be used for optimizing fairness measures. However, though effective for queries with a moderate number of repeating sessions, textbackslashacPL optimization has room for improvement for queries with a small number of repeating sessions. In this paper, we present a novel way of representing permutation distributions, based on the notion of permutation graphs. Similar totextasciitildetextbackslashacPL, our distribution representation, calledtextasciitildetextbackslashacPPG, can be used for black-box optimization of fairness. Different fromtextasciitildetextbackslashacPL, where pointwise logits are used as the distribution parameters, intextasciitildetextbackslashacPPG pairwise inversion probabilities together with a reference permutation construct the distribution. As such, the reference permutation can be set to the best sampled permutation regarding the objective function, makingtextasciitildetextbackslashacPPG suitable for both deterministic and stochastic rankings. Our experiments show thattextasciitildetextbackslashacPPG, while comparable totextasciitildetextbackslashacPL for larger session repetitions (i.e., stochastic ranking), improves overtextasciitildetextbackslashacPL for optimizing fairness metrics for queries with one session (i.e., deterministic ranking). Additionally, when accurate utility estimations are available, e.g., in tabular models, the performance of textbackslashacPPG in fairness optimization is significantly boosted compared to lower quality utility estimations from a learning to rank model, leading to a large performance gap with PL. Finally, the pairwise probabilities make it possible to impose pairwise constraints such as "item $d_1$ should always be ranked higher than item $d_2$.'' Such constraints can be used to simultaneously optimize the fairness metric and control another objective such as ranking performance.

Generale, A.; Blume, T.; Cochez, M.

Scaling R-GCN Training with Graph Summarization Proceedings Article

In: WWW '22: Companion Proceedings of the Web Conference 2022, pp. 1073–1082, Association for Computing Machinery, Virtual Event, Lyon, France, 2022, ISBN: 978-1-4503-9130-6, (arXiv preprint arXiv:2203.02622).

Alivanistos, D.; Berrendorf, M.; Cochez, M.; Galkin, M.

Query Embedding on Hyper-Relational Knowledge Graphs Proceedings Article

In: The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, OpenReview.net, 2022.

71 entries « ‹ 1 of 2 › »

People

Partners

Vrije Universiteit Amsterdam (VU) is a broad research and teaching university in Amsterdam founded in 1880.

‍University of Amsterdam (UvA) is the Netherlands’ largest university, offering the widest range of academic programmes.

‍Elsevier is a global information analytics company that helps institutions and professionals progress science, advance healthcare and improve performance.

Newsletter

Stay in the loop