Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Standard

Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics. / Bhargava, Prajjwal; Drozd, Aleksandr; Rogers, Anna.

Proceedings of the Second Workshop on Insights from Negative Results in NLP. Online and Punta Cana, Dominican Republic : Association for Computational Linguistics (ACL), 2021. p. 125-135.

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Harvard

Bhargava, P, Drozd, A & Rogers, A 2021, Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics. in Proceedings of the Second Workshop on Insights from Negative Results in NLP. Association for Computational Linguistics (ACL), Online and Punta Cana, Dominican Republic, pp. 125-135. <https://aclanthology.org/2021.insights-1.18>

APA

Bhargava, P., Drozd, A., & Rogers, A. (2021). Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics. In Proceedings of the Second Workshop on Insights from Negative Results in NLP (pp. 125-135). Association for Computational Linguistics (ACL). https://aclanthology.org/2021.insights-1.18

Vancouver

Bhargava P, Drozd A, Rogers A. Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics. In Proceedings of the Second Workshop on Insights from Negative Results in NLP. Online and Punta Cana, Dominican Republic: Association for Computational Linguistics (ACL). 2021. p. 125-135

Author

Bhargava, Prajjwal ; Drozd, Aleksandr ; Rogers, Anna. / Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics. Proceedings of the Second Workshop on Insights from Negative Results in NLP. Online and Punta Cana, Dominican Republic : Association for Computational Linguistics (ACL), 2021. pp. 125-135

Bibtex

@inproceedings{222844cdf1d041a596ec723837a773d8,
title = "Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics",
abstract = "Much of recent progress in NLU was shown to be due to models' learning dataset-specific heuristics. We conduct a case study of generalization in NLI (from MNLI to the adversarially constructed HANS dataset) in a range of BERT-based architectures (adapters, Siamese Transformers, HEX debiasing), as well as with subsampling the data and increasing the model size. We report 2 successful and 3 unsuccessful strategies, all providing insights into how Transformer-based models learn to generalize.",
keywords = "t/generalization, task/NLI",
author = "Prajjwal Bhargava and Aleksandr Drozd and Anna Rogers",
year = "2021",
month = nov,
day = "1",
language = "English",
pages = "125--135",
booktitle = "Proceedings of the Second Workshop on Insights from Negative Results in NLP",
publisher = "Association for Computational Linguistics (ACL)",
address = "United States",

}

RIS

TY - GEN

T1 - Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics

AU - Bhargava, Prajjwal

AU - Drozd, Aleksandr

AU - Rogers, Anna

PY - 2021/11/1

Y1 - 2021/11/1

N2 - Much of recent progress in NLU was shown to be due to models' learning dataset-specific heuristics. We conduct a case study of generalization in NLI (from MNLI to the adversarially constructed HANS dataset) in a range of BERT-based architectures (adapters, Siamese Transformers, HEX debiasing), as well as with subsampling the data and increasing the model size. We report 2 successful and 3 unsuccessful strategies, all providing insights into how Transformer-based models learn to generalize.

AB - Much of recent progress in NLU was shown to be due to models' learning dataset-specific heuristics. We conduct a case study of generalization in NLI (from MNLI to the adversarially constructed HANS dataset) in a range of BERT-based architectures (adapters, Siamese Transformers, HEX debiasing), as well as with subsampling the data and increasing the model size. We report 2 successful and 3 unsuccessful strategies, all providing insights into how Transformer-based models learn to generalize.

KW - t/generalization

KW - task/NLI

M3 - Article in proceedings

SP - 125

EP - 135

BT - Proceedings of the Second Workshop on Insights from Negative Results in NLP

PB - Association for Computational Linguistics (ACL)

CY - Online and Punta Cana, Dominican Republic

ER -

ID: 285387385