DataDecide: How to Predict Best Pretraining Data with Small Experiments (ICML, 2025). PDF

Ian Magnusson*, Nguyen Tai*, Ben Bogin*, David Heineman, Jena D. Hwang, Luca Soldaini, Akshita Bhagia, Jiacheng Liu, Dirk Groeneveld, Oyvind Tafjord, Noah A. Smith, Pang Wei Koh, Jesse Dodge

SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories (EMNLP, 2024). PDF Code

Outstanding Paper Award

Ben Bogin, Kejuan Yang, Shashank Gupta, Kyle Richardson, Erin Bransom, Peter Clark, Ashish Sabharwal, Tushar Khot

AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks? (EMNLP, 2024). PDF Code

Ori Yoran, Samuel Joseph Amouyal, Chaitanya Malaviya, Ben Bogin, Ofir Press, Jonathan Berant

Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research (ACL, 2024). PDF Code

Best Paper Award

Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A. Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, Kyle Lo

Leveraging Code to Improve In-context Learning for Semantic Parsing (NAACL, 2024). PDF Code

Ben Bogin*, Shivanshu Gupta*, Peter Clark, Ashish Sabharwal

Answering Questions by Meta-Reasoning over Multiple Chains of Thought (EMNLP, 2023). PDF Code

Ori Yoran*, Tomer Wolfson*, Ben Bogin, Uri Katz, Daniel Deutch, Jonathan Berant

Diverse Demonstrations Improve In-context Compositional Generalization (ACL, 2023). PDF Code

Itay Levy*, Ben Bogin*, Jonathan Berant

Unobserved Local Structures Make Compositional Generalization Hard (EMNLP, 2022). PDF Code

Ben Bogin, Shivanshu Gupta, Jonathan Berant

COVR: A Test-Bed for Visually Grounded Compositional Generalization with Real Images (EMNLP, 2021). PDF Code Project

Ben Bogin, Shivanshu Gupta, Matt Gardner, Jonathan Berant

Text-to-SQL in the Wild: A Naturally-Occurring Dataset Based on Stack Exchange Data (Workshop on Natural Language Processing for Programming @ ACL, 2021). PDF Code

Moshe Hazoom, Vibhor Malik, Ben Bogin

An autonomous debating system (Nature, 2021). PDF

Noam Slonim, Yonatan Bilu, Carlos Alzate, Roy Bar-Haim, Ben Bogin, Francesca Bonin, Leshem Choshen, Edo Cohen-Karlik, Lena Dankin, Lilach Edelstein, Liat Ein-Dor, Roni Friedman-Melamed, Assaf Gavron, Ariel Gera, Martin Gleize, Shai Gretz, Dan Gutfreund, Alon Halfon, Daniel Hershcovich, Ron Hoory, Yufang Hou, Shay Hummel, Michal Jacovi, Charles Jochim, Yoav Kantor, Yoav Katz, David Konopnicki, Zvi Kons, Lili Kotlerman, Dalia Krieger, Dan Lahav, Tamar Lavee, Ran Levy, Naftali Liberman, Yosi Mass, Amir Menczel, Shachar Mirkin, Guy Moshkowich, Shila Ofek-Koifman, Matan Orbach, Ella Rabinovich, Ruty Rinott, Slava Shechtman, Dafna Sheinwald, Eyal Shnarch, Ilya Shnayderman, Aya Soffer, Artem Spector, Benjamin Sznajder, Assaf Toledo, Orith Toledo-Ronen, Elad Venezian, Ranit Aharonov

Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering (TACL, 2020). PDF Code

Ben Bogin, Sanjay Subramanian, Matt Gardner, Jonathan Berant

Obtaining Faithful Interpretations from Compositional Neural Networks (ACL, 2020). PDF Code

Sanjay Subramanian*, Ben Bogin*, Nitish Gupta*, Tomer Wolfson, Sameer Singh, Jonathan Berant, Matt Gardner

Evaluating nlp models via contrast sets (Findings of EMNLP, 2020). PDF Code

Matt Gardner, Yoav Artzi, Victoria Basmova, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hanna Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, Ben Zhou

MedICaT: A Dataset of Medical Images, Captions, and Textual References (Findings of ACL, 2020). PDF

Sanjay Subramanian, Lucy Lu Wang, Ben Bogin, Sachin Mehta, Madeleine van Zuylen, Sravanthi Parasa, Sameer Singh, Matt Gardner, Hannaneh Hajishirzi

Global Reasoning over Database Structures for Text-to-SQL Parsing (EMNLP, 2019). PDF Code

Ben Bogin, Matt Gardner, Jonathan Berant

Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing (ACL, 2019). PDF Code

Ben Bogin, Matt Gardner, Jonathan Berant

Grammar-based Neural Text-to-SQL Generation (2019). PDF

Kevin Lin, Ben Bogin, Mark Neumann, Jonathan Berant, Matt Gardner

Emergence of Communication in an Interactive World with Consistent Speakers (Emergent Communication Workshop at NIPS, 2018). PDF Code

Ben Bogin, Mor Geva, Jonathan Berant

Towards an argumentative content search engine using weak supervision (COLING, 2018). PDF

Ran Levy*, Ben Bogin*, Shai Gretz*, Ranit Aharonov, Noam Slonim

Language Generation with Recurrent Generative Adversarial Networks without Pre-training (Subword and Character level models in NLP Workshop, EMNLP, 2017). PDF Code

Ofir Press*, Amir Bar*, Ben Bogin*, Jonathan Berant, Lior Wolf