Batch Prompting: Efficient Inference with Large Language Model APIs

19 January 2023

Tao Yu

Papers citing "Batch Prompting: Efficient Inference with Large Language Model APIs"

50 / 54 papers shown

Title
MAC-Tuning: LLM Multi-Compositional Problem Reasoning with Enhanced Knowledge Boundary Awareness Junsheng Huang Zhitao He Sandeep Polisetty Q. Wang May Fung KELM 45 0 0 30 Apr 2025
Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization Minsu Kim Seongmin Hong RyeoWook Ko S. Choi Hunjong Lee Junsoo Kim Joo-Young Kim Jongse Park 57 0 0 24 Mar 2025
Efficient but Vulnerable: Benchmarking and Defending LLM Batch Prompting Attack Murong Yue Ziyu Yao SILM AAML 58 0 0 18 Mar 2025
LexGenie: Automated Generation of Structured Reports for European Court of Human Rights Case Law T. Y. S. S. Santosh Mahmoud Aly O. Ichim Matthias Grabmair AILaw ELM 90 0 0 05 Mar 2025
BatchGEMBA: Token-Efficient Machine Translation Evaluation with Batched Prompting and Prompt Compression Daniil Larionov Steffen Eger VLM MQ 77 0 0 04 Mar 2025
Improving LLM-as-a-Judge Inference with the Judgment Distribution Victor Wang Michael J.Q. Zhang Eunsol Choi 58 1 0 04 Mar 2025
Can LLMs Help Uncover Insights about LLMs? A Large-Scale, Evolving Literature Analysis of Frontier LLMs Jungsoo Park Junmo Kang Gabriel Stanovsky Alan Ritter 57 0 0 26 Feb 2025
MEG: Medical Knowledge-Augmented Large Language Models for Question Answering Laura Cabello Carmen Martin-Turrero Uchenna Akujuobi Anders Søgaard Carlos Bobed AI4MH 154 1 0 06 Nov 2024
EcoAct: Economic Agent Determines When to Register What Action Shaokun Zhang Jieyu Zhang Dujian Ding Mirian Hipolito Garcia Ankur Mallick Daniel Madrigal Menglin Xia Victor Rühle Qingyun Wu Chi Wang LLMAG 50 4 0 03 Nov 2024
BATON: Enhancing Batch-wise Inference Efficiency for Large Language Models via Dynamic Re-batching Peizhuang Cong Qizhi Chen Haochen Zhao Tong Yang KELM 29 1 0 24 Oct 2024
Expanding Chatbot Knowledge in Customer Service: Context-Aware Similar Question Generation Using Large Language Models Mengze Hong Yuanfeng Song Di Jiang Lu Wang Zichang Guo Yuanqin He Zhiyang Su Qing Li 37 2 0 16 Oct 2024
Auto-Demo Prompting: Leveraging Generated Outputs as Demonstrations for Enhanced Batch Prompting Longyu Feng Mengze Hong Chen Jason Zhang 39 2 0 02 Oct 2024
A Pair Programming Framework for Code Generation via Multi-Plan Exploration and Feedback-Driven Refinement Huan Zhang Wei Cheng Yuhan Wu Wei Hu LLMAG 41 5 0 08 Sep 2024
Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoning Xinglin Wang Shaoxiong Feng Yiwei Li Peiwen Yuan Y. Zhang Boyuan Pan Heda Wang Yao Hu Kan Li LRM 40 17 0 24 Aug 2024
From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future Haolin Jin Linghan Huang Haipeng Cai Jun Yan Bo Li Huaming Chen 78 27 0 05 Aug 2024
Hybrid Querying Over Relational Databases and Large Language Models T. Pham Cody T. Reynolds A. El Abbadi 33 1 0 01 Aug 2024
PromptIntern: Saving Inference Costs by Internalizing Recurrent Prompt during Large Language Model Fine-tuning Jiaru Zou Mengyu Zhou Tao Li Shi Han Dongmei Zhang 51 6 0 02 Jul 2024
Many-Shot In-Context Learning in Multimodal Foundation Models Yixing Jiang Jeremy Irvin Ji Hun Wang Muhammad Ahmed Chaudhry Jonathan H. Chen Andrew Y. Ng VLM 60 27 0 16 May 2024
Position: Leverage Foundational Models for Black-Box Optimization Xingyou Song Yingtao Tian Robert Tjarko Lange Chansoo Lee Yujin Tang Yutian Chen 42 5 0 06 May 2024
Symbolic Prompt Program Search: A Structure-Aware Approach to Efficient Compile-Time Prompt Optimization Tobias Schnabel Jennifer Neville LRM 29 6 0 02 Apr 2024
SMART: Automatically Scaling Down Language Models with Accuracy Guarantees for Reduced Processing Fees Saehan Jo Immanuel Trummer 20 0 0 11 Mar 2024
AmbigNLG: Addressing Task Ambiguity in Instruction for NLG Ayana Niwa Hayate Iso 36 4 0 27 Feb 2024
Multi-Task Inference: Can Large Language Models Follow Multiple Instructions at Once? Guijin Son Sangwon Baek Sangdae Nam Ilgyun Jeong Seungone Kim ELM LRM 37 14 0 18 Feb 2024
CliqueParcel: An Approach For Batching LLM Prompts That Jointly Optimizes Efficiency And Faithfulness Jiayi Liu Tinghan Yang Jennifer Neville 26 10 0 17 Feb 2024
Towards Unified Alignment Between Agents, Humans, and Environment Zonghan Yang An Liu Zijun Liu Kai Liu Fangzhou Xiong ... Zhenhe Zhang Fuwen Luo Zhicheng Guo Peng Li Yang Liu 32 4 0 12 Feb 2024
CigaR: Cost-efficient Program Repair with LLMs Dávid Hidvégi K. Etemadi Sofia Bobadilla Martin Monperrus 28 20 0 09 Feb 2024
Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives Chen Gao Xiaochong Lan Nian Li Yuan Yuan Jingtao Ding Zhilun Zhou Fengli Xu Yong Li LLMAG AI4CE LM&Ro 44 105 0 19 Dec 2023
Urban Generative Intelligence (UGI): A Foundational Platform for Agents in Embodied City Environment Fengli Xu Jun Zhang Chen Gao J. Feng Yong Li AI4CE LLMAG 26 29 0 19 Dec 2023
Cost-Effective In-Context Learning for Entity Resolution: A Design Space Exploration Meihao Fan Xiaoyue Han Ju Fan Chengliang Chai Nan Tang Guoliang Li Xiaoyong Du 29 5 0 07 Dec 2023
Jellyfish: A Large Language Model for Data Preprocessing Haochen Zhang Yuyang Dong Chuan Xiao Masafumi Oyamada 35 26 0 04 Dec 2023
Just Ask One More Time! Self-Agreement Improves Reasoning of Language Models in (Almost) All Scenarios Lei Lin Jiayi Fu Pengli Liu Qingyang Li Yan Gong Junchen Wan Fuzheng Zhang Zhongyuan Wang Di Zhang Kun Gai LRM 27 7 0 14 Nov 2023
Interpretable-by-Design Text Understanding with Iteratively Generated Concept Bottleneck Josh Magnus Ludan Qing Lyu Yue Yang Liam Dugan Mark Yatskar Chris Callison-Burch 35 4 0 30 Oct 2023
Tree-Planner: Efficient Close-loop Task Planning with Large Language Models Mengkang Hu Yao Mu Xinmiao Yu Mingyu Ding Shiguang Wu Wenqi Shao Qiguang Chen Bin Wang Yu Qiao Ping Luo LLMAG 44 33 0 12 Oct 2023
Diversity of Thought Improves Reasoning Abilities of LLMs Ranjita Naik Varun Chandrasekaran Mert Yuksekgonul Hamid Palangi Besmira Nushi LRM 34 6 0 11 Oct 2023
Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoning Murong Yue Jie Zhao Min Zhang Liang Du Ziyu Yao LRM 35 55 0 04 Oct 2023
BatchPrompt: Accomplish more with less Jianzhe Lin Maurice Diesendruck Liang Du Robin Abraham LRM 24 9 0 01 Sep 2023
Large Language Models as Data Preprocessors Haochen Zhang Yuyang Dong Chuan Xiao Masafumi Oyamada 35 29 0 30 Aug 2023
FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering Zhenyu Li Sunqi Fan Yu Gu Xiuxing Li Zhichao Duan Bo Dong Ning Liu Jianyong Wang 17 70 0 23 Aug 2023
Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrations Wei-Lin Chen Cheng-Kuang Wu Yun-Nung Chen Hsin-Hsi Chen 23 27 0 24 May 2023
ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models Binfeng Xu Zhiyuan Peng Bowen Lei Subhabrata Mukherjee Yuchen Liu Dongkuan Xu KELM LLMAG LRM 32 90 0 23 May 2023
Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline Zangwei Zheng Xiaozhe Ren Fuzhao Xue Yang Luo Xin Jiang Yang You 42 54 0 22 May 2023
Reducing Sequence Length by Predicting Edit Operations with Large Language Models Masahiro Kaneko Naoaki Okazaki 21 4 0 19 May 2023
Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learning Wenhao Li Dan Qiao Baoxiang Wang Xiangfeng Wang Bo Jin H. Zha 38 5 0 18 May 2023
Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark Alexander Pan Chan Jun Shern Andy Zou Nathaniel Li Steven Basart Thomas Woodside Jonathan Ng Hanlin Zhang Scott Emmons Dan Hendrycks 26 126 0 06 Apr 2023
Binding Language Models in Symbolic Languages Zhoujun Cheng Tianbao Xie Peng Shi Chengzu Li Rahul Nadkarni ... Dragomir R. Radev Mari Ostendorf Luke Zettlemoyer Noah A. Smith Tao Yu LMTD 122 198 0 06 Oct 2022
ReAct: Synergizing Reasoning and Acting in Language Models Shunyu Yao Jeffrey Zhao Dian Yu Nan Du Izhak Shafran Karthik Narasimhan Yuan Cao LLMAG ReLM LRM 267 2,510 0 06 Oct 2022
Ask Me Anything: A simple strategy for prompting language models Simran Arora A. Narayan Mayee F. Chen Laurel J. Orr Neel Guha Kush S. Bhatia Ines Chami Frederic Sala Christopher Ré ReLM LRM 223 208 0 05 Oct 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 314 3,273 0 21 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 339 12,003 0 04 Mar 2022
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts Stephen H. Bach Victor Sanh Zheng-Xin Yong Albert Webson Colin Raffel ... Khalid Almubarak Xiangru Tang Dragomir R. Radev Mike Tian-Jian Jiang Alexander M. Rush VLM 225 339 0 02 Feb 2022