Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery

7 February 2023

Papers citing "Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery"

50 / 197 papers shown

Title
Open-Vocabulary Attention Maps with Token Optimization for Semantic Segmentation in Diffusion Models Pablo Marcos-Manchón Roberto Alcover-Couso Juan C. Sanmiguel Jose M. Martínez VLM 49 18 0 21 Mar 2024
Helpful or Harmful? Exploring the Efficacy of Large Language Models for Online Grooming Prevention Ellie Prosser Matthew Edwards 42 6 0 14 Mar 2024
PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement Zhijie Wang Yuheng Huang Da Song Lei Ma Tianyi Zhang DiffM 50 58 0 06 Mar 2024
Regeneration Based Training-free Attribution of Fake Images Generated by Text-to-Image Generative Models Meiling Li Zhenxing Qian Xinpeng Zhang 39 2 0 03 Mar 2024
Accelerating Greedy Coordinate Gradient via Probe Sampling Yiran Zhao Wenyue Zheng Tianle Cai Xuan Long Do Kenji Kawaguchi Anirudh Goyal Michael Shieh 51 11 0 02 Mar 2024
Personalized Federated Instruction Tuning via Neural Architecture Search Peng Zhang Yingbo Zhou Ming Hu Junxian Feng Jiawen Weng Mingsong Chen FedML 38 4 0 26 Feb 2024
Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing Jiabao Ji Bairu Hou Alexander Robey George J. Pappas Hamed Hassani Yang Zhang Eric Wong Shiyu Chang AAML 50 41 0 25 Feb 2024
Fast Adversarial Attacks on Language Models In One GPU Minute Vinu Sankar Sadasivan Shoumik Saha Gaurang Sriramanan Priyatham Kattakinda Atoosa Malemir Chegini S. Feizi MIALM 43 34 0 23 Feb 2024
Coercing LLMs to do and reveal (almost) anything Jonas Geiping Alex Stein Manli Shu Khalid Saifullah Yuxin Wen Tom Goldstein AAML 48 43 0 21 Feb 2024
A User-Friendly Framework for Generating Model-Preferred Prompts in Text-to-Image Synthesis Nailei Hei Qianyu Guo Zihao Wang Yan Wang Haofen Wang Wenqiang Zhang DiffM 36 2 0 20 Feb 2024
MORL-Prompt: An Empirical Analysis of Multi-Objective Reinforcement Learning for Discrete Prompt Optimization Yasaman Jafari Dheeraj Mekala Rose Yu Taylor Berg-Kirkpatrick 53 7 0 18 Feb 2024
Benchmarking Knowledge Boundary for Large Language Models: A Different Perspective on Model Evaluation Xunjian Yin Xu Zhang Jie Ruan Xiaojun Wan ELM 41 18 0 18 Feb 2024
A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents Lingbo Mo Zeyi Liao Boyuan Zheng Yu-Chuan Su Chaowei Xiao Huan Sun AAML LLMAG 51 15 0 15 Feb 2024
Efficient Prompt Optimization Through the Lens of Best Arm Identification Chengshuai Shi Kun Yang Zihan Chen Jundong Li Jing Yang Cong Shen 50 6 0 15 Feb 2024
How Secure Are Large Language Models (LLMs) for Navigation in Urban Environments? Congcong Wen Jiazhao Liang Shuaihang Yuan Hao Huang Yi Fang AAML 38 5 0 14 Feb 2024
Attacking Large Language Models with Projected Gradient Descent Simon Geisler Tom Wollschlager M. H. I. Abdalla Johannes Gasteiger Stephan Günnemann AAML SILM 49 50 0 14 Feb 2024
Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space Leo Schwinn David Dobre Sophie Xhonneux Gauthier Gidel Stephan Gunnemann AAML 51 38 0 14 Feb 2024
COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability Xing-ming Guo Fangxu Yu Huan Zhang Lianhui Qin Bin Hu AAML 117 73 0 13 Feb 2024
Rapid Optimization for Jailbreaking LLMs via Subconscious Exploitation and Echopraxia Guangyu Shen Shuyang Cheng Kai-xian Zhang Guanhong Tao Shengwei An Lu Yan Zhuo Zhang Shiqing Ma Xiangyu Zhang 20 11 0 08 Feb 2024
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal Mantas Mazeika Long Phan Xuwang Yin Andy Zou Zifan Wang ... Nathaniel Li Steven Basart Bo Li David A. Forsyth Dan Hendrycks AAML 35 327 0 06 Feb 2024
Intent-based Prompt Calibration: Enhancing prompt optimization with synthetic boundary cases Elad Levi Eli Brosh Matan Friedmann 27 8 0 05 Feb 2024
Vision-Language Models Provide Promptable Representations for Reinforcement Learning William Chen Oier Mees Aviral Kumar Sergey Levine VLM LM&Ro 54 23 0 05 Feb 2024
Jailbreaking Attack against Multimodal Large Language Model Zhenxing Niu Haoxuan Ji Xinbo Gao Gang Hua Rong Jin 50 61 0 04 Feb 2024
Object-Driven One-Shot Fine-tuning of Text-to-Image Diffusion with Prototypical Embedding Jianxiang Lu Cong Xie Hui Guo DiffM 108 5 0 28 Jan 2024
Black-Box Access is Insufficient for Rigorous AI Audits Stephen Casper Carson Ezell Charlotte Siegmann Noam Kolt Taylor Lynn Curtis ... Michael Gerovitch David Bau Max Tegmark David M. Krueger Dylan Hadfield-Menell AAML 38 78 0 25 Jan 2024
Fluent dreaming for language models T. B. Thompson Zygimantas Straznickas Michael Sklar VLM 35 2 0 24 Jan 2024
Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmers Aleksandar Stanić Sergi Caelles Michael Tschannen LRM VLM 27 9 0 03 Jan 2024
Prompt Expansion for Adaptive Text-to-Image Generation Siddhartha Datta Alexander Ku Deepak Ramachandran Peter Anderson DiffM 47 9 0 27 Dec 2023
Black-Box Tuning of Vision-Language Models with Effective Gradient Approximation Zixian Guo Yuxiang Wei Ming-Yu Liu Zhilong Ji Jinfeng Bai Yiwen Guo Wangmeng Zuo VLM 36 8 0 26 Dec 2023
Prompting Hard or Hardly Prompting: Prompt Inversion for Text-to-Image Diffusion Models Shweta Mahajan Tanzila Rahman Kwang Moo Yi Leonid Sigal DiffM 43 17 0 19 Dec 2023
Silent Guardian: Protecting Text from Malicious Exploitation by Large Language Models Jiawei Zhao Kejiang Chen Xianjian Yuan Yuang Qi Weiming Zhang Neng H. Yu 67 8 0 15 Dec 2023
MotionCrafter: One-Shot Motion Customization of Diffusion Models Yuxin Zhang Fan Tang Nisha Huang Haibin Huang Chongyang Ma Weiming Dong Changsheng Xu DiffM VGen 46 14 0 08 Dec 2023
Make Them Spill the Beans! Coercive Knowledge Extraction from (Production) LLMs Zhuo Zhang Guangyu Shen Guanhong Tao Shuyang Cheng Xiangyu Zhang 41 13 0 08 Dec 2023
Analyzing the Inherent Response Tendency of LLMs: Real-World Instructions-Driven Jailbreak Yanrui Du Sendong Zhao Ming Ma Yuhan Chen Bing Qin 26 15 0 07 Dec 2023
Tree of Attacks: Jailbreaking Black-Box LLMs Automatically Anay Mehrotra Manolis Zampetakis Paul Kassianik Blaine Nelson Hyrum Anderson Yaron Singer Amin Karbasi 44 211 0 04 Dec 2023
VA3: Virtually Assured Amplification Attack on Probabilistic Copyright Protection for Text-to-Image Generative Models Xiang Li Qianli Shen Kenji Kawaguchi 27 4 0 29 Nov 2023
DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer Junyuan Hong Jiachen T. Wang Chenhui Zhang Zhangheng Li Bo-wen Li Zhangyang Wang 53 29 0 27 Nov 2023
CatVersion: Concatenating Embeddings for Diffusion-Based Text-to-Image Personalization Ruoyu Zhao Mingrui Zhu Shiyin Dong Nannan Wang Xinbo Gao DiffM 35 13 0 24 Nov 2023
Hijacking Large Language Models via Adversarial In-Context Learning Yao Qiang Xiangyu Zhou Dongxiao Zhu 34 32 0 16 Nov 2023
Do Physicians Know How to Prompt? The Need for Automatic Prompt Optimization Help in Clinical Note Generation Zonghai Yao Ahmed Jaafar Beining Wang Zhichao Yang Hong-ye Yu LM&MA 32 2 0 16 Nov 2023
Stealthy and Persistent Unalignment on Large Language Models via Backdoor Injections Yuanpu Cao Bochuan Cao Jinghui Chen 34 24 0 15 Nov 2023
How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilities Lingbo Mo Boshi Wang Muhao Chen Huan Sun 31 27 0 15 Nov 2023
Prompt have evil twins Rimon Melamed Lucas H. McCabe T. Wakhare Yejin Kim H. H. Huang Enric Boix-Adsera 38 3 0 13 Nov 2023
In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering Sheng Liu Haotian Ye Lei Xing James Y. Zou 26 87 0 11 Nov 2023
AntifakePrompt: Prompt-Tuned Vision-Language Models are Fake Image Detectors You-Ming Chang Chen Yeh Wei-Chen Chiu Ning Yu VPVLM VLM 78 23 0 26 Oct 2023
PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization Xinyuan Wang Chenxi Li Zhen Wang Fan Bai Haotian Luo Jiayou Zhang Nebojsa Jojic Eric P. Xing Zhiting Hu 34 102 0 25 Oct 2023
AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models Sicheng Zhu Ruiyi Zhang Bang An Gang Wu Joe Barrow Zichao Wang Furong Huang A. Nenkova Tong Sun SILM AAML 30 41 0 23 Oct 2023
Language Model Unalignment: Parametric Red-Teaming to Expose Hidden Harms and Biases Rishabh Bhardwaj Soujanya Poria ALM 57 16 0 22 Oct 2023
InferDPT: Privacy-Preserving Inference for Black-box Large Language Model Meng Tong Kejiang Chen Jie Zhang Yuang Qi Weiming Zhang Neng H. Yu Tianwei Zhang Zhikun Zhang SILM 42 2 0 18 Oct 2023
To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now Yimeng Zhang Jinghan Jia Xin Chen Aochuan Chen Yihua Zhang Jiancheng Liu Ke Ding Sijia Liu DiffM 30 83 0 18 Oct 2023