SEA4DQ '24: "Evaluating the Quality of ..."
Evaluating the Quality of Open Source Ansible Playbooks: An Executability Perspective
Pemsith Mendis, Wilson Reeves, Muhammad Ali Babar, Yue Zhang, and Akond Rahman
(Auburn University, USA; University of Adelaide, Australia)
Infrastructure as code (IaC) is the practice of automatically managing computing platforms, such as Internet of Things (IoT) platforms. IaC has gained popularity in recent years, yielding a plethora of software artifacts, such as Ansible playbooks that are available on social coding platforms. Despite the availability of open source software (OSS) Ansible playbooks, there is a lack of empirical research on the quality of these playbooks, which can hinder the progress of IaC-related research. To that end, we conduct an empirical study with 2,952 OSS Ansible playbooks where we evaluate the quality of OSS playbooks from the perspective of executability, i.e., if publicly available OSS Ansible playbooks can be executed without failures. From our empirical study, we observe 71.5% of the mined 2,952 Ansible playbooks cannot be executed as is because of four categories of failures.
@InProceedings{SEA4DQ24p2,
author = {Pemsith Mendis and Wilson Reeves and Muhammad Ali Babar and Yue Zhang and Akond Rahman},
title = {Evaluating the Quality of Open Source Ansible Playbooks: An Executability Perspective},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {2--5},
doi = {10.1145/3663530.3665019},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "A Hitchhiker’s Guide to ..."
A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering
Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Kailong Wang
(Nanyang Technological University, Singapore; UNSW, Australia; Institute of Information Engineering at Chinese Academy of Sciences, China; Virginia Tech, USA; Huazhong University of Science and Technology, China)
Natural language prompts serve as an essential interface between users and Large Language Models (LLMs) like GPT-3.5 and GPT-4, which are employed by ChatGPT to produce outputs across various tasks. However, prompts crafted with malicious intent, known as jailbreak prompts, can circumvent the restrictions of LLMs, posing a significant threat to systems integrated with these models. Despite their critical importance, there is a lack of systematic analysis and comprehensive understanding of jailbreak prompts. Our paper aims to address this gap by exploring key research questions to enhance the robustness of LLM systems: 1) What common patterns are present in jailbreak prompts? 2) How effectively can these prompts bypass the restrictions of LLMs? 3) With the evolution of LLMs, how does the effectiveness of jailbreak prompts change? To address our research questions, we embarked on an empirical study targeting the LLMs underpinning ChatGPT, one of today's most advanced chatbots. Our methodology involved categorizing 78 jailbreak prompts into 10 distinct patterns, further organized into three jailbreak strategy types, and examining their distribution. We assessed the effectiveness of these prompts on GPT-3.5 and GPT-4, using a set of 3,120 questions across 8 scenarios deemed prohibited by OpenAI. Additionally, our study tracked the performance of these prompts over a 3-month period, observing the evolutionary response of ChatGPT to such inputs. Our findings offer a comprehensive view of jailbreak prompts, elucidating their taxonomy, effectiveness, and temporal dynamics. Notably, we discovered that GPT-3.5 and GPT-4 could still generate inappropriate content in response to malicious prompts without the need for jailbreaking. This underscores the critical need for effective prompt management within LLM systems and provides valuable insights and data to spur further research in LLM testing and jailbreak prevention.
@InProceedings{SEA4DQ24p12,
author = {Yi Liu and Gelei Deng and Zhengzi Xu and Yuekang Li and Yaowen Zheng and Ying Zhang and Lida Zhao and Tianwei Zhang and Kailong Wang},
title = {A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {12--21},
doi = {10.1145/3663530.3665021},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "A Pilot Study in Surveying ..."
A Pilot Study in Surveying Data Challenges of Automatic Software Engineering Tasks
Liming Dong, Qinghua Lu, and Liming Zhu
(CSIRO’s Data61, Australia; UNSW, Australia)
The surge in automatic SE research aims to boost development efficiency and quality while reducing costs. However, challenges such as limited real-world project data and inadequate data conditions constrain the effectiveness of these methods. To systematically understand these challenges, our pilot study reviews prevalent data challenges across various SE tasks. Despite these challenges, thanks to the advances of large language model offers promising performance on SE tasks. Overall, this pilot survey focused on provide a quick retrospective review on SE data challenges and introduce practical LLM solutions from the SE community to mitigate these challenges.
@InProceedings{SEA4DQ24p6,
author = {Liming Dong and Qinghua Lu and Liming Zhu},
title = {A Pilot Study in Surveying Data Challenges of Automatic Software Engineering Tasks},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {6--11},
doi = {10.1145/3663530.3665020},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "A Hitchhiker’s Guide to ..."
A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering
Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Kailong Wang
(Nanyang Technological University, Singapore; UNSW, Australia; Institute of Information Engineering at Chinese Academy of Sciences, China; Virginia Tech, USA; Huazhong University of Science and Technology, China)
Natural language prompts serve as an essential interface between users and Large Language Models (LLMs) like GPT-3.5 and GPT-4, which are employed by ChatGPT to produce outputs across various tasks. However, prompts crafted with malicious intent, known as jailbreak prompts, can circumvent the restrictions of LLMs, posing a significant threat to systems integrated with these models. Despite their critical importance, there is a lack of systematic analysis and comprehensive understanding of jailbreak prompts. Our paper aims to address this gap by exploring key research questions to enhance the robustness of LLM systems: 1) What common patterns are present in jailbreak prompts? 2) How effectively can these prompts bypass the restrictions of LLMs? 3) With the evolution of LLMs, how does the effectiveness of jailbreak prompts change? To address our research questions, we embarked on an empirical study targeting the LLMs underpinning ChatGPT, one of today's most advanced chatbots. Our methodology involved categorizing 78 jailbreak prompts into 10 distinct patterns, further organized into three jailbreak strategy types, and examining their distribution. We assessed the effectiveness of these prompts on GPT-3.5 and GPT-4, using a set of 3,120 questions across 8 scenarios deemed prohibited by OpenAI. Additionally, our study tracked the performance of these prompts over a 3-month period, observing the evolutionary response of ChatGPT to such inputs. Our findings offer a comprehensive view of jailbreak prompts, elucidating their taxonomy, effectiveness, and temporal dynamics. Notably, we discovered that GPT-3.5 and GPT-4 could still generate inappropriate content in response to malicious prompts without the need for jailbreaking. This underscores the critical need for effective prompt management within LLM systems and provides valuable insights and data to spur further research in LLM testing and jailbreak prevention.
@InProceedings{SEA4DQ24p12,
author = {Yi Liu and Gelei Deng and Zhengzi Xu and Yuekang Li and Yaowen Zheng and Ying Zhang and Lida Zhao and Tianwei Zhang and Kailong Wang},
title = {A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {12--21},
doi = {10.1145/3663530.3665021},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "A Hitchhiker’s Guide to ..."
A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering
Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Kailong Wang
(Nanyang Technological University, Singapore; UNSW, Australia; Institute of Information Engineering at Chinese Academy of Sciences, China; Virginia Tech, USA; Huazhong University of Science and Technology, China)
Natural language prompts serve as an essential interface between users and Large Language Models (LLMs) like GPT-3.5 and GPT-4, which are employed by ChatGPT to produce outputs across various tasks. However, prompts crafted with malicious intent, known as jailbreak prompts, can circumvent the restrictions of LLMs, posing a significant threat to systems integrated with these models. Despite their critical importance, there is a lack of systematic analysis and comprehensive understanding of jailbreak prompts. Our paper aims to address this gap by exploring key research questions to enhance the robustness of LLM systems: 1) What common patterns are present in jailbreak prompts? 2) How effectively can these prompts bypass the restrictions of LLMs? 3) With the evolution of LLMs, how does the effectiveness of jailbreak prompts change? To address our research questions, we embarked on an empirical study targeting the LLMs underpinning ChatGPT, one of today's most advanced chatbots. Our methodology involved categorizing 78 jailbreak prompts into 10 distinct patterns, further organized into three jailbreak strategy types, and examining their distribution. We assessed the effectiveness of these prompts on GPT-3.5 and GPT-4, using a set of 3,120 questions across 8 scenarios deemed prohibited by OpenAI. Additionally, our study tracked the performance of these prompts over a 3-month period, observing the evolutionary response of ChatGPT to such inputs. Our findings offer a comprehensive view of jailbreak prompts, elucidating their taxonomy, effectiveness, and temporal dynamics. Notably, we discovered that GPT-3.5 and GPT-4 could still generate inappropriate content in response to malicious prompts without the need for jailbreaking. This underscores the critical need for effective prompt management within LLM systems and provides valuable insights and data to spur further research in LLM testing and jailbreak prevention.
@InProceedings{SEA4DQ24p12,
author = {Yi Liu and Gelei Deng and Zhengzi Xu and Yuekang Li and Yaowen Zheng and Ying Zhang and Lida Zhao and Tianwei Zhang and Kailong Wang},
title = {A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {12--21},
doi = {10.1145/3663530.3665021},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "A Pilot Study in Surveying ..."
A Pilot Study in Surveying Data Challenges of Automatic Software Engineering Tasks
Liming Dong, Qinghua Lu, and Liming Zhu
(CSIRO’s Data61, Australia; UNSW, Australia)
The surge in automatic SE research aims to boost development efficiency and quality while reducing costs. However, challenges such as limited real-world project data and inadequate data conditions constrain the effectiveness of these methods. To systematically understand these challenges, our pilot study reviews prevalent data challenges across various SE tasks. Despite these challenges, thanks to the advances of large language model offers promising performance on SE tasks. Overall, this pilot survey focused on provide a quick retrospective review on SE data challenges and introduce practical LLM solutions from the SE community to mitigate these challenges.
@InProceedings{SEA4DQ24p6,
author = {Liming Dong and Qinghua Lu and Liming Zhu},
title = {A Pilot Study in Surveying Data Challenges of Automatic Software Engineering Tasks},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {6--11},
doi = {10.1145/3663530.3665020},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "Responsible AI Engineering ..."
Responsible AI Engineering from a Data Perspective (Keynote)
Qinghua Lu
(CSIRO’s Data61, Australia)
The rapid advancements in AI, particularly with the emergence of large language models (LLMs) and their diverse applications, have attracted huge global interest and raised significant concerns on responsible AI and AI safety. While LLMs are impressive examples of AI models, it is the compound AI systems, which integrate these models with other key components for functionality and quality/risk control, that are ultimately deployed and have real-world impact. These AI systems, especially autonomous LLM agents and those involving multi-agent interacting, require system-level engineering to ensure responsible AI and AI safety. On the other hand, data is the lifeblood of AI systems, cross-cutting different components in AI systems. There are various challenges associated with the data collected, used, and generated by AI systems, as well as their engineering processes. In this talk, I will introduce a responsible AI engineering approach to address system-level responsible AI challenges. This includes engineering/governance methods, practices, tools, and platforms to ensure responsible AI and AI safety. Specially, I will focus on how the responsible AI engineering approach tackles data challenges within the context of responsible AI.
@InProceedings{SEA4DQ24p1,
author = {Qinghua Lu},
title = {Responsible AI Engineering from a Data Perspective (Keynote)},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {1--1},
doi = {10.1145/3663530.3676516},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "Evaluating the Quality of ..."
Evaluating the Quality of Open Source Ansible Playbooks: An Executability Perspective
Pemsith Mendis, Wilson Reeves, Muhammad Ali Babar, Yue Zhang, and Akond Rahman
(Auburn University, USA; University of Adelaide, Australia)
Infrastructure as code (IaC) is the practice of automatically managing computing platforms, such as Internet of Things (IoT) platforms. IaC has gained popularity in recent years, yielding a plethora of software artifacts, such as Ansible playbooks that are available on social coding platforms. Despite the availability of open source software (OSS) Ansible playbooks, there is a lack of empirical research on the quality of these playbooks, which can hinder the progress of IaC-related research. To that end, we conduct an empirical study with 2,952 OSS Ansible playbooks where we evaluate the quality of OSS playbooks from the perspective of executability, i.e., if publicly available OSS Ansible playbooks can be executed without failures. From our empirical study, we observe 71.5% of the mined 2,952 Ansible playbooks cannot be executed as is because of four categories of failures.
@InProceedings{SEA4DQ24p2,
author = {Pemsith Mendis and Wilson Reeves and Muhammad Ali Babar and Yue Zhang and Akond Rahman},
title = {Evaluating the Quality of Open Source Ansible Playbooks: An Executability Perspective},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {2--5},
doi = {10.1145/3663530.3665019},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "Evaluating the Quality of ..."
Evaluating the Quality of Open Source Ansible Playbooks: An Executability Perspective
Pemsith Mendis, Wilson Reeves, Muhammad Ali Babar, Yue Zhang, and Akond Rahman
(Auburn University, USA; University of Adelaide, Australia)
Infrastructure as code (IaC) is the practice of automatically managing computing platforms, such as Internet of Things (IoT) platforms. IaC has gained popularity in recent years, yielding a plethora of software artifacts, such as Ansible playbooks that are available on social coding platforms. Despite the availability of open source software (OSS) Ansible playbooks, there is a lack of empirical research on the quality of these playbooks, which can hinder the progress of IaC-related research. To that end, we conduct an empirical study with 2,952 OSS Ansible playbooks where we evaluate the quality of OSS playbooks from the perspective of executability, i.e., if publicly available OSS Ansible playbooks can be executed without failures. From our empirical study, we observe 71.5% of the mined 2,952 Ansible playbooks cannot be executed as is because of four categories of failures.
@InProceedings{SEA4DQ24p2,
author = {Pemsith Mendis and Wilson Reeves and Muhammad Ali Babar and Yue Zhang and Akond Rahman},
title = {Evaluating the Quality of Open Source Ansible Playbooks: An Executability Perspective},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {2--5},
doi = {10.1145/3663530.3665019},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "Evaluating the Quality of ..."
Evaluating the Quality of Open Source Ansible Playbooks: An Executability Perspective
Pemsith Mendis, Wilson Reeves, Muhammad Ali Babar, Yue Zhang, and Akond Rahman
(Auburn University, USA; University of Adelaide, Australia)
Infrastructure as code (IaC) is the practice of automatically managing computing platforms, such as Internet of Things (IoT) platforms. IaC has gained popularity in recent years, yielding a plethora of software artifacts, such as Ansible playbooks that are available on social coding platforms. Despite the availability of open source software (OSS) Ansible playbooks, there is a lack of empirical research on the quality of these playbooks, which can hinder the progress of IaC-related research. To that end, we conduct an empirical study with 2,952 OSS Ansible playbooks where we evaluate the quality of OSS playbooks from the perspective of executability, i.e., if publicly available OSS Ansible playbooks can be executed without failures. From our empirical study, we observe 71.5% of the mined 2,952 Ansible playbooks cannot be executed as is because of four categories of failures.
@InProceedings{SEA4DQ24p2,
author = {Pemsith Mendis and Wilson Reeves and Muhammad Ali Babar and Yue Zhang and Akond Rahman},
title = {Evaluating the Quality of Open Source Ansible Playbooks: An Executability Perspective},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {2--5},
doi = {10.1145/3663530.3665019},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "A Hitchhiker’s Guide to ..."
A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering
Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Kailong Wang
(Nanyang Technological University, Singapore; UNSW, Australia; Institute of Information Engineering at Chinese Academy of Sciences, China; Virginia Tech, USA; Huazhong University of Science and Technology, China)
Natural language prompts serve as an essential interface between users and Large Language Models (LLMs) like GPT-3.5 and GPT-4, which are employed by ChatGPT to produce outputs across various tasks. However, prompts crafted with malicious intent, known as jailbreak prompts, can circumvent the restrictions of LLMs, posing a significant threat to systems integrated with these models. Despite their critical importance, there is a lack of systematic analysis and comprehensive understanding of jailbreak prompts. Our paper aims to address this gap by exploring key research questions to enhance the robustness of LLM systems: 1) What common patterns are present in jailbreak prompts? 2) How effectively can these prompts bypass the restrictions of LLMs? 3) With the evolution of LLMs, how does the effectiveness of jailbreak prompts change? To address our research questions, we embarked on an empirical study targeting the LLMs underpinning ChatGPT, one of today's most advanced chatbots. Our methodology involved categorizing 78 jailbreak prompts into 10 distinct patterns, further organized into three jailbreak strategy types, and examining their distribution. We assessed the effectiveness of these prompts on GPT-3.5 and GPT-4, using a set of 3,120 questions across 8 scenarios deemed prohibited by OpenAI. Additionally, our study tracked the performance of these prompts over a 3-month period, observing the evolutionary response of ChatGPT to such inputs. Our findings offer a comprehensive view of jailbreak prompts, elucidating their taxonomy, effectiveness, and temporal dynamics. Notably, we discovered that GPT-3.5 and GPT-4 could still generate inappropriate content in response to malicious prompts without the need for jailbreaking. This underscores the critical need for effective prompt management within LLM systems and provides valuable insights and data to spur further research in LLM testing and jailbreak prevention.
@InProceedings{SEA4DQ24p12,
author = {Yi Liu and Gelei Deng and Zhengzi Xu and Yuekang Li and Yaowen Zheng and Ying Zhang and Lida Zhao and Tianwei Zhang and Kailong Wang},
title = {A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {12--21},
doi = {10.1145/3663530.3665021},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "A Hitchhiker’s Guide to ..."
A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering
Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Kailong Wang
(Nanyang Technological University, Singapore; UNSW, Australia; Institute of Information Engineering at Chinese Academy of Sciences, China; Virginia Tech, USA; Huazhong University of Science and Technology, China)
Natural language prompts serve as an essential interface between users and Large Language Models (LLMs) like GPT-3.5 and GPT-4, which are employed by ChatGPT to produce outputs across various tasks. However, prompts crafted with malicious intent, known as jailbreak prompts, can circumvent the restrictions of LLMs, posing a significant threat to systems integrated with these models. Despite their critical importance, there is a lack of systematic analysis and comprehensive understanding of jailbreak prompts. Our paper aims to address this gap by exploring key research questions to enhance the robustness of LLM systems: 1) What common patterns are present in jailbreak prompts? 2) How effectively can these prompts bypass the restrictions of LLMs? 3) With the evolution of LLMs, how does the effectiveness of jailbreak prompts change? To address our research questions, we embarked on an empirical study targeting the LLMs underpinning ChatGPT, one of today's most advanced chatbots. Our methodology involved categorizing 78 jailbreak prompts into 10 distinct patterns, further organized into three jailbreak strategy types, and examining their distribution. We assessed the effectiveness of these prompts on GPT-3.5 and GPT-4, using a set of 3,120 questions across 8 scenarios deemed prohibited by OpenAI. Additionally, our study tracked the performance of these prompts over a 3-month period, observing the evolutionary response of ChatGPT to such inputs. Our findings offer a comprehensive view of jailbreak prompts, elucidating their taxonomy, effectiveness, and temporal dynamics. Notably, we discovered that GPT-3.5 and GPT-4 could still generate inappropriate content in response to malicious prompts without the need for jailbreaking. This underscores the critical need for effective prompt management within LLM systems and provides valuable insights and data to spur further research in LLM testing and jailbreak prevention.
@InProceedings{SEA4DQ24p12,
author = {Yi Liu and Gelei Deng and Zhengzi Xu and Yuekang Li and Yaowen Zheng and Ying Zhang and Lida Zhao and Tianwei Zhang and Kailong Wang},
title = {A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {12--21},
doi = {10.1145/3663530.3665021},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "A Hitchhiker’s Guide to ..."
A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering
Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Kailong Wang
(Nanyang Technological University, Singapore; UNSW, Australia; Institute of Information Engineering at Chinese Academy of Sciences, China; Virginia Tech, USA; Huazhong University of Science and Technology, China)
Natural language prompts serve as an essential interface between users and Large Language Models (LLMs) like GPT-3.5 and GPT-4, which are employed by ChatGPT to produce outputs across various tasks. However, prompts crafted with malicious intent, known as jailbreak prompts, can circumvent the restrictions of LLMs, posing a significant threat to systems integrated with these models. Despite their critical importance, there is a lack of systematic analysis and comprehensive understanding of jailbreak prompts. Our paper aims to address this gap by exploring key research questions to enhance the robustness of LLM systems: 1) What common patterns are present in jailbreak prompts? 2) How effectively can these prompts bypass the restrictions of LLMs? 3) With the evolution of LLMs, how does the effectiveness of jailbreak prompts change? To address our research questions, we embarked on an empirical study targeting the LLMs underpinning ChatGPT, one of today's most advanced chatbots. Our methodology involved categorizing 78 jailbreak prompts into 10 distinct patterns, further organized into three jailbreak strategy types, and examining their distribution. We assessed the effectiveness of these prompts on GPT-3.5 and GPT-4, using a set of 3,120 questions across 8 scenarios deemed prohibited by OpenAI. Additionally, our study tracked the performance of these prompts over a 3-month period, observing the evolutionary response of ChatGPT to such inputs. Our findings offer a comprehensive view of jailbreak prompts, elucidating their taxonomy, effectiveness, and temporal dynamics. Notably, we discovered that GPT-3.5 and GPT-4 could still generate inappropriate content in response to malicious prompts without the need for jailbreaking. This underscores the critical need for effective prompt management within LLM systems and provides valuable insights and data to spur further research in LLM testing and jailbreak prevention.
@InProceedings{SEA4DQ24p12,
author = {Yi Liu and Gelei Deng and Zhengzi Xu and Yuekang Li and Yaowen Zheng and Ying Zhang and Lida Zhao and Tianwei Zhang and Kailong Wang},
title = {A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {12--21},
doi = {10.1145/3663530.3665021},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "A Hitchhiker’s Guide to ..."
A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering
Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Kailong Wang
(Nanyang Technological University, Singapore; UNSW, Australia; Institute of Information Engineering at Chinese Academy of Sciences, China; Virginia Tech, USA; Huazhong University of Science and Technology, China)
Natural language prompts serve as an essential interface between users and Large Language Models (LLMs) like GPT-3.5 and GPT-4, which are employed by ChatGPT to produce outputs across various tasks. However, prompts crafted with malicious intent, known as jailbreak prompts, can circumvent the restrictions of LLMs, posing a significant threat to systems integrated with these models. Despite their critical importance, there is a lack of systematic analysis and comprehensive understanding of jailbreak prompts. Our paper aims to address this gap by exploring key research questions to enhance the robustness of LLM systems: 1) What common patterns are present in jailbreak prompts? 2) How effectively can these prompts bypass the restrictions of LLMs? 3) With the evolution of LLMs, how does the effectiveness of jailbreak prompts change? To address our research questions, we embarked on an empirical study targeting the LLMs underpinning ChatGPT, one of today's most advanced chatbots. Our methodology involved categorizing 78 jailbreak prompts into 10 distinct patterns, further organized into three jailbreak strategy types, and examining their distribution. We assessed the effectiveness of these prompts on GPT-3.5 and GPT-4, using a set of 3,120 questions across 8 scenarios deemed prohibited by OpenAI. Additionally, our study tracked the performance of these prompts over a 3-month period, observing the evolutionary response of ChatGPT to such inputs. Our findings offer a comprehensive view of jailbreak prompts, elucidating their taxonomy, effectiveness, and temporal dynamics. Notably, we discovered that GPT-3.5 and GPT-4 could still generate inappropriate content in response to malicious prompts without the need for jailbreaking. This underscores the critical need for effective prompt management within LLM systems and provides valuable insights and data to spur further research in LLM testing and jailbreak prevention.
@InProceedings{SEA4DQ24p12,
author = {Yi Liu and Gelei Deng and Zhengzi Xu and Yuekang Li and Yaowen Zheng and Ying Zhang and Lida Zhao and Tianwei Zhang and Kailong Wang},
title = {A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {12--21},
doi = {10.1145/3663530.3665021},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "Evaluating the Quality of ..."
Evaluating the Quality of Open Source Ansible Playbooks: An Executability Perspective
Pemsith Mendis, Wilson Reeves, Muhammad Ali Babar, Yue Zhang, and Akond Rahman
(Auburn University, USA; University of Adelaide, Australia)
Infrastructure as code (IaC) is the practice of automatically managing computing platforms, such as Internet of Things (IoT) platforms. IaC has gained popularity in recent years, yielding a plethora of software artifacts, such as Ansible playbooks that are available on social coding platforms. Despite the availability of open source software (OSS) Ansible playbooks, there is a lack of empirical research on the quality of these playbooks, which can hinder the progress of IaC-related research. To that end, we conduct an empirical study with 2,952 OSS Ansible playbooks where we evaluate the quality of OSS playbooks from the perspective of executability, i.e., if publicly available OSS Ansible playbooks can be executed without failures. From our empirical study, we observe 71.5% of the mined 2,952 Ansible playbooks cannot be executed as is because of four categories of failures.
@InProceedings{SEA4DQ24p2,
author = {Pemsith Mendis and Wilson Reeves and Muhammad Ali Babar and Yue Zhang and Akond Rahman},
title = {Evaluating the Quality of Open Source Ansible Playbooks: An Executability Perspective},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {2--5},
doi = {10.1145/3663530.3665019},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "A Hitchhiker’s Guide to ..."
A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering
Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Kailong Wang
(Nanyang Technological University, Singapore; UNSW, Australia; Institute of Information Engineering at Chinese Academy of Sciences, China; Virginia Tech, USA; Huazhong University of Science and Technology, China)
Natural language prompts serve as an essential interface between users and Large Language Models (LLMs) like GPT-3.5 and GPT-4, which are employed by ChatGPT to produce outputs across various tasks. However, prompts crafted with malicious intent, known as jailbreak prompts, can circumvent the restrictions of LLMs, posing a significant threat to systems integrated with these models. Despite their critical importance, there is a lack of systematic analysis and comprehensive understanding of jailbreak prompts. Our paper aims to address this gap by exploring key research questions to enhance the robustness of LLM systems: 1) What common patterns are present in jailbreak prompts? 2) How effectively can these prompts bypass the restrictions of LLMs? 3) With the evolution of LLMs, how does the effectiveness of jailbreak prompts change? To address our research questions, we embarked on an empirical study targeting the LLMs underpinning ChatGPT, one of today's most advanced chatbots. Our methodology involved categorizing 78 jailbreak prompts into 10 distinct patterns, further organized into three jailbreak strategy types, and examining their distribution. We assessed the effectiveness of these prompts on GPT-3.5 and GPT-4, using a set of 3,120 questions across 8 scenarios deemed prohibited by OpenAI. Additionally, our study tracked the performance of these prompts over a 3-month period, observing the evolutionary response of ChatGPT to such inputs. Our findings offer a comprehensive view of jailbreak prompts, elucidating their taxonomy, effectiveness, and temporal dynamics. Notably, we discovered that GPT-3.5 and GPT-4 could still generate inappropriate content in response to malicious prompts without the need for jailbreaking. This underscores the critical need for effective prompt management within LLM systems and provides valuable insights and data to spur further research in LLM testing and jailbreak prevention.
@InProceedings{SEA4DQ24p12,
author = {Yi Liu and Gelei Deng and Zhengzi Xu and Yuekang Li and Yaowen Zheng and Ying Zhang and Lida Zhao and Tianwei Zhang and Kailong Wang},
title = {A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {12--21},
doi = {10.1145/3663530.3665021},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "A Hitchhiker’s Guide to ..."
A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering
Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Kailong Wang
(Nanyang Technological University, Singapore; UNSW, Australia; Institute of Information Engineering at Chinese Academy of Sciences, China; Virginia Tech, USA; Huazhong University of Science and Technology, China)
Natural language prompts serve as an essential interface between users and Large Language Models (LLMs) like GPT-3.5 and GPT-4, which are employed by ChatGPT to produce outputs across various tasks. However, prompts crafted with malicious intent, known as jailbreak prompts, can circumvent the restrictions of LLMs, posing a significant threat to systems integrated with these models. Despite their critical importance, there is a lack of systematic analysis and comprehensive understanding of jailbreak prompts. Our paper aims to address this gap by exploring key research questions to enhance the robustness of LLM systems: 1) What common patterns are present in jailbreak prompts? 2) How effectively can these prompts bypass the restrictions of LLMs? 3) With the evolution of LLMs, how does the effectiveness of jailbreak prompts change? To address our research questions, we embarked on an empirical study targeting the LLMs underpinning ChatGPT, one of today's most advanced chatbots. Our methodology involved categorizing 78 jailbreak prompts into 10 distinct patterns, further organized into three jailbreak strategy types, and examining their distribution. We assessed the effectiveness of these prompts on GPT-3.5 and GPT-4, using a set of 3,120 questions across 8 scenarios deemed prohibited by OpenAI. Additionally, our study tracked the performance of these prompts over a 3-month period, observing the evolutionary response of ChatGPT to such inputs. Our findings offer a comprehensive view of jailbreak prompts, elucidating their taxonomy, effectiveness, and temporal dynamics. Notably, we discovered that GPT-3.5 and GPT-4 could still generate inappropriate content in response to malicious prompts without the need for jailbreaking. This underscores the critical need for effective prompt management within LLM systems and provides valuable insights and data to spur further research in LLM testing and jailbreak prevention.
@InProceedings{SEA4DQ24p12,
author = {Yi Liu and Gelei Deng and Zhengzi Xu and Yuekang Li and Yaowen Zheng and Ying Zhang and Lida Zhao and Tianwei Zhang and Kailong Wang},
title = {A Hitchhiker’s Guide to Jailbreaking ChatGPT via Prompt Engineering},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {12--21},
doi = {10.1145/3663530.3665021},
year = {2024},
}
Publisher's Version
SEA4DQ '24: "A Pilot Study in Surveying ..."
A Pilot Study in Surveying Data Challenges of Automatic Software Engineering Tasks
Liming Dong, Qinghua Lu, and Liming Zhu
(CSIRO’s Data61, Australia; UNSW, Australia)
The surge in automatic SE research aims to boost development efficiency and quality while reducing costs. However, challenges such as limited real-world project data and inadequate data conditions constrain the effectiveness of these methods. To systematically understand these challenges, our pilot study reviews prevalent data challenges across various SE tasks. Despite these challenges, thanks to the advances of large language model offers promising performance on SE tasks. Overall, this pilot survey focused on provide a quick retrospective review on SE data challenges and introduce practical LLM solutions from the SE community to mitigate these challenges.
@InProceedings{SEA4DQ24p6,
author = {Liming Dong and Qinghua Lu and Liming Zhu},
title = {A Pilot Study in Surveying Data Challenges of Automatic Software Engineering Tasks},
booktitle = {Proc.\ SEA4DQ},
publisher = {ACM},
pages = {6--11},
doi = {10.1145/3663530.3665020},
year = {2024},
}
Publisher's Version