A Collection is a selection of features, articles, comments and opinions on any given theme or topic. It allows you to stay up‑to‑date with what interests you most.
Login here to access your saved articles and followed authors.
We have sent you an email so you can reset your password.
Sorry, we had a problem.
Tags related to this article
Published 28 febrero 2023
AI language models (a type of artificial intelligence that is designed to process and understand human language) like Chat GPT have the potential to revolutionise communication and information exchange. Chat GPT, short for ‘Chat Generative Pre-trained Transformer’, is a language based bot that can generate sophisticated human-like text and which can be used for a number of purposes. Indeed, amongst other things, it has been used to pass exams, write letters and even song lyrics. It is unsurprising that tech giants like Google and Amazon have taken a keen interest in AI chatbot technology, with Microsoft implementing it into its Bing search engine as a separate chatbot function. AI language models have allowed for significant advances in natural language processing and have become increasingly important in a wide range of industries, from healthcare and finance to education and entertainment.
Whilst the technology is not perfect (it has been known to provide inaccurate information and offensive responses to queries), it is important to acknowledge the current and future cyber risks posed by this emerging technology particularly as AI chatbot technology will almost certainly improve. Our team decided to explore first-hand Chat GPT’s current capabilities.
Phishing attacks often use social engineering techniques to trick victims into clicking on a link or downloading an attachment, which can install malware or lead to a fake login page where they are asked to provide their personal information. Armed with this information, threat actors can access a victim’s organisation’s IT estate and freely move through it and access sensitive data and deploy malware and ransomware. One of the key giveaways that a communication is phishing is when there are typos and unnatural language expressions in the body of the email or text.
Whilst in theory, Chat GPT, like other AI language models, is intended to reject inappropriate user requests, it can potentially be used by hackers to circumvent any language deficiencies of their phishing content, by generating highly-convincing phishing messages.
Our team put this to the test and asked the platform to draft a convincing phishing message targeting a law firm. While it initially refused to do so, based on its ethical limitation (see Figure 1), after some convincing, we were able to obtain the body of a believable email message (see Figure 2).
Figure 1: Chat GPT’s ethics policy preventing us from generating a phishing email
Figure 2: Our team successfully circumventing the protection of the Chat GPT ethics policy and engaging ChatGPT to draft a phishing email (content blurred)
Chat GPT has been praised by programmers worldwide for its ability to create software code, in various programming languages, therefore revolutionising the software industry. This ability of the platform to generate code, poses the risk of threat actors using it to create malicious software of all kinds like viruses, works, Trojans, ransomware and wipers, to name a few.
As reported by Techradar in late 2022, cybersecurity researchers were able to use the platform to create a “weaponised” Excel file with nothing more than a simple command for the chatbot. However, our team’s much less subtle attempt to convince it to replicate the source code for the world’s first computer virus, the Creeper, were unsuccessful (see Figure 3).
Figure 3: The Chat GPT ethics policy preventing us from getting the source code of a virus
Finally, there is a risk of Chat GPT being used to create deepfake content. Deepfakes are videos or images that are altered to appear as though they depict a real person, while in reality, being entirely or partially computer-generated. While AI language models could be used to generate false media content, there are several machine learning models that are capable of generating such content.
For more information on the commercial risks borne out of malicious use of deepfakes, see some of our latest articles on the subject.
We consider that this AI technology is intrinsically a good thing, which can be used for a wide range of useful purposes. However, it does not come without risks and it is important that steps are taken to protect the technology from being exploited. To mitigate against cyber risks borne out of the use of AI language models, it is important to ensure that they are designed with robust security features and incorporate ethical policies, which cannot be circumvented by end users. It is also important that the platform is monitored for any suspicious activity and that there is an awareness of the technology’s associated risks.
+44 (0)161 934 3167
London - Walbrook
+44(0)20 7894 6703
+44 (0) 20 7894 6443
Hans Allnutt, Patrick Hill, Jade Kowalski
Christopher Air, Stuart Hunt
Patrick Hill, Sonali Malhotra
Jade Kowalski, Amanda Mackenzie
Jade Kowalski, Stuart Hunt
Jade Kowalski, Mathew Rutter, Zoë Carpenter
Hans Allnutt, Astrid Hardy
Patrick Hill, Camilla Elliot
Jade Kowalski, Alexander Dimitrov
Rowena McCormack, Aidan Healy, Charlotte Burke
Jade Kowalski, Astrid Hardy, Kelsey Farish
Christopher Air, Omar Kamal
Igor Pinedo Garcia, Aidan Healy, Stuart Hunt
Patrick Hill, Hans Allnutt, Jade Kowalski
Jade Kowalski, Omar Kamal