Generative artificial intelligence (AI) has undeniably revolutionised lives and businesses. From source code development to autonomous content generation, the apogee of the tool’s expansive potential is only limited by our imagination.
At the same time, the tool’s immense power has also attracted the attention of hackers and bad actors in the cybersecurity arms race, raising concerns over its potential weaponisation. Research recently published by cybersecurity firm SlashNext revealed the discovery of WormGPT, a blackhat “alternative” to ChatGPT that is being sold to cybercriminal operators on the dark web.
As generative AI continues to develop, scams have also evolved to become increasingly sophisticated. Businesses and cybersecurity firms now face a pressing challenge to protect their organisations from falling prey to such cyberthreats. According to Accenture’s Pulse of Change survey, 93% of organisations plan to allocate a significant portion of their technology investment budget towards generative AI initiatives.
While the excitement and aspirations surrounding generative AI are undeniable, business leaders must be highly attuned to accompanying operational and security risks – and how those risks can be minimised.
The rise of new-age cyberthreats
While concerns around data privacy are not new, generative AI has opened up Pandora’s box and introduced new complications. Take chatbots, for example, which routinely gather personal data, such as IP addresses, browser information, and browsing activities across websites without users’ knowledge. When employees interact with such AI systems, they often unintentionally disclose sensitive information that becomes part of the public domain. Cyberhaven, a data security company, found that 11% of the data employees paste into ChatGPT is confidential.
The way that generative AI can be misused to weaken authentication systems based on voice and/or facial recognition is evident in how it has simplified the creation of deepfakes, enabling malicious actors to steal or manipulate a person’s voice and biometric information, incorporating them into images or videos. AI algorithms can also analyse online behaviour to create fake identities, impersonate individuals, and commit fraud. For instance, in 2020, scammers utilised generative AI technology to mimic a company director’s voice, successfully duping a bank manager in Hong Kong into authorising a transfer of HK$ 35 million.
Moreover, scammers can now misuse AI tools such as ChatGPT to mimic the language style of specific organisations, creating highly realistic copies of scam messages or fake websites. Consequently, detecting phishing attacks becomes increasingly challenging for email recipients, as these messages are well-written and lack the traditional typos and grammatical errors typically associated with such scams. In 2022, Singapore alone reported over SG$660 million lost in scam cases, a staggering 32.6% increase from the previous year, with a majority of these scams originating from one of the most primitive forms of cyberthreat – email phishing. This is set to proliferate further as bad actors harness the power of generative AI tools.
Even with some protections against misuse built into the tools, Accenture has recognised that hackers are already selling workarounds that ‘jailbreak’ GPTs to allow for malware creation. Thus, increasing the need for robust guardrails such as a shift in security mechanisms, people processes, and technology to establish and maintain trust in business, government, and communities.
Generative AI can be secure AI
Despite the risks, businesses should not shun Generative AI; rather, they should exercise caution and develop robust security strategies to leverage its potential. These steps could include:
- Harnessing AI and machine learning technologies such as sophisticated behavioural analytics and fraud detection scores, businesses can train fraud models and enhance their detection capabilities.
- Workforce training programmes are crucial to combat the dissemination of misinformation, ensuring that employees understand the business and security risks involved while acquiring knowledge of best practices. In many organisations, employees are required to complete training modules that cover various aspects, such as cloud computing, security, and operations, including mandatory modules on responsible AI. Through comprehensive training programmes, employees can prevent the inadvertent leakage of confidential information by being mindful of the data they unknowingly provide to AI systems.
- Implementing a “trust by design” approach can help organisations create a trusted environment and minimise the risk of data loss. As the risks of data leakage lie primarily at the application layer rather than the chat layer, companies can build a custom front end that replaces the ChatGPT interface to leverage the chat API directly. Going one step further, they can add filters, reduce bias, and safeguard data by isolating data within a sandbox environment. Ensuring that there are necessary applications capable of creating a trusted environment also requires the reinvention of cybersecurity practices. Key areas of focus include preparing for adversarial attacks, data privacy and security loopholes, organisational trustworthiness and accountability, and threat detection capabilities.
Diving into trustworthiness, the risk of biased or fabricated answers derived by GPT tools can also compromise the integrity of the generated content. The White House has asserted that organisations developing these pioneering technologies have a profound obligation to behave responsibly and ensure their products are safe. This further underscores the imperative for organisations to add in the human element as a guardrail to prevent rogue use of the tool. - Adopting a “human in the loop” approach, where human intervention is integrated into AI processes, adds an extra layer of security and facilitates a sanity check on responses. Generative AI, fundamentally, bases its decision-making and algorithm from fed data. In this regard, reinforcement learning from human feedback can tune the model based on human responses generated from the same prompt to enhance trust and safety, and minimise harm by eliminating bias.
As AI continues to advance, bad actors will undoubtedly evolve their techniques to exploit AI further. It is imperative for businesses to proactively identify emerging threats and promptly take the necessary precautions to defend against new-age cyberthreats. With a well-planned security strategy, comprehensive employee training, and a combined human-AI approach, businesses can effectively navigate the challenges posed by generative AI. Vigilance and adaptability would also represent key traits for organisations to harness the expansive potential and immense power of generative AI, while safeguarding their operations, reputation, and sensitive information from the evolving landscape of cybersecurity threats. Despite its unpredictable nature, the tool that has undeniably revolutionised lives and businesses need not be a cause for anxiety. Ultimately, these guardrails can help lay the foundations for using AI for good and defending AI for all.
