The use of AI-generated code in software is on the rise, promising a fast and efficient way to automate manual work. However, this comes with a major downside: the code produced by artificial intelligence is not secure and can expose your company to malicious attacks. It’s essential to recognize that while these tools may seem too good to be true, they often produce code that is vulnerable to critical security issues.
In this blog post, we discuss the main risks of using AI tools to generate software code and the steps you need to take to keep your code secure.
AI-generated code is here to stay — and the risks should not be overlooked
As software development continues to grow as a popular career choice, many developers are starting to use AI tools in their junior roles or even during their studies. Unfortunately, beginner programmers often lack the necessary expertise to identify and fix code vulnerabilities, leading to serious security risks for the company they work for.
GitHub’s Copilot is one of the most popular tools among developers who want to write code more efficiently. However, as the article, Analyzing GitHub Copilot suggestions points out, the suggestions provided by GitHub Copilot should always be carefully analyzed and reviewed for security vulnerabilities before being implemented in production. Their analysis found serious deficiencies in the code written by Copilot, such as:
- SQL injection: The SQL query stored in the variable $query is constructed in a manner that is susceptible to SQL injection, as it takes user-supplied values and uses them directly in the query.
- Sensitive information disclosure: A form field uses auto-completion, which allows some browsers to retain sensitive information in their history.
- Session fixation: The session name is predictable (set to the username), exposing the user to session fixation attacks.
- Cross-site scripting (XSS): The value of the username parameter is shown directly on the page, which creates a vulnerability for reflected cross-site scripting (XSS).
- Weak hashing algorithm: Password encryption is weak as it uses unsalted MD5 hash which is vulnerable and can be cracked in seconds, leaving the password unprotected.
A blog post by Elaine Atwell, GitHub Copilot isn’t worth the risk also found GitHub’s Copilot a high-risk tool and suggested that while it may seem convenient to use, it’s definitely not worth the potential security risks. The article features the alarming results of a study that investigated code written by Copilot:
„Experts have reported that Copilot often suggests code with security flaws. In one study, researchers produced 1,689 programs with Copilot, of which 40% were vulnerable to attack”
Moreover, GitHub’s Copilot has even been the subject of a lawsuit by the American Institute of Mathematics (AIM) with the plaintiffs alleging that Copilot infringes on their copyright, by using code from a language model that was trained on math textbooks authored by the plaintiffs. The lawsuit raises important questions about the use of AI in software development and its potential for copyright infringement.
These examples highlight why companies should always prioritize security over efficiency. While GitHub Copilot and its newer alternatives like AWS’s CodeWhisperer can be useful and convenient in certain use cases, the code written by these tools should always be double-checked and not replace the expertise of experienced security professionals.
The 3 main steps to keep your codebase safe
- Always check the code thoroughly, especially if it’s copied from somewhere else
It is important to thoroughly review any code that you copy from another source before integrating it into your projects. While copying and pasting code can be a time-saving strategy, it can also introduce potential vulnerabilities and errors. By checking the code carefully, you can ensure that it is secure, well-written, and compatible with your project requirements. Taking the extra time to review and validate code can help you develop more reliable and effective software in the long run and avoid unexpected issues.
- Use code analysis tools to identify issues
Code analytics tools are designed to analyze code for potential vulnerabilities and errors, providing insights and recommendations for improving code quality and reducing security risks. Some of the best code analytics tools like SonarQube and Veracode use a range of techniques such as static code analysis, dynamic code analysis, and fuzz testing to identify potential issues and provide actionable insights for developers. By using code analytics tools, you can quickly and efficiently identify and fix vulnerabilities, improving the overall security and stability of your software.
- Use Ethical Hacking Platforms, such as Hackrate
Ethical hacking platforms like Hackrate provide a controlled and safe environment for ethical hackers to test and identify potential vulnerabilities in your system. By using these platforms, you can simulate real-world hacking scenarios and identify weaknesses in your security measures, allowing you to proactively address them before malicious hackers can exploit them. Additionally, ethical hacking platforms can help you comply with industry regulations and standards, such as SOC 2 or ISO 27001, which require continuous security testing and vulnerability assessments.
Testing ChatGPT-written code for vulnerabilities
We were curious to see a real-life example of an AI-generated vulnerability, so we asked ChatGPT to create a generic SQL query in C#.
The result speaks for itself. Upon the first prompt, the AI tool’s code already shows a serious issue with SQL injection: it concatenates user input into SQL queries without properly sanitizing it. This could allow an attacker to inject malicious SQL code into the query and potentially access or modify sensitive data, including personally identifiable information (PII), financial records, user credentials, and other confidential data.
By injecting malicious SQL statements, attackers can modify, delete, or insert unauthorized data into the database. For example, an attacker could modify the prices of products, delete customer orders, or tamper with financial records.
Depending on the nature of the data exposed or compromised, organizations may face legal consequences for failing to protect sensitive information adequately.
In conclusion
The previous examples underline the importance of being vigilant about vulnerabilities in code generated by AI, as it can pose significant security risks to organizations. Hackers can exploit these vulnerabilities to gain unauthorized access to sensitive data, cause damage to the system, or compromise the privacy of users.
Therefore, companies should not rely on AI-generated code alone. It’s also crucial to take steps to secure the codebase, such as regularly testing the code for vulnerabilities, using code analysis tools, and taking advantage of ethical hacking platforms. By taking proactive steps to keep the codebase safe, you can protect your business from potential security breaches and serious consequences.
We hope you found this article helpful. Let us know if you have any specific questions about how you can improve the security of your codebase, or book a demo to see how Hackrate works!
