Awesome GPT + Security

A curated list of awesome security tools, experimental case or other interesting things with LLM or GPT.

Contents

• Tools
  • Audit
  • Reconnaissance
  • Offensive
  • Detecting
  • Preventing
  • Social Engineering
  • Reverse Engineering
  • Fix
  • Assessment
• Cases
  • Experimental
  • Academic
  • Blogs
  • Fun
• GPT Security
  • Bypass Security Policy
  • Bug Bounty
  • Crack
• Contributing

Tools

🧰

Audit

• SourceGPT - prompt manager and source code analyzer built on top of ChatGPT as the oracle
• ChatGPTScanner - A white box code scan powered by ChatGPT
• chatgpt-code-analyzer - ChatGPT Code Analyzer for Visual Studio Code
• hacker-ai - An online tool using AI to detect vulnerabilities in source code
• audit_gpt - Fine-tuning GPT for Smart Contract Auditing

Reconnaissance

• GPT_Vuln-analyzer - Uses ChatGPT API, Python-Nmap, DNS Recon modules and uses the GPT3 model to create vulnerability reports based on Nmap scan data, and DNS scan information. It can also perform subdomain enumeration to a great extent
• SubGPT - SubGPT looks at subdomains you have already discovered for a domain and uses BingGPT to find more.
• Navi - A QA based Reconnaissance Tool with GPT

Offensive

• PentestGPT - A GPT-empowered penetration testing tool
• burpgpt - A Burp Suite extension that integrates OpenAI's GPT to perform an additional passive scan for discovering highly bespoke vulnerabilities, and enables running traffic-based analysis of any type.
• ReconAIzer - A Burp Suite extension to add OpenAI (GPT) on Burp and help you with your Bug Bounty recon to discover endpoints, params, URLs, subdomains and more!
• PassGAN - A Deep Learning Approach for Password Guessing. HomeSecurityHeroes land a Product, and you can test how much time an AI would need to crack your password here.
• nuclei_gpt - Only need to submit the relevant Request and Response and the description of the vulnerability to generate a Nuclei PoC.

Detecting

• k8sgpt - a tool for scanning your Kubernetes clusters, diagnosing, and triaging issues in simple English.
• cloudgpt - Vulnerability scanner for AWS customer managed policies using ChatGPT
• IATelligence - About IATelligence is a Python script that will extract the IAT of a PE file and request GPT to get more information about the API and the ATT&CK matrix related

Preventing

Social Engineering

• ChatGPT-Web-Setting-Funny-Abuse - Play with ChatGPT-Web and found the HTML rendering in description settings.

Reverse Engineering

• gpt-wpre - Whole-Program Reverse Engineering with GPT-3
• G-3PO - A Script that Solicits GPT-3 for Comments on Decompiled Code

Fix

• wolverine - Auto fix the bugs in your Python Script/Code

Assessment

• falco-gpt - AI-generated remediations for Falco audit events
• selefra - an open-source policy-as-code software that provides analytics for multi-cloud and SaaS.

---

Cases

🌰

Experimental

• Lost in ChatGPT's memories: escaping ChatGPT-3.5 memory issues to write CVE PoCs
• I built a Zero Day virus with undetectable exfiltration using only ChatGPT prompts
• Experimenting with GPT-3 for Detecting Security Vulnerabilities in Code
• We put GPT-4 in Semgrep to point out false positives & fix code
• A Practical, AI-Generated Phishing PoC With ChatGPT
• Capturing the Flag with GPT-4
• I Used GPT-3 to Find 213 Security Vulnerabilities in a Single Codebase
• Using ChatGPT to generate encoder and supporting WebShell
• Using OpenAI Chat to Generate Phishing Campaigns -- Include Phishing Platform
• Chat4GPT Experiments for Security

Academic

• GPT-4 Technical Report -- OpenAI's own security assessment and mitigation of the model
• Ignore Previous Prompt: Attack Techniques For Language Models -- Pioneering work of Prompt Injection
• More than you've asked for: A Comprehensive Analysis of Novel Prompt Injection Threats to Application-Integrated Large Language Models
• RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
• Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks
• Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
• Can We Generate Shellcodes via Natural Language? An Empirical Study

Blogs

• Dissecting redis CVE-2023-28425 with chatGPT as assistant
• Security Code Review With ChatGPT
• ChatGPT happy to write ransomware, just really bad at it

Fun

• shortest prompt that will enable GPT to protect the secret key
• a CTF-like game that teaches how to bypass LLM using language hacks

---

GPT Security

🚨

Bypass Security Policy

• Chat GPT "DAN" (and other "Jailbreaks")
• ChatGPT Prompts for Bug Bounty & Pentesting

Bug Bounty

• Building A Virtual Machine inside ChatGPT -- deprecated but interesting
• LangChain vulnerable to code injection -- CVE-2023-29374

Crack

• gpt4free -- Just API's from some language model sites.
• EdgeGPT -- Reverse engineered API of Microsoft's Bing Chat AI

Contributing

Your contributions are always welcome! Please take a look at the contribution guidelines first.

---

If you have any question about this opinionated list, do not hesitate to open an issue on GitHub.