Evaluating Zero-Shot Chatgpt Performance on Predicting CVE Data From Vulnerability Descriptions
Costin, A., Turtiainen, H., Yousefnezhad, N., Bogulean, V., & Hämäläinen, T. (2024). Evaluating Zero-Shot Chatgpt Performance on Predicting CVE Data From Vulnerability Descriptions. In M. Lehto, & M. Karjalainen (Eds.), Proceedings of the 23rd European Conference on Cyber Warfare and Security (23, pp. 576-584). Academic Conferences International Ltd. Proceedings of the European Conference on Cyber Warfare and Security. https://doi.org/10.34190/eccws.23.1.2285
Julkaistu sarjassa
Proceedings of the European Conference on Cyber Warfare and SecurityTekijät
Päivämäärä
2024Tekijänoikeudet
© 2024 European Conference on Cyber Warfare and Security
2024:52 | 2025:10
Vulnerability management is a critical industry activity driven by compliance and regulations aiming to allocate best-fitted resources to address vulnerabilities efficiently. The increasing number of vulnerabilities reported and discovered by a diverse community results in varying quality of the reports and differing perspectives. To tackle this, machine learning (ML) has shown promise in automating vulnerability assessments. While some existing ML approaches have demonstrated feasibility, there is room for improvement. Additionally, gaps remain in the literature to understand how the specific terminology used in vulnerability databases and reports influences ML interpretation. Large Language Model (LLM) systems, such as ChatGPT, are praised for their versatility and high applicability to any domain. However, how well or poorly a state-of-the-art LLM system performs on existing vulnerability datasets at a large scale and across different scoring metrics needs to be clarified or well-researched. This paper aims to close several such gaps and present a more precise and comprehensive picture of how ChatGPT performs on predicting vulnerability metrics based on NVD's CVE vulnerability database. We analyze the responses from ChatGPT on a set of 113,228 (~50% out of all NVD vulnerabilities) CVE vulnerability descriptions and measure its performance against NVD-CVE as ground truth. We measure and analyze the predictions for several vulnerabilities in metadata and calculate performance statistics.
...
Julkaisija
Academic Conferences International LtdKonferenssi
European Conference on Cyber Warfare and SecurityKuuluu julkaisuun
Proceedings of the 23rd European Conference on Cyber Warfare and SecurityISSN Hae Julkaisufoorumista
2048-8602Asiasanat
Julkaisu tutkimustietojärjestelmässä
https://converis.jyu.fi/converis/portal/detail/Publication/220855002
Metadata
Näytä kaikki kuvailutiedotKokoelmat
Lisätietoja rahoituksesta
Hannu Turtiainen thanks the Finnish Cultural Foundation / Suomen Kulttuurirahasto (www.skr.fi) for supporting his Ph.D. dissertation work and research (grant decision no. 00231412). (Part of) This work was supported by the European Commission under the Horizon Europe Programme, as part of the project LAZARUS (https://lazarus-he.eu/) (Grant Agreement no. 101070303).Lisenssi
Samankaltainen aineisto
Näytetään aineistoja, joilla on samankaltainen nimeke tai asiasanat.
-
Process‐Informed Neural Networks : A Hybrid Modelling Approach to Improve Predictive Performance and Inference of Neural Networks in Ecology and Beyond
Wesselkamp, Marieke; Moser, Niklas; Kalweit, Maria; Boedecker, Joschka; Dormann, Carsten F. (Wiley, 2024)Despite deep learning being state of the art for data-driven model predictions, its application in ecology is currently subject to two important constraints: (i) deep-learning methods are powerful in data-rich regimes, but ... -
Emotions and Activity Recognition System Using Wearable Device Sensors
Rumiantcev, Mikhail (FRUCT Oy, 2021)Nowadays machines have become extremely smart, there are a lot of existing services that seemed to be unexpectable and futuristic decades or even a few years ago. However, artificial intelligence is still far from human ... -
Tekoälykäs viestintä
Niittymaa, Jukka; Luoma-aho, Vilma (ProCom - Viestinnän ammattilaiset ry, 2024)Organisaatioiden viestintä, markkinointi ja asiakaspalvelu siirtyivät uuteen tekoälykkääseen aikaan loppuvuodesta 2022, kun yhdysvaltalainen OpenAI lanseerasi ChatGPT:n. Tekoälykäs viestintä, eli tässä artikkelissa ... -
AI as a User of AI : Towards Responsible Autonomy
Shukla, Amit K.; Terziyan, Vagan; Tiihonen, Timo (Elsevier, 2024)Recent advancements in Artificial Intelligence (AI), particularly in generative language models and algorithms, have led to significant impacts across diverse domains. AI capabilities to address prompts are growing beyond ... -
Can ChatGPT Challenge the Scientific Impact of Published Research, Particularly in the Context of Industry 4.0 and Smart Manufacturing?
Terziyan, Vagan; Kaikova, Olena; Golovianko, Mariia; Vitko, Oleksandra (Elsevier, 2024)The released ChatGPT as a powerful language model is capable of assisting with a wide range of tasks, including answering questions, summarizing, paraphrasing, proofreading, classifying, and integrating texts. In this ...
Ellei toisin mainittu, julkisesti saatavilla olevia JYX-metatietoja (poislukien tiivistelmät) saa vapaasti uudelleenkäyttää CC0-lisenssillä.