Advancing a Speech Act-Based Model to Improve Future Quality of Information Security Policies Using Large Language Models
DOI:
https://doi.org/10.7250/csimq.2026-46.03Keywords:
Information Security Policy, Cybersecurity Policy, Speech Act, Large Language ModelAbstract
Employee compliance with Information Security Policies (ISPs) depends on communicating clear and comprehensible content. However, existing research has shown that many ISPs are of poor communicative quality. Large Language Models (LLMs) could enhance ISPs if fine-tuned on high-quality data, but to do such fine-tuning requires a conceptual model for classifying the data and evaluating the resulting text. Therefore, as a step in this direction, the aim of this article is to develop a conceptual model of ISPs using Speech Act Theory as a theoretical lens to enable assessments of the communicative quality of ISPs. We used conceptual modeling and document analysis to develop the model based on 600 ISP statements from ten British National Health Service ISPs. We used selected parts from the SEQUAL framework to evaluate the model. The evaluation pointed to potential areas for improving the model’s semantic, empirical, physical, and deontic qualities. By incorporating these improvements, the final class diagram contains 21 classes, six of which address ISP statement quality as speech acts.
References
B. Kör and B. Metin, “Understanding human aspects for an effective information security management implementation,” International Journal of Applied Decision Sciences, vol. 14, no. 2, pp. 105–122, 2021. Available: https://doi.org/10.1504/IJADS.2021.113532 DOI: https://doi.org/10.1504/IJADS.2021.113532
M. J. Culnan and C. C. Williams, “How Ethics Can Enhance Organizational Privacy: Lessons from the Choicepoint and TJX Data Breaches,” MIS Quarterly, vol. 33, no. 4, pp. 673–687, 2009. Available: https://doi.org/10.2307/20650322 DOI: https://doi.org/10.2307/20650322
E. Kolkowska, F. Karlsson, and K. Hedström, “Towards analysing the rationale of information security noncompliance: Devising a Value-Based Compliance analysis method,” Journal of Strategic Information Systems, vol. 26, no. 1, pp. 39–57, 2017. Available: https://doi.org/10.1016/j.jsis.2016.08.005 DOI: https://doi.org/10.1016/j.jsis.2016.08.005
G. Dhillon, Information Security – Text & Cases. Edition 2.0 ed. Burlington, USA: Prospect Press, 2017.
Truesec, “Threat Intelligence Report 2023,” Truesec, Stockholm, Sweden, 2023.
Crowdstrike, “2025 Global Threat Report,” Crowdstrike Inc, 2025.
S. Chatterjee, X. Gao, S. Sarkar, and C. Uzmanoglu, “Reacting to the scope of a data breach: The differential role of fear and anger,” Journal of Business Research, vol. 101, pp. 183–193, 2019. Available: https://doi.org/10.1016/j.jbusres.2019.04.024 DOI: https://doi.org/10.1016/j.jbusres.2019.04.024
K. D. Loch, H. H. Carr, and M. E. Warkentin, “Threats to information systems: today’s reality, yesterday’s understanding,” MIS Quarterly, vol. 16, no. 2, pp. 173–186, 1992. Available: https://doi.org/10.2307/249574 DOI: https://doi.org/10.2307/249574
N. H. Chowdhury, M. T. Adam, and G. Skinner, “The impact of time pressure on cybersecurity behaviour: a systematic literature review,” Behav. Inf. Technol., vol. 38, no. 12, pp. 1290–1308, 2019. Available: https://doi.org/10.1080/0144929X.2019.1583769 DOI: https://doi.org/10.1080/0144929X.2019.1583769
K. Höne and J. H. P. Eloff, “Information security policy – what do international information security standards say?” Computers & Security, vol. 21, no. 5, pp. 402–409, 2002. Available: https://doi.org/10.1016/S0167-4048(02)00504-7 DOI: https://doi.org/10.1016/S0167-4048(02)00504-7
S. Goel and I. N. Chengalur-Smith, “Metrics for characterizing the form of security policies,” Journal of Strategic Information Systems, vol. 19, no. 4, pp. 281–295, 2010. Available: https://doi.org/10.1016/j.jsis.2010.10.002 DOI: https://doi.org/10.1016/j.jsis.2010.10.002
R. Baskerville and M. Siponen, “An information security meta-policy for emergent organizations,” Logistics Information Management, vol. 15, no. 5/6, pp. 337–346, 2002. Available: https://doi.org/10.1108/09576050210447019 DOI: https://doi.org/10.1108/09576050210447019
W. A. Cram, J. G. Proudfoot, and J. D’Arcy, “Organizational information security policies: a review and research framework,” European Journal of Information Systems, vol. 26, no. 6, pp. 605–641, 2017. Available: https://doi.org/10.1057/s41303-017-0059-9 DOI: https://doi.org/10.1057/s41303-017-0059-9
M. E. Whitman, “Security Policy – From Design to Maintenance,” in Information Security – Policy, Processes, and Practices, 2008, pp. 123–151.
S. V. Flowerday and T. Tuyikeze, “Information security policy development and implementation: The what, how and who,” Computers & Security, vol. 61, pp. 169–183, 2016. Available: https://doi.org/10.1016/j.cose.2016.06.002 DOI: https://doi.org/10.1016/j.cose.2016.06.002
M. Siponen and A. Vance, “Neutralization: New Insights into the Problem of Employee Information Systems Security Policy Violations,” MIS Quarterly, vol. 34, no. 3, pp. 487–502, 2010. Available: https://doi.org/10.2307/25750688 DOI: https://doi.org/10.2307/25750688
Ponemon, “Cost of insider threats global report,” Ponemon Institute, North Traverse City, 2020. Available: https://www.ibm.com/downloads/documents/us-en/107a02e94cc8f836
E. Rostami, F. Karlsson, E. Kolkowska, and S. Gao, “Towards software for tailoring information security policies to organisations’ different target groups,” Computers & Security, vol. 159, Article 104687, 2025. Available: https://doi.org/10.1016/j.cose.2025.104687 DOI: https://doi.org/10.1016/j.cose.2025.104687
K. Höne and J. H. P. Eloff, “What makes an effective information security policy?” Network Security, vol. 6, no. 1, pp. 14–16, 2002. Available: https://doi.org/10.1016/S1353-4858(02)06011-7 DOI: https://doi.org/10.1016/S1353-4858(02)06011-7
I. Lopes and P. Oliveira, “Applying Action Research in the Formulation of Information Security Policies,” in New Contributions in Information Systems and Technologies. Advances in Intelligent Systems and Computing, vol. 353, pp. 513–522, 2015. Available: https://doi.org/10.1007/978-3-319-16486-1_50 DOI: https://doi.org/10.1007/978-3-319-16486-1_50
F. Karlsson, K. Hedström, and G. Goldkuhl, “Practice-based discourse analysis of information security policies,” Computers & Security, vol. 67, pp. 267–279, 2017. Available: https://doi.org/10.1016/j.cose.2016.12.012 DOI: https://doi.org/10.1016/j.cose.2016.12.012
E. Rostami and F. Karlsson, “Qualitative Content Analysis of Actionable Advice in Information Security Policies – Introducing the Keyword Loss of Specificity Metric,” Information & Computer Security, vol. 32, no. 4, pp. 492–508, 2024. Available: https://doi.org/10.1108/ICS-10-2023-0187 DOI: https://doi.org/10.1108/ICS-10-2023-0187
B. C. Stahl, N. F. Doherty, and M. Shaw, “Information security policies in the UK healthcare sector: a critical evaluation,” Information Systems Journal, vol. 22, pp. 77–94, 2012. Available: https://doi.org/10.1111/j.1365-2575.2011.00378.x DOI: https://doi.org/10.1111/j.1365-2575.2011.00378.x
ISO, “ISO/IEC 27002:2022 Information security, cybersecurity and privacy protection – Information security controls,” International Organization for Standardization (ISO), 2022.
C. Sundt, “Information security and the law,” Information Security Technical Report, vol. 11, no. 1, pp. 2–9, 2006. Available: https://doi.org/10.1016/j.istr.2005.11.003 DOI: https://doi.org/10.1016/j.istr.2005.11.003
I. Trummer, “From BERT to GPT-3 codex: harnessing the potential of very large language models for data management,” in Proceedings of the VLDB Endowment, vol. 15, no. 12, 2022, pp. 3770–3773. Available: https://doi.org/10.14778/3554821.3554896 DOI: https://doi.org/10.14778/3554821.3554896
M. Abdullah, A. Madain, and Y. Jararweh, “ChatGPT: Fundamentals, Applications and Social Impacts,” 2022 Ninth International Conference on Social Networks Analysis, Management and Security (SNAMS), 2022, pp. 1–8. Available: https://doi.org/10.1109/SNAMS58071.2022.10062688 DOI: https://doi.org/10.1109/SNAMS58071.2022.10062688
F. Karlsson, S. Gao, J. Krogstie, and L. Aro-Sati, “Towards a Speech Act-Based Model to Enable Future Quality Improvements of Information Security Policies Using Large Language Models,” Perspectives in Business Informatics Research. BIR 2025. Lecture Notes in Business Information Processing, vol. 562, 2025, pp. 349–364. Available: https://doi.org/10.1007/978-3-032-04375-7_22 DOI: https://doi.org/10.1007/978-3-032-04375-7_22
J. R. Searle, “A Classification of Illocutionary Acts,” Language in Society, vol. 5, no. 1, pp. 1–23, 1976. Available: https://doi.org/10.1017/S0047404500006837 DOI: https://doi.org/10.1017/S0047404500006837
M. Alshaikh, S. B. Maynard, A. Ahmad, and S. Chang, “Information Security Policy: A Management Practice Perspective,” Australasian Conference on Information Systems, 2015.
N. Doherty and H. Fulford, “Aligning the information security policy with the strategic information systems plan,” Computer & Security, vol. 25, no. 1, pp. 55–63, 2006. Available: https://doi.org/10.1016/j.cose.2005.09.009 DOI: https://doi.org/10.1016/j.cose.2005.09.009
E. Rostami, F. Karlsson, and G. Shang, “Policy components – a conceptual model for modularizing and tailoring of information security policies,” Information & Computer Security, vol. 31, no. 3, pp. 331–352, 2023. Available: https://doi.org/10.1108/ICS-10-2022-0160 DOI: https://doi.org/10.1108/ICS-10-2022-0160
E. Rostami, M. Hanif, F. Karlsson, and S. Gao, “Defining Actionable Advice in Information Security Policies - Guiding Employees to Strengthen Digital Sovereignty of Organizations,” Procedia Computer Science, vol. 254, pp. 30–38, 2025. Available: https://doi.org/10.1016/j.procs.2025.02.061 DOI: https://doi.org/10.1016/j.procs.2025.02.061
E. Rostami, F. Karlsson, and S. Gao, “Requirements for computerized tools to design information security policies,” Computers & Security, vol. 99, Article 102063, 2020. Available: https://doi.org/10.1016/j.cose.2020.102063 DOI: https://doi.org/10.1016/j.cose.2020.102063
S. Diver, “Information Security Policy – A Development Guide for Large and Small Companies,” SANS Institute, 2021.
NIST, “Information Security Handbook: A Guide for Managers,” National Institute of Standards and Technology, Gaithersburg, USA, 2006.
T. R. Peltier, Information Security Policies and Procedures – A Practitioner’s Reference. Second Edition, Boca Raton, 2004. Available: https://doi.org/10.1201/9780203488737 DOI: https://doi.org/10.1201/9780203488737
C. R. Smith, “The Definitive Guide to Writing Effective Information Security Policies and Procedures,” Createspace, 2010.
Y. Yao, J. Duan, K. Xu, Y. Cai, Z. Sun, and Y. Zhang, “A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly,” High-Confidence Computing, vol. 4, no. 2, Article 100211, 2024. Available: https://doi.org/10.1016/j.hcc.2024.100211 DOI: https://doi.org/10.1016/j.hcc.2024.100211
J. Yang et al., “Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond,” ACM Transactions on Knowledge Discovery from Data, vol. 18, no. 6, pp. 1–32, 2024. Available: https://doi.org/10.1145/3649506 DOI: https://doi.org/10.1145/3649506
L. Yun, S. Yun, and H. Xue, “Improving citizen-government interactions with generative artificial intelligence: Novel human-computer interaction strategies for policy understanding through large language models,” PLoS ONE, vol. 19, no. 12, 2024. Available: https://doi.org/10.1371/journal.pone.0311410 DOI: https://doi.org/10.1371/journal.pone.0311410
S. Lawal, X. Zhao, A. Rios, R. Krishnan, and D. Ferraiolo, “Translating Natural Language Specifications into Access Control Policies by Leveraging Large Language Models,” 2024 IEEE 6th International Conference on Trust, Privacy and Security in Intelligent Systems, and Applications (TPS-ISA), pp. 361–370, 2024. Available: https://doi.org/10.1109/TPS-ISA62245.2024.00048 DOI: https://doi.org/10.1109/TPS-ISA62245.2024.00048
E. Quevedo et al., “Creation and Analysis of a Natural Language Understanding Dataset for DoD Cybersecurity Policies (CSIAC-DoDIN V1. 0),” 2023 International Conference on Computational Science and Computational Intelligence (CSCI), pp. 91–98, 2023. Available: https://doi.org/10.1109/CSCI62032.2023.00021 DOI: https://doi.org/10.1109/CSCI62032.2023.00021
S. Deldari et al., “AuditNet: A Conversational AI-based Security Assistant,” Adjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction (MobileHCI ’24 Adjunct), pp. 1–4, 2024. Available: https://doi.org/10.1145/3640471.3680444 DOI: https://doi.org/10.1145/3640471.3680444
D. Najafali, J. M. Camacho, E. Reiche, L. G. Galbraith, S. D. Morrison, and A. H. Dorafshar, “Truth or lies? The pitfalls and limitations of ChatGPT in systematic review creation,” Aesthetic Surgery Journal, vol. 43, no. 8, pp. NP654-NP655, 2023. Available: https://doi.org/10.1093/asj/sjad108 DOI: https://doi.org/10.1093/asj/sjad093
G. Goldkuhl and E. Braf, “Organisational Ability: Constituents and Congruencies,” in Knowledge Management in the SocioTechnical World, pp. 30–42, 2002. Available: https://doi.org/10.1007/978-1-4471-0187-1_4 DOI: https://doi.org/10.1007/978-1-4471-0187-1_4
J. L. Austin, How to do Things with Words. Cambridge: Oxford University Press, 1962.
J. R. Searle, Speech Acts: An Essay in the Philosophy of Language. Cambridge: Cambridge University Press, 1969. Available: https://doi.org/10.1017/CBO9781139173438 DOI: https://doi.org/10.1017/CBO9781139173438
J. R. Searle and D. Vanderveken, “Speech acts and illocutionary logic,” in Logic, Thought and Action. Logic, Epistemology, and the Unity of Science, vol. 2, pp. 109–132, 1985. Available: https://doi.org/10.1007/1-4020-3167-X_5 DOI: https://doi.org/10.1007/1-4020-3167-X_5
T. Holtgraves, “The production and perception of implicit performatives,” Journal of Pragmatics, vol. 37, no. 12, pp. 2024–2043, 2005. Available: https://doi.org/10.1016/j.pragma.2005.03.005 DOI: https://doi.org/10.1016/j.pragma.2005.03.005
J. R. Searle, Expression and Meaning: Studies in the Theory of Speech Acts. Cambridge: Cambridge University Press, 1979. Available: https://doi.org/10.1017/CBO9780511609213 DOI: https://doi.org/10.1017/CBO9780511609213
R. Gasparatou, “How to do things with words: Speech acts in education,” Educational Philosophy and Theory, vol. 50, no. 5, pp. 510–518, 2018. Available: https://doi.org/10.1080/00131857.2017.1382353 DOI: https://doi.org/10.1080/00131857.2017.1382353
J. V. Schmidt, “Can Artificial Agents be Authors?” Philosophy & Technology, vol. 38, no. 1, pp. 1–25, 2025. Available: https://doi.org/10.1007/s13347-025-00839-y DOI: https://doi.org/10.1007/s13347-025-00839-y
O. I. Lindland, G. Sindre, and A. Solvberg, “Understanding quality in conceptual modeling,” IEEE Software, vol. 11, no. 2, pp. 42–49, 1994. Available: https://doi.org/10.1109/52.268955 DOI: https://doi.org/10.1109/52.268955
J. Krogstie, Quality in Business Process Modeling. Springer, 2016. Available: https://doi.org/10.1007/978-3-319-42512-2 DOI: https://doi.org/10.1007/978-3-319-42512-2
J. Krogstie, Model-Based Development and Evolution of Information Systems: A Quality Approach. Springer, 2012. Available: https://doi.org/10.1007/978-1-4471-2936-3 DOI: https://doi.org/10.1007/978-1-4471-2936-3
B. Thalheim, “The Science of Conceptual Modelling,” Database and Expert Systems Applications. DEXA 2011. Lecture Notes in Computer Science, vol. 6860, 2011, pp. 12–26. Available: https://doi.org/10.1007/978-3-642-23088-2_2 DOI: https://doi.org/10.1007/978-3-642-23088-2_2
G. A. Bowen, “Document Analysis as a Qualitative Research Method,” Qualitative Research Journal, vol. 9, no. 2, pp. 27–40, 2009. Available: https://doi.org/10.3316/QRJ0902027 DOI: https://doi.org/10.3316/QRJ0902027
M. E. Duffy, “Methodological triangulation: a vehicle for merging quantitative and qualitative research methods,” Image: The Journal of Nursing Scholarship, vol. 19, no. 3, pp. 130–133, 1987. Available: https://doi.org/10.1111/j.1547-5069.1987.tb00609.x DOI: https://doi.org/10.1111/j.1547-5069.1987.tb00609.x
E. Rostami, “Tailoring information security policies - computerized tool and a design theory,” Ph.D. dissertation, Department of Informatics, Örebro University, Örebro, 2023.
E. G. Guba and Y. S. Lincoln, Fourth Generation Evaluation. SAGE Publications, 1989.
Y. S. Lincoln and E. G. Guba, Naturalistic Inquiry. Sage Publications, 1985. DOI: https://doi.org/10.1016/0147-1767(85)90062-8
D. S. Collingridge and E. E. Gantt, “The Quality of Qualitative Research,” American Journal of Medical Quality, vol. 23, no. 5, pp. 389–395, 2008, Available: https://doi.org/10.1177/1062860608320646 DOI: https://doi.org/10.1177/1062860608320646
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Fredrik Karlsson, Shang Gao, John Krogstie, Leila Aro-Sati (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
How to Cite
