Federal AI Security: Gaps in Enterprise Protection
Artificial Intelligence (AI) is being used within businesses to assist in answering customer queries, analysing data, and writing reports. But there are security concerns associated with using AI in a company, which can include exposing private data, releasing unsafe outputs, and providing new avenues of attack for hackers. Currently, in the US, NIST and CISA, along with other federal agencies, release guidance to help firms protect themselves from any security risks with AI. This can help companies understand how to secure their AI and can offer them more secure systems. However, the federal government’s advice may not cover all of the security threats from AI that a business faces. What should businesses be aware of? What isn’t covered in our advice and how can you take action? Implement strong policies, make sure your security measures are regularly tested, and promote safe use of AI.
What Is Federal AI Security Guidance?
Federal AI security guidance helps organizations understand AI-related risks and responsibly use AI. In the US, NIST offers resources to help organizations understand and manage these risks. Resources are available to help organizations evaluate risk and implement controls for human oversight, access controls, data accuracy, and data privacy. Resources are also available to help organizations implement processes and policies to govern AI acquisition, development, and use. AI security guidance is not always required. But some applicable laws and regulations require certain organizations to implement specific security controls. It is important for organizations to recognize what actions they need to take and choose the relevant guidance to appropriately secure AI systems.
Why Businesses Need Strong AI Security
Many businesses will utilise digital systems for storing data such as customer information, financial data, staff information and company documentation. The introduction of AI into these systems can cause extra security concerns, such as employees uploading sensitive data to an unauthorised AI chatbot. Or companies may connect an AI assistant to files that contain private customer data. If there are few restrictions on accessing the system, then important data could be accessed by users who shouldn’t have access. Attackers may also target AI services to steal information and cause disruption. Having strong AI security can help decrease these risks for companies. It can help improve customer confidence, protect the company’s reputation, and make organisations use AI more responsibly.
The Gap Between Federal Guidance and Real Threats
Federal AI security guidance can provide businesses with important security principles. In reality, the attacker’s tactics constantly change, and the time between the arrival of a new AI tool and a new corporate product can vary greatly. As both AI systems and attack methods get more sophisticated, businesses need to be constantly scanning their security controls for evolving risks. Not all businesses can see how to practically test their AI against generic guidance. Businesses may dismiss the risks from AI because they are almost exclusively concerned with conventional cybersecurity. Guidance can require organisations to interpret how to apply it themselves. These guidelines may be intimidating for some companies with small budgets and/or small information security teams. A company should implement the right controls, based on these high-level guidelines.
Unapproved AI Tools Create Security Risks
An important area of concern is if staff are using AI without permission. Employees may be using public chatbots to allow them to summarise documents, create better emails, and speed up reporting processes. As they do not know how the service is handling their information, there could be issues with privacy and confidentiality if they upload sensitive business data. It can also mean that security teams are unable to monitor how company data is flowing through different systems. If staff are working to federal security standards, they can still access AI tools without authorisation, further raising security concerns. Company-specific lists of available AI tools will have to be provided. Businesses should offer approved tools to use along with easy training. Security teams should monitor the use of AI to ensure that employees can complete their tasks safely.
Weak Access Controls Can Expose Business Data
Sometimes AI systems connect to your company’s documents, databases, emails, and more. This can cause issues if there are poor access controls. If organizations give an AI assistant more permissions than necessary, it can access data that it should not be able to view. If the account has been compromised, it could cause issues. To overcome this, businesses should follow the principle of least privilege. This is where every user and system only has access to what they need to do their job. Check permissions often to remove any access that isn’t needed. Important accounts should also be protected with multi-factor authentication that requires extra measures apart from the password. This helps to prevent unintended access to these accounts and helps make AI systems safer.
Prompt Injection and Manipulated AI Systems
What is prompt injection? Prompt injection is the insertion of commands into a prompt or external content, in an effort to manipulate an AI system. Another attacker could embed malicious code into the web page or email being opened by the AI assistant. That code could deceive the AI assistant into exposing information or performing malicious actions on the system. Guard against these attacks by adhering to our Federal best practices for using AI and conducting regular security audits on AI-enabled devices. Limit the AI’s permissions, treat all inputs as untrusted, and require explicit approval before allowing certain actions to prevent this attack. If you have your system tested regularly, this will assist your security team in identifying any risks to your system.
Data Privacy and AI Model Security
Data protection is an important part of enterprise security, due to the reliance of AI systems on large amounts of data. Most of the time, companies will have customer data, along with internal documentation and reports also present. Improper handling of data leaves data open to being viewed by employees it shouldn’t be or external people attempting to manipulate model behavior or gain access to private data. Know what data is sensitive and ensure your AI tools are only labeling valid data. Read your AI vendors’ privacy policies and data storage practices, and if training your own models, protect your training data, model files, and environment from attack or leak.
Third-Party AI Vendors Create Additional Risks
Many companies will use third-party vendors to offer a variety of services, including AI, cloud, software, and data. However, this can introduce additional security risks. If the vendor makes a change to their service, it may cause some security concerns. Businesses may also rely on other vendors that use different security controls.Review vendors’ security controls and assess their products and services for security, privacy, data protection, data management, and breach response. Ask whether the vendors test, secure, and patch their AI systems to address the vulnerabilities.
Why Traditional Cybersecurity Is Not Enough
Can AI be a back door for attack? Yes, traditional cybersecurity does matter, but AI can also possibly be a back door for attack. While the available security solutions could keep a device, an employee account, or an enterprise system safe, most companies will struggle to identify every AI-based risk. Even if an AI assistant simply pulls data out of a valid employee account, the assistant might still take a prohibited action. When protecting an AI model, companies need to extend that protection to the full AI system, performing system testing, mapping the connections between the model and other systems, testing the AI’s behavior and impact, and spotting anomalous activity. By combining traditional security with AI testing, companies will have a better approach to securing their enterprise.
How Businesses Can Close AI Security Gaps
Companies will have to take inventory of the AI tools, models, vendors, and infrastructure they’ve implemented. Furthermore, they need to have a clear understanding of what data is sensitive and what amount of risk a security attack would pose. Define data-use and sharing policies, access to data, and escalation to a human. Secure data assets and implement a checklist of systems to identify, and perform multiple tests on, such as: adversarial attack; idiosyncratic behaviour or unauthorised use; log analysis; software patches.Organizations can draw on resources and guidance designed for public bodies, such as the NIST AI Risk Management Framework, to govern AI risk and enable innovation. Organizations should tailor their guidance to their existing capabilities, available data, and risk tolerance.
Employee Training and Clear Company Policies
Employees can also help ensure the secure use of AI. Employees who lack awareness of AI security risks and common scams may accidentally reveal sensitive information, believe false claims, and act on misleading information. Make sure you teach all your staff about the dangers associated with AI. Give them a few examples of actual security breaches and then teach them what they could be doing, what information they should never give away, and when they should come to see you. Tell them what might indicate suspicious activity and what might signal a data leak. Proper employee training will allow employees to make smarter decisions in the face of new AI tools and threats.
The Role of Human Oversight
Humans, in the loop, can be a safeguard against taking an important decision without human oversight. Even though AI seems to ‘think’ for itself, it is still possible for it to make errors or act in an unexpected manner. For this reason, it’s worth having some understanding of what sorts of validation you may require. This could include any actions related to sensitive information, financial actions, customer accounts, and critical changes within systems. Although AI assistants can handle security incidents, trained professionals should review all high-impact actions. Consider tracking major decisions and taking steps to prevent unsafe behaviour. People can catch mistakes, and your organization is accountable for how you use AI.
Measuring and Improving AI Security
AI security is constantly changing, so it’s important to regularly review the controls you have in place. Ensure you’re measuring the effectiveness of your security controls by testing access rights, reviewing incident reports, looking at any anomalous activity within the system, and carrying out controlled security testing. Also make sure you are tracking how long problems take to resolve. If there has been an identified weakness, you should be documenting what actions you’re taking to address this weakness and checking that they are working. Monitor your plan to spot security gaps. Modify your plan for new tools, data sources, or large system changes.
Conclusion
Federal AI security guidance may also help enterprises seek out and mitigate dangers. Standard policies cannot in all cases safeguard all organizations from these threats. Protect your business from unauthorized AI use, weak access controls, prompt injection, data breaches, and third-party vendor risks. Use cybersecurity, testing, training, and human oversight as means of defending yourself. Also, identify which federal rules or laws apply to your organization. Strong enterprise security requires businesses to turn relevant guidance into practical security measures. Businesses are able to spot any weaknesses early on and improve their controls over time so that they can reduce any risk associated with using AI. This can protect businesses’ sensitive data and allow them to make use of AI more confidently.
