Blog
Thought Leadership
Generative AI: How Darktrace AI protects 8,400 customers from security and privacy risks







生成AIや大規模言語モデル(LLM)は今年、一般の人々の意識の主流になりました。OpenAIのChatGPTやGoogle Bardを、ウェブ検索の支援から職場の効率化を促進するためのAI機能に至るまで、あらゆることに利用しています。
Darktrace では、AI が現代において最も変革的な技術的機会のひとつとなる可能性を、長い間理解してきました。ケンブリッジにある当社のサイバーAIリサーチセンターは、10年以上にわたってAIツールの研究開発を行ってきました。Darktrace DETECTやRESPONDのようなツールは、さまざまなAI技術を駆使して、世界中の8,400社の顧客をサイバー破壊から保護しています。
AIのパイオニアとして、また世界を変える可能性を理解する者として、私たちは2023年、ついにAIの魔神が瓶から出てしまったと認識しています。AIツールは急速に私たちの日常生活の一部となりつつあるのです。
アクティブな顧客の74%において、従業員が職場で生成AIツールを使用している [1]
生成AIツールは生産性を向上させ、人間の創造性を補強する力を持つ一方で、企業はイノベーションのペースに遅れないように迅速に対応する必要があります。これらのツールは、使い方を誤ったり、ビジネス特有のニーズに合った適切なポリシーが適用されなかったりすると、潜在的なプライバシーリスクやセキュリティリスクを誘発し、CISOにとっての悩みの種となります。
生成AIによるプライバシーとセキュリティのリスク
英国の国家サイバーセキュリティセンター(NCSC)のような政府機関はすでに、生成AIツールやその他のLLMを職場で使用する際のリスク管理の必要性に関するガイダンスを発表しています。米国では、サイバーセキュリティインフラストラクチャー庁(CISA)も、生成AIのセキュリティへの影響について懸念を表明しています。
その理由のひとつは、LLMがプロンプトから学習し、入力された情報を保存してデータセットの訓練に使用できるからです。そのデータがシステムにあれば、誰かが正しいプロンプトを入力すれば、LLMはその問い合わせに対して貴社のデータを利用できる可能性があるのです。
また、入力した情報に知的財産やノウハウ、財務報告書、社内の機密文書、販売数などの機密ファイルやデータが含まれている場合、適切なガードレールが設置されていなければ、第三者のAIモデルの一部となって他者が利用できる可能性があり、プライバシー、知的財産、セキュリティ上のリスクが発生します。
Darktrace が生成AIの利用を管理するのに役立つ方法
生成AIツールの利用が拡大していることを受け、Darktrace のお客様が知的財産の損失やデータ漏えいのリスクに関する懸念に対応するための新しいリスクおよびコンプライアンスモデルを発表しました。
私たちは、これらの生成AIツールが、人々や企業の効率的な作業を支援する機能を備えた非常に強力なものであることに興奮していますが、他のテクノロジーと同様に、正しく管理または監視されない場合、不注意に悪用される可能性があるというリスクも理解しています。そのため、Darktrace DETECTとRESPONDの新しいリスクとコンプライアンスモデルは、AutoGPT、ChatGPT、Stable Diffusion、Claudeなどの生成AIとLLMツールのアクティビティと接続を監視し、必要に応じて対応するためのガードレールを設置することを容易にします。
生成AIツールに関連する独自の明確なポリシーとニーズは各々の企業が擁しており、我々もまた、お客様が監視するツールのリストを独自に追加することが容易になりました。
Darktraceの自己学習型AIは、会社のポリシーやベストプラクティスから逸脱する可能性のある生成AIの活動を検知することができます。当社は、お客様のデータにAIを導入し、すべてのユーザー、資産、およびデバイスの日々の動作を学習し、お客様のビジネス固有の「生活パターン」の理解を構築します。 そのため、ビジネスへの脅威を示すわずかな異常もリアルタイムに検知し、自律的に対応することができ、数秒で脅威を封じ込めることができるのです。
2023年5月、Darktrace の自己学習型AIが、ある顧客の生成AIツールに対する1GBを超えるデータのアップロードを検知し、阻止しました。 [2]
これらのガードレールがあることで、Darktrace のお客様は、潜在的なセキュリティ、IP、およびプライバシーのリスクから保護されたまま、生成AIとLLMを使用する機会をフル活用することができます。
安全で責任あるAIの活用
Darktraceでは、最近の生成AI と LLM の進歩は、サイバーセキュリティを変革する AI 技術の武器になる重要な追加要素だと考えています。LLMや生成AIを含むAI技術は、長年にわたり当社の全製品で活用されており、インシデントのリアルタイム分析を行うCyber AI Analystを含め、Darktrace のお客様がAIの力を駆使してサイバー脅威から保護され続けることを支援しています。
しかし、当社はまた、様々なAI技術の責任ある開発と展開を信じています。だからこそ当社はお客様が安全かつ責任を持ってAIを使用するために必要なツールを提供しています。
Darktraceの自己学習型AIは、過去10年間にわたり、すでに8,400社以上の企業がサイバー脅威や混乱に対処し、自らを保護するのに役立っています。これらの新しいツールを使用することで、CISOは潜在的なセキュリティリスクを心配することなく、生成AIによって生産性を確実に向上させることができます。当社のAIは、リアルタイムかつ常時ビジネスを学習しており、セキュリティの成果向上へのインパクトは非常に大きいものです。
自己学習型AIは、DarktraceのサイバーAIループ(サイバー攻撃を予防、検知、遮断、修復するための継続的なフィードバックループを作成するために自律的に連携して動作する、動的に関連する機能の相互接続された包括的なセット)に反映されます。データ、人、企業がサイバー脅威から完全自律的に確実に保護されるようにする当社のミッションです。
.png)
参考文献
2023年6月2日に取得した、Call Homeを有効にしたアクティブなお客様の環境から、Darktrace がある時点で生成AIの活動を検知したデータに基づきます。
2023年6月2日に取得した、Call Homeを有効にしたアクティブなお客様の自社環境から、Darktrace がある時点で生成AIの活動を検知したデータに基づきます。
Like this and want more?
More in this series
Blog
Inside the SOC
PurpleFox in a Henhouse: How Darktrace Hunted Down a Persistent and Dynamic Rootkit



Versatile Malware: PurpleFox
As organizations and security teams across the world move to bolster their digital defenses against cyber threats, threats actors, in turn, are forced to adopt more sophisticated tactics, techniques and procedures (TTPs) to circumvent them. Rather than being static and predictable, malware strains are becoming increasingly versatile and therefore elusive to traditional security tools.
One such example is PurpleFox. First observed in 2018, PurpleFox is a combined fileless rootkit and backdoor trojan known to target Windows machines. PurpleFox is known for consistently adapting its functionalities over time, utilizing different infection vectors including known vulnerabilities (CVEs), fake Telegram installers, and phishing. It is also leveraged by other campaigns to deliver ransomware tools, spyware, and cryptocurrency mining malware. It is also widely known for using Microsoft Software Installer (MSI) files masquerading as other file types.
The Evolution of PurpleFox
The Original Strain
First reported in March 2018, PurpleFox was identified to be a trojan that drops itself onto Windows machines using an MSI installation package that alters registry values to replace a legitimate Windows system file [1]. The initial stage of infection relied on the third-party toolkit RIG Exploit Kit (EK). RIG EK is hosted on compromised or malicious websites and is dropped onto the unsuspecting system when they visit browse that site. The built-in Windows installer (MSIEXEC) is leveraged to run the installation package retrieved from the website. This, in turn, drops two files into the Windows directory – namely a malicious dynamic-link library (DLL) that acts as a loader, and the payload of the malware. After infection, PurpleFox is often used to retrieve and deploy other types of malware.
Subsequent Variants
Since its initial discovery, PurpleFox has also been observed leveraging PowerShell to enable fileless infection and additional privilege escalation vulnerabilities to increase the likelihood of successful infection [2]. The PowerShell script had also been reported to be masquerading as a .jpg image file. PowerSploit modules are utilized to gain elevated privileges if the current user lacks administrator privileges. Once obtained, the script proceeds to retrieve and execute a malicious MSI package, also masquerading as an image file. As of 2020, PurpleFox no longer relied on the RIG EK for its delivery phase, instead spreading via the exploitation of the SMB protocol [3]. The malware would leverage the compromised systems as hosts for the PurpleFox payloads to facilitate its spread to other systems. This mode of infection can occur without any user action, akin to a worm.
The current iteration of PurpleFox reportedly uses brute-forcing of vulnerable services, such as SMB, to facilitate its spread over the network and escalate privileges. By scanning internet-facing Windows computers, PurpleFox exploits weak passwords for Windows user accounts through SMB, including administrative credentials to facilitate further privilege escalation.
Darktrace detection of PurpleFox
In July 2023, Darktrace observed an example of a PurpleFox infection on the network of a customer in the healthcare sector. This observation was a slightly different method of downloading the PurpleFox payload. An affected device was observed initiating a series of service control requests using DCE-RPC, instructing the device to make connections to a host of servers to download a malicious .PNG file, later confirmed to be the PurpleFox rootkit. The device was then observed carrying out worm-like activity to other external internet-facing servers, as well as scanning related subnets.
Darktrace DETECT™ was able to successfully identify and track this compromise across the cyber kill chain and ensure the customer was able to take swift remedial action to prevent the attack from escalating further.
While the customer in question did have Darktrace RESPOND™, it was configured in human confirmation mode, meaning any mitigative actions had to be manually applied by the customer’s security team. If RESPOND had been enabled in autonomous response mode at the time of the attack, it would have been able to take swift action against the compromise to contain it at the earliest instance.
攻撃の概要

Initial Scanning over SMB
On July 14, 2023, Darktrace detected the affected device scanning other internal devices on the customer’s network via port 445. The numerous connections were consistent with the aforementioned worm-like activity that has been reported from PurpleFox behavior as it appears to be targeting SMB services looking for open or vulnerable channels to exploit.
This initial scanning activity was detected by Darktrace DETECT, specifically through the model breach ‘Device / Suspicious SMB Scanning Activity’. Darktrace’s Cyber AI Analyst™ then launched an autonomous investigation into these internal connections and tied them into one larger-scale network reconnaissance incident, rather than a series of isolated connections.

As Darktrace RESPOND was configured in human confirmation mode, it was unable to autonomously block these internal connections. However, it did suggest blocking connections on port 445, which could have been manually applied by the customer’s security team.

特権昇格
The device successfully logged in via NTLM with the credential, ‘administrator’. Darktrace recognized that the endpoint was external to the customer’s environment, indicating that the affected device was now being used to propagate the malware to other networks. Considering the lack of observed brute-force activity up to this point, the credentials for ‘administrator’ had likely been compromised prior to Darktrace’s deployment on the network, or outside of Darktrace’s purview via a phishing attack.
Exploitation
Darktrace then detected a series of service control requests over DCE-RPC using the credential ‘admin’ to make SVCCTL Create Service W Requests. A script was then observed where the controlled device is instructed to launch mshta.exe, a Windows-native binary designed to execute Microsoft HTML Application (HTA) files. This enables the execution of arbitrary script code, VBScript in this case.


There are a few MSIEXEC flags to note:
- /i : installs or configures a product
- /Q : sets the user interface level. In this case, it is set to ‘No UI’, which is used for “quiet” execution, so no user interaction is required
Evidently, this was an attempt to evade detection by endpoint users as it is surreptitiously installed onto the system. This corresponds to the download of the rootkit that has previously been associated with PurpleFox. At this stage, the infected device continues to be leveraged as an attack device and scans SMB services over external endpoints. The device also appeared to attempt brute-forcing over NTLM using the same ‘administrator’ credential to these endpoints. This activity was identified by Darktrace DETECT which, if enabled in autonomous response mode would have instantly blocked similar outbound connections, thus preventing the spread of PurpleFox.

Installation
On August 9, Darktrace observed the device making initial attempts to download a malicious .PNG file. This was a notable change in tactics from previously reported PurpleFox campaigns which had been observed utilizing .MOE files for their payloads [3]. The .MOE payloads are binary files that are more easily detected and blocked by traditional signatured-based security measures as they are not associated with known software. The ubiquity of .PNG files, especially on the web, make identifying and blacklisting the files significantly more difficult.
The first connection was made with the URI ‘/test.png’. It was noted that the HTTP method here was HEAD, a method similar to GET requests except the server must not return a message-body in the response.
The metainformation contained in the HTTP headers in response to a HEAD request should be identical to the information sent in response to a GET request. This method is often used to test hypertext links for validity and recent modification. This is likely a way of checking if the server hosting the payload is still active. Avoiding connections that could possibly be detected by antivirus solutions can help keep this activity under-the-radar.


The server responds with a status code of 200 before the download begins. The HEAD request could be part of the attacker’s verification that the server is still running, and that the payload is available for download. The ‘/test.png’ HEAD request was sent twice, likely for double confirmation to begin the file transfer.

Subsequent analysis using a Packet Capture (PCAP) tool revealed that this connection used the Windows Installer user agent that has previously been associated with PurpleFox. The device then began to download a payload that was masquerading as a Microsoft Word document. The device was thus able to download the payload twice, from two separate endpoints.
By masquerading as a Microsoft Word file, the threat actor was likely attempting to evade the detection of the endpoint user and traditional security tools by passing off as an innocuous text document. Likewise, using a Windows Installer user agent would enable threat actors to bypass antivirus measures and disguise the malicious installation as legitimate download activity.
Darktrace DETECT identified that these were masqueraded file downloads by correctly identifying the mismatch between the file extension and the true file type. Subsequently, AI Analyst was able to correctly identify the file type and deduced that this download was indicative of the device having been compromised.
In this case, the device attempted to download the payload from several different endpoints, many of which had low antivirus detection rates or open-source intelligence (OSINT) flags, highlighting the need to move beyond traditional signature-base detections.



If Darktrace RESPOND was enabled in autonomous response mode at the time of the attack it would have acted by blocking connections to these suspicious endpoints, thus preventing the download of malicious files. However, as RESPOND was in human confirmation mode, RESPOND actions required manual application by the customer’s security team which unfortunately did not happen, as such the device was able to download the payloads.
結論
The PurpleFox malware is a particularly dynamic strain known to continually evolve over time, utilizing a blend of old and new approaches to achieve its goals which is likely to muddy expectations on its behavior. By frequently employing new methods of attack, malicious actors are able to bypass traditional security tools that rely on signature-based detections and static lists of indictors of compromise (IoCs), necessitating a more sophisticated approach to threat detection.
Darktrace DETECT’s Self-Learning AI enables it to confront adaptable and elusive threats like PurpleFox. By learning and understanding customer networks, it is able to discern normal network behavior and patterns of life, distinguishing expected activity from potential deviations. This anomaly-based approach to threat detection allows Darktrace to detect cyber threats as soon as they emerge.
By combining DETECT with the autonomous response capabilities of RESPOND, Darktrace customers are able to effectively safeguard their digital environments and ensure that emerging threats can be identified and shut down at the earliest stage of the kill chain, regardless of the tactics employed by would-be attackers.
Credit to Piramol Krishnan, Cyber Analyst, Qing Hong Kwa, Senior Cyber Analyst & Deputy Team Lead, Singapore
付録
Darktraceによるモデル検知
- Device / Increased External Connectivity
- Device / Large Number of Connections to New Endpoints
- Device / SMB Session Brute Force (Admin)
- Compliance / External Windows Communications
- Anomalous Connection / New or Uncommon Service Control
- Compromise / Unusual SVCCTL Activity
- Compromise / Rare Domain Pointing to Internal IP
- Anomalous File / Masqueraded File Transfer
RESPOND Models
- Antigena / Network / Significant Anomaly / Antigena Breaches Over Time Block
- Antigena / Network / External Threat / Antigena Suspicious Activity Block
- Antigena / Network / Significant Anomaly / Antigena Significant Anomaly from Client Block
- Antigena / Network / Significant Anomaly / Antigena Enhanced Monitoring from Client Block
- Antigena / Network / External Threat / Antigena Suspicious File Block
- Antigena / Network / External Threat / Antigena File then New Outbound Block
IoC一覧
IoC - Type - Description
/C558B828.Png - URI - URI for Purple Fox Rootkit [4]
5b1de649f2bc4eb08f1d83f7ea052de5b8fe141f - File Hash - SHA1 hash of C558B828.Png file (Malware payload)
190.4.210[.]242 - IP - Purple Fox C2 Servers
218.4.170[.]236 - IP - IP for download of .PNG file (Malware payload)
180.169.1[.]220 - IP - IP for download of .PNG file (Malware payload)
103.94.108[.]114:10837 - IP - IP from Service Control MSIEXEC script to download PNG file (Malware payload)
221.199.171[.]174:16543 - IP - IP from Service Control MSIEXEC script to download PNG file (Malware payload)
61.222.155[.]49:14098 - IP - IP from Service Control MSIEXEC script to download PNG file (Malware payload)
178.128.103[.]246:17880 - IP - IP from Service Control MSIEXEC script to download PNG file (Malware payload)
222.134.99[.]132:12539 - IP - IP from Service Control MSIEXEC script to download PNG file (Malware payload)
164.90.152[.]252:18075 - IP - IP from Service Control MSIEXEC script to download PNG file (Malware payload)
198.199.80[.]121:11490 - IP - IP from Service Control MSIEXEC script to download PNG file (Malware payload)
MITRE ATT&CK マッピング
Tactic - Technique
Reconnaissance - Active Scanning T1595, Active Scanning: Scanning IP Blocks T1595.001, Active Scanning: Vulnerability Scanning T1595.002
Resource Development - Obtain Capabilities: Malware T1588.001
Initial Access, Defense Evasion, Persistence, Privilege Escalation - Valid Accounts: Default Accounts T1078.001
Initial Access - Drive-by Compromise T1189
Defense Evasion - Masquerading T1036
Credential Access - Brute Force T1110
Discovery - Network Service Discovery T1046
Command and Control - Proxy: External Proxy T1090.002
参考文献
- https://blog.360totalsecurity.com/en/purple-fox-trojan-burst-out-globally-and-infected-more-than-30000-users/
- https://www.trendmicro.com/en_us/research/19/i/purple-fox-fileless-malware-with-rookit-component-delivered-by-rig-exploit-kit-now-abuses-powershell.html
- https://www.akamai.com/blog/security/purple-fox-rootkit-now-propagates-as-a-worm
- https://www.foregenix.com/blog/an-overview-on-purple-fox
- https://www.trendmicro.com/en_sg/research/21/j/purplefox-adds-new-backdoor-that-uses-websockets.html
Blog
OT
$70 Million in Cyber Security Funding for Electric Cooperatives & Utilities



What is the Bipartisan Infrastructure Deal?
The Bipartisan Infrastructure Law passed by congress in 2021 aimed to upgrade power and infrastructure to deliver clean, reliable energy across the US to achieve zero-emissions. To date, the largest investment in clean energy, the deal will fund new programs to support the development and deployment of clean energy technology.
Why is it relevant to electric municipalities?
Section 40124 of the Bipartisan Infrastructure Law allocates $250 million over a 5-year period to create the Rural and Municipal Utility Cybersecurity (RMUC) Program to help electric cooperative, municipal, and small investor-owned utilities protect against, detect, respond to, and recover from cybersecurity threats.1 This act illuminates the value behind a full life-cycle approach to cyber security. Thus, finding a cyber security solution that can provide all aspects of security in one integrated platform would enhance the overall security posture and ease many of the challenges that arise with adopting multiple point solutions.
On November 16, 2023 the Office of Cybersecurity, Energy Security, and Emergency Response (CESER) released the Advanced Cybersecurity Technology (ACT) for electric utilities offering a $70 million funding opportunity that aims to enhance the cybersecurity posture of electric cooperative, municipal, and small investor-owned utilities.
Funding Details
10 projects will be funded with application submissions due November 29, 2023, 5:00 pm ET with $200,000 each in cash prizes in the following areas:
- Direct support for eligible utilities to make investments in cybersecurity technologies, tools, training, and improvements in utility processes and procedures;
- Funding to strengthen the peer-to-peer and not-for-profit cybersecurity technical assistance ecosystem currently serving eligible electric utilities; and
- Increasing access to cybersecurity technical assistance and training for eligible utilities with limited cybersecurity resources. 2
To submit for this award visit: https://www.herox.com/ACT1Prize
How can electric municipalities utilize the funding?
While the adoption of hybrid working patterns increase cloud and SaaS usage, the number of industrial IoT devices also continues to rise. The result is decrease in visibility for security teams and new entry points for attackers. Particularly for energy and utility organizations.
Electric cooperatives seeking to enhance their cyber security posture can aim to invest in cyber security tools that provide the following:
Compliance support: Consider finding an OT security solution that maps out how its solutions and features help your organization comply with relevant compliance mandates such as NIST, ISA, FERC, TSA, HIPAA, CIS Controls, and more.
Anomaly based detection: Siloed security solutions also fail to detect attacks that span
the entire organization. Anomaly-based detection enhances an organization’s cyber security posture by proactively defending against potential attacks and maintaining a comprehensive view of their attack surface.
Integration capabilities: Implementation of several point solutions that complete individual tasks runs the risk of increasing workloads for operators and creates additional challenges with compliance, budgeting, and technical support. Look for cyber security tools that integrate with your existing technologies.
Passive and active asset tracking: Active Identification offers accurate enumeration, real time updates, vulnerability assessment, asset validation while Passive Identification eliminates the risk of operational disruption, minimizes risk, does not generate additional network traffic. It would be ideal to find a security solution that can do both.
Can secure both IT and OT in unison: Given that most OT cyber-attacks actually start in IT networks before pivoting into OT, a mature security posture for critical infrastructure would include a single solution for both IT and OT. Separate solutions for IT and OT present challenges when defending network boundaries and detecting incidents when an attacker pivots from IT to OT. These independent solutions also significantly increase operator workload and materially diminish risk mitigation efforts.
Darktrace/OT for Electric Cooperatives and Utilities
For smaller teams with just one or two dedicated employees, Darktrace’s Cyber AI Analyst and Investigation features allow end users to spend less time in the platform as it compiles critical incidents into comprehensive actionable event reports. AI Analyst brings all the information into a centralized view with incident reporting in natural language summaries and can be generated for compliance reports specific to regulatory requirements.
For larger teams, Darktrace alerts can be forwarded to 3rd party platforms such as a SIEM, where security team decision making is augmented. Additionally, executive reports and autonomous response reduce the alert fatigue generally associated with legacy tools. Most importantly, Darktrace’s unique understanding of normal allows security teams to detect zero-days and signatureless attacks regardless of the size of the organization and how alerts are consumed.
Key Benefits of Darktrace/OT
- Anomaly-based detection and real-time response
- Secures IT, OT, and IoT in unison
- Active and Passive Asset Identification
- Automated security reporting
- Attack surface management and vulnerability assessment
- Covers all levels of the Purdue Model
.png)
参考文献
