On Monday, October 16, Cisco’s Talos group published a blog on an active threat campaign exploiting CVE-2023-20198, a “previously unknown” zero-day vulnerability in the web UI component of Cisco IOS XE software. IOS XE is an operating system that runs on a wide range of Cisco networking devices, including routers, switches, wireless controllers, access points, and more. Successful exploitation of CVE-2023-20198 allows a remote, unauthenticated attacker to create an account on an affected device and use that account to obtain full administrator privileges, effectively enabling a complete takeover of the system.
There is no patch for CVE-2023-20198 as of October 17, 2023. As Cisco Talos noted in their blog, it is being actively exploited in the wild. There appear to be a significant number of devices running IOS XE on the public internet as of October 17. Estimates of internet-exposed devices running IOS XE vary, but the attack surface area does appear to be relatively large; one estimate puts the exposed device population at 140K+.
In the activity Cisco observed, attackers created (malicious) local user accounts from suspicious IP addresses. Additional activity has included deployment of an implant that allows the attacker to execute arbitrary commands at the system level or IOS level. Cisco has an extensive description of the malicious behavior they’ve observed here.
Affected products
Cisco’s public advisory on CVE-2023-20198 merely says that Cisco IOS XE software is vulnerable if the web UI feature is enabled (the UI is enabled through the ip http server or ip http secure-server commands). Cisco does not offer a list of products that definitively run IOS XE, but their product page for IOS XE lists some, including the Catalyst, ASR, and NCS families.
According to the advisory, customers can determine whether the HTTP Server feature is enabled for a system, by logging into the system and using the show running-config | include ip http server|secure|active command in the CLI to check for the presence of the ip http server command or the ip http secure-server command in the global configuration. The presence of either command or both commands in the system configuration indicates that the web UI feature is enabled (and that the system is therefore vulnerable).
Cisco’s advisory also specifies that if the ip http server command is present and the configuration also contains ip http active-session-modules none, the vulnerability is not exploitable over HTTP. If the ip http secure-server command is present and the configuration also contains ip http secure-active-session-modules none, the vulnerability is not exploitable over HTTPS.
Mitigation guidance
In lieu of a patch, organizations should disable the web UI (HTTP Server) component on internet-facing systems on an emergency basis. To disable the HTTP Server feature, use the no ip http server or no ip http secure-server command in global configuration mode. Per Cisco’s advisory, if both the HTTP server and HTTPS server are in use, both commands are required to disable the HTTP Server feature. Organizations should also avoid exposing the web UI and management services to the internet or to untrusted networks.
Disabling the web UI component of IOS XE systems and limiting internet exposure reduces risk from known attack vectors, but notably does not mitigate risk from implants that may have already been successfully deployed on vulnerable systems. Rapid7 recommends invoking incident response procedures where possible to prioritize hunting for indicators of compromise Cisco has shared, listed below.
Cisco-observed attacker behavior
The Cisco Talos blog on CVE-2023-21098 has a full analysis of the implant they’ve observed being deployed as part of this threat campaign. We strongly recommend reading the analysis in its entirety. The implant is saved under the file path /usr/binos/conf/nginx-conf/cisco_service.conf that contains two variable strings made up of hexadecimal characters. While the implant is not persistent (a device reboot will remove it), the attacker-created local user accounts are.
Cisco observed the threat actor exploiting CVE-2021-1435, which was patched in 2021, to install the implant after gaining access to a device vulnerable to CVE-2023-20198. Talos also notes that they have seen devices fully patched against CVE-2021-1435 getting the implant successfully installed “through an as of yet undetermined mechanism.”
Rapid7-observed attacker behavior
Rapid7 MDR has so far identified a small number of instances where CVE-2023-20198 was exploited in customer environments, including multiple instances of exploitation within the same customer environment on the same day. The indicators of compromise our team has identified with available evidence indicate the use of techniques similar to those described by Cisco Talos.
Rapid7 identified variations of techniques in the course of our investigations. The first malicious activity performed on the system post-exploitation was associated with the admin account. The following is an excerpt from this log file:
%SYS-5-CONFIG_P: Configured programmatically by process SEP_webui_wsma_http from console as admin on vty1
The threat actor created the local account cisco_support using the command username cisco_support privilege 15 algorithm-type sha256 secret * under user context admin. The threat actor then authenticated to the system using this newly created cisco_support account and began running several commands, including the following:
show running-config
show voice register global
show dial-peer voice summary
show platform
show flow monitor
show platform
show platform software iox-service
show iox-service
dir bootflash:
dir flash:
clear logging
no username cisco_support
no username cisco_tac_admin
no username cisco_sys_manager
Upon completion of these commands, the threat actor deleted the account cisco_support. The accounts cisco_tac_admin and cisco_sys_manager were also deleted, but Rapid7 did not observe account creation commands associated with these accounts within available logs.
The threat actor also executed the clear logging command to clear system logging and cover their tracks. Rapid7 identified logging for the second exploitation on October 12, 2023, but could not review logs for the first intrusion because the logs had been cleared.
Evidence indicated that the last action performed by the threat actor relates to a file named aaa:
%WEBUI-6-INSTALL_OPERATION_INFO: User: cisco_support, Install Operation: ADD aaa
When comparing the two intrusions that occurred within the same environment on October 12, there are slight differences in observed techniques. For example, log clearing was only performed within the first exploitation, while the second exploitation included additional directory viewing commands.
Indicators of compromise
The Cisco Talos blog on CVE-2023-20198 directs organizations to look for unexplained or newly created users on devices running IOS XE. One way of identifying whether the implant observed by Talos is present is to run the following command against the device, where the "DEVICEIP” portion is a placeholder for the IP address of the device to check:
curl -k -X POST "https[:]//DEVICEIP/webui/logoutconfirm.html?logon_hash=1"
The command above will execute a request to the device’s Web UI to see if the implant is present. If the request returns a hexadecimal string, the implant is present (note that the web server must have been restarted by the attacker after the implant was deployed for the implant to have become active). Per Cisco’s blog, the above check should use the HTTP scheme if the device is only configured for an insecure web interface.
Additional Cisco IOCs
- 5.149.249[.]74
- 154.53.56[.]231
Usernames:
- cisco_tac_admin
- cisco_support
Cisco Talos also advises performing the following checks to determine whether a device may have been compromised:
Check the system logs for the presence of any of the following log messages where “user” could be cisco_tac_admin, cisco_support or any configured, local user that is unknown to the network administrator:
-
%SYS-5-CONFIG_P: Configured programmatically by process SEP_webui_wsma_http from console as user on line
-
%SEC_LOGIN-5-WEBLOGIN_SUCCESS: Login Success [user: user] [Source: source_IP_address] at 03:42:13 UTC Wed Oct 11 2023
Note: The %SYS-5-CONFIG_P message will be present for each instance that a user has accessed the web UI. The indicator to look for is new or unknown usernames present in the message.
Organizations should also check the system logs for the following message where filename is an unknown filename that does not correlate with an expected file installation action:
%WEBUI-6-INSTALL_OPERATION_INFO: User: username, Install Operation: ADD filename
Rapid7 customers
InsightVM and Nexpose customers can assess their exposure to CVE-2023-20198 with an authenticated vulnerability check that looks for Cisco IOS XE devices with the web UI enabled. The check is available in today’s (October 17) content release.
InsightIDR and Rapid7 MDR customers have existing detection coverage through Rapid7's expansive library of detection rules. The following detection rules are deployed and alerting on activity related to this vulnerability via the IP addresses provided by Cisco:
- Network Flow - CURRENT_EVENTS Related IP Observed
- Suspicious Connection - CURRENT_EVENTS Related IP Observed
Updates
October 17, 2023: Updated with Rapid7-observed attacker behavior and IOCs.
