CERT^® Advisory CA-2002-27 Apache/mod_ssl Worm

A complete revision history can be found at the end of this file.

Systems Affected

• Linux systems running Apache with mod_ssl accessing SSLv2-enabled OpenSSL 0.9.6d or earlier on Intel x86 architectures

Overview

The CERT/CC has received reports of self-propagating malicious code which exploits a vulnerability (VU#102795) in OpenSSL. This malicious code has been referred to as Apache/mod_ssl worm, linux.slapper.worm and bugtraq.c worm. Reports received by the CERT/CC indicate that the Apache/mod_ssl worm has already infected thousands of systems. There are currently at least three known variants of this worm in circulation.

I. Description

The Apache/mod_ssl worm is self-propagating malicious code that exploits the OpenSSL vulnerability described in VU#102795. This vulnerability was the among the topics discussed in CA-2002-23 Multiple Vulnerabilities In OpenSSL. While this OpenSSL server vulnerability exists on a wide variety of platforms, the Apache/mod_ssl worm appears to work only on Linux systems running Apache with the OpenSSL module (mod_ssl) on Intel architectures.

The Apache/mod_ssl worm scans for potentially vulnerable systems on 80/tcp using an invalid HTTP GET request. When a potentially vulnerable Apache system is detected, the worm attempts to connect to the SSL service via 443/tcp in order to deliver the exploit code. If successful, a copy of the malicious source code is then placed on the victim server, where the attacking system tries to compile and run it. Once infected, the victim server begins scanning for additional hosts to continue the worm's propagation.

Additionally, the Apache/mod_ssl worm can act as an attack platform for distributed denial-of-service (DDoS) attacks against other sites by building a network of infected hosts. During the infection process, the attacking host instructs the newly-infected victim to initiate traffic on 2002/udp (newer variants have been reported using 1978/udp or 4156/udp) back to the attacker. Once this communications channel has been established, the infected system becomes part of the Apache/mod_ssl worm's DDoS network. Infected hosts can then share information on other infected systems as well as attack instructions. Thus, this UDP traffic can be used by a remote attacker as a communications channel between infected systems to coordinate attacks on other sites.

Reports to the CERT/CC indicate that the high volume of 1978/udp, 2002/udp, or 4156/udp traffic generated between hosts infected with the Apache/mod_ssl worm may itself lead to performance issues (including possible denial-of-service conditions) on networks with infected hosts. Furthermore, since repairing an infected host does not remove its IP address from the Apache/mod_ssl worm's Peer-to-Peer network, sites that have had hosts infected with the Apache/mod_ssl worm and subsequently patched them may continue to see significant levels of 1978/udp, 2002/udp, or 4156/udp traffic directed at those formerly infected systems.

Identifying infected hosts

During the infection process of the "A" variant of the Apache/mod_ssl worm, an encoded version of the worm's source code is placed in /tmp/.uubugtraq. This file is then decoded into /tmp/.bugtraq.c, compiled with gcc, and the executable binary is subsequently stored at /tmp/.bugtraq. More recent variants follow a similar (but not identical) pattern of infection, and leave behind different files. Because all three variants exploit the same system vulnerabilities, it is possible that systems infected with one variant may also become infected with the others. Therefore, presence of any of the following files on Linux systems running Apache with OpenSSL is indicative of compromise.

Variant "A"

/tmp/.uubugtraq

/tmp/.bugtraq.c

/tmp/.bugtraq

Variant "B"

/tmp/.unlock.c

/tmp/.update.c

Variant "C"

/tmp/.cinik

/tmp/.cinik.c

/tmp/.cinik.go

/tmp/.cinik.goecho

/tmp/.cinik.uu

The probing phase of the attack may show up in web server log as shown in the example below. It is important to note that there may be other causes of such log entries, so the appearance of entries matching (or similar to) these in a web server log should not be construed as evidence of compromise. Rather, their presence is indicative that further investigation may be warranted.

Example: Initial probe to identify web server software version

GET / HTTP/1.1

Note: Based on initial reports received by the CERT/CC, earlier versions of this Advisory mentioned other SSL error messages that might be logged on potentially vulnerable hosts. On further analysis, we have concluded that these log messages were unrelated to the the Apache/mod_ssl worm. An explanation of one possible cause of those other mod_ssl error messages was provided by Inktomi and appears in Appendix A below.

Hosts found to be listening for or transmitting data on 1978/udp (variant "C"), 2002/udp (variant "A"), or 4156/udp (variant "B") are also indicative of compromise by the Apache/mod_ssl worm.

In addition to communicating with other infected hosts via 4156/udp, the "B" variant of the Apache/mod_ssl worm creates a backdoor listening on 1052/tcp.

Detecting Apache/mod_ssl worm activity on the network

Infected systems are readily identifiable on a network by the following traffic characteristics:

• Probing -- Scanning on 80/tcp
• Propagation -- Connections to 443/tcp
• DDoS -- Transmitting or receiving datagrams with both source and destination ports 1978/udp, 2002/udp, or 4156/udp. This traffic is used as a communications channel between infected systems to coordinate attacks on other sites.
• Backdoor ("B" variant only) -- Listening on 1052/tcp.

Additionally, infected hosts that are actively participating in DDoS attacks against other systems may generate unusually high volumes of attack traffic using various protocols (e.g., TCP, UDP, ICMP)

II. Impact

Compromise by the Apache/mod_ssl worm indicates that a remote attacker can execute arbitrary code as the apache user on the victim system. It may be possible for an attacker to subsequently leverage a local privilege escalation exploit in order to gain root access to the victim system. The high volume of 2002/udp traffic (1978/udp or 4156/udp in newer variants) generated between hosts infected with the Apache/mod_ssl worm may itself lead to performance issues on networks with infected or formerly infected hosts. Furthermore, the DDoS capabilities included in the Apache/mod_ssl worm allow victim systems to be used as platforms to attack other systems.

III. Solution

Apply a patch

Administrators of all systems running OpenSSL are encouraged to review CA-2002-23 and VU#102795 for detailed vendor recommendations regarding patches. Additional vendor information is available in Appendix A below.

Note that while the vulnerability exploited by the Apache/mod_ssl worm was fixed beginning with OpenSSL version 0.9.6e, as of this writing the latest version of OpenSSL is 0.9.6g. Administrators may wish to upgrade to that version instead.

The following is reproduced in part from CA-2002-23

Upgrade to version 0.9.6e of OpenSSL

Upgrade to version 0.9.6e of OpenSSL to resolve the issues addressed in this advisory. As noted in the OpenSSL advisory, separate patches are available:

Combined patches for OpenSSL 0.9.6d:
http://www.openssl.org/news/patch_20020730_0_9_6d.txt

After either applying the patches above or upgrading to 0.9.6e, recompile all applications using OpenSSL to support SSL or TLS services, and restart said services or systems. This will eliminate all known vulnerable code.

Sites running OpenSSL pre-release version 0.9.7-beta2 may wish to upgrade to 0.9.7-beta3, which corrects these vulnerabilities. Separate patches are available as well:

Combined patches for OpenSSL 0.9.7 beta 2:
http://www.openssl.org/news/patch_20020730_0_9_7.txt

Disable SSLv2

SSLCipherSuite ALL:!ADH:RC4+RSA:+HIGH:+SSLv2

which allows SSLv2 can be changed to

SSLCipherSuite ALL:!ADH:RC4+RSA:+HIGH:!SSLv2

However, systems may still be susceptible to the other vulnerabilities described in CA-2002-23.

Ingress/Egress filtering

The following steps are only effective in limiting the damage that systems already infected with the Apache/mod_ssl worm can do. They provide no protection whatsoever against the initial infection of systems. As a result, these steps are only recommended in addition to the preventative steps outlined above, not in lieu thereof.

Ingress filtering manages the flow of traffic as it enters a network under your administrative control. Servers are typically the only machines that need to accept inbound traffic from the public Internet. In the network usage policy of many sites, external hosts are only permitted to initiate inbound traffic to machines that provide public services on specific ports. Thus, ingress filtering should be performed at the border to prohibit externally initiated inbound traffic to non-authorized services.

Egress filtering manages the flow of traffic as it leaves a network under your administrative control. There is typically limited need for machines providing public services to initiate outbound connections to the Internet.

In the case of the Apache/mod_ssl worm, employing ingress and egress filtering can help prevent systems on your network from participating in the worm's DDoS network and attacking systems elsewhere. Blocking UDP datagrams with both source and destination ports 1978, 2002 and 4156 from entering or leaving your network reduces the risk of external infected systems communicating with infected hosts inside your network.

Recovering from a system compromise

If you believe a system under your administrative control has been compromised, please follow the steps outlined in

Steps for Recovering from a UNIX or NT System Compromise

Reporting

The CERT/CC is interested in receiving reports of this activity. If machines under your administrative control are compromised, please send mail to cert@cert.org with the following text included in the subject line: "[CERT#23820]".

Appendix A. - Vendor Information

This appendix contains information provided by vendors for this advisory. As vendors report new information to the CERT/CC, we will update this section and note the changes in our revision history. If a particular vendor is not listed below, we have not received their comments.

Apple Computer, Inc.

The vulnerability described in this report has been addressed by

• Security Update 2002-08-23 for Mac OS X 10.2 (Jaguar), and by
• Security Update 2002-08-02 for Mac OS X 10.1.5.

Covalent Technologies

Covalent Technologies has been informed by RSA Security that the BSAFE libraries used in Covalent's SSL implementations are potentially vulnerable to the SSL V2 negotiation issue detailed in VU 102795 and the related CA-2002-23 and CA-2002-27 advisories. All Covalent products using SSL are affected. Covalent has product updates and additional information available at: http://www.covalent.net/products/rotate.php?page=110

Debian Project

The Debian project has released DSA 136 a while ago which fixes this vulnerability. Here's the link:

http://www.debian.org/security/2002/dsa-136

Inktomi

As noted in the advisory, server log messages such as

GET /mod_ssl:error:HTTP-request HTTP/1.0

do not necessarily indicate access by a compromised system. Any HTTP request to a port expecting to serve HTTPS requests will generate this log message. The Inktomi web crawler follows URL links published on public web pages and is sometimes incorrectly directed to https servers. The crawler does not use Apache nor mod_ssl (nor any kind of SSL), so it is not subject to the compromise described in this advisory. But crawler requests can match two of the listed symptoms of the Apache/mod_ssl worm:

• Probing -- Scanning on 80/tcp
• Propagation -- Connections to 443/tcp

The crawler does not use port 2002 nor UDP. Port 80 access or HTTPS handshake errors from an Inktomi web crawler do not represent an attack on your web server.

Inktomi crawler systems have hostnames of the form

j[1-9][0-9][0-9][0-9].inktomisearch.com
si[1-9][0-9][0-9][0-9].inktomisearch.com

The IP addresses of Inktomi crawler hosts will reverse-DNS resolve to a name of this form.

Red Hat Inc.

Versions of OpenSSL that are not vulnerable to this issue have been available from Red Hat since 29th July 2002. Customers who have kept their systems up to date by applying fixes or using the Red Hat Network are not impacted by this worm. Updates for all affected Red Hat products are available; details and links to the individual advisories can be found at

http://www.redhat.com/support/alerts/linux_slapper_worm.html

Feedback can be directed to the author: Allen Householder

CERT/CC Contact Information

Email: cert@cert.org
Phone: +1 412-268-7090 (24-hour hotline)
Fax: +1 412-268-6989
Postal address:

CERT Coordination Center
Software Engineering Institute
Carnegie Mellon University
Pittsburgh PA 15213-3890
U.S.A.

CERT/CC personnel answer the hotline 08:00-17:00 EST(GMT-5) / EDT(GMT-4) Monday through Friday; they are on call for emergencies during other hours, on U.S. holidays, and on weekends.

Using encryption

We strongly urge you to encrypt sensitive information sent by email. Our public PGP key is available from

http://www.cert.org/CERT_PGP.key

If you prefer to use DES, please call the CERT hotline for more information.

Getting security information

CERT publications and other security information are available from our web site

http://www.cert.org/

* "CERT" and "CERT Coordination Center" are registered in the U.S. Patent and Trademark Office.

NO WARRANTY
Any material furnished by Carnegie Mellon University and the Software Engineering Institute is furnished on an "as is" basis. Carnegie Mellon University makes no warranties of any kind, either expressed or implied as to any matter including, but not limited to, warranty of fitness for a particular purpose or merchantability, exclusivity or results obtained from use of the material. Carnegie Mellon University does not make any warranty of any kind with respect to freedom from patent, trademark, or copyright infringement.

Copyright 2002 Carnegie Mellon University.

Revision History

September 14, 2002:  Initial release
September 16, 2002:  Updated details on 2002/udp traffic in Description, Impact sections
September 16, 2002:  Clarified example web server log entries in Description section
September 16, 2002:  Added Ingress/Egress filtering section to Solutions section
September 17, 2002:  Clarified example log entries seen by probed servers
September 17, 2002:  Added Apple vendor statement made 9/16/2002 1:48:09 PM (UTC-0700)
September 17, 2002:  Removed references to "GET /mod_ssl:error:HTTP-request" appearing in server logs
September 17, 2002:  Added Inktomi vendor statement made 9/16/2002 09:16:09 AM (UTC-0700)
September 17, 2002:  Added Covalent Technologies vendor statement made 9/16/2002 09:59:00 AM (UTC-0700)
September 19, 2002:  Added Red Hat Inc. vendor statement made 2002-09-18 09:44:14 (UTC+0100)
September 23, 2002:  Added mention of 1978/udp and 4156/udp in use by newer variants of the worm
September 23, 2002:  Added additional details on the "B" and "C" variants of the worm.
October 11, 2002:    Added Debian vendor statement made 10/08/2002 (DSA-136 posted 07/30/2002)