More information on RST cookies

    This Knowledge Base article applies to:
      Amaranten Firewall,all versions

Quick info

"RST cookies" is a weak form of SYN flood protection that works by eliciting a bogus TCP ACK in response
to a TCP SYN, and expecting a TCP RST back. In theory, this should prevent people from lying about their
source IP adress, since the attacker would have to see the bogus ACK packet in order to be able to reply to
it. To Amaranten's knowledge, the Axent Raptor/Symantec Enterprise Firewall is the only commercial firewall currently encouraging the use of the RST cookie SYN flood protection scheme. The result on your end

Amaranten Firewall refuses to send RSTs in response to these bogus ACKs. Rather, it generates a log entry and drops the packet. The result is that users behind an Amaranten Firewall cannot connect to machines behind a Axent Raptor/Symantec Enterprise Firewall with SYN flood protection enabled. Why Amaranten
does not want to support RST cookies

There are many others that have discarded RST cookies as a viable approach to SYN flood protection. For
example, the Linux kernel previously implemented an optional SYN flood protection via RST cookies. The
open-source community saw it fit to remove the RST cookie scheme in recent kernels, for several reasons:

  • It doesn't work very well with a lot of firewalls
  • It still consumes resources on the server end, which is exactly what a SYN flood is designed to do.
  • It's a fairly weak form of SYN flood protection; there are other approaches that are by far more
    preferable.
  • It does not prevent you from actually SYN flooding the server / firewall. It only (supposedly)
    prevents you from doing so from a spoofed (fake) IP address.
  • If you are spoofing an existing host, that host will happily respond to the bogus ACK with a RST, so
    it is always entirely possible to SYN flood someone spoofing the IP address of another (existing)
    host.
Sidenote on the Raptor/SEF RST cookies

The Raptor/SEF RST cookies are even weaker than the designers of the RST cookie scheme intended for
them to be. The "secret" in the bogus ACKs generated by Raptor/SEF firewalls is simply the sequence
number of the original SYN (that the attacker sends), plus 1 000 000.

Hence, it is entirely possible to send a SYN from a spoofed source IP to a Raptor/SEF firewall, and
immediately afterwards send a RST with the sequence number of the original SYN, plus 1 000 000.

Conclusion

Given the ineffectiveness of the RST cookie scheme, Amaranten would much rather see that Symantec /
Axent redesign their SYN flood protection than having to support it in Amaranten Firewall. Supporting it
generates unnecessary complexity, and quite possibly opens up avenues of exploitation that we are quite
sure that our users could do without.

Workarounds

There are no workarounds on your end. The only workaround is to turn off the RST cookies in the firewall / server causing the problems. If you are having problems accessing a site employing a RST cookie SYN flood protection scheme, we recommend that you contact the site admin and ask him/her to turn the "protection"
off; it isn't doing them any good. Also, be reassured, you are far from the only one having problems
accessing that site, since this protection scheme causes problems with several other firewalling solutions.

How do I know if RST cookies are causing my problems?

If you are seeing DROP log entries with a rule of "LogStateViolations", concering a connection in
"SYN_SENT" state, and the dropped packet is TCP with the ACK flag set, you can be fairly certain that you're witnessing the result of the RST cookie protection scheme.