Why are my scans being blocked?
In the overview card, you can see which of your scans get blocked by WAF technology:
If you can see any of your domains listed in this view, it means that the WAF Technology you are using deems Detectify traffic suspicious and blocks our scanner from doing its job on these domains. This may be, for example, due to high request volume from our scanner IPs or unexpected requests - such as those with unusual payloads - that violate your firewall’s security rules.
Please note that the total count of affected domains viewed by Technology is the total number of unique domains being blocked, hence if the same domain is being blocked twice by the same WAF technology (both in Surface Monitoring and Application Scanning), it will be counted as 1.
Should I scan with or without WAF?
Scanning with WAF enabled may be treated as a test for the effectiveness of the WAF itself rather than the underlying application. While it can be useful to verify that potentially suspicious traffic is being blocked the first time the scans run, scans that are being blocked may not produce further interesting results. We recommended allowing the scanner's traffic to reach the application directly to ensure a comprehensive assessment. In that way, you actually test the state of your application standalone and do not risk missing critical vulnerabilities.
How to allow Detectify to perform scans on assets blocked by WAF?
In this article, you can see the IPs from which our traffic originates in order to allow them on your end.
As an example, if you are using Cloudflare and want to allow Detectify's IPs, you need to:
Go to Domain’s Security view
Choose WAF -> create a Custom rule to allow requests matching Detectfy's IPs
Choose the field IP Source Address and paste Detectify’s scanner IP addresses as values
Choose Skip as an action to take and tick all the WAF components listed, including the ones listed under “More components to skip”
Click Deploy
How does it work?
To determine whether or not we are getting blocked, we start by sending the following payload:
example.com/x
This payload is then followed by a handful of other simpler payloads, including but not limited to the following:
XSS - /x?wafcheck=%3Cscript%3Ealert%281%29%3C%2Fscript%3E
XXE - /x?wafcheck=%3C%21ENTITY%20xxe%20SYSTEM%20%22file%3A%2F%2F%2Fetc%2Fshadow%22%3E%5D%3E%3Cpwn%3E%26hack%3B%3C%2Fpwn%3E
SQL Injection - /x?wafcheck=-1'+UNION+SELECT+ALL+*+FROM+information_schema.columns+WHERE+x=SLEEP(5)+OR+'
Path Traversal - /x?wafcheck=../../../../../../../../../../etc/passwd
RCE - /x?wafcheck=;cat+/etc/passwd%60ping+127.0.0.1%26curl+example.com
We compare the result of each of these with the result of the initial payload. If we see any discrepancies in the results, we will mark the domain as blocked by WAF. If we can fingerprint the responses to a specific vendor, we will mark them as such, e.g., CloudFlare. If we cannot identify the differing result, e.g. /x generates a 404 whilst /x?wafcheck=... generates a 403 Forbidden, we mark it as a WAF being present marked as “Unidentified”.