For your first test, we always recommend you to add an apex (e.g. example.com without subdomains) to get an overview of your entire domain. Depending on the size of your domain and the number of subdomains, the test may take a bit longer than our average scan time of 3 hours. Even though we do our best to crawl all of your subdomains, we can’t guarantee that this will be the case. We don’t cap on number of URLs to visit, but we do limit the crawl time to 9 hours. If we exceed these 9 hours, our scanner will automatically move forward from the crawling phase and go on to the next phase. More about this under “Why you need to break your domain down”.
After your first test
When your first test is done, you can look at your report and start dealing with the highest critical findings. During this process, you may also discover that some of your subdomains have more vulnerabilities than others. Does your blog.example.com contain many high severity findings? Then it might be a good idea to start breaking your domain down and add separate profiles for the subdomains that are most critical to you.
Breaking your domain down into smaller scan profiles
There are three Information findings (in green) in your report that are useful when you start breaking your domain down into smaller scan profiles: Crawled URLs, Discovered Hosts and Fingerprinted Software. Below you can find more information on how these findings can help you with your account setup.
In the “Crawled URLs” finding, you will find the URLs we’ve crawled for each scan. After clicking the link under “Found at”, you will find a downloadable CSV file at the bottom that you can go through to make sure we have visited all the relevant parts of your site.
Another useful finding to get an overview of what data we’ve found is “Discovered Hosts”. In this finding, you’ll find subdomains that have been detected during your scan.
You might find some unexpected findings, such as systems that you didn’t know existed. From this list, you can start selecting the most crucial applications and add them as a separate scan profile for a more in-depth scan.
The “Fingerprinted Software” finding shows you which technologies we’ve fingerprinted your system for. This finding can be used to make sure that not too many technologies are covered under the same scan profile.
Why you need to break down your domain into smaller scan profiles
We try to make it as easy as possible for our users to scan their web applications. With that said, there are some things that you should keep in mind when setting up your account.
Time caps for scans
As mentioned earlier, we do not cap the number of URLs per scan. Instead, we cap time for each of the scan phases. The reason we do this is to make sure to catch any anomalies for our scans. For example, if a scan goes beyond 9 hours in the crawling phase, we assume that something has gone wrong (since that’s the case for most sites) and will push the scanner into the next phase. However, the reason we cap the 9 hour time limit may also be that the scope of your scan is too big. If you are trying to scan, for example, google.com with all of its subdomain, we won’t be able to visit all of the pages for this domain. This is simply because we can’t gather that much data and analyse it in a consistent way.
Different technologies for your subdomains
Let’s say that you have 10 subdomains for your site (*.example.com). In turn, the subdomains are built on different technologies and CMSes: Your blog (blog.example.com) is running WordPress. Your landing site (www.example.com) is custom built with XYZ. Your e-commerce (shop.example.com) is built upon Magento. When you start a test for this domain, we will fingerprint all these different technologies. In turn, this will tell our scanner to activate the modules for exploits affecting WordPress, Magento and XYZ. This will lead to an increase in scan time due to more tests being activated, which in turn can lead to caps on the different phases in the scan process which may affect the accuracy of the result. Sounds a bit inefficient, right?
How to think about your account setup
Now we have covered how our scanner and crawler work, let’s get down to some concrete suggestions for best possible account setup.
The smaller the scope for each scan profile, the more accurate and consistent the result.
Scanning system2.example.com will give you a more in-depth test than scanning *.example.com.
Same technologies/frameworks for each scan profile.
This will make it easier for our scanner to only run relevant tests for each of your scan profiles.
Example account setup
Below is an example of how a scan setup could look for an average SaaS company:
General test for your entire domain to get an overview (*.example.com)
This will give you a general overview of your application
One profile for your landing site (www.example.com)
One profile for your blog (blog.example.com)
One profile for your application (app.example.com) authenticated