Why does the list of crawled URLs differ from scan to scan?

There are many reasons as to why you may see inconsistencies in your results. Put simply, this is due to the fact that the state of the application is not exactly the same every time you run a scan. When you commit an update to your website or when we update the Detectify scanner, the URLs we crawl and the order in which we crawl them might be affected.
For efficiency reasons we will scan pages in accordance with their response time so another instance where potential discrepancies could occur is if certain pages time out during one scan but not during the next, which subsequently would result in the crawl tree looking different.
Other factors that can affect which URLs get crawled are time caps placed on crawl time and the various actions we perform on your website during the scan, including link clicks and testing input forms and fields. We also have a smart filter in place to exclude repetitive content. Finally, we don't reuse any data between scans, which effectively means everything is redone from scratch.
If you have a large scope with many assets, the solution we recommend is to break it down by dividing the domain into smaller scan profiles and disallowing certain paths. This way you can ensure that your most important assets are always being scanned. You can find additional information on how to optimize your scan results here: 
Optimize your results