How to Spider Web Applications using Burpsuite

Hello friends! Today we are doing web penetration testing using burp suite spider which very rapidly crawls entire web application and dumps the formation of targeted website.

Burp Spider is a tool for automatically crawling web applications. While it is generally preferable to map applications manually, you can use Burp Spider to partially automate this process for very large applications, or when you are short of time.

Source: https://portswigger.net/burp/help/spider.html

Let’s begin!!

The first attacker needs to configure the browser and burp proxy to work properly, www.tetphp.vulnweb.com will my targeted web site for enumeration.

The form is given below screenshot you can see currently there is no targeted website inside site map of burp suite. To add your targeted web site inside it you need to fetch the http request sent by the browser to the web application server, using intercept option of the proxy tab.

Click on the Proxy tab and turn on intercept in order to catch http request.

Here you can observe that I had fetched the http request of www.tetphp.vulnweb.com; now send to spider with help of action tab.

Confirm your action by making click on YES; Burp will alter the existing target scope to include the preferred item, and all sub-items contained by the site map tree.

Now choose spider tab for a further step, here you will find two subcategories control tab and option.

Burp Spider – Control Tab

This tab is used to start and stop Burp Spider, monitor its progress, and define the spidering scope.

 Spider Status

Use these settings to monitor and control Burp Spider:

• Spider is paused/running– This toggle button is used to start and stop the Spider. While the Spider is stopped it will not make any requests of its own, although it will continue to process responses generated via Burp Proxy (if passive spidering is enabled), and any newly-discovered items that are within the spidering scope will be queued to be requested if the Spider is restarted.
• Clear queues– If you want to reprioritize your work, you can completely clear the currently queued items, so that other item can be added to the queue. Note that the cleared items may be re-queued if they remain in-scope and the Spider’s parser encounters new links to the items.

 Spider Scope

This panel lets you define exactly what is in the scope for the Spider to request.

The best way to handle spidering scope is normally using the suite-wide target scope, and by default, the Spider will use that scope.

Burp Spider Options

This tab contains options for the basic crawler settings, passive spidering, form submission, application login, the Spider engine, and HTTP request headers.

You can monitor the status of the Spider when running, via the Control tab. Any newly discovered content will be added to the Target site map.

When spidering a selected branch of the site map, Burp will carry out the following actions (depending on your settings):

• Request any unrequested URLs already present within the branch.
• Submit any discovered forms whose action URLs lay within the branch.
• Re-request any items in the branch that previously returned 304 status codes, to retrieve fresh (uncached) copies of the application’s responses.
• Parse all content retrieved to identify new URLs and forms.
• Recursively repeat these steps as new content is discovered.
• Continue spidering all in-scope areas until no new content is discovered.

Hence you can see the targeted website has been added inside the site map as a new scope for web crawling. Choose spider this host option by making right click on selected URL which automatically starts web crawling.

When you click on preferred target site map further content which has been discovering by the spider will get added inside it as shown in the given image below.

Form screenshot you can see its dump all items of web site even by throwing request and response of the host.