Web Scraping

Web Scraping

Web Scraping is a technique used for extracting and structuring large amounts of data from websites. The purpose of Web Scraping is to automatically collect information from the internet and store it in a structured way. 

We use Web Scraping for example in lead generation or competitive pricing analysis. In the Data Analytics Crash Course, we teach how to scrape a website. After following the course, you can actually perform a simple Web Scraping project yourself.

The Process

Prior to scraping a website, it is very important that the Terms and Conditions do not prohibit the use of data from the website. Then the first step in the process is to locate the data of interest and load or fetch it from a website.

After the data is fetched, it can be extracted by inspecting the underlying HTML code of the source. The extracted data can then be structured and stored in a database. Afterwards, it can be passed along to and used by another process such as lead generation.