How to Get Data from Google Maps in Three Simple Steps Using the Web Scraping API

How to Get Data from Google Maps in Three Simple Steps Using the Web Scraping API
Posted on
Sep 19, 2022

Scraping in the modern world is an essential part of any business. It allows you not to waste employees' time manually collecting the necessary information, whether it is data on competitors or the dynamics of price changes on the trading floor.

Often scrapers are used to collect leads. They allow you to manage current databases or update old ones quickly.

Read more: How to Use Web Scraping to Generate High-Quality Leads and Grow Your Business

Ways to get Data

To use such an advanced tool as scraping, you must decide what you are ready to pay for: time, money, or programming knowledge.

Start Google Maps Scraping in Minutes

Don't waste time manually clicking through Google Maps to find what you're looking for. Let our API do it for you in a fraction of the time!

  • 1,000 Free API credits
  • No Credit Card Required
  • 30-Day Trial

Here we will consider precisely the option of using the Web Scraping API. And you can even try it for free - at least the first 1000 requests.

Ways to Get Data
Ways to Get Data

If you think programming is too complex, there is an alternative. For example, you can use Zapier for visual customization and integration.

Read more about using Zapier for scraping: The Complete Guide to Scrape Google Maps for Lead Generation with No-Code

Scraping Google Maps Using web Scraping API

To use the web scraping API, sign up for Scrape-It.Cloud. To do this, click "Try for Free" and fill in the login information. Now you can create a simple request to scrape some data from Google Maps. To do this, go to the "Google Maps Builder" tab and enter the keyword for crawling in the keyword field:

Google Maps Scraper Builder
Google Maps Scraper Builder

Run the request from the site by clicking "Run Script.” It will return the first 20 rows of results from Google Maps.

Algorithm of Scraper Creating

We will write the entire script in the Python programming language. To use it, you need to install an interpreter.

Read more about scraping in Python: Web Scraping with Python: from Fundamentals to Practice

Now, to more accurately imagine the final result, let's describe the algorithm of what we will consider next.

  1. Try an example of a simple google maps scraper.
  2. Get a list of keywords from an excel document.
  3. Get the start value and several positions from the command line.
  4. Save data to a new document.
  5. Creating a Python file with a simple google maps scraper is the easiest part. To do this, go to Google Maps Builder and select the code type - Python. Then copy the resulting code into a file with the *.py extension.

However, then the HTTP.client library will be used. But for further convenience, we should use the requests library:

import requests
import json

url = "https://api.scrape-it.cloud/scrape/google/locals"

payload = json.dumps({
  "country": "US",
  "domain": "com",
  "keyword": "Pizza",
  "ll": "@40.7455096,-74.0083012,14z"
})
headers = {
  'x-api-key': 'YOUR-API-KEY',
  'Content-Type': 'application/json'
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

It is an example of the simplest scraper. So we can go to the next step.

It's Never Been Easier to Harvest Data from Google Maps

If you can copy/paste, you can scrape 1000+ businesses in less than 2 minutes. It's that easy!

  • 1,000 rows included
  • No Credit Card Required
  • 30-Day Trial

Get a list of Keywords from a Document

Create a file that will store the keywords. Let's name it "keywords.csv.” Therefore, every time the program goes to a new line and sees a new keyword, it should re-execute the query.

Add a line that will open the keywords.csv file for reading and specify a line-by-line cycle:

with open("keywords.csv", "r+") as f:
  for keyword in f:

Place here the request body, where instead of the given keyword, there will be a variable:

 temp = """{
      "country": "US",
      "domain": "com",
      "keyword": """+"\""+keyword.strip()+"\""+""",
      "ll": "@40.7455096,-74.0083012,14z"
    }
    """
    payload = json.loads(json.dumps(temp))

To ensure everything is fine and working, you can run the file on the command line.

Set Rows Amount

Let's ask on the command line for the scraping start position and the number of pages from that position. We will understand 20 lines by one page since the API returns 20 rows, starting from the specified.

To do this, add a request to enter the starting position and enter it into the variable start:

start = int(input("Start position: "))

And also, add a request for the number of pages:

pages = int(input("Number of pages: "))

And with this in mind, let’s change the request body and add a loop for scraping the specified number of pages starting from the position start:

    for i in range(start, 20*pages, 20):
      temp = """{
        "country": "US",
        "domain": "com",
        "keyword": """+"\""+keyword.strip()+"\""+""",
        "ll": "@40.7455096,-74.0083012,14z",
        "start": """+str(i)+"""
      }
      """

It is important to remember that we can use only string variables in the request body, but we need numeric variables for the loop.

Save results in an Excel Document

For the data to be conveniently processed, let's save it to an excel file. Every time the program starts, it shouldn’t add the data in an existing file to the end. In this case, we will delete the old data. We add a file overwrite at the beginning of the script. We will also name the columns in which we will save the data:

with open("result.csv", "w") as f:
  f.write("position; title; phone; address; website; rating; reviews; type\n")

We will place data saving in the try..except block so that if the request returns an empty response, the script doesn’t stop its work completely:

      try:
        for item in data["scrapingResult"]["locals"]:
          with open("result.csv", "a") as f:
            f.write(str(item["position"])+"; "+str(item["title"])+"; "+str(item["phone"])+"; "+str(item["address"])+"; "+str(item["website"])+"; "+str(item["rating"])+"; "+str(item["reviews"])+"; "+str(item["type"])+"\n")
      except Exception as e:
        print("There are no locals")

Let's try to run the resulting script and check the generated result.csv file:

Result of Scraping Google Maps
Result of Scraping Google Maps

Find Leads Fast

Now you can get thousands of prospects with the click of a button. Our web scrapers can work hard for you and get you to the right leads.

  • 1,000 rows included
  • No Credit Card Required
  • 30-Day Trial

Full Scraper Code

import requests
import json

url = "https://api.scrape-it.cloud/scrape/google/locals"

headers = {
  'x-api-key': 'YOUR-API-KEY',
  'Content-Type': 'application/json'
}

with open("result.csv", "w") as f:
  f.write("position; title; phone; address; website; rating; reviews; type\n")

start = int(input("Start position: "))
pages = int(input("Number of pages: "))
with open("keywords.csv", "r+") as f:
  for keyword in f:
    for i in range(start, 20*pages, 20):
      temp = """{
        "country": "US",
        "domain": "com",
        "keyword": """+"\""+keyword.strip()+"\""+""",
        "ll": "@40.7455096,-74.0083012,14z",
        "start": """+str(i)+"""
      }
      """
      payload = json.loads(json.dumps(temp))
      response = requests.request("POST", url, headers=headers, data=payload)
      data = json.loads(response.text)
      try:
        for item in data["scrapingResult"]["locals"]:
          with open("result.csv", "a") as f:
            f.write(str(item["position"])+"; "+str(item["title"])+"; "+str(item["phone"])+"; "+str(item["address"])+"; "+str(item["website"])+"; "+str(item["rating"])+"; "+str(item["reviews"])+"; "+str(item["type"])+"\n")
      except Exception as e:
        print("There are no locals")

Conclusion and Takeaways

The web scraping API allows the developer to not worry about proxies and captcha solving and to focus on creating the main product. And using the google maps scraping API allows you to save time when collecting leads or suppliers.

Using Python, the task of creating a full-fledged scraper is quite simple. Similarly to the option discussed in the article, you can also change coordinates or the country for which you will perform scraping.

Tired of getting blocked while scraping the web?

Try out Web Scraping API with proxy rotation, CAPTCHA bypass, and Javascript rendering.

  • 1,000 Free API Credits
  • No Credit Card Required
  • 30-Day Trial
Try now for free

Get structured data in the format you need!

We offer customized web scraping solutions that can provide any data you need, on time and with no hassle!

  • Regular, custom data delivery
  • Pay after you receive sample dataset
  • A range of output formats
Get a Quote
Valentina Skakun
Valentina Skakun
Valentina Skakun’s profile

I'm a technical writer who believes that data parsing can help in getting and analyzing data. I'll tell about what parsing is and how to use it.

Request a Quote

Tell us more about you and your project information.