How To Crawl As Googlebot Smartphone With Screaming Frog

Transcription: 

Hi, I’m Chris Long with Go Fish Digital and welcome to another episode of “60 Second SEO”

Today we’re going to talk about how you can change your Screaming Frog settings to crawl as Googlebot Smartphone.

As most of you are probably aware, Google uses mobile-first indexing. It’s going to take a look at the mobile version of your site first. That means when we perform a crawl of our site, to get the best representation of what Google sees, we need to be crawling as if we’re using Googlebot Smartphone.

So here I’m on Mattel site and let’s say I wanted to perform a crawl. What I can do is I can go to Screaming Frog and I can go up to “Configuration” and then click on “User Agent”. If you haven’t changed this setting, it might be preset to Screaming Frog SEO Spider. I can click this drop-down here and I can just click Googlebot Smartphone and then click “OK”, I can start my crawl.

And now Screaming Frog is going to crawl as if it was the Googlebot Smartphone user agent, giving me a much more accurate representation of how Google might actually be crawling my site, especially if we know that Google is using mobile-first indexing for our site.