SEO 101: A V9 Version of Google's Webmaster Guidelines March 2016 The Basics Google’s Webmaster Guidelines have been published for years, but they recently underwent some major changes. They outline basics on what you need to do to get your site indexed, ranked and not tanked. But what if I don’t wanna follow the guidelines? A rebel, eh? Well, some of the guidelines are best practices that can help Google crawl and index your site - not always required, but a good idea. Other guidelines (specifically quality guidelines), if ignored, can earn you a hefty algorithmic or manual action - something that can lower your ranking or get you kicked out of the SERPs entirely. Google breaks the guidelines up into three sections - we’ll do the same. ● Design and content guidelines ● Technical guidelines ● Quality guidelines Before doing anything, take a gut check: ● Would I do this if search engines didn’t exist? ● Am I making choices based on what would help my users? ● Could I explain this strategy to my competitor or my mama and with confidence? ● If my site went away tomorrow, what would the world be missing? Am I offering something unique and valuable? 1 Updated March 2016 Now, Let’s Get Your Site Indexed Make Sure Your Site is Crawlable ● Check www.yourdomain.com/robots.txt ● Make sure you don’t see: User-agent: * Disallow: / ● A robots.txt file that allows bots to crawl everything would look like: User-agent: * Disallow: ● Learn more! Make Sure You’re Allowing Google to Include Pages in their Index ● Check for and remove any meta tags that noindex pages: <meta name="robots" content="noindex" /> ● Do a Fetch as Google in Search Console and check the header for x-robots: noindex directives that need to be removed X-Robots-Tag: noindex ● Learn more! 2 Updated March 2016 Submit Your Site to Google When ready, check out Google’s options for submitting content. For websites, the best way to do this is to create an XML sitemap of all of the live and indexable pages on your website and submit the sitemap in Search Console. ● Build an XML sitemap ● In Search Console, navigate to Crawl > Sitemaps ● Add the URL & submit ● Monitor the dashboard, but keep in mind the data can be delayed Common Questions About Getting a Site Indexed Are XML sitemaps required to be indexed? No. But they can help Google crawl your pages faster. Do you have to re-submit a sitemap to Google for each new page added? No. Ideally your XML sitemap would be kept up to date, but once you’ve submitted it to Google there’s no need to keep resubmitting unless you have major changes. You also do not need to fetch and submit new pages - Google will crawl them naturally. Don’t harass the bots. 3 Updated March 2016 Do XML sitemaps help ranking? No. But they are a great crawling asset and can be a diagnostic tool if you’re having indexation issues. Plus, ‘XML’ just sounds cool. Design and Content Guidelines Design-Specific Guidelines ● Take the time to plan out your site structure. Logical hierarchies make both users and bots alike swoon and can support changes and growth. ● Bots crawl via links so make sure all pages are easily navigable and well- linked. ● Sitemaps are not just for bots! Add user-friendly html sitemap. ● Keeps links to a page reasonable. There’s no ultimate number here - but think like a user. ○ Too few places to go: *sad trombone* ○ Too many links with no clear next place to go: *sad trombone* ● Crawl the site and fix broken links regularly. Reward yourself afterward. You deserve it. ● Consider using structured data to give bots more information about your content and potentially earn rich snippets. This won’t boost your ranking, but it’s icing on the cake (and cake is delicious). 4 Updated March 2016 ● If your site uses dynamic pages (?) or Ajax (#!), know that these can sometimes cause indexing issues. Use sparingly and test often. ● [V9 Exclusive] Design/UI does matter because user-experience matters. Make the site enjoyable to visit. Content-Specific Guidelines Content is the single most important factor when it comes to ranking. You can muck up a lot on a site and still do well if the content is awesome. You cannot do well with a tightly designed/functional site that is ultimately weak on content. ● Ok now, be honest. Step back and try to read your content critically and as a visitor. ○ Is this content useful and easy to read? ○ Is the page rich and include relevant info? ○ Do I trust that the content is accurate? ● Forget using “keywords” for Google. Instead, think about how customers might describe your products or services and use the words they’d use. Write like you’re talking to an actual person. ○ For example, hotels may call their offerings “accommodations”, but a person usually is looking for “hotel rooms”. ● Images are hard for bots to understand. Make sure your important content is crawlable text instead of images. ○ For your images, follow Google’s snappy image best practices. ○ Oh hey, best practices are on video too! If you ever find yourself writing content specifically for Google… back away from the keyword, take a deep breath, and call the Oops I Might Be Spamming hotline (1-888-BAD- WEBZ). We’ll get through this crisis together! REMEMBER: Focus on the user! Focus on the user! 5 Updated March 2016 Technical Guidelines Technical Guidelines Overview ● Make sure your pages are crawlable. ● Keep Google focused on important, unique pages on the site. ● Keep it simple with clean URLs and limited conditional behavior. ● Make sure the site loads fast. Users and bots will love you. <xoxo> Technical Guidelines Details ● Allow Google to crawl all rendering assets (CSS, JS, HTML, images). Google checks to see how your pages render, especially when determining your mobile friendliness. Let ‘em see your lovely site! ○ If you’re concerned about these files getting indexed, you can add a noindex directive to the files. ○ To test what Google can crawl us the Fetch as Google function or the Robots.txt tester in Search Console. ● Don’t use session IDs or tracking tools that change URLs for bots crawling the site. Let them crawl the clean, preferred URLs so they don’t get confused about what pages to index. Keep those URLs clean! ● Ask your developer or check to see if your server supports the If-Modified- Since HTTP header. This is a tag that just tells Google the last time the page was updated - it can help save bandwidth with bots crawling. ● Use the robots.txt file, but use it wisely. ○ Don’t block rendering assets or stuff that was indexed that you don’t want indexed. ○ Do block directories/files that the bot should never crawl, like search pages or auto-generated pages. ○ Learn more about how to tell Google to crawl the site. ● If you have ads on the site, do your best to make sure they don’t affect ranking. Block the ads from being crawled or use a nofollow tag on the ad link. 6 Updated March 2016 ● Make sure your site looks awesome, regardless of browser or device. ● Fast sites win. ○ Keep an eye on load times and performance regularly. Use tools like Google’s PageSpeed Insights, Pingdom and WebPageTest. ○ Remember that many users have slow internet connections and mobile users often rely on cellular data to load pages. ○ Consistently fast sites make users happy - and Google is in the business of sending users to websites that make them happy. Quality Guidelines This is where offenses go from potentially clumsy mistakes to “oopsies” that can get you in hot water. If you’re breaking these guidelines it’s not if you’ll get caught, it’s when. Pay attention! Potential consequences include: ● Manual Actions: these show up in Search Console & require a human review ● Algorithm Issues: you’ll get no confirmation and you clean up the issue and wait Key takeaways ● Don’t be deceptive. ● Avoid tricks and don’t try to manipulate your ranking. There are no shortcuts! ● Focus on users - not search engines. ● You’ll see some examples below, but the list is not complete. Focus on intention. 7 Updated March 2016 Examples of Big Fat No-No’s ● Publishing auto-generated content. ○ This includes machine-translated content or spun content. ● Participating in link schemes. ○ Links are often spammed, so be careful. ○ Safe links are high quality/relevant links that you didn’t build. ○ “Nofollow” all links you had a hand in creating. “Nofollow” doesn’t mean bad. ○ Get rid of (remove or disavow) low-quality, followed irrelevant links. ● Creating pages with little or no original content. ● Cloaking - serving different content to Google/users. ● Sneaky redirects - sending users to different pages/sites unexpectedly. 8 Updated March 2016 ● Hiding text or links so Google can see them, but users can’t easily see the content. ● Doorway pages - read more about what these are on our blog. ● Scraped content - come on, you’re better than taking content from other sites! ● Participate in affiliate programs without doing enough to make your content awesome and add significant value. ● Trying to rank for irrelevant keywords that don’t match your content. ● Doing downright nasty malicious stuff (phishing, malware, unwanted software, etc.). ● Abusing rich snippets markup like adding fake reviews. ● Sending automated queries to Google to scrape rankings or otherwise harass the big G. Earn good karma - do good things ● Keep your site safe and secure, prevent hacks or get ‘em cleaned up quickly.