Apr 18, 2018 10:35:16 PM
Search engine optimization, better known as SEO, is now considered a staple of digital marketing. Used to drive web presence, expand visibility, and improve rankings, SEO plays an integral role in virtually every aspect of modern web-based strategies.
However, this wasn't always the case. In the initial days of the web, SEO was a distant thought years into the future. The earliest algorithms did incorporate a limited application of current search tactics, like employment of keywords, but the hows and whys were still largely unknown. In time, as marketing pros and web developers alike learned more about the mechanics behind search engines, this changed, slowly building to the current climate marketers know and embrace. Here's how SEO evolved – and what you can learn from the history of digital marketing.
The Birth of the Search Engine
The web as we know it today kicked off in 1991 with the launch of the first ever public website, but others weren't far behind. As sites grew in number, the need for search functionality did, too; with no real way to explore and evaluate websites, the functionality of the web was vastly compromised. However, the tools available did little to offer a dynamic, precise, and valuable search experience. In fact, a structured approach to web search didn't truly take off until the mid-1990s, providing a first look at the landscape ahead.
In 1993, six Stanford students came together to found Architext, which later became the search engine Excite. Yahoo was born in 1994 out of the minds of two Stanford students, with Google's first incarnation, BackRub, coming two years later in 1996 as the brainchild of Larry Page and Sergey Brin. At this time, the internet was still small and easily manageable; only 18,000,000 American homes were online by 1995, but only 3% of online users had ever connected to the World Wide Web. This number rose to 70,000,000 by 1997, the year in which Ask Jeeves debuted. A human-powered search engine, this concept used actual human searchers to answer questions asked in basic language.
The first search engines were less engine and more directory, providing a list of useful and applicable websites that users could explore for particular purposes. In many cases, webmasters submitted their own sites to the major early directories, like Yahoo: the first clear attempts to increase visibility.
Searching based on the relevancy of keywords was a brand new idea with Excite, in which sites were organized based on content and backend optimization. BackRub then took this idea a step further, including popularity and link relevancy to refine the data available to searchers. These developments ultimately laid the foundation for the framework that remains in place today.
The Earliest Days of SEO
As search engines continued to gain traction, webmasters and businesses using the Internet began dreaming up ways to gain prominence in early search engines. Thus, the anything-goes style that dominated the mid to late 1990s was born: everything that worked, from keyword stuffing to spammy backlinks intended solely to boost rankings, was fair game.
However, many search industry experts believe that the original guiding principle behind modern search was born in this period of rapid growth. The paper "The Anatomy of a Large-Scale Hypertextual Web Search Engine" published in 1998 at Stanford by Page and Brin of Google, jumpstarted the trends that would evolve over the next two decades.
In their own words, the masterminds behind search engines stated "Human maintained lists cover popular topics effectively but are subjective, expensive to build and maintain, slow to improve, and cannot cover all esoteric topics. Automated search engines that rely on keyword matching usually return too many low quality matches. [...] We have built a large-scale search engine which addresses many of the problems of existing systems. It makes especially heavy use of the additional structure present in hypertext to provide much higher quality search results." With this, the ideals and goals behind everything Google has worked to accomplish are broken down clearly, paving the way for the next generation of search.
This paper also introduced the concept of PageRank – Google's first and best-known algorithm. Per the proposal, " "PageRank", an objective measure of [...] citation importance that corresponds well with people's subjective idea of importance. Because of this correspondence, PageRank is an excellent way to prioritize the results of web keyword searches." While still in its earliest iteration at the time, PageRank is ostensibly what set Google apart from its compatriots.
Google's Rise to Prominence
The search engine game in the 1990s was competitive, with countless contenders in the arena. However, as the first decade of the World Wide Web gave way to a more streamlined, refined Internet in the 2000s, many of the smaller players fell by the wayside. AltaVista became a part of Yahoo by way of Overture Services, a keyword-based advertising service that rivaled Google. Excite was purchased by @Home Network in 1999, which then filed for bankruptcy in 2001 and sold to Ask Jeeves in 2004. Ask Jeeves, which moved away from the human search element in the mid-2000s, transformed into Ask.com, a Q&A platform. One by one, Google's competitors continued to fail, leaving only one clear leader in the search engine game.
While Google existed in the 1990s – the company picked up its now-ubiquitous name in 1997 during the writing of the aforementioned paper at Stanford – the search engine didn't rise to prominence until the early 2000s. In fact, in Google's soar to meteoric heights almost didn't happen: the company tried to sell its product to Excite for under $1,000,000 in 1999.
The real party behind Google's presence as a household name, however, was Yahoo. In 2000, the two search giants came together as Yahoo chose Google to power all of its search results with the bold tagline "Powered by Google." At the time, Yahoo was a bigger name, but the presence of Google's brand name on Yahoo's results made a significant influence on the associations between Google and quality search. In 2002, the CEO of Yahoo even offered to buy Google for $3 billion, but the startup wanted more. After determining that $5 billion was too big an ask, the deal was pushed aside, paving the way for Google to take the reins.
As Google's presence grew, so did its influence in search results. In the early 2000s, even in advance of its 2004 IPO, Google began to tout the concept of white hat SEO, or search engine principles that didn't use shady or scammy workarounds to boost rankings. These tactics were intended to help webmasters rank organically and naturally without using subversive tactics to land high up in the SERPs. There was just one problem: Google wasn't taking these kinds of mechanics into account, leaving the door open for less-than-honest individuals to continue dominating results pages.
Progressive Algorithm Changes
While Google had been tweaking search engine principles for years, most changes were small and lengthy, leading to adjustments that took months to implement and didn't notably affect how sites ranked. In November of 2003, this all changed. The introduction of Florida, Google's first major named algorithm, altered the face of search for the foreseeable future. In this adjustment, Google made dedicated strides to reduce the impact of keyword stuffing, multiple sites under the same brand, and even hidden text, leading to a cleaner, more accurate list of results.
The outcome was shocking. With a change to how affiliate links were ranked, many retailers saw their traffic – and, subsequently, revenue – slide, leading to temporary outages and permanent closures. This led to wide-scale outrage, but only for a short time. Once the dust settled, those left still standing realized that one major change had to happen: improved quality. Slowly, little by little, businesses and webmasters began to adapt to the idea of higher quality content and design, investing more into the user experience instead of tactics designed to simply rank.
This first major change ultimately laid the foundation for what would prove to be a busy decade in search.
The Shifting Landscape of 2005 and Beyond
2003's Florida proved to carry serious firepower, but it was nothing in comparison to what 2005 and the years to follow had in store.
In early 2005, Google combined with Yahoo and MSN to create the Nofollow Attribute, a function intended to reduce the number of spam links available online. Later that year came the inception of personalized search, a search engine feature that used search history to customize web results. Then, in the fall, Google Analytics was rolled out. A dashboard to track website rankings and traffic, Google gave websites the power to analyze performance and potential in real time.
In this time, Matt Cutts and the webspam team also drew attention; Google was the first search engine to appoint a team intended solely to fight spam results and facilitate white hat practices. This lead to an actual patent in the field of fighting spam, ushering in a new wave of search results with a focus on legitimacy and the white-hat methodology our SEO agency adheres to.
While the landscape of the Internet continued to grow and change over the next few years, particularly with the launch of Facebook, Twitter, and the spread of what we now know as social media, the search engine game quietly moved forward until 2009. This year welcomed the threat of a first major competitor for Google in Bing, as well as the next big algorithm change. Bing, of course, did little to quash Google's growth – today, Bing's Alexa rank is 42 versus Google's 1 – but at the time, a new search engine meant new changes to stay competitive.
2009 also saw a shift to the content Google's search engine presented. As of this time, social media posts and breaking news were included at the top of search pages, leading to an increased focus on relevancy over basic keyword use. This was essentially a continuation of 2007 and 2008's Universal Search, an approach that blended images, news, and more into the front page of Google's results.
2010 to the Present: Modern Search Engine Results
The start of the '10s kicked off with more features that added to the user experience but did little to affect SEO, like Google Instant and the further integration of PageRank into social media. It wasn't until 2011's Panda that search made huge strides. One of Google's largest ever algorithm shifts, Panda targeted content farms, or pages inundated with low quality content and a high ad to content ratio, plunging these kinds of sites to the bottom of the SERPs. Penalizing sites deemed not good enough wasn't a new concept – Google already had a reputation for down-ranking sites that attempted to work around their approved tactics – but something on this scale was the first of its kind.
Quick on Panda's heels came Penguin, an update in 2012 that penalized sites that utilize link schemes or keyword stuffing to manipulate rankings. Google also returned to its anti-ad roots in this time; pages with significant advertising "above the fold," so to speak, were punished. In 2014, the Payday Loan algorithm was implemented, addressing search queries, like payday loans, that were likely to lead to spam results.
Other algorithm changes rolled out in the next few years, targeting things like local search, https encryption, and social media results, but 2015's Mobilegeddon update created lasting changes still felt today. As of April 2015, sites that did not utilize mobile-friendly or responsive designs were penalized in mobile search results, leading to a huge shift in optimization for mobile specifically. This update came at a good time: the scales were tipping in mobile's favor, with mobile overtaking desktop for the first time in categories like digital media consumption.
Around the same time, Google announced RankBrain, its machine-learning artificial intelligence system that sorts, optimizes, and refines search results. Per Google, RankBrain's capabilities make its conclusions one of the three most important ranking features, the other two being content and links, (although I would say User Experience is #4). RankBrain is used globally, working behind the scenes to improve search accuracy.
Over the next few years up until now, many updates occurred. In November 2017, metadata underwent its first major change, with increased description character amount restraints from 160 to over 250 (depending where Google wants to cut if off). A Google spokesperson said, "We recently made a change to provide more descriptive and useful snippets, to help people better understand how pages are relevant to their searches. This resulted in snippets becoming slightly longer, on average." A recent and notable update that was announced on March 26, 2018 was Google's mobile-first indexing. Per Google, this update "means that we'll use the mobile version of the page for indexing and ranking, to better help our – primarily mobile – users find what they're looking for."
Despite the huge strides search has made since the first website in 1991, Google's widespread search engine domination shows no signs of stopping. These kinds of updates and algorithms roll out all the time, some announced to much fanfare with others sliding below the radar, but as history shows, the field of SEO is ever-evolving. Regardless of audience awareness, one fact remains: Google is always seeking to improve, and marketers are always striving to follow.
This article first appeared on LinkedIn.
Posts by Topic
- Internet Marketing
- Marketing Agency
- Search Engine Optimization (SEO)
- Content Creation
- Inbound Marketing
- Link Building & Acquisition
- Inbound Marketing Agency
- Website Analytics
- Lead Management
- Paid Search Marketing (PPC)
- Usability & Conversion Optimization
- Social Media Marketing
- Content Marketing
- Lead Generation
- Automated Marketing
- Increase Conversions
- Marketing Data
- Sales Generation
- Social Media
- Web Design & Development
- Keyword Data
- Landing Pages
- Persona Development
- marketing research
- Agile Marketing
- Growth Hacking
- Voice Search Optimization
- Accelerated Mobile Pages
- Email Marketing
- Google Penalty
- Link Removal
- Local SEO
- Manual Action
- Marketing Automation
- Mobile Marketing
- Public Relations
- Voice Search Optimization (VSO)
- Windows Cortana
- instagram ads
- online marketing