Every wonder how search engines work? Here’s you chance to find out. In the video below, I’ll explain the basic concepts behind how search engines work and what some of the differences are between the various search engines.
Full Transcript:
Hi, I’m Kurt Steinbrueck with OurChurch.com. In this video, I’m going to describe how search engines work, just give a brief overview of that.
Search engines really have four basic components.
- The first is that they gather together a list of web pages;
- then they try to figure out what those web pages are about;
- the third thing is they find a way to allow users to submit questions or search for keywords;
- and finally, they produce a list of results that’s based off of what they think are the best results for that search.
This is basically how all search engines work, but of course, not all search engines work the same, so it’s how do each one of these search engines do each of these four things is what separates each of these search engines and what makes them unique.
Creating a List of Websites:
Let’s look at the first thing: creating a list of websites. Now, how do they do that? Well, if you’re looking at a smaller site, like a niche search engine, maybe a Christian search engine, a lot of those search engines rely on people coming to them and submitting their site to the search engine. That’s how they get their list. It’s just a manual thing; you go in there and you enter in the information.
If you’re looking at the bigger search engines, your Googles and Bings and some of the others, they still accept the submissions, but they find most of their web pages by going out and looking for them themselves, with little software that they call bots. Basically this software goes and views a web page, it finds all the links on the web page, and then follows those links to find new web pages. It just keeps doing this, over and over and over, and Google and Bing are literally crawling around and finding millions and millions of web pages every single day. So that’s how they create their lists.
What is the Web Page About?
Then the next thing: how do they figure out what a web page is about? Well, with your smaller search engines, your niches, Christian search engines, they usually rely on the user, the person submitting the site, to tell them what the site is about. So you’ll enter in some keyword phrases, you may put in the description, the title of your site, things like that, and they rely on that. The problem with this, of course, is that you could lie. The person submitting the information could lie about it, and so a lot of times this is not the most reliable way to determine what a page is about.
Your larger search engines that have a lot more resources, they actually try to figure it out themselves, and they have very complex software programs that look at the site and they try to figure out what each page is about. How do they do that? Well, a lot of times it’s by looking at the content, the text on the site, and looking at what words are being used and what words aren’t being used and looking at the links that are pointing to the site, what words are being used to describe the site when someone links to it. All these types of things come together to tell the search engines what a page is about.
Now, one of the newer developments in this, especially with Google, is that they’re now moving beyond just looking at how many times does a word appear on a page to what is the concept that this page is trying to be about? They’re trying to look at all of the content that’s there and just put it all together to figure out what’s the idea here. Then they’re going to rank that website for things related to that idea. It’s what’s called semantic search, and they’re moving beyond words to concepts.
Submitting Queries:
So that’s figuring out what a page is about; the next thing is allowing people to submit their search queries, their search terms. Most of the time this is done just by a little bar that’s on the screen and you type in your question or your search terms and it returns the results. But of course, there are different ways that you can do this. You may speak in a search term; there are some places where you click a link to view a search. So there’s a variety of different ways that people do these searches.
The interesting thing about this is that Google especially is doing the same thing that they’re doing with trying to understand what pages are about, they’re trying to do with queries. So they’re trying to understand the concepts that people are searching for, more than just the words. Before, you would search for four different words in a search phrase, and it would look at each of those four different words as four separate things, and then put them “here’s the results for this word, the results for this word, the results for this word,” kind of put it all together and figure out what was best. Now it actually tries to understand the relationship of each of those four words to each other, so that it can understand the concept of the search and then return results for that concept. That’s been a newer development for especially Google.
Returning Results and Ranking Pages:
Then the big one. They return results. How do they figure out what order to return results? Because there can only be one result that’s at the top, so there’s always going to be an order to these results. How do they figure that out? They do this in a variety of ways, and the primary things are relevance and authority. Is a site really relevant to this search term? Is it about this concept or about this search term? They’ll go and look at what is this page about, and they’ll apply that to the question that was asked, and that’s what’s going to get you into the list initially.
Then to figure out if there’s 10 pages that all are relevant, what’s the order for those? In your smaller search engines, there’s a variety of different ways that they may figure this out. It may be based off of when the site was submitted; there may be a voting system on the site that allows different sites to move up as they’re voted for; they may have places where you can pay, so you can actually pay to be higher in their search rankings.
With the major search engines, they primarily do this by looking at how many links are coming to a site and what kinds of sites are linking to them. It’s not just the number; it’s also the quality of those sites and whether those sites are relevant to the search term. All of that kind of comes together, and they’re looking at a lot of different signals. Social, are people talking about the site? Are people sharing this page? Is this something that people are excited about in the various social networks? If you’re Google, you’re looking at author rank and things like that. The person who wrote the page, are they an authoritative person based off of that authorship?
There you have it.
So there’s a lot of things that go into that, but they’re basically looking is this page relevant to the search result and how authoritative is this page to determine what that order is going to be. That is the general idea of how search engines work.
3 Comments
Great overview Kurt!
I learnt a lot thanks
Can you maybe explain why some search engines work better than others at finding the info about what you want to know about in a future article?