Web Governance
Algorithms Behind AI Search
This is the third blog post in a series of blogs from Yext, highlighting how adding natural language powered AI search to your website can help to provide a best-in-class web experience. If you missed the previous blogs, check out the first one and the second one!
Your Drupal-powered website is chock full of useful information about your locations, content, products, services, and more but are you customers actually finding it? One of the best ways to connect customers to that content faster is to make it searchable. More and more, customers expect to be able to search for exactly what they need without having to wade through a sea of links. But the search landscape isn't what it used to be. Today, a simple keyword-matching algorithm won't cut it because users expect to find structured content, direct answers, and clear calls to action like they do on major search engines.
Artificial Intelligence (AI) is critical in the next generation of search, allowing you to efficiently parse through all of the different types of data you host on your Drupal site, like articles, people, page content, places, and events.
Top search engines like Google, search huge swaths of data from a variety of sources the entire web and its famed knowledge graph in order to provide users the exact answers they're looking for. However, there isn't a one-size-fits-all approach that works for any type of data. Instead, Google uses different algorithms, all powered by AI and deep learning, to search different data types.
The best site search experiences take the same approach. To create that search engine-like search experience on your Drupal site, one algorithm isn't enough to search all of the different data types. Even after structuring your data in a Knowledge Graph, the data will still live in a number of different states there is no one algorithm that can answer every question.
Data can be structured, semi-structured, or unstructured. Each type of data requires a different approach to efficiently search that's where multiple algorithms come into play! Let's dive into the effective algorithms for each data structure.
Structured data follows a predictable pattern, with each entity (a real world data point such as a location, person, event, etc.) containing a number of fields that provide more information. For example, people' entities might have a name, address, and phone number.
For structured data, you can utilize a Named Entity Recognition (NER) algorithm, which identifies words in each query and classifies them. At Yext, our NER algorithm is based on Google's open source machine learning framework BERT (Bidirectional Encoder Representations from Transformers) to detect potential filters and show structured results from Yext's Knowledge Graph.
For example, Answers will automatically turn the query "Financial Advisor in New York who speaks Mandarin" into a structured graph query with the following filters:
We can see how NER is helpful for searching structured data, like a financial advisor entity. If your Drupal site contains other structured data like products, locations, menu items, or professionals, you can trust NER to parse out and return the exact result your customer wanted to find.
Semi-Structured data follows a predictable short-form format, but has a bit more flexibility than structured data. For example, semi-structured data may look like an FAQ, the title of a help article, or the name of a blog post.
For semi-structured data, you can leverage a form of Semantic Text Search. This method uses AI to understand what the query really means, beyond just matching on individual keywords. Instead of relying on keywords, you use an algorithm to identify the most closely related entities within your Knowledge Graph.
At Yext, we trained Google's BERT to better understand what a customer is really looking for. Unlike keyword-based search algorithms, Semantic Text Search knows that someone searching send back shoes is looking to kick off a return process, and someone asking about a dislocated shoulder probably needs a doctor, specifically an orthopedist.
When a user enters a query, Semantic Text Search encodes it as a vector, creating a numerical representation of the query. Then, it looks for the data in the Knowledge Graph that is closest to the query in vector space.
Two vectors that are close in space share meaning more than two that are far apart. Semantic Text Search places a query in vector space, and it locates the content that is closest to it, giving your customers the answers they might not have otherwise found based on simple differences of word choice or phrasing.
These vectors can be huge! At Yext, they have hundreds of dimensions which is impossible for us to wrap our heads around but you can visualize the process in two dimensions here:
While the machine learning power behind Semantic Text Search is complex, the benefits for your users couldn't be simpler: answers to the questions they really asked.
Some of the data your users want to search might not be structured or even semi-structured at all. What can you do for the long-form content on your Drupal site, like articles or blog posts?
For unstructured data, the answer is Document Search! This algorithm allows you to identify the most relevant documents and answers. With Extractive Question Answering (also known as Extractive QA), you can extract relevant snippets from the documents that answer the query posed.
Extractive QA uses BERT to identify the excerpts from long documents that best answer the user's question. If it finds a good answer in the text, the algorithm displays it as a featured snippet no additional clicking or searching needed!
In addition to providing a direct answer at the top of the page, Extractive QA also highlights relevant context as in-line snippets on each result down the page. These snippets provide a great search experience and help searchers quickly find the best answers to their queries.
Building the multiple algorithms required for modern AI Search is a daunting task. Luckily, Yext's integrations with Drupal make it easy to apply a multi-algorithm approach to all the amazing content you already have on your site.
With the Drupal Data Connectors, you can easily pull built-in content from your Drupal site into a Yext Knowledge Graph to make that content easily searchable. Our five new out-of-the-box connector apps allow you to pull in Drupal articles, people, pages, places, and events data from your Drupal site. Additionally, if you have custom content types on Drupal, we have a new connector source you can use to pull in all content from your site. If you already use a Yext solution, you can instantly access these in your account by clicking into Apps from our navigation panel. Once you have your Drupal data in your Yext account, you can easily add AI Search to your Drupal site.
Sign up for a 30-day free trial to see how the Yext and Drupal integrations can help you boost on-site conversion, reduce customer support costs, and gain new customer intelligence.
Discover more here and learn how to easily integrate Answers with Drupal with Yext's Hitchhiker's guide.
Web Governance