Context made clear with natural language processing

Most of us rely every day on the ability to quickly find the answers and tools we’re looking for, an effective search functionality is a cornerstone of ecommerce websites, intranets, knowledge platforms and much more. With the tsunami of available textual information mounting with each passing day, sifting through it takes real intelligence. Natural language processing, or NLP, gives machines the ability to infer the meaning behind our queries – even when we’re not sure exactly what we’re looking for.

“NLP is actually a subform of AI,” explains Inez Van Laer, NLP expert at delaware. “It involves the use of algorithms to understand text, but not simply the value of the words themselves as strings of letters.”

With traditional search tools, users type search queries into the input field and the computer finds exact matches by searching through tables and metadata. “It’s very basic, and in order for it to be effective, you have to know exactly what you are searching for,” she continues. “If you type ‘healthy food’ into a standard search tool, you’ll only get results that contain ‘healthy’ and ‘food’ in their titles and metadata – not very helpful if there are ‘vegetarian’, ‘low carb’ and other healthy results that don’t contain your search terms. You’re missing out on a lot of relevant results.”

With semantic search – an application of NLP – the context of the query is captured and analyzed, and relevant information is found by the algorithm. Inez: “Even if you’re not sure what you’re looking for, the model understands the idea behind your query. Searches are exponentially more effective using NLP because they present highly relevant results that aren’t exact matches.”

“Over the last handful of years, we’ve moved from standard keyword searches to vector-based semantic search,” says Inez. “Now, delaware is on the cutting edge of semantic search with our sentence vector-based NLP models.”

These powerful tools map out relationships between the ideas contained in full sentences in a virtual multidimensional space and then measures the distance between them to determine their relationships to each other.

“This is revolutionary,” Inez goes on to say, “because the model is dealing with fully fledged ideas, not words. This means that the very same model can be applied to data sets in different languages, and even to identify relevant results among multiple languages. In a multinational, multilingual world, this ability will become ever more essential in our approach to information management and structuring.”

The applications of vector-based semantic search go far beyond the universal quest for tasty recipes. Think of online shoppers looking for the perfect products, or company employees in search of specific legal agreements.

Inez: “We built an ecommerce website for a large retail company. For companies that sell vast quantities of different products, the effectiveness of the search functionality is business critical. Even more, you have to fine tune the metrics to weigh results not only for their relevance, but also recentness, popularity, total product sales and more.”

Different challenges emerge when it isn’t different products but different information sources that users are searching through.

“In another NLP use case, we developed a semantic search tool capable of hunting through huge volumes of legal documents in many languages,” she continues. “Even more, it had to be able to find relevant results in languages other than the search language, enabling users to obtain answers even if they weren’t available in their native tongue.”

In addition to the web and intranet search functionalities that we’re familiar with, NLP drives emerging technologies that are transforming the way businesses interact with employees and customers.

“If you think about it, chatbots are really just responsive search tools,” Inez elaborates. “These tools have to be capable of understanding the ideas behind and contexts around the questions that users ask, making NLP their driving engine.”

Even today, NLP is still a young domain in the field of AI, with lots of different applications. “delaware is unique as a solutions provider in that we’re truly pushing the boundaries of what can be done using available NLP technologies – and developing completely novel methods of responding to challenges in human-computer interaction,” Inez concludes.

Ready to optimize the way your employees, collaborators or customers find products, tools or information? Get in touch with our NLP team today or schedule an ideation session.