This post was originally published by Sander Gansen at Medium [AI]
Over the years, I have wasted hundreds of hours on knowledge management — be it texts, images or various types of data.
I have been organising knowledge by trying to come up with universally understood names, forming elaborate folder structures and distributing access to those files.
Often I have even maintained reorganised copies of the whole company’s knowledge base to ensure that I would be able to find everything when needed, potentially creating a security risk.
I know that other people are behaving similarly. Like me, they have spent too much time on knowledge management, perhaps believing that it must be the new normal.
But is it?
Or is there another way?
In this article, I am going to discuss the following:
- Why are people wasting time on knowledge management?
- How are we organising knowledge at the moment?
- What else can we do to fix the situation?
- What is semantic search?
- What are we building at Kiri?
In the age of data, we generate a vast amount of information. All this needs to be stored somewhere for future use. And that is where different knowledge management systems, such as intranets, wikis and other shared storage solutions come in play.
However, there are only a limited number of tools that can help us analyse unstructured data. For that reason, we must organise the knowledge for its intended use in the future.
That said, I have identified two main reasons why some of us have to waste time on manual knowledge management:
- People generating files are usually storing them in unstructured folders with random names. Or at least that is how it might seem to another person.
- Most knowledge management systems do not understand the content. More so, they are unable to help us in organising the knowledge or finding the relevant files via search when needed.
The most common method for knowledge management is having someone keep an eye on all content manually — renaming files and changing folder structures as necessary.
In an ideal world, there would be rules in place, and all people would follow them when storing information.
Yet in reality, a generally benevolent employee or someone specially hired to do so has to spend actual time on manually tweaking the data generated by others. And even then we might not find the knowledge when needed.
Of course, some specific collaboration tools, like Notion, simplify the process of finding relevant information.
But this means that we ought to change our existing knowledge base to something new. All while even these solutions expect us to link or tag each piece of content manually.
What can we do if we cannot get people to store knowledge in a universally understood way? Or do not have any team members who enjoy tweaking files all day long? Or cannot change our knowledge management system?
The first obvious choice would be to hire someone or outsource the task. Based on Twitter, it seems that both individuals and businesses are willing to pay even $ 1,000 for someone to organise their digital files. In IBM’s case, this would total up to $350 million spent on a one time job. So this might not be a sustainable choice, but it is one nevertheless.
The second option is to keep looking for a better tool.
However, what should the tool do?
That is precisely the question I asked myself this Spring. Because by that time, I realised that I was already wasting too much time on knowledge management.
At that moment, I was maintaining three copies of the same files within one of my organisations. Mostly to ensure that everyone in the team and our clients would see a structure that makes sense for them.
Meanwhile, I was also maintaining multiple duplicated knowledge bases for other companies, so I would always know where different files were. Because people using the original systems kept misplacing files.
So I began to search. Only to realise that I never needed a better storage solution. Instead, I needed a better enterprise search solution. Because with a reliable search solution, I do not have to know where the files are, as I can just search for them.
However, most search systems lean on having a clean file structure and the use of specific keywords. Thus if we want to find something, we currently need to know its location or the exact name to search for it.
But what if we did not have to know any of that?
What if the tools we use could fetch anything we need them to, regardless of the keywords we use? With Natural Language Processing (NLP) algorithms becoming smarter, that has become a reality in the form of semantic search.
As described in the Search Engine Journal, a semantic search represents a search engine’s attempt to generate the most accurate results by understanding the following:
- Searcher intent.
- Query context.
- The relationships between words.
In essence, semantic search attempts to understand your queries the way a person would.
For example, if you asked a friend “What is the largest mammal?” and then followed that question up with “How big is it?” your friend would understand that “it” refers to the largest mammal: a blue whale.
However, understanding the query is just one side of the equation. To eliminate the need for manual knowledge management, the algorithm behind semantic search also needs to understand the meaning behind the content.
That is where advanced NLP algorithms play an essential role again — in this case, helping search engines index data and subsequently being able to seek relevant answers. All without having anyone tweak the content manually.
In layman terms, such a solution would understand the context of both the query and the answer. And thus be able to find an answer even when no exact word is used anywhere inside the content.
For example, if you search for an “elephant” on a government website, then most likely there is no mention of the animal anywhere. Simultaneously, there might be a mention of managing a zoo or keeping a pet. The engine will know that an elephant is an animal that mostly belongs in the zoo but can also be a pet. Thus, it will return you those files, ranking the one you might have intended to search higher.
So with NLP at our use, we can finally stop wasting time on manual knowledge management. In exchange, we can rely on semantic search to fetch everything based on our intention, exactly when we need it.
This is precisely what Kiri is building, so we could stop wasting time on knowledge management, and just use the knowledge.
At Kiri, we are building enterprise search solutions using semantic algorithms. Meaning that we are using trained models to understand the meaning behind the queries without any manual tagging — in 15 different languages, and counting. This way, we are helping organisations of all sizes to make their knowledge more easily findable.
Kiri connects to your information via an API and learns on the go. We can thus decrease the integration time to merely hours, while reducing the effort from your side to the minimum. No need to manually tweak the search engine ranking, run A/B tests or write synonym sets for the content.
Furthermore, we know how critical security, reliability and compliance are for organisations around the world. Notably, when dealing with the most precious resource of any company — its data. Thus, all data within Kiri is encrypted, so that even our team cannot access it. Meanwhile, we store everything within European servers that have SLA 99.9% and are fully GDPR compliant.
We also provide our clients with a full-service solution which is fully hosted by us. Once you decide to use Kiri, we will be taking care of the rest — integration, content crawling, data monitoring and in-time tips shared with your team. We’ll always guide you and help you reach your goals.
Kiri is an enterprise search solution done right.
We are still in beta. At present, we are working with a few choice companies testing the suitability of our product for their individual needs. As we are looking to expand on these results, we invite others to collaborate with us — reach out to me personally here.
This post was originally published by Sander Gansen at Medium [AI]