Concept Searching is pleased to announce that UCare, has deployed conceptClassifier for SharePoint in a SharePoint 2013 environment and has immediately seen an improvement in search results. UCare is an independent, nonprofit health plan providing health coverage and services to more than 400,000 members in Minnesota and western Wisconsin. UCare serves more people from diverse cultures and more people with disabilities enrolled in Medical Assistance than any other health plan in Minnesota.
Our CEO/CTO, John Challis, was recently asked the above question for an article. Here is his response.
At the height of the dot-com boom (15 years ago) it seemed likely that enterprise search was ready to move beyond simple keyword and Boolean searching being replaced by more sophisticated techniques developed by companies such as: Autonomy, Verity, Convera and FAST. These products offered a variety of new algorithms: automatic query expansion, compound term processing, semantic networks, linguistic analysis, concept searching, feature extraction, custom term weighting, etc. What actually happened was that each of these companies was acquired and their products either killed off by their new owners (Verity, Convera, FAST) or otherwise fatally injured (Autonomy). Today, enterprise search is dominated by Microsoft and Google with other leading vendors including: Oracle, Solr/Lucene and ISYS. None of the current offerings moves significantly beyond keyword and Boolean searching. In my opinion the enterprise search market today has little appetite for more sophisticated products the likes of which we have seen come and go in recent years.
WDYT? Agree, or disagree? Comments?
I just posted a blog about the answer our CEO/CTO, John Challis, provided for an article. The question was, What is your assessment of today’s enterprise search industry? A second question asked was ‘What do you think the future of ‘search’ will look like?
John’s answer to “What do you think the future of ‘search will look like” was as follows:
The future of enterprise search seems destined to continue with simple keyword and Boolean searching, augmented by faceted navigation based on metadata. The main driver for this is the World Wide Web. Virtually every e-commerce web site today offers guided navigation based on metadata. When you enter a simple text query into Amazon or eBay, or virtually any other shopping site, you see filters for “vendor”, “price range”, “colour”, “size”, etc. This ubiquitous model now appears in most of the leading enterprise search products and users immediately understand how a simple text query can quickly be focused to a specific domain by clicking on a metadata filter. This updated search model is increasing demand for auto-classification products which can generate descriptive metadata automatically based on an analysis of the document’s unstructured content.
What do you think about this statement? Disagreements accepted!
Migration isn’t often viewed as an IT project with a hefty ROI. In fact, in can bring shudders to IT professionals. Of course the devil is in the details. Is the objective just to move content? Say from SharePoint to Office 365? Or is it to ‘intelligently’ migrate content, regardless of the platform so information is more usable, organized, and efficiently retrieved? All questions, only the organization can answer and what the specific project requires. According to a snippet I found, 84% of migration projects fail (Bloor).
At the basic level, migration is needed to move content, but at the deeper level to enable access for business users who need to find the information to support their job functions. That’s where the ROI can be achieved. This places most of the effort on pre-migration activities, such as identifying content to be used, cleansing the content, and moving it to an organized hierarchy that will enable it to be used after migration. Our approach is to use auto-classification to one or more taxonomies before the migration. This enables the ‘tweaking’ of the taxonomy to ensure documents are migrated accurately during the migration process.
This does represent an ROI which is based on the same criteria for calculating an ROI for enterprise search. Pre-migration activities are critically important and many organizations can achieve long term benefits and not simply moving content from one repository to another.
Do you prepare an ROI for migrations?
I’ve commented on this before, but the pendulum of enterprise search is moving, in my view, the wrong direction. In an effort to compete, vendors are delivering bells and whistles, that don’t necessarily improve enterprise search, in fact, can degrade the accuracy and relevancy of search to fit the current assumptions of what business users need and want.
At the end of the day, what they want is the ability to find the information they are seeking, in the right context, as quickly as possible. The ability to ‘like’, ‘rank’, ‘action terms’, ‘promoted results’, and automatically return these documents in a higher order has removed accuracy and relevancy from the search game.
Do business users get bogged down in the tricks of the trade so to speak, and therefore become less productive? What do you think of these bells and whistles? Do your business users see them as valuable? Are these features able to deliver high accuracy and relevancy?
Don’t you wish sometimes your enterprise search engine could tell you that it doesn’t understand what the heck you are asking for?
Although the following is from 2009, which is probably leap years in technology, “Using the Internet: Skill Related Problems in User Online Behavior”; van Deursen & van Dijk the following statistics were generated:
- Searchers do not know “how to search”
- 56% constructed poor queries
- Proficiency with the machine does not translate into proficiency with the software
- Searchers get lost in the data
- 33% had difficulty navigating/orienting search results
- 28% had difficulty maintaining orientation on a website
- Loss of capacity for discernment
- 36% did not go beyond the first 3 search results (not pages…results on page 1)
- 91% did not go beyond the first page of search results
- 55% selected irrelevant results 1 or more times
I personally think that enterprise search is actually even harder to use. Yes, even for our savvy younger generation. Part of the problem is too much information and a lack of patience. Knowledge workers want the information they need – and now. Almost like instant gratification.
Do you agree that those statistics would still be valid today? If so, why or why not. What other enterprise search problems have you encountered?
Just like Dorothy and Toto, following the yellow brick road to productive, accurate, and relevant enterprise search results is fraught with challenges. Unfortunately, there is no wizard at the end of the road that magically solves the problem. Although Dorothy didn’t know that she could have gone home any time she wanted. Maybe it’s the same with enterprise search.
Part of the issue with enterprise search is it is viewed as a stand-alone application. This isn’t true, and education in the marketplace is needed to present ancillary technologies that don’t replace enterprise search engines, but improve them.
There is a difference between human retrieval and machine retrieval. Most of the time, business users cannot necessarily articulate what they are looking for, therefore can’t find it without a lot of effort. The search engine technology is machine driven and takes our searches quite literally and doesn’t know how to interpret what we are really looking for. And then, of course, there is the ambiguity in language which search engines typically don’t understand.
The use of a taxonomy as the backbone, (and if implemented correctly, an enterprise metadata repository) can present information and offer search techniques that are not typically embedded in a search application. This results in a user focused approach as opposed to a machine approach. The business user then has the opportunity to explore topics by the hierarchy in the taxonomy. In this way, it enables them to drill down the specific information, in the context of what they seeking and not necessarily how the search was constructed. This provides human relevance as opposed to machine relevance.
I am curious to know how many of you use a taxonomy, or ancillary tools to improve your enterprise search? If so, what are they and have they improved enterprise search?
Ok, we are picking on government today. It seems that most federal agencies are not confident in their ability to handle eDiscovery. I would argue that many organizations are also unable to handle the challenges associated with it. What is the basic goal? To have certainty that your information is accurate, accessible, complete and trustworthy, according to a recent Deloitte Survey. I would add one more standard and that is the information needed can be found.
The problem, which is legitimate, is the overwhelming amount of information that continues to accumulate. In fact, now the term Big Data is creeping into the Discovery process. Federal agencies do have more rigorous demands and more minutiae that must be saved. But the problem still exists for us ordinary organizations.
The following is the graphic from the Deloitte Survey. If nothing else, worth a look.
Do you feel that Discovery or even litigation support is becoming more difficult due to the unabated growth of content? From an organizational view, have you changed processes to accommodate?
Seems like a duh statement. Many organizations do recognize the importance of search and will change their search engine more frequently than is necessary. Others make do with what they have. But the purpose in deploying a new search solution is not necessarily to obtain new search features, but to improve the relevancy of retrieving accurate and timely information from a typically unorganized morass of content. This is the cause of failure and I think sometimes organizations muddle the differences on what the real problem is.
The primary problem in poor search results is the lack of metadata associated with the content and the relating of content in one system that is similar to or equivalent to content in a different system. There is a growing need to generate far richer metadata and manage it effectively to provide simplified access to these resources by business users.
You search is as good as your metadata. It’s as simple as that. The metadata identifies what words and phrases are meaningful in what context. By automatically generating conceptual metadata and supplying the terms to the search engine index, the descriptors ensure more relevant results. To make content fully transparent and go beyond system or user generated metadata, identifying ‘concepts in context’ from your content transforms it into business assets.
Without meaningful metadata, content will remain unmanaged and content will lose value and context. Thoughts?
Organizations are beginning to recognize that search is not a stand-alone technology or application, but must be integrated with business processes and corporate objectives as a key infrastructure component. Why? Poor search costs money.
The touch points of search are far reaching and illustrated below:
The costs of poor search have been documented again and again (and again). The statistics battered around include:
- 15% of a knowledge workers time is spent recreating information
- 26% of their time is spent searching
- 40% can’t find the information needed to do their jobs
- The cost to a 500 employee company is $2.4 million per year in inefficiencies and lost productivity
A good search engine will provide end users the means to easily find the information they are seeking and shields them from the complexities of accessing content assets from multiple repositories and LOB applications. This also obviates the need for training as the search process integrates all relevant information and ideally is designed for ease-of-use. This process value substantially reduces the time required to complete the search (i.e. find the information to do your job). This reduction can have a positive ripple effect where it can span departments and the organization itself. Therefore the business benefits can be viewed as organizational as well as individual.
Clients using our products
Join our team
- Ucare Deploys conceptClassifier for SharePoint
- Protiviti Deploys conceptClassifier
- German Bundeswehr to Deploy Concept Searching
- Concept Searching Welcomes XL Group as a New Client
- Research Platform to Use conceptSearch
- Medical Council of Canada Case Study Now Published
- Largest Bank in Japan Selects conceptClassifier for SharePoint
- Intergovernmental Organization to Deploy conceptClassifier
- conceptClassifier Selected at the Town of Fishers
- Welcome Canadian Department of Aboriginal Affairs and Northern Development
- NASA Safety Center Recognized by KMWorld
- Leading Global Manufacturer of Auto-parts Has Selected Concept Searching Technology Plat...
Recent Blog Posts
- Bye Bye SharePoint – You can’t say you weren’t warned!
- Really, you want to do text analytics? And get results?
- Text Analytics – OK Sherlock, what’s next?
- Social Media – Who really cares how many Tweets you get?
- Text Analytics and the Eureka Moment
- Using Text Analytics in SharePoint or Office 365
- Text Analytics – How to Get from Here to There
- If you have 10K Records, Your chance of a material data breach is 22% @ $145 per record
- How Much Dark Data is Out There? Do you care?
- Cloud Ready? Think Again – It’s a free-for-all out there!
- Cloud Adoption – a Sponsor’s Perspective? It’s called ‘massaging’ ...
- What does the IT, Finance, and Healthcare Industries All Share? A penchant for malware a...