The DoD and military face unique challenges. The impact of budget constraints and the push for modernization and cloud use can’t be ignored. Technology continues to improve while siloed repositories, lack of information transparency, and inadequate security continue to hamper success. Achieving interoperability requires a cultural change as well as the introduction of technologies that are fundamental in achieving the full potential of transformation.
Concept Searching delivers an enterprise framework to mitigate risk, automate processes, securely manage information, ensure privacy, and address compliance issues, enabling enterprise information governance. This is accomplished at a basic level with the ability to transparently tag content, classify it to organizational taxonomies, preserve and protect information through the automatic identification of records and privacy data, and act as a migration tool in any environment.
Concept Searching technologies serve multiple purposes and are platform independent. The end result is a metadata framework that provides flexibility, and enables an organization to implement governance policies and operations incrementally, with each step directly correlated to mission value.Core Technology Technology Benefits
The rate at which technology advancements are being made means that federal agencies that lack the agility to embrace the advances at an equal rate are being left behind. Typically these entities have not kept up with technology capabilities. Challenges embedded in the institutional culture, a cumbersome regulatory and acquisition environment, and legacy applications that require modernization have contributed to the problem. The explosion of unstructured content, the increase in security breaches, the mandate for transitioning to the cloud, stagnant records management practices that continue unabated, increased FOIA and eDiscovery, and to some extent retiring baby boomers, all present ongoing challenges to the government.
Lack of Information Transparency (FOIA and eDiscovery)
Noncompliance with Records Management Policies
Increasing Volume of Data Exposure Events
At a basic level, noncompliance with records management policies, lack of metadata for improving search, and increasing volumes of data exposure events are directly related to insufficient metadata. The limiting factor in all these scenarios is the human element, as it relates to behavior and training.
The fundamental technology problems that can improve the performance of the organization as a whole are addressed via Concept Searching product platforms. The challenges solved include:
At the crux of all these applications and business process bottlenecks is metadata. The elimination of human metadata tagging, replaced with the automatic generation of multi-term semantic metadata, auto-classification, and taxonomy tools, enormous amounts of time and resources can be saved and allocated to moving the organization forward.
Concept Searching technologies are deployed today across a wide number of organizations on both the SIPR and NIPR networks, as well as the DISA network, and has defense clients with more than 70,000 users accessing a single application globally. Concept Searching also has a current enterprise Authority to Operate (ATO) with the US Air Force and an enterprise Certificate of Networthiness (CoN) for the US Army.
Below are the fundamental challenges our clients have faced, followed by information on how they were able to address their challenges.
The components consist of Metadata, Insight, Risk, Policy, and Action. The Metadata building block is the development of a single repository of organizationally relevant metadata that is available to any application that requires its use. Insight provides the ability to find and deliver the most relevant and granular results from large, heterogeneous repositories. The third building block, Risk, is determined by the organization to identify high profile risk factors, and analyze the impact and cost for non-compliance. Policy is the response to organizational risk and includes organizational and individual approaches to mitigate or eliminate the risk. Action, the final building block is the execution and interactive management of the policies and subsequent processes that ensures all unstructured and semi-structured content is processed in a manner that achieves the information governance objectives.
This US Army command organization currently manages a budget of more than $12 billion and cares for more than 1.8 million beneficiaries – active-duty members of all services, retirees and their family members. In an effort to address enterprise issues with information, records, and knowledge management, the organization conducted a Joint Capabilities Integration and Development System (JCIDS) analysis and selected the Concept Searching technology platform to deliver a robust organizationally aligned taxonomy structure for defense healthcare, to significantly improve information transparency, reduce sensitive information breaches, and assist with the preservation and storage of records in line with federal guidelines.
This US Army command deployed the conceptClassifier for SharePoint platform within its SharePoint internal portal. The organization plans, conducts, and reports operational tests, assessments, and experiments, in order to provide essential information for the acquisition and fielding of war fighting systems. It implemented the technology to organize and structure its content, and perform auto-classification to the structure in the process, applying metadata that can be filtered upon by the Microsoft Search engine, significantly improving search and retrieval through the ability to search on concepts. The result is greater transparency and improved project collaboration.
Reaching out to impact eDiscovery and litigation support, and unauthorized access to confidential information, the results can increase costs as well as organizational risk. A core component in all Concept Searching platforms is the ability to automatically generate compound term metadata, eliminating end user tagging. Identifying relevant content during a search effectively removes ambiguity and enables the retrieval of information based on the concepts within the document, increasing both precision and recall.
All Concept Searching platforms provide the ability to automatically identify, secure, and prohibit portability of organizationally defined vocabulary and descriptors. Since the metadata generation is not restricted to keywords, it is highly accurate in identifying potential security exposures, in real time, as content is ingested or created and then routed to a secure repository for disposition. Full support for standard descriptors is included, and additional value is provided by the ability for the organization, or functional group, to define confidential information that falls outside of standard descriptors.
The conceptClassifier for SharePoint platform has been deployed at numerous US Air Force bases, to enable compliance with data privacy and security guidelines associated with the Federal Information Security Management Act (FISMA), Privacy Act Program, Health Insurance Portability and Accountability Act (HIPAA), Joint Commission on the Accreditation of Healthcare Organizations (JCAHO), and Payment Card Industry (PCI).The DoD and military have often complicated, multiple levels of security, that must be applied to every piece of content. The flexibility of the technology enables the organization to not only identify standard descriptors included in the base product, but also create their own descriptors and vocabulary to identify any type of potential security breach.
This US Air Force support organization has a budget of $6.9 billion and runs 75 hospitals and clinics, providing care to over 2.6 million beneficiaries. It deployed conceptClassifier for SharePoint to: increase information retrieval precision on its intranet; enable subject-matter experts to develop business rules for information management; eliminate the need to manually meta tag documents; and provide automatic classification of documents and records, based on contextually relevant and domain specific information contained within the bodies of documents. It is currently using the technologies for records management, identification and protection of data privacy, search, and migration.
In the DoD and military it is almost overwhelming. The biggest stumbling block has always been end user tagging. With complex file plans, and the fact that human tagging is often haphazard at best, organizations can often unwittingly fall into the trap of noncompliance. The Concept Searching platforms, are being used in records management to automatically identify documents of record, auto-classify the content to a taxonomy that mirrors the file plan, and route directly to the records management application.
The best defense is the ongoing management of data to ensure preparedness. Unmanaged information carries a great deal of risk, as it can be used in unintended way. During the eDiscovery or FOIA process the same basic problem of finding relevant information occurs, which is costly and unproductive. The volumes can be unprecedented, and facilitating the eDiscovery or FOIA process is a drain on human resources and carries risk and well as increasing costs.
This agency deployed conceptClassifier for SharePoint with the Digital Asset Finder™ records management solution from COMPU-DATA International, to enable the proper disposition of documents that had been declared records, based on federal records management policies. Over 20 data sources and databases were consolidated into a few repositories. It currently loads millions of records per day, and automatically classifies these against multiple taxonomies simultaneously. The solution is easily expandable through its modular and scalable architecture. Professional users are able to find highly granular information that typically would not be found, reducing the time, costs, and increases productivity.
The increased spotlight on intelligence agencies in the past few years, has highlighted the constant need for agencies to be supplied with accurate and timely information, that enables them to make the best decisions possible. This agency contributes to the security of New Zealand through the provision of foreign intelligence to government, assisting government departments and agencies to protect their electronic information resources and communications systems. To improve these initiatives, the organization rolled out the conceptClassifier for SharePoint platform and conceptTaxonomyWorkflow integrated with SharePoint 2010 and FAST Search, to automatically harvest large amounts of data, classify in real time, identify specific content and take action based upon classification metadata, thereby supporting the intelligence service within New Zealand.
One of the biggest benefits of the Concept Searching platforms is the ability to aggregated content from file shares, SharePoint, websites, and diverse applications to provide a single point of reference for knowledge workers. Supporting navigational, concept-based, and discovery type searches, the user can directly access the information needed, or can be guided to relevant content, identifying relationships between content, and offering topics and information that typically would not be found.
Data is machine driven, whereas unstructured content is driven by people, which makes the nuances, insights, relationships of disparate content, sentiment, and knowledge capital much more difficult to extract. Using Concept Searching product platforms before text analysis the quantity of content can be reduced through an initial cleansing. Once the ‘noise’ has been eliminated, the ability to analyze the content at a very granular level produces actionable results that can be used to solve problems, reduce costs, and improve decision making.
The UK government department responsible for promoting British interests overseas and supporting UK citizens and businesses around the world deployed the conceptClassifier for SharePoint platform, with the optional Concept Searching search engine, conceptSearch. The department has large data repositories, many of them with sensitive information, and the technology platform, integrated with SharePoint, has enabled the development and maintenance of organizational and mission aligned taxonomies and the classification of large data sets, improving transparency and collaboration on sensitive information.
The Concept Searching platforms are used in all industries to migrate content to provide the basis for a re-usable enterprise repository to address metadata challenges and enable intelligent metadata enabled solutions. As an example, a professional services client using conceptClassifier for SharePoint identified 66,000 duplicates out of a total of 270,000 documents, representing a 24% reduction in disk space. In another example, a global supplier of automotive parts primary objective was to implement conceptClassifier for SharePoint to improve search for 147,000 business users. The first project was to migrate several millions documents. conceptClassifier for SharePoint was used for the pre and post migration, and for enabling concept based searching integrated with their existing search engine after the migration. The US Air Force has been a client for over twelve years and was one of the first clients to use the platforms for migration, and has successfully used the capabilities in multiple and diverse migration scenarios.
To migrate document collections effectively, the text content of each document needs to be searched to determine its value. This cannot be done manually, as the volume is too high, and the consistency of human review and decision making is unreliable as well as costly. Using Concept Searching platforms, an intelligent approach to migration can be achieved. As content is migrated it is analyzed for organizationally defined descriptors and vocabularies, that will automatically classify the content to taxonomies, or optionally the SharePoint Term Store, and automatically apply organizationally defined workflows to process the content to the appropriate repository for review and disposition. Concept Searching migration capabilities include migrating from file stores to file stores, file stores to SharePoint, SharePoint to SharePoint, and provide the ability to define a custom action through the conceptTaxonomyWorkflow product.
But moving to a cloud environment, or even a hybrid environment, poses challenges with regards to applying a consistent information governance strategy across all environments. The Concept Searching platforms have been deployed in the cloud, including Office 365 and Amazon Web Services, and enables effective management of unstructured content by enforcing and extending an enterprises’ on-premises information governance policies within the cloud environment. Metadata driven policy actions on content used in search, migration, identifying and securing sensitive information, or in the automatic declaration of documents of record, are some of the key capabilities available in a cloud environment. conceptTaxonomyWorkflow, an add-on product, provides the workflow capabilities and is available in all product platforms.
The CIO or CTO in any industry faces both strategic and tactical challenges. In intelligence and military agencies, the responsibilities are broad and challenging. The strategic challenges must be carefully addressed with a focus on the impact at a tactical level. Budgetary constraints can change the priorities, and force a ‘pick and choose’ philosophy to improve performance. One of the key differentiators in the Concept Searching platforms is the re-usability of the core technologies to solve multiple content management and business process challenges. This provides cost benefits in terms of the elimination of separate solutions to address key application challenges, a reduction in staff to implement and maintain the applications, and transparent usability from the end user perspective. The flexibility of the intelligent metadata enabled solutions can address search, FOIA and eDiscovery, records management, eliminate data privacy and data breaches, provide intelligent migration, social media applications, text analytics capabilities, and over all content management. The ability to leverage the existing infrastructure to improve these challenges assists the CIO/CTO in addressing multiple issues. An additional key component in the Concept Searching platforms is the ability to deploy in an on-premise environment, cloud, or hybrid environment.
The management of knowledge is a significant and ongoing task for knowledge managers, librarians, or taxonomy specialists. With content continually in a state of flux, the management of knowledge uses a multidimensional approach that must capture, make available, share, and maximize the value of content, to achieve organizational knowledge. Managed effectively, it leverages content assets as well as captures intellectual assets, that ultimately drives innovation. Using conceptTaxonomyManager, a core component in all platforms, the development and management of taxonomies was designed for subject-matter experts and provides an easy-to-use interactive interface. The product provides a rule-based engine that eliminates the need for training sets and highly specialized human resources, that is unique to conceptTaxonomyManager. Other unique features in the product include automatic taxonomy node clue suggestion, dynamic screen updating to immediately see impact of changes in the taxonomy, and document movement feedback to see cause and effect of changes without re-indexing.
End users need to access relevant information to their specific role which not only increases productivity, but also contributes to organizational performance. This can be a challenge as content delivered during the search process must address the needs of each individual user, multiple communities, geographies, locations, who all need information that is only relevant to them. Because of the ability to generate compound term metadata that represents a concept, the ambiguity in search is removed. Eliminating the need for often subjective and erroneous metadata, metadata is automatically generated and classified to the taxonomies, where it can be managed and refined. Working with any search engine, the metadata is fed to the search index to enable concept-based search across the organization. Users no longer need to know ‘how to search’, as the combination of the metadata and taxonomies can return relevant information, or guide them to the information they need. Since the products aggregate content from diverse repositories, users are provided with a holistic view of content regardless of where it resides, based on their access rights.
Regardless of whether you are a SharePoint administrator or a CTO, Concept Searching technologies make the management of your metadata easy. Integrated natively with the SharePoint Term Store, Concept Searching platforms remove the manual processes of metadata management, eliminate end user adoption issues, and align IT with business objectives. Installed quickly and easily, you can start classifying content in a matter of hours. Without a taxonomy expert on staff, the interactive, powerful, yet easy-to-use taxonomy tools can be used by business or IT professionals. Organizations can build an enterprise metadata repository, and provide the re-usable technology to any application that requires the use of metadata.
At any level of government, the organization is highly vulnerable to internal and external data privacy exposures and breaches, where repercussions can be costly. The benefits of the Concept Searching platforms is the ability for the organization to define what is confidential information and should be protected, either internally or externally. Standard descriptors, such as social security number are included in the base platforms. Content that is deemed confidential, is easily added as a rule to address specific organizational requirements. Content containing the vocabulary or descriptors is automatically flagged at creation or ingestion and routed to a secure repository, made unavailable during the search process by unauthorized users, and protected against portability.
In records management, automatic declaration of documents of record, elimination of end user tagging, automatic application of content types, and routing of the document of record to the records management application reduces organizational risk, and improves information lifecycle management. Concept Searching product platforms eliminate end user tagging, automatically declares documents of record, automates document workflow for storage, preservation, access, and usage controls, and automatically enforces protection of the records integrity, throughout the individual document lifecycle, eliminating many of the fundamental obstacles faced by records professionals on a day-to-day basis.
For both eDiscovery and FOIA, professional users all need accurate and rapid access to relevant information. The key is to facilitate the search process, and reduce unproductive queries to try to ‘find’ the relevant information. Since content is automatically tagged with relevant, conceptual metadata, it can be retrieved by specifying concepts or descriptors. A significant advantage is the elimination of complex queries and, through vocabulary normalization, ambiguities are eliminated and related content is identified even though the specific vocabulary wasn’t specified in the query.
Governments can cross global boundaries, as well as within the country. Additional security and compliance mandates are critical to the process of managing the compliance challenges associated with unstructured information. The resources and time drain is enormous when staying current with changing mandates, as well as multinational content asset protection. The value of the Concept Searching platforms is the ability to rapidly develop information governance and compliance rules that can immediately identify any compliance violation at the time of content creation or ingestion. These violations can then be flagged and automatically routed to a repository for disposition.