Intelligent Metadata Enabled Solutions for Defense and Intelligence

Information transparency, compliance with federal records management mandates, and information security represent major disruptions that must be addressed to achieve modernization. Without access to highly accurate and relevant information, the mission is bound to fail.

Sector White Paper USAFMS Case Study  Request a Demo

  • OverviewChallenges and SolutionsUse CasesJob FunctionKnowledge Center

    Achieving interoperability in a net-centric environment

    The DoD and military face unique challenges. The impact of budget constraints and the push for modernization and cloud use can’t be ignored. Technology continues to improve while siloed repositories, lack of information transparency, and inadequate security continue to hamper success. Achieving interoperability requires a cultural change as well as the introduction of technologies that are fundamental in achieving the full potential of transformation.

  • The Concept Searching Difference

  • enterprise-search-information-transparency

    Information Transparency


    Compliance with Federal Records Management Mandates


    Eliminate Data Exposure Events


    Eliminate Manual Tagging


    Control Access and Distribution of Information

    Concept Searching delivers an enterprise framework to mitigate risk, automate processes, securely manage information, ensure privacy, and address compliance issues, enabling enterprise information governance. This is accomplished at a basic level with the ability to transparently tag content, classify it to organizational taxonomies, preserve and protect information through the automatic identification of records and privacy data, and act as a migration tool in any environment.


    Are you concerned you are compliant with all federal programs relating to the protection of privacy and confidential assets?


    Are you confident that all records have been correctly tagged and preserved according to regulatory guidelines and federal mandates?


    Is information transparency hampered due to inaccessible, siloed repositories and poor search results?


    Are you concerned about the rising cost of finding information, as a result of documents being mistagged or not tagged at all?


    Are you concerned about information dissemination by role, inter-agency, ITAR compliance, and forensic analysis?
    Key Features

    Key Features

    Concept Searching technologies serve multiple purposes and are platform independent. The end result is a metadata framework that provides flexibility, and enables an organization to implement governance policies and operations incrementally, with each step directly correlated to mission value.

    Core Technology Technology Benefits
    • Automatic generation of compound term metadata as it is created or ingested, eliminating end user tagging
    • Provides the conceptual metadata to any search engine index to improve findability, with real-time classification of content from internal and external repositories
    • Automatic assignment of records retention codes and updating of content types based on file plan
    • Identifies location of sensitive information – PII, PHI, confidential information – within a document, and migrates it to a secure location
    • Secures information access internally and externally
    • Provides taxonomy testing, validation, and management with industry unique features
    • Concept-based searching improves accessibility to information that may not have been found and the understanding of the information presented
    • Provides navigational aids to subject/topic to direct the user to the right information in the context they are seeking, and supports multiple search methods with no training required
    • Data security through the identification of sensitive information – PII, PHI, OPSEC – and controlling both access to and distribution of sensitive organizational information
    • Ensures compliance with federal records management mandates, auto-tagging content and migrating to a records platform for storage, preservation and disposition
    • Semantic metadata is captured at source and automatically classified to the enterprise taxonomies to enable enterprise policy management and information governance
    • Flexible technologies that solve metadata management, improve search, records management, compliance, data privacy, migration, and text analytics with a single solution
    Key Benefits

    Key Benefits

    Deployed on the Microsoft and AWS government clouds, on the NIPR and SIPR and DISA networks, and FedRAMP compliant. Available on GSA Schedule 70, and SEWP IV contract vehicles, and holds a Current CON with the US Army and an ATO with the Air Force.

    Client Use Cases RMDA Case Study

    Challenges and Solutions

    The rate at which technology advancements are being made means that federal agencies that lack the agility to embrace the advances at an equal rate are being left behind. Typically these entities have not kept up with technology capabilities. Challenges embedded in the institutional culture, a cumbersome regulatory and acquisition environment, and legacy applications that require modernization have contributed to the problem. The explosion of unstructured content, the increase in security breaches, the mandate for transitioning to the cloud, stagnant records management practices that continue unabated, increased FOIA and eDiscovery, and to some extent retiring baby boomers, all present ongoing challenges to the government.

    What are the Challenges?

    Lack of Information Transparency (FOIA and eDiscovery)

    • Untagged data assets equal untapped resources – information cannot be accessed or found because it is either mistagged or not tagged
    • Time gap between information requests and discovery is directly proportional to the volume of data, and the longer it takes to find information the more costly that information becomes
    • People do not ‘tag’ for transparency, despite the fact that every federal agency has a ‘data transparency’ directive or regulation – DoDD 8320, net-centric data sharing

    Noncompliance with Records Management Policies

    • Documents of record are stored in the wrong location
    • Information is not preserved in accordance with regulatory guidelines
    • People do not ‘tag’ with records retention codes, despite the fact that every federal agency has a records management directive or regulation – DoD 5015,  records management program

    Increasing Volume of Data Exposure Events

    • Every federal agency experiences violations associated with the Privacy Act Program, Federal Information Management Security Act, HIPAA, and payment card industry guidelines
    • PII, PHI, organizational confidential, and sensitive information all put federal organizations and their support contractors at risk

    At a basic level, noncompliance with records management policies, lack of metadata for improving search, and increasing volumes of data exposure events are directly related to insufficient metadata. The limiting factor in all these scenarios is the human element, as it relates to behavior and training.

    The fundamental technology problems that can improve the performance of the organization as a whole are addressed via Concept Searching product platforms. The challenges solved include:

    • Search – inability to deliver relevant content in context, up-to-date content, exclude redundant information, and the inability to accommodate multiple search engines
    • Records Management – the inability to accommodate the human factor, typically responsible for tagging and processing documents of record, resulting in undeclared records and impacting the information lifecycle of individual pieces of content
    • Security – the inability to define vocabulary and descriptors used for the real-time identification of confidential/data privacy exposures, remove it from unauthorized access, and prevent portability of the content
    • Migration – inability to migrate content based on semantic metadata tagging to simplify pre-migration activities and improve the organization of content post-migration
    • Hybrid Search – inability to provide a consistent enterprise search interface regardless of the platform, unsure how to address on-premise risks that also apply to the cloud, such as security and records identification
    • Text Analytics – inability to reduce the quantity of information to be analyzed, cleanse the content, and find the precise information needed to make informed decisions and gain insight into content

    At the crux of all these applications and business process bottlenecks is metadata. The elimination of human metadata tagging, replaced with the automatic generation of multi-term semantic metadata, auto-classification, and taxonomy tools, enormous amounts of time and resources can be saved and allocated to moving the organization forward.

    Use Cases

    Concept Searching technologies are deployed today across a wide number of organizations on both the SIPR and NIPR networks, as well as the DISA network, and has defense clients with more than 70,000 users accessing a single application globally. Concept Searching also has a current enterprise Authority to Operate (ATO) with the US Air Force and an enterprise Certificate of Networthiness (CoN) for the US Army.

    Below are the fundamental challenges our clients have faced, followed by information on how they were able to address their challenges.

  • Information Governance

    Information Governance is defined in the Smart Content Framework™ as a five prong process that provides the roadmap to optimize the value of information, while simultaneously minimizing the associated risks and costs.

    The components consist of Metadata, Insight, Risk, Policy, and Action. The Metadata building block is the development of a single repository of organizationally relevant metadata that is available to any application that requires its use. Insight provides the ability to find and deliver the most relevant and granular results from large, heterogeneous repositories. The third building block, Risk, is determined by the organization to identify high profile risk factors, and analyze the impact and cost for non-compliance. Policy is the response to organizational risk and includes organizational and individual approaches to mitigate or eliminate the risk. Action, the final building block is the execution and interactive management of the policies and subsequent processes that ensures all unstructured and semi-structured content is processed in a manner that achieves the information governance objectives.

    Case Study

    Information Transparency, Records, and Security

    document-downloadsThis US Army command organization currently manages a budget of more than $12 billion and cares for more than 1.8 million beneficiaries – active-duty members of all services, retirees and their family members. In an effort to address enterprise issues with information, records, and knowledge management, the organization conducted a Joint Capabilities Integration and Development System (JCIDS) analysis and selected the Concept Searching technology platform to deliver a robust organizationally aligned taxonomy structure for defense healthcare, to significantly improve information transparency, reduce sensitive information breaches, and assist with the preservation and storage of records in line with federal guidelines.

  • Case Study

    Achieving Precision Search and Retrieval

    document-downloadsThis US Army command deployed the conceptClassifier for SharePoint platform within its SharePoint internal portal. The organization plans, conducts, and reports operational tests, assessments, and experiments, in order to provide essential information for the acquisition and fielding of war fighting systems. It implemented the technology to organize and structure its content, and perform auto-classification to the structure in the process, applying metadata that can be filtered upon by the Microsoft Search engine, significantly improving search and retrieval through the ability to search on concepts. The result is greater transparency and improved project collaboration.


    Enterprise search is an infrastructure component and the impact of poor search reaches far beyond the retrieval of information.

    Reaching out to impact eDiscovery and litigation support, and unauthorized access to confidential information, the results can increase costs as well as organizational risk. A core component in all Concept Searching platforms is the ability to automatically generate compound term metadata, eliminating end user tagging. Identifying relevant content during a search effectively removes ambiguity and enables the retrieval of information based on the concepts within the document, increasing both precision and recall.

  • Security

    Sensitive, confidential, and data privacy information exists in documents, scanned items, faxed items, and emails that could be in any unstructured or semi-structured content.

    All Concept Searching platforms provide the ability to automatically identify, secure, and prohibit portability of organizationally defined vocabulary and descriptors. Since the metadata generation is not restricted to keywords, it is highly accurate in identifying potential security exposures, in real time, as content is ingested or created and then routed to a secure repository for disposition. Full support for standard descriptors is included, and additional value is provided by the ability for the organization, or functional group, to define confidential information that falls outside of standard descriptors.

    Case Study

    Eliminating Security Breaches

    document-downloadsThe conceptClassifier for SharePoint platform has been deployed at numerous US Air Force bases, to enable compliance with data privacy and security guidelines associated with the Federal Information Security Management Act (FISMA), Privacy Act Program, Health Insurance Portability and Accountability Act (HIPAA), Joint Commission on the Accreditation of Healthcare Organizations (JCAHO), and Payment Card Industry (PCI).The DoD and military have often complicated, multiple levels of security, that must be applied to every piece of content. The flexibility of the technology enables the organization to not only identify standard descriptors included in the base product, but also create their own descriptors and vocabulary to identify any type of potential security breach.

  • Case Study

    Eliminating End User Tagging in Records Management

    This US Air Force support organization has a budget of $6.9 billion and runs 75 hospitals and clinics, providing care to over 2.6 million beneficiaries. It deployed conceptClassifier for SharePoint to: increase information retrieval precision on its intranet; enable subject-matter experts to develop business rules for information management; eliminate the need to manually meta tag documents; and provide automatic classification of documents and records, based on contextually relevant and domain specific information contained within the bodies of documents. It is currently using the technologies for records management, identification and protection of data privacy, search, and migration.

    Automatic Identification of Documents of Record

    Regardless of whether you are using the SharePoint Records Center or a third party application, records management and the lifecycle of documents of record are typically challenging, in any industry.

    In the DoD and military it is almost overwhelming. The biggest stumbling block has always been end user tagging. With complex file plans, and the fact that human tagging is often haphazard at best, organizations can often unwittingly fall into the trap of noncompliance. The Concept Searching platforms, are being used in records management to automatically identify documents of record, auto-classify the content to a taxonomy that mirrors the file plan, and route directly to the records management application.

  • FOIA and eDiscovery

    eDiscovery and litigation support is costly, time-consuming, and risky. Although the real costs of eDiscovery and litigation support can be estimated, they are often hard to determine, let alone plan for.

    The best defense is the ongoing management of data to ensure preparedness. Unmanaged information carries a great deal of risk, as it can be used in unintended way. During the eDiscovery or FOIA process the same basic problem of finding relevant information occurs, which is costly and unproductive. The volumes can be unprecedented, and facilitating the eDiscovery or FOIA process is a drain on human resources and carries risk and well as increasing costs.

    Case Study

    Classifying Terabytes of Data Improves FOIA Processing

    document-downloadsThis agency deployed conceptClassifier for SharePoint with the Digital Asset Finder™ records management solution from COMPU-DATA International, to enable the proper disposition of documents that had been declared records, based on federal records management policies. Over 20 data sources and databases were consolidated into a few repositories. It currently loads millions of records per day, and automatically classifies these against multiple taxonomies simultaneously. The solution is easily expandable through its modular and scalable architecture. Professional users are able to find highly granular information that typically would not be found, reducing the time, costs, and increases productivity.

  • Case Study

    Improving intelligence and decision making in New Zealand

    document-downloadsThe increased spotlight on intelligence agencies in the past few years, has highlighted the constant need for agencies to be supplied with accurate and timely information, that enables them to make the best decisions possible. This agency contributes to the security of New Zealand through the provision of foreign intelligence to government, assisting government departments and agencies to protect their electronic information resources and communications systems. To improve these initiatives, the organization rolled out the conceptClassifier for SharePoint platform and conceptTaxonomyWorkflow integrated with SharePoint 2010 and FAST Search, to automatically harvest large amounts of data, classify in real time, identify specific content and take action based upon classification metadata, thereby supporting the intelligence service within New Zealand.

    Knowledge Portal

    Regardless of what term an agency uses, a knowledge resources repository typically exists to capture not only past and current knowledge assets, but also the intellectual assets based on the staff’s expertise and knowledge, for re-use and knowledge sharing.

    One of the biggest benefits of the Concept Searching platforms is the ability to aggregated content from file shares, SharePoint, websites, and diverse applications to provide a single point of reference for knowledge workers. Supporting navigational, concept-based, and discovery type searches, the user can directly access the information needed, or can be guided to relevant content, identifying relationships between content, and offering topics and information that typically would not be found.

  • Text Analytics

    The ability to aggregate very precise information from diverse sources and then analyze them can be invaluable in decision making and reducing costs.

    Data is machine driven, whereas unstructured content is driven by people, which makes the nuances, insights, relationships of disparate content, sentiment, and knowledge capital much more difficult to extract. Using Concept Searching product platforms before text analysis the quantity of content can be reduced through an initial cleansing. Once the ‘noise’ has been eliminated, the ability to analyze the content at a very granular level produces actionable results that can be used to solve problems, reduce costs, and improve decision making.

    Case Study

    Intelligence – Text Analytics Mining

    document-downloadsThe UK government department responsible for promoting British interests overseas and supporting UK citizens and businesses around the world deployed the conceptClassifier for SharePoint platform, with the optional Concept Searching search engine, conceptSearch. The department has large data repositories, many of them with sensitive information, and the technology platform, integrated with SharePoint, has enabled the development and maintenance of organizational and mission aligned taxonomies and the classification of large data sets, improving transparency and collaboration on sensitive information.

  • Case Study

    Pre and Post Migration Solutions

    document-downloadsThe Concept Searching platforms are used in all industries to migrate content to provide the basis for a re-usable enterprise repository to address metadata challenges and enable intelligent metadata enabled solutions. As an example, a professional services client using conceptClassifier for SharePoint identified 66,000 duplicates out of a total of 270,000 documents, representing a 24% reduction in disk space. In another example, a global supplier of automotive parts primary objective was to implement conceptClassifier for SharePoint to improve search for 147,000 business users. The first project was to migrate several millions documents. conceptClassifier for SharePoint was used for the pre and post migration, and for enabling concept based searching integrated with their existing search engine after the migration. The US Air Force has been a client for over twelve years and was one of the first clients to use the platforms for migration, and has successfully used the capabilities in multiple and diverse migration scenarios.

    Intelligent Migration

    All Concept Searching clients use the product platforms to migrate content. The US Air Force sites have been using it to migrate to 2010, and now to 2013.

    To migrate document collections effectively, the text content of each document needs to be searched to determine its value. This cannot be done manually, as the volume is too high, and the consistency of human review and decision making is unreliable as well as costly. Using Concept Searching platforms, an intelligent approach to migration can be achieved. As content is migrated it is analyzed for organizationally defined descriptors and vocabularies, that will automatically classify the content to taxonomies, or optionally the SharePoint Term Store, and automatically apply organizationally defined workflows to process the content to the appropriate repository for review and disposition. Concept Searching migration capabilities include migrating from file stores to file stores, file stores to SharePoint, SharePoint to SharePoint, and provide the ability to define a custom action through the conceptTaxonomyWorkflow product.

  • Cloud

    Moving to the cloud can provide a strong return on investment and enable ubiquitous access for end users, regardless of where they are located.

    But moving to a cloud environment, or even a hybrid environment, poses challenges with regards to applying a consistent information governance strategy across all environments. The Concept Searching platforms have been deployed in the cloud, including Office 365 and Amazon Web Services, and enables effective management of unstructured content by enforcing and extending an enterprises’ on-premises information governance policies within the cloud environment. Metadata driven policy actions on content used in search, migration, identifying and securing sensitive information, or in the automatic declaration of documents of record, are some of the key capabilities available in a cloud environment. conceptTaxonomyWorkflow, an add-on product, provides the workflow capabilities and is available in all product platforms.

    Why Metadata Matters to You


    The CIO or CTO in any industry faces both strategic and tactical challenges. In intelligence and military agencies, the responsibilities are broad and challenging. The strategic challenges must be carefully addressed with a focus on the impact at a tactical level. Budgetary constraints can change the priorities, and force a ‘pick and choose’ philosophy to improve performance. One of the key differentiators in the Concept Searching platforms is the re-usability of the core technologies to solve multiple content management and business process challenges. This provides cost benefits in terms of the elimination of separate solutions to address key application challenges, a reduction in staff to implement and maintain the applications, and transparent usability from the end user perspective. The flexibility of the intelligent metadata enabled solutions can address search, FOIA and eDiscovery, records management, eliminate data privacy and data breaches, provide intelligent migration, social media applications, text analytics capabilities, and over all content management. The ability to leverage the existing infrastructure to improve these challenges assists the CIO/CTO in addressing multiple issues. An additional key component in the Concept Searching platforms is the ability to deploy in an on-premise environment, cloud, or hybrid environment.

    Knowledge Managers

    The management of knowledge is a significant and ongoing task for knowledge managers, librarians, or taxonomy specialists. With content continually in a state of flux, the management of knowledge uses a multidimensional approach that must capture, make available, share, and maximize the value of content, to achieve organizational knowledge. Managed effectively, it leverages content assets as well as captures intellectual assets, that ultimately drives innovation. Using conceptTaxonomyManager, a core component in all platforms, the development and management of taxonomies was designed for subject-matter experts and provides an easy-to-use interactive interface. The product provides a rule-based engine that eliminates the need for training sets and highly specialized human resources, that is unique to conceptTaxonomyManager. Other unique features in the product include automatic taxonomy node clue suggestion, dynamic screen updating to immediately see impact of changes in the taxonomy, and document movement feedback to see cause and effect of changes without re-indexing.

    Knowledge Users

    End users need to access relevant information to their specific role which not only increases productivity, but also contributes to organizational performance. This can be a challenge as content delivered during the search process must address the needs of each individual user, multiple communities, geographies, locations, who all need information that is only relevant to them. Because of the ability to generate compound term metadata that represents a concept, the ambiguity in search is removed. Eliminating the need for often subjective and erroneous metadata, metadata is automatically generated and classified to the taxonomies, where it can be managed and refined. Working with any search engine, the metadata is fed to the search index to enable concept-based search across the organization. Users no longer need to know ‘how to search’, as the combination of the metadata and taxonomies can return relevant information, or guide them to the information they need. Since the products aggregate content from diverse repositories, users are provided with a holistic view of content regardless of where it resides, based on their access rights.

    Information Technology Professionals

    Regardless of whether you are a SharePoint administrator or a CTO, Concept Searching technologies make the management of your metadata easy. Integrated natively with the SharePoint Term Store, Concept Searching platforms remove the manual processes of metadata management, eliminate end user adoption issues, and align IT with business objectives. Installed quickly and easily, you can start classifying content in a matter of hours. Without a taxonomy expert on staff, the interactive, powerful, yet easy-to-use taxonomy tools can be used by business or IT professionals. Organizations can build an enterprise metadata repository, and provide the re-usable technology to any application that requires the use of metadata.

    Privacy Officer/Security Professionals

    At any level of government, the organization is highly vulnerable to internal and external data privacy exposures and breaches, where repercussions can be costly. The benefits of the Concept Searching platforms is the ability for the organization to define what is confidential information and should be protected, either internally or externally. Standard descriptors, such as social security number are included in the base platforms. Content that is deemed confidential, is easily added as a rule to address specific organizational requirements. Content containing the vocabulary or descriptors is automatically flagged at creation or ingestion and routed to a secure repository, made unavailable during the search process by unauthorized users, and protected against portability.

    Records Managers

    In records management, automatic declaration of documents of record, elimination of end user tagging, automatic application of content types, and routing of the document of record to the records management application reduces organizational risk, and improves information lifecycle management. Concept Searching product platforms eliminate end user tagging, automatically declares documents of record, automates document workflow for storage, preservation, access, and usage controls, and automatically enforces protection of the records integrity, throughout the individual document lifecycle, eliminating many of the fundamental obstacles faced by records professionals on a day-to-day basis.

    eDiscovery and FOIA

    For both eDiscovery and FOIA, professional users all need accurate and rapid access to relevant information. The key is to facilitate the search process, and reduce unproductive queries to try to ‘find’ the relevant information. Since content is automatically tagged with relevant, conceptual metadata, it can be retrieved by specifying concepts or descriptors. A significant advantage is the elimination of complex queries and, through vocabulary normalization, ambiguities are eliminated and related content is identified even though the specific vocabulary wasn’t specified in the query.

    Compliance Professionals

    Governments can cross global boundaries, as well as within the country. Additional security and compliance mandates are critical to the process of managing the compliance challenges associated with unstructured information. The resources and time drain is enormous when staying current with changing mandates, as well as multinational content asset protection. The value of the Concept Searching platforms is the ability to rapidly develop information governance and compliance rules that can immediately identify any compliance violation at the time of content creation or ingestion. These violations can then be flagged and automatically routed to a repository for disposition.