Defense and Intelligence

Mission Support Through Technology Innovation

More so than the commercial sector, the DoD and military face unique challenges. The impact of budgetary constraints requires thoughtful decisions and a strategic plan to modernize the current way of doing business and improve organizational performance. The typical government approach has been to purchase and customize systems for individual programs at each agency. This approach has resulted in information and operations silos that end up constraining government’s efficiency and effectiveness.

Traditional approaches have delivered benefits in the previous environment of large monolithic systems development. These approaches are no longer viable options for today’s environment, where rapid development cycles and capabilities can be quickly be developed and deployed with the understanding that they will interoperate with the organization as a whole. Achieving interoperability in a net-centric environment is fundamental to realizing the full potential of transformation.

Concept Searching’s Smart Content Framework™, product platforms, and intelligent metadata enabled solutions, address and solve content management and information governance challenges with one set of technologies. The products have been deployed in multiple locations within diverse agencies on both the NIPR and SIPR networks including:

  • US Air Force
  • US Army
  • UK Ministry of Defence
  • NATO
  • The German Bundeswehr
  • Development, Concepts and Doctrine, Ministry of Defence Centre, UK

What do all these entities have in common?
The need to transform information into actionable knowledge, and to apply information governance policies across dispersed locations and missions. From an informational perspective, immediate access to up-do-date and accurate information is now a requirement for intelligent decision making. This should be a transparent process and result in a holistic view of knowledge assets regardless of where the information is stored or the location of staff. The DoD and military suffer from silos of redundant data located in a variety of legacy applications and diverse repositories that were designed to support the individual needs and requirements for a specific community of users.

Issues with search, records identification, the potential of internal or external security breaches, migration, and to some extent text analytics are all challenges that surround the inability to manage content. The ‘Cloud-First Mandate’ presents a new set of complicated challenges including retirement of legacy systems and migration to more flexible technologies. These factors impact the mission, degrade decision making, and impede many government entities from moving forward.

How is this Solved?
The Concept Searching product platforms can deliver intelligent metadata enabled solutions for any challenge that requires the use of metadata. These individual challenges can be solved using one set of technologies including:

The still unique technology transparently identifies the meaning of ‘content in context’ enabling the use of rich semantic metadata to be automatically generated, without end user intervention, auto-classified, and managed through highly sophisticated and interactive taxonomy tools.

What are some of the benefits our government clients have achieved?

  • Improved information transparency and utilization of enterprise information
  • Provided immediate and actionable visibility of relevant content from diverse repositories
  • Eliminated potential data security exposures, removed confidential information from unauthorized access, and prohibited portability
  • Automation of records identification and routing to records management application, eliminating end user tagging
  • Enabled intelligent migration to move content based on conceptual metadata, and classify against a taxonomy, providing long term benefits of effective content management and concept based searching
  • Improved FOIA and eDiscovery processing, reducing the time and effort typically required
  • Used to build and support enterprise knowledge portals and accessed via concept based search, regardless of search engine
  • Reduced the noise in aggregated content sets for text analytics, and improved outcomes with the ability to identify concepts, nuances, sentiment from heterogeneous sets of documents

Smart Content Framework™

Transforming relevant information into actionable knowledge has three intuitively significant benefits. For leadership, they are able to rapidly organize their organization’s explicit and implicit knowledge to facilitate more effective communication and decision-making. For staff, cross-functional operating units are able to push relevant information to interested persons, reduce process timeline, utilize untapped resources, and enhance outcome quality. For the organization, it avails contemporary and relevant information that assists and expedites task performance and decision making, advancing individual and group performance via enhanced situational and issue-specific knowledge.

At a basic level, non-compliance with records management policies, lack of metadata for improving search, and increasing volumes of data exposure events are directly related to insufficient metadata. The limiting factor in all of these scenarios is the human element as it relates to behavior and training.

At the crux of all these challenges is metadata. In alignment with the Smart Content Framework™, intelligent metadata enabled solutions can only be achieved through the automatic generation of multi-term metadata, is specific to the various vocabularies and nomenclature of the organization, and serves as the foundation for any application that requires metadata.

This infrastructure framework assists agencies in reducing costs and increasing productivity. It allows knowledge workers to effectively query, use, and reuse agency content improving the speed and efficiency of operations in an environment where information can be shared and leveraged throughout the operations cycle. The elimination of inconsistent tagging and different publishing conventions across multiple content stores provides access to relevant content from internal and external sources.

For the agency, significant benefits can be achieved by removing the ambiguity in content through the identification of concepts within a large corpus of information. Concept Searching’s solutions can be the catalysts to improve access to unstructured information, encourage innovation, and deliver real benefits to defense and intelligence entities, their constituents, and stakeholders.

The Smart Content Framework™ addresses information governance as it applies to unstructured and semi-structured assets. It consists of best practices, technologies, and people. This combination enables the organization to identify key enterprise challenges, develop policy surrounding the challenges, and enforce and measure the results.

This framework assists in defining major organizational risk factors, and results in the tactical changes needed in business processes. The building blocks include:

  • Metadata – the development of an enterprise metadata repository that is flexible to address multiple business challenges with one set of technologies, and provides the ability to apply policy across diverse business groups, repositories, and applications
  • Insight – improves any interactive metadata application such as search, eDiscovery, litigation support, FOIA, text analytics, social tagging, and collaboration, and is used as the data to analyze and quantify risk
  • Risk – defined by staff to determine the organizational risk factors, and effectively identify and validate the specific risk, to analyze cost versus benefit
  • Policy – defined by the organization to implement and enforce policy based on the risk factors
  • Action – fulfills the organizational policies, enforceable and adaptable by end users, and can be quantifiable and able to be measured

The end result is the ability to deploy intelligent metadata enabled solutions that share the same framework for information governance. Instead of recreating the wheel for each challenge, with the framework in place, organizations can become more agile and improve business results more quickly and with less effort.

Industry Challenges

The rate at which technology advancements are being made means that federal agencies that lack the agility to embrace the advances at an equal rate are being left behind. Typically these entities have not kept up with technology capabilities. Challenges embedded in the institutional culture, a cumbersome regulatory and acquisition environment, and legacy applications that require modernization have contributed to the problem. The explosion of unstructured content, the increase in security breaches, the mandate for transitioning to the cloud, stagnant records management practices that continue unabated, increased FOIA and eDiscovery, and to some extent retiring baby boomers, all present on-going challenges to the government.

What are the challenges?

Lack of Information Transparency (FOIA and eDiscovery)

  • Untagged Data Assets equal Untapped Resources. Information cannot be accessed or found because it is either mistagged or not tagged
  • Time gap between information requests and discovery is directly proportional to the volume of data, and the longer it takes to find information the more costly that information becomes
  • People do not “tag” for transparency despite the fact that every federal agency has a “Data Transparency” directive or regulation (e.g.: DoDD 8320 (Net-Centric Data Sharing)

Non-Compliance with Records Management Policies

  • Documents of record are stored in the wrong location
  • Information is not preserved in accordance with regulatory guidelines
  • People do not “tag” with records retention codes, despite the fact that every federal agency has a “Records Management” directive or regulation (e.g.: DoD 5015 (Records Management Program)

Increasing volume of Unplanned Data Exposure Events

  • Every federal agency experiences violations associated with the Privacy Act Program, Federal Information Management Security Act, HIPAA, and Payment Card Industry Guidelines
  • PII, PHI, organizational confidential and sensitive information, all put federal organizations and their support contractors at risk

At a basic level, non-compliance with records management policies, lack of metadata for improving search, and increasing volumes of data exposure events are directly related to insufficient metadata. The limiting factor in all of these scenarios is the human element as it relates to behavior and training.

The fundamental technology problems that can improve the performance of the organization as a whole are delivered via Concept Searching product platforms. The challenges solved include:

  • Search – inability to deliver relevant content in context, up-to-date content, exclude redundant information, and the inability to accommodate multiple search engines
  • Records Management – the inability to accommodate the human factor, typically responsible for tagging and processing documents of record resulting in undeclared records and impacting the information lifecycle of individual pieces of content
  • Security – the inability to define vocabulary and descriptors used for the real-time identification of confidential/data privacy exposures, remove it from unauthorized access, and prevent portability of the content
  • Migration – inability to migrate content based on semantic metadata tagging to simplify pre-migration activities and improve the organization of content post-migration
  • Cloud – inability to provide a consistent enterprise search interface regardless of the platform, unsure how to address on-premise risks that also apply to the cloud, such as security and records identification
  • Text Analytics – inability to reduce the quantity of information to be analyzed, cleanse the content, and find the precise information needed to make informed decisions and gain insight into content

At the crux of all these applications and business process bottlenecks is metadata. The elimination of human metadata tagging, replaced with the automatic generation of multi-term semantic metadata, auto-classification, and taxonomy tools, enormous amounts of time and resources can be saved and allocated to moving the organization forward.


Concept Searching technologies are deployed today across a wide number of organizations both on the SIPR and NIPR networks, as well as the DISA network, and has defense clients with more than 70,000 users accessing a single application globally. Concept Searching also has a current enterprise Authority to Operate (ATO) with the US Air Force and an enterprise Certificate of Networthiness (CoN) for the US Army.

Below are the fundamental challenges our clients have faced followed by information on how they were able to address their challenges.

Download our Challenges and Solutions Collateral to read mini case studies

Information Governance

Information Governance is defined in the Smart Content Framework™ as a five prong process that provides the roadmap to optimize the value of information, while simultaneously minimizing the associated risks and costs. The components consist of Metadata, Insight, Risk, Policy, and Action. The Metadata building block is the development of a single repository of organizationally relevant metadata that is available to any application that requires its use. Insight provides the ability to find and deliver the most relevant and granular results from large, heterogeneous repositories. The third building block, Risk, is determined by the organization to identify high profile risk factors, and analyze the impact and cost for non-compliance. Policy is the response to organizational risk and includes organizational and individual approaches to mitigate or eliminate the risk. Action, the final building block is the execution and interactive management of the policies and subsequent processes that ensures all unstructured and semi-structured content is processed in a manner that achieves the information governance objectives.

Case Study

Information Transparency, Records, and Security
This US Army command organization currently manages a budget of more than $12 billion and cares for more than 1.8 million beneficiaries – active-duty members of all services, retirees and their family members. In an effort to address enterprise issues with information, records, and knowledge management, the organization conducted a Joint Capabilities Integration and Development System (JCIDS) analysis and selected the Concept Searching technology platform to deliver a robust organizationally aligned taxonomy structure for defense healthcare, to significantly improve information transparency, reduce sensitive information breaches, and assist with the preservation and storage of records in line with federal guidelines.


Enterprise search is an infrastructure component and the impact of poor search reaches far beyond the retrieval of information. Reaching out to impact eDiscovery and litigation support, and unauthorized access to confidential information, the results can increase costs as well as organizational risk. A core component in all Concept Searching platforms is the ability to automatically generate compound term metadata, eliminating end user tagging. Identifying relevant content during a search effectively removes ambiguity and enables the retrieval of information based on the concepts within the document, increasing both precision and recall.

Case Study

Achieving Precision Search and Retrieval
This US Army command deployed the conceptClassifier for SharePoint platform within its SharePoint internal portal. The organization plans, conducts, and reports operational tests, assessments, and experiments, in order to provide essential information for the acquisition and fielding of war fighting systems. It implemented the technology to organize and structure its content, and perform auto-classification to the structure in the process, applying metadata that can be filtered upon by the Microsoft Search engine, significantly improving search and retrieval through the ability to search on concepts. The result is greater transparency and improved project collaboration.


Sensitive, confidential, and data privacy information exists in documents, scanned items, faxed items, emails, and could be in any unstructured or semi-structured content. All Concept Searching platforms provide the ability to automatically identify, secure, and prohibit portability of organizationally defined vocabulary and descriptors. Since the metadata generation is not restricted to keywords, it is highly accurate in identifying potential security exposures, in real-time, as content is ingested or created and then routed to a secure repository for disposition. Full support for standard descriptors is included, and additional value is provided by the ability for the organization, or functional group, to define confidential information that falls outside of standard descriptors.

Case Study

Eliminating Security Breaches
The conceptClassifier for SharePoint platform has been deployed at numerous US Air Force bases, to enable compliance with data privacy and security guidelines associated with the Federal Information Security Management Act (FISMA), Privacy Act Program, Health Insurance Portability and Accountability Act (HIPAA), Joint Commission on the Accreditation of Healthcare Organizations (JCAHO), and Payment Card Industry (PCI).The DoD and military have often complicated, multiple levels of security, that must be applied to every piece of content. The flexibility of the technology enables the organization to not only identify standard descriptors included in the base product, but also create their own descriptors and vocabulary to identify any type of potential security breach.

Automatic Identification of Documents of Record

Regardless if you are using the SharePoint Records Center, or a third party application, records management and the lifecycle of documents of record is typically a challenge, in any industry In the DoD and military it is almost overwhelming. The biggest stumbling block has always been end user tagging. With complex file plans, and the fact that human tagging is often haphazard at best, organizations can often unwittingly fall into the trap of non-compliance. The Concept Searching platforms, are being used in records management to automatically identify documents of record, auto-classify the content to a taxonomy that mirrors the file plan, and route directly to the records management application.

Case Study:

Eliminating End User Tagging in Records Management
This US Air Force support organization has a budget of $6.9 billion and runs 75 hospitals and clinics, providing care to over 2.6 million beneficiaries. It deployed conceptClassifier for SharePoint to: increase information retrieval precision on its intranet; enable Subject Matter Experts to develop business rules for information management; eliminate the need to manually meta tag documents; and provide automatic classification of documents and records, based on contextually relevant and domain specific information contained within the bodies of documents. It is currently using the technologies for records management, identification and protection of data privacy, search, and migration.

FOIA and eDiscovery

eDiscovery and Litigation support is costly, time-consuming, and risky. Although the real costs of eDiscovery and litigation support can be estimated, they are often hard to determine, let alone plan for. The best defense is the on-going management of data to ensure preparedness. Unmanaged information carries a great deal of risk as it can be used in unintended way. During the eDiscovery or FOIA process the same basic problem of finding relevant information occurs, which is costly and unproductive. The volumes are also bordering on unprecedented, and facilitating the eDiscovery or FOIA process is a drain on human resources, and carries risk and well as increasing costs.

Case Study

Classifying Terabytes of Data Improves FOIA Processing
This agency deployed conceptClassifier for SharePoint with the Digital Asset Finder™ records management solution from COMPU-DATA International, to enable the proper disposition of documents that had been declared records, based on federal records management policies. Over 20 data sources and databases were consolidated into a few repositories. It currently loads millions of records per day, and automatically classifies these against multiple taxonomies simultaneously. The solution is easily expandable through its modular and scalable architecture. For the professional user, they are able to find highly granular information that typically would not be found, reducing the time, costs, and increases productivity.

Knowledge Portal

Regardless of what term the agency uses, a knowledge resources repository typically exists to capture not only past and current knowledge assets, but also the intellectual assets based on the staff’s expertise and knowledge, for re-use and knowledge sharing. One of the biggest benefits of the Concept Searching platforms is the ability to aggregated content from file shares, SharePoint, web sites, and diverse applications to provide a single point of reference for knowledge workers. Supporting navigational, concept based, and discovery type searches, the user can directly access the information needed, or the user can be guided to relevant content, identifying relationships between content, and offering topics and information that typically would not be found.

Case Study

Improving intelligence and decision making in New Zealand
The increased spotlight on intelligence agencies in the past few years, has highlighted the constant need for agencies to be supplied with accurate and timely information, that enables them to make the best decisions possible. This agency contributes to the security of New Zealand through the provision of foreign intelligence to government, assisting government departments and agencies to protect their electronic information resources and communications systems. To improve these initiatives, the organization rolled out the conceptClassifier for SharePoint platform and conceptTaxonomyWorkflow integrated with SharePoint 2010 and FAST Search, to automatically harvest large amounts of data, classify in real time, identify specific content and take action based upon classification metadata, thereby supporting the intelligence service within New Zealand.

Text Analytics

The ability to aggregate very precise information from diverse sources and then analyze them can be invaluable in decision making and reducing costs. Data is machine driven, whereas unstructured content is driven by people, which makes the nuances, insights, relationships of disparate content, sentiment, and knowledge capital much more difficult to extract. Using Concept Searching product platforms before text analysis, the quantity of content can be reduced through an initial cleansing. Once the ‘noise’ has been eliminated, the ability to analyze the content at a very granular level produces actionable results that can be used to solve problems, reduce costs, and improve decision making.

Case Study

Intelligence – Text Analytics Mining
The UK government department responsible for promoting British interests overseas and supporting UK citizens and businesses around the world deployed the conceptClassifier for SharePoint platform, with the optional Concept Searching search engine, conceptSearch. The department has large data repositories, many of them with sensitive information, and the technology platform, integrated with SharePoint, has enabled the development and maintenance of organizational and mission aligned taxonomies and the classification of large datasets, improving transparency and collaboration on sensitive information.

Intelligent Migration

All Concept Searching clients use the product platforms to migrate content. The US Air Force sites have been using it to migrate to 2010, and now to 2013. To migrate document collections effectively, the text content of each document needs to be searched to determine its value. This cannot be done manually, as the volume is too high, and the consistency of human review and decision making is unreliable as well as costly. Using Concept Searching platforms, an intelligent approach to migration can be achieved. As content is migrated it is analyzed for organizationally defined descriptors and vocabularies, that will automatically classify the content to taxonomies, or optionally the SharePoint Term Store, and automatically apply organizationally defined workflows to process the content to the appropriate repository for review and disposition. Concept Searching migration capabilities include migrating from File Stores to File Stores, File Stores to SharePoint, SharePoint to SharePoint, and provides the ability to define a Custom Action through the conceptTaxonomyWorkflow product.

Case Study

Pre and Post Migration Solutions
The Concept Searching platforms are used in all industries to migrate content to provide the basis for a re-usable enterprise repository to address metadata challenges and enable intelligent metadata enabled solutions. As an example, a professional services client using conceptClassifier for SharePoint identified 66,000 duplicates out of a total of 270,000 documents, representing a 24% reduction in disk space. In another example, a global supplier of automotive parts primary objective was to implement conceptClassifier for SharePoint to improve search for 147,000 business users. The first project was to migrate several millions documents. conceptClassifier for SharePoint was used for the pre and post migration, and for enabling concept based searching integrated with their existing search engine after the migration. The US Air Force has been a client for over twelve years and was one of the first clients to use the platforms for migration, and has successfully used the capabilities in multiple and diverse migration scenarios.


Moving to the cloud can provide a strong return on investment and enable ubiquitous access for end users regardless of where they are located. But moving to a cloud environment, or even a hybrid environment, poses challenges with regards to applying a consistent information governance strategy across all environments. The Concept Searching platforms have been deployed in the cloud, including Office 365 and Amazon AWS, and enables effective management of unstructured content by enforcing and extending an enterprises’ on-premise information governance policies within the cloud environment. Metadata driven policy actions on content used in search, migration, identifying and securing sensitive information, or in the automatic declaration of documents of record, are some of the key capabilities available in a cloud environment. conceptTaxonomyWorkflow, an add-on product, provides the workflow capabilities and is available in all product platforms.

Why Metadata Matters to You


The CIO or CTO in any industry faces both strategic and tactical challenges. In intelligence and military agencies the responsibilities are broad and challenging. The strategic challenges must be carefully addressed with a focus on the impact at a tactical level. Budgetary constraints can change the priorities, and force a ‘pick and choose’ philosophy to improve performance. One of the key differentiators in the Concept Searching platforms is the re-usability of the core technologies to solve multiple content management and business process challenges. This provides a cost benefit in terms of the elimination of separate solutions to address key application challenges, a reduction in staff to implement and maintain the applications, and transparent usability from the end user perspective. The flexibilty of the intelligent metadata enabled solutions can address search, FOIA and eDiscovery, records management, eliminate data privacy and data breaches, provide intelligent migration, social media applications, text analytics capabilities, and over all content management. The ability to leverage the existing infrastructure to improve these challenges assists the CIO/CTO in addressing multiple issues. An additional key component in the Concept Searching platforms is the ability to deploy in an on-premise environment, cloud, or hybrid environment.

Knowledge Managers

The management of knowledge is a significant and on-going task for Knowledge Managers, librarians, or taxonomy specialists. With content continually in a state of flux, the management of knowledge uses a multidimensional approach that must capture, make available, share, and maximize the value of content, to achieve organizational knowledge. Managed effectively, it leverages content assets as well as captures intellectual assets, that ultimately drives innovation. Using conceptTaxonomyManager, a core component in all platforms, the development and management of taxonomies was designed for Subject Matter Experts and provides an easy-to-use interactive interface. The product provides a rules based engine that eliminates the need for training sets and highly specialized human resources, that is unique to conceptTaxonomyManager. Other unique features in the product include automatic taxonomy node clue suggestion, dynamic screen updating to immediately see impact of changes in the taxonomy, and document movement feedback to see cause and effect of changes without re-indexing.

Knowledge Users

End users need to access relevant information to their specific role which not only increases productivity, but also contributes to organizational performance. This can be a challenge as content delivered during the search process must address the needs of each individual user, multiple communities, geographies, locations, who all need information that is only relevant to them. Because of the ability to generate compound term metadata that represents a concept, the ambiguity in search is removed. Eliminating the need for often subjective and erroneous metadata, metadata is automatically generated and classified to the taxonomies, where it can be managed and refined. Working with any search engine, the metadata is fed to the search index to enable concept based search across the organization. Users no longer need to know ‘how to search’, as the combination of the metadata and taxonomies can return relevant information, or guide them to the information they need. Since the products aggregate content from diverse repositories, users are provided with a holistic view of content regardless of where it resides, based on their access rights.

Information Technology Professionals

Regardless if you are a SharePoint Administrator, or the CTO, Concept Searching technologies make your management of metadata easy. Integrated natively with the SharePoint Term Store, Concept Searching platforms remove the manual processes of metadata management, eliminates end user adoption issues, and aligns IT with business objectives. Installed quickly and easily, you can start classifying content in a matter of hours. Without a taxonomy expert on staff, the interactive, powerful, yet easy-to-use taxonomy tools can be used by business or IT professionals. Organizations can build an enterprise metadata repository, and provide the re-usable technology to any application that requires the use of metadata.

Privacy Officer/Security Professionals

At any level of government, the organization is highly vulnerable to internal and external data privacy exposures and breaches, where repercussions can be costly. The benefits of the Concept Searching platforms is the ability for the organization to define what is confidential information and should be protected, either internally or externally. Standard descriptors such as social security number are included in the base platforms. Content that is deemed confidential, is easily added as a rule to address specific organizational requirements. Content containing the vocabulary or descriptors is automatically flagged at creation or ingestion and routed to a secure repository, made unavailable during the search process by unauthorized users, and protected against portability.

Records Managers

In records management, automatic declaration of documents of record, elimination of end user tagging, automatic application of content types, and routing of the document of record to the records management application reduces organizational risk, and improves information lifecycle management. Concept Searching product platforms eliminate end user tagging, automatically declares documents of record, automates document workflow for storage, preservation, access, and usage controls, and automatically enforces protection of the records integrity, throughout the individual document lifecycle eliminating many of the fundamental obstacles faced by records professionals on a day-to-day basis.

eDiscovery and FOIA

For both eDiscovery and FOIA, professional users all need accurate and rapid access to relevant information. The key is to facilitate the search process, and reduce unproductive queries to try to ‘find’ the relevant information. Since content is automatically tagged with relevant, conceptual metadata, it can be retrieved by specifying concepts or descriptors. A significant advantage is the elimination of complex queries and, through vocabulary normalization, ambiguities are eliminated and related content is identified even though the specific vocabulary wasn’t specified in the query.

Compliance Professionals

The government can cross global boundaries, as well as within the country. Additional security and compliance mandates are critical to the process of managing the compliance challenges associated with unstructured information. The resources and time drain is enormous to stay current in regards to, changing mandates, as well as multinational content asset protection. The value of the Concept Searching platforms is the ability to rapidly develop information governance and compliance rules that can immediately identify any compliance violation at the time of content creation or ingestion. These violations can then be flagged and automatically routed to a repository for disposition.