Document Capture & Analytics for Developers

Using Web Services as a Hidden Automation Layer

The benefits of intelligent document capture are well documented, and its impact on efficiency can be a quick win for any business processing a significant volume of documents. Next-generation capture adds a new dimension of automation, and provides capture functionality to any application or system. So now, the ERP, Copier, iPad, CRM or Document Management System can be capture-enabled through its own interface, without the need to switch windows, open a new application or “send files” to a processing location. This “invisible” automation layer requires no end-user expertise, and as far as they know, they are doing business as usual.

Below are some core benefits to leveraging Capture as a Service (CaaS):

Minimal Impact to Operations

Current process: the end-user uploads a received file through the CRM interface, and then enters some notes and metadata about the file: customer name, date of contract, salesperson, and region. Capture as a Service enabled process: the end-user uses the same upload process, but in the background, the invisible capture automation layer classifies the document, extracts all the data and enters it into the CRM automatically. This is the power of document capture web services. With no impact to current process or operations, you gain efficiency and reduce errors, driving speed of transactions, reduced response times and the costly fixing of erroneous data.

Salesforce Upload Classification and Extraction

Low to No Training

Because end users are using the applications they use every day, there is almost zero training required. That’s the beauty of the invisible capture layer. If deployed correctly, users just perform their process the same way, and manual data entry and steps are totally eliminated. The lack of a training requirement minimizes any lost work days spent on costly training, and provides value immediately, starting day one.

Workflow Enhancement

With capture web services, you can add intelligence to any existing workflow product. Just about every workflow engine in the market provides simple, no code hooks into any RESTful web service. With Ephesoft’s web services, you have both macro and micro functions ready. Need to read a barcode? Use a micro service. Need to classify, extract and OCR? Use a macro processor. Capture becomes an efficiency booster to any document-centric process.

GDPR SharePoint Workflow in Microsoft Flow

Solve the Plague of Windows

More and more, IT and Business staff alike are looking to streamline and consolidate, and reduce the number of applications required to do business. Creating that single interactive interface is the end goal. And a web services automation layer can provide functionality that would normally require the addition of an interactive app.

Maximum Efficiency

It is said that 50% of document-intensive process labor is spent on 5% of your documents. Why? The cost of problems and fixing errors. Automating any data extraction and document classification process, coupled with data validation techniques, and reduce errors to almost 0, and drive the maximum efficiency possible in your organization.

Just a brief note on some thoughts and trends I am seeing in the marketplace. Thoughts?

If you want to see Ephesoft Transact Web Services in action, take a peek at Ephesoft Transact Web Services API.


6 Trends in Intelligent Document Capture

6 Trends in Intelligent Document Capture

The New Requirements for Capture 2.0 in the Enterprise

In the early 2000’s, the Nokia 1110 was the most popular cell phone on the planet.  It was a great, functional phone at the time, but within a few years, it was obsolete.  If you forced your users to work with that phone today, you would have a mass rebellion.  So why do organizations still use legacy technology from the same decade for document automation and capture?  The answer: Many are quickly moving to modern, newly designed technology, and reaping great benefits.  For organizations looking to capitalize on the enhanced efficiency and productivity modern intelligent capture can provide, here are some trends within the broader capture and document analytics market.

 

Web scan browser

  1. The Browser is the UI of Choice

    As organizations look to refresh their capture technology and migrate to modern platforms, the thick client is going the way of the Dodo.  Web-enabled applications provide simple access through any of the major browsers, ease the burden on IT staff from a deployment and management perspective and allow an easy transition to the cloud.  Providing both end-user and administrator access through a web app, also allows for broad reaching distributed capture solutions, a must in today’s global, connected Enterprise.

OCR API web service

  1. RESTful Web Service APIs are now King (and Queen)

    As organizations drive for maximum efficiency and productivity, capture-enabled line of business systems are the weapon of choice.  From the ERP solution that auto-classifies a document on upload, to the workflow app that uses data extraction to make branching decisions, “transparent” capture that just happens, offers a seamless way to leverage the power of document automation.  In addition, two types of APIs are on the street:

    • Macro-services -  APIs that perform multiple tasks provide efficiency in code, and allow single calls to process from start to finish.  These all in one services will classify, extract data and return a searchable PDF, and in one swoop.
    • Micro-services – having an expansive toolkit available, through web services, provides an all-encompassing document-centric platform. This platform can service end users through the browser, provide broad-reaching macro functionality where needed, but also provide niche solutions.  Examples might include the ability to convert an image to searchable PDF or read a barcode from a photo.

Cloud document scanning

  1. True Cloud is necessary

    Being cloud-ready means quite a bit more than just running a VM with software in the cloud. A platform that is built from the ground up to take advantage of the key benefits of the cloud:

    • Automatic Scale-up/Scale-down
    • Native Web Browser support
    • Cluster-awareness
    • Support for Cloud DB architectures
    • Core-based licensing

    Having a structured, tiered offering that can support all types of organizations and usage scenarios is also key.

Open source OCR extraction

  1. Open, modular architectures

    Just like the fat client, the locked down, proprietary architecture of the past is over and done.  CIOs are demanding open interfaces in all their apps for ease of integration, and standard software languages for extending functionality.  Modularity with an open “plugin” type architecture allows administrators and developers to only use what they need, when they need it.

OCR machine learning

  1. Machine learning provides the ultimate in efficiency

    Basic pattern matching and templates are long gone, and have outlived their useful days.  OCR and provided text are now just input for the intelligent, learning application.  The emphasis is now on automated learning, not only during configuration, but also supervised learning provided by the end user/knowledge worker.  The system gets smarter as you use it, always working towards zero end-user interaction.

Document analytics

  1. Companion analytics

    There is so much information that is buried within an organization’s document content.  Providing that content in an “analytics ready” state optimizes processing, and provides ready access to “dark” data.  Today’s modern capture platforms can feed the big data beast, and provide powerful, once hidden information to those that can benefit in the organization.

The last two points are the future of the market, and forward-looking companies are focusing on vendors that provide a vision and path to take full advantage of the unstructured information that lies within document content.


Boxworks Primer: Machine Learning and Analytics with Ephesoft

Automating Box with Ephesoft

As we prepare for our first BoxWorks event, I wanted to put a concise and clear outline together on the advantages and key use case for Ephesoft Technology when coupled with Box. There are 3 key questions you need to ask yourself when working with Box at a strategic level:

  1. How do I add content and associated metadata with the least amount of effort?
  2. How can I find the content I need quickly and easily?
  3. How do I glean value from the dark data that lies within my repository?

In the following sections, I will outline how Ephesoft addresses each of these key questions, and can provide enormous value to any Box repository.

Using Intelligent Capture to Add Content to Box

At its most basic level, Ephesoft Transact is an onramp for adding documents to Box. You can ingest from the following sources:

  • Scanners (Browser-based scanning)
  • MFPs/Copiers
  • Network Folders
  • Email (Body and Attachments)
  • Mobile (Through our SDK and App)
  • Legacy ECM systems (CMIS import)
  • Custom code (RESTful Web Service)

Import from a variety of sources is just the first step. Once imported, documents pass through our analytics and machine-learning engine and are classified, and then data extraction rules are applied. This extracted data is mapped to Box metadata fields/attributes. Along with our application, macro and micro services are also available for a broad variety of functions, and can accessed through code in Box, or other applications.

Box document extraction

Document Metadata in Box

Enhancing Search with OCR and Metadata

Ephesoft Transact can provide Optical Character Recognition (OCR) for Box, and provide the converted text in a number of ways:

  • All documents from an Ephesoft process are full text, searchable PDFs.
  • As mentioned, extracted metadata to Box metadata mapping

Box OCR search PDFs

Enhancing Box Search Capabilities with Searchable PDFs

Adding searchable attributes, along with providing a searchable PDF from image, allows broad flexibility in search, and insures you can find exactly what you need. To go a little deeper, Ephesoft has extensive data extraction features that can service a number of unique use cases. Here are some examples:

  • Line item extraction for AP Invoice processing
  • Paragraph extraction for contracts processing
  • Cross section extraction for financial statements
  • Signature detection for any signed document

This extraction capability allows Ephesoft to be leveraged beyond just the repository, and it can be a solution for complex data extraction needs.

Unstructured to Structured: Harvesting Data from Large Volumes of Documents

It is estimated that over 80% of the data in an organization is unstructured, and this inaccessible data can create big problems for any size business. It is estimated that poor data, and lack of structure can cost a business 20-30% of their operating revenue (FATHOM – “Big Data” Facts and Statistics That Will Shock You). Ephesoft Insight can help you tame the unstructured data in the form of documents. In essence, Insight is a big data technology that can crawl multiple, disparate content repositories. Through patented machine learning and analytics algorithms, models can be built not only to classify documents and extract data, but also to visualize, correlate and create a data map for your documents. From a Box perspective, Insights crawlers can harvest your Box content, and create meaning through machine learning. So, here are some example uses:

  • GDPR Discovery
  • Anti-money Laundering and Fraud Detection
  • Know Your Customer (KYC) Analysis
  • Contract Analysis

Box OCR Analytics

Document Relationship Mapping in Insight

So, all in all, Ephesoft solutions reduce labor, eliminate manual data entry, improve efficiency and productivity, and help you tame your unstructured data. When tied into Box, the sky is the limit when you use our platform to enhance your document workflows and analyze your content.

Ephesoft is an exhibitor at BoxWorks this week. Stop by Booth 31 for more information and an overview of our solutions.


GDPR

A Document-Centric Strategy for GDPR Compliance

With the effective date for the new General Data Protection Regulation (GDPR) fast approaching, now is the time to put in a solid strategy when it comes to documents and images. Organizations not only need to implement process and procedure for handling private information, but also need a firm evaluation of "current state" to understand high risk areas of their business and their understand their exposure. Below are the four key steps, as outlined by Microsoft's GDPR Strategy, and how you can incorporate a document-centric view within your plan:

Discover

Discovery will probably be the most challenging step when it comes to documents and GDPR. When it comes to the enterprise, the majority has many document repositories. Just think of the modern workplace and all the locations where documents reside:

  • Network folders
  • Local folders
  • Sync technologies like Box, OneDrive, Dropbox, Google Drive
  • Corporate Enterprise Content Management (ECM) and Document Management (DM) systems
  • Line of Business systems that house documents
  • Emails and attachments

The ability to crawl and identify high risk entities within these locations is critical for compliance. Here is a checklist of required functionality when it comes to a technical solution:

  • Two-phase Identification - most of the technologies on the market just use pattern matching to identify personal information within documents. This can be problematic, and burden staff with false-positives, and require immense time requirements to validate. With two-phase identification systems (like Ephesoft), documents are first classified as a specific type: agreement, application, correspondence, etc. This classification can be configured for an organization's specific document requirements, and can immediately ID a document as high risk. The second phase of risk identification is pattern matching, fuzzy DB correlation and key value searching. This two-phase approach is required for accuracy and high confidence.
  • Optical Character Recognition (OCR) - images can be a very high risk type of document. To properly evaluate an image for risk, there needs to be a text conversion process. It goes much further than that, the application also needs a voting and confidence engine. Images vary in quality, and a fax or "copy of a copy" can be problematic. With a confidence flag on both the overall document and identified private information, images can be graded on overall quality, and quality of data.
  • Open Architecture - proprietary systems cannot meet all the requirements that will be necessary for GDPR Discovery, and most organizations will need ultimate flexibility to modify and customize software for their unique needs and requirements. Using modular and open platforms will guarantee the best solution and fit for your needs.
  • Machine Learning - using a system that gets smarter with each day of use is required in today's modern world. A data capture solution with machine learning capabilities can learn new
  • high risk documents, and evolve as an organization changes. Using ML tools can help organizations detect and access data that is at risk with GDPR.

Manage

Once a GDPR document inventory is complete and an organization understands their areas of document risk and exposure, a plan can be put in place to manage and govern the assets of their data subjects. This phase or step within your GDPR document strategy can include the following:

  • Migrating high risk documents to a managed repository - if high risk documents exist outside of a governed and managed repository, the same tool that can help in discovery can also help with migration. As documents are classified, metadata can also be extracted, and the document moved into a new or existing system of record. You can see an example of contract migration to SharePoint online here: Migrating Contracts and Data to SharePoint.
  • Implementing an intelligent document transport layer - creating a repeatable, standardized process for document ingestion and processing can flag new documents as they enter an organization's digital realm. This insures proper governance, and placement of high risk assets.

Protect

In the protection step, organizations need to put security controls on all documents deemed as high risk. But the protection step also requires thought on future documents, and protecting new private assets. As outlined in the "Manage" section above, an effective document transport technology will identify and route newly ingested documents to a protected location. Organizations also need to implement real-time controls for high risk identification and classification. Here are some examples:

  • Continuously discover - you can protect those documents that are in your managed repository, but what about newly generated personal data? As new policies and procedures are implemented, organizations need to use their discovery technology to constantly monitor and find new high risk entities.
  • Embed classification technology - enabling detection in your everyday applications can reduce risk, and insure compliance. Modern classification platforms have web services enabled in cloud and on premises solutions to help. You can see an example here: Real-time GDPR Scanning and Detection in SharePoint.

Report

The new GDPR standard is all about accurate record keeping, which provides transparency and overall accountability. Knowing all the document types that can be classified as having personal information, and the processes around them, are critical to insure compliance. An audit of policies and procedures is sure to require records of document creation, or ingestion, how it was handled, and where it was ultimately placed under management. All of the technologies mentioned in this article have broad reporting and analytics capabilities.

GDPR Dashboard in Ephesoft Insight

With the complexities of GDPR, standard reporting will not suffice in most cases, and the ability to perform deep analytics to track and identify key data and documents will be a requirement. For more information on GDPR and how Ephesoft can help you in your strategy, please contact info@ephesoft.com.


Ephesoft Microsoft Integrations

Your Guide to Intelligent Automation using Microsoft and Ephesoft

Ephesoft Microsoft IntegrationsWe are ramping up our team for the Microsoft Inspire Conference at Booth 1237 in Washington, DC on July 9-13. The timing is right to share ideas on the power of Ephesoft technology when combined with Microsoft technologies. Ephesoft has been working with several Microsoft Teams (Azure, SharePoint, Flow) to bring intelligent automation solutions to market and provide extensive document-centric solutions to their partner and customer ecosystem. So how do we fit?

 

Who is Ephesoft?

Ephesoft was founded in 2010 by leaders from the document capture industry that wanted to drive innovation and disrupt the legacy document automation space. The company has shown explosive growth through its unique perspective on taming unstructured content using patented, complex analytics and machine learning. Its technology has garnered broad interest, and investment from top-tier firms like Fujitsu and In-Q-Tel.

Ephesoft has two products:

  1. Ephesoft Transact - a transaction document capture platform for day-to-day document processing.
  2. Ephesoft Insight - a document analytics platform for ingesting large volumes of existing unstructured content and extracting meaning.

 

How does Ephesoft technology work?

At the heart of the Ephesoft platform is an engine that provides automated document classification and data extraction. Feed it documents from any source (fax, scanners, copiers, folders, legacy ECM systems, mobile devices and repositories) and the system will do all the heavy lifting – sorting, separating, classifying and getting you the data you need to drive efficiency, productivity, automation and decision-making with minimal end-user intervention. Providing SaaS and PaaS solutions, along with availability on-premises or in the cloud, the Ephesoft platform can provide great value to any size organization.

 

How does Ephesoft fit with Microsoft?

Think of Ephesoft as an added intelligent document automation layer that can be placed on top of other technologies as a catalyst for automation. Below is a list of core technologies from Microsoft and how Ephesoft fits from a business perspective.

 

  • Microsoft SharePoint and Ephesoft

Microsoft SharepointWith SharePoint, Ephesoft Transact can be an intelligent onramp for documents into SharePoint libraries. As a front-end loader, Transact can auto-identify and route documents from almost any source. Then, the system exports the documents into the right library, as a searchable PDF, with all the important metadata extracted. It provides a standardized, repeatable process for adding any type of document to Microsoft SharePoint.

With Ephesoft Insight, SharePoint libraries can be consumed and leveraged for Document Analytics. Insight provides the “document side” of the analytics equation.

You can get more information here:

Ephesoft/SharePoint Integration

Email Classification with SharePoint

 

  • Microsoft Flow and Ephesoft (Add in product logos for each Microsoft)

Microsoft FlowUtilizing Ephesoft Web Services in the cloud, users can add intelligence to any Microsoft Flow workflow. Using the classification or extraction services, you can use Ephesoft Transact technology to "open up" documents mid-process as well as make workflow branching decisions based on what you find. An example of a Flow use case here:

Ephesoft and Microsoft Flow with SharePoint Online

Scanning to Microsoft Flow

 

  • Microsoft Dynamics and Ephesoft

Microsoft Dynamics 365ERP and Accounting systems can leverage the power of Ephesoft in many ways. As a processing engine, Ephesoft Transact can extract data from critical documents, like invoices or sales orders, and pass the information onto Dynamics. Employees will no longer have to hand key information, wasting precious time. Along with time savings, data entry errors can now be eliminated through Ephesoft Transact’s validation and exception processing capabilities. For more information:

Ephesoft Accounting ERP Solutions

 

  • Microsoft Azure and Ephesoft

Microsoft AzureDocument capture and automation is a great fit for the cloud. Ephesoft’s web-based technology and RESTful APIs are cloud ready, including Microsoft Azure. As a Cloud Infrastructure partner, Ephesoft has worked diligently to ensure compatibility with Azure, and to take advantage of all the cloud offers from a scalability and availability perspective. Read more on Ephesoft's cloud platform:

Ephesoft Capture in the Cloud

 

This is just a short list of possibilities. Ephesoft's products are built for partners and have an open architecture to facilitate the building of portable solutions to add value and drive revenue. Visit us at Booth 1237 or reach out to us directly for more information: Contact Us.

 


OCR 2.0

OCR 2.0 - Using Adaptive OCR Technology & Analytics to Drive Efficiency

Optical Character Recognition technology, or OCR, has been around for quite some time.

OCR 2.0It became mainstream back in the '70s when a man named Ray Kurzweil developed a technology to help the visually impaired.    He quickly realized the broad commercial implications of the technology, and so did Xerox, who purchased his company.   From there, the technology experienced broad adoption across all types of use cases.

At its simplest, OCR is a means to take an image and convert recognized characters to text.  In the Enterprise Content Management (ECM) world, it is this technology that provides a broad range of metadata and content collection methods as documents are scanned and processed.   Here are the basic legacy forms of OCR that can be leveraged:

  • Full-Text OCR- converts the entire document image to text, allowing full-text search capabilities.  Using this OCR type, documents are typically converted to an Image+Text PDF, which can be crawled, and the content made fully searchable.
  • Zone OCR- Zoning provides the ability to extract text from a specific location on the page.  In this form of "templated" processing, specific OCR metadata can be extracted and mapped to an ECM system index field or column.  This method is appropriate for structured documents that have the data in the same location.
  • Pattern Matching OCR- pattern matching is purely a method to filter, or match patterns within OCR text.  This technique can provide some capabilities when it comes to extracting data from unstructured, or non-homogeneous documents.  For example, you could extract a Social Security Number pattern (XXX-XX-XXXX) from the OCR text.

These forms of OCR are deemed as legacy methods of extraction, and although they can provide some value when utilized with any document process, they are purely data driven at the text level.

In steps OCR 2.0.  Today, at Ephesoft  we leverage OCR as the very bottom of our document analytics and intelligence stack.   The OCR text is now pushed through algorithms that create meaning out of all types of dimensions: location, size, font, patterns, values, zones, numbers, and more (You can read about this patented technology here: Document Analytics and Why It Matters in Capture and OCR ).  So rather than just being completely data-centric, or functioning at the text layer, we now create a high-functioning intelligence layer that can be used beyond just text searching and metadata.  Moreover, the best part?  This technology has been extended to non-scanned files like Office documents.

Examples?  See below:

  • Multi-dimensional Classification- using that analysis capability (with OCR as algorithm input), and all the collected dimensions of the document, document type or content type can now be accurately identified.  As documents are fed into any system, they can be intelligently classified, and that information is now actionable with workflows, retention policies, security restrictions and more.  You can see more on this topic in this video on Multidimensional Classification Technology: Machine Learning and Classification of Documents
  • Machine Learning- legacy OCR technology provided no means or method to "get smarter" as documents were processed.  Just looking at the pure text, it either recognized it or not.  With a machine learning layer, you now have a system that gets more efficient the more you use it.   The key here is that learned intelligence must span documents, it cannot be tied to any one item.  It's this added efficiency that can drive usage and adoption through ease of use.  You can see more on machine learning in the videos below:

Machine Learning and OCR Data Extraction

Machine Learning and External Data

  • Document Analytics, Accuracy, and Extraction- with legacy OCR, extracting the information you need can be problematic at best.  How do you raise confidence that the information you have is accurate?  With an analysis engine, we look not just at the text, but where it sits, what surrounds it, and know patterns or libraries.  This added layer provides the ability to express higher confidence in data extraction, and make sure you are putting the right data into your backend systems.