The application
This is an application demonstrating the capabilities of the off-line diagram recognizer trained for flowcharts. The user can draw a flowchart using a pen and paper and then take a scan or a photo of the drawing. After the image file is loaded by the application, the recognition may commence. The recognizer combines strokes reconstruction from images with on-line recognition techniques (on-line recognizer). The image is binarized and skeletonized before the strokes are reconstructed. The adapted on-line recognizer then interprets the diagram.
Output of the individual steps of the recognition pipeline is shown in corresponding tabs. Images and strokes may be saved into PNG/InkML files via a pop-up menu triggered by right mouse clicking on the image or the canvas with strokes.
The recognition result is visualized in such way that individual symbols are colored according to their class and their bounding boxes are displayed.
Additionally, the reconstructed strokes and recognized symbols are mapped and drawn into the input image.
Finally, the diagram is beautified using Graphviz. When the last tab is selected, the user can choose whether the layout of symbols should correspond to the drawn layout or if the layout manager of GraphViz should be used.
The application supports the following flowchart symbols: connection, data, decision, process, terminator. The symbols may be connected by arrows. Each symbol can be labeled by text. The on-line text recognizer in .NET cannot recognize the text reliably, because the reconstructed strokes are quite distorted and their natural order was lost. We ordered strokes in each text block from left to right. However, the text is recognized incorrectly in most of the cases. One option to solve this problem would be to use an off-line text recognizer. The text block is known and a corresponding image patch can be easily obtained from the input image. Unfortunately, we did not find a suitable free library for off-line text recognition.
The demo application is provided with a set of samples stored in folder data, which can be used as an inspiration and examples what is the supported structure of diagrams. There are three types of samples in three sub-folders: 1) test - the samples taken from the test dataset of the off-line flowchart database. 2) scans - scans (All-in-One KONICA MINOLTA bizhub C3110) of several flowcharts drawn on paper with a pen, 3) photos - photos (iPhone 6) of the same sketched diagrams taken from various angles. The recognition difficulty is increasing from 1) to 3).
System requirements
The application is implemented in C# and thus requires the following:
- Microsoft Windows Vista operating system or higher
- .NET 4.0 or higher
Download
The demo application is freely available for experimentations with the off-line diagram recognizer.
file: flowchart_recognizer_offline_demo_1.0.zip
References
The application implements an interface for the recognizer introduced in the following paper:
| [1] | Bresler M., Průša D., Hlaváč V.: Recognizing Off-line Flowcharts by Reconstructing Strokes and Using On-line Recognition Techniques. Yet unpublished manuscript. |
Acknowledgement
The creation of this application was supported by the Technology Agency of the Czech Republic under the project TE01020197, the Grant Agency of the CTU under the project SGS13/205/OHK3/3T/13.