Abstract:

Motivation:
Mathematical formula recognition has a wide range of applications in smart teaching tasks, such as photo search for questions, automatic scoring, and question bank construction. Most of the mathematical formulas in these applications exist in the form of pictures. Therefore, identifying mathematical formulas in pictures has become one of the most critical research issues in smart education. Due to the complex structure of the mathematical formula structure, the recognition of mathematical formulas from pictures is much more complicated than ordinary isolated symbol recognition. Current research mostly uses methods based on machine learning. Traditional mathematical formula recognition technology requires complex recognition processes, and the recognition accuracy is not high. To solve these problems, we propose a method of end-to-end handwritten mathematical formula recognition to recognizes two-dimensional mathematical formula pictures and outputs them with one-dimensional character sequences in LaTeX, which is an edited format for easier human-computer interaction.

Method:
In this paper, we introduce a neural network model of the attention-based end-to-end. Convolutional neural network (CNN) as a feature extractor maps formula images to high-level features. The decoder is a Gated Recurrent Unit (GRU) that converts these high-level features into output sequences, word by word. For each predicted word, the attention mechanism built into the decoder scans the entire input formula image and chooses the most relevant region to describe a predicted digit or symbol. Moreover, the decoder and the encoder are jointly trained, the decoder could provide contextual information to tune the encoder and guide the attention.

Result:
We validated the proposed method on the data released by the CROHME competition. Results showed that the proposed method achieves the best-reported results with an expression recognition accuracy of 61.16% on CROHME 2014 and 57.02% on CROHME 2016.

Conclusions:
The paper proposed a deep learning-based formula recognition method for smart education. The proposed method will be applied to smart education for artificial intelligence technology to solve the problems existing in teaching tasks further, such as low accuracy when searching for questions with photos and low efficiency in automatic scoring and question bank construction.

Keywords:

Mathematical formula recognition, end-to-end, smart education, artificial intelligence, teaching tasks.