The contest's main objective is to recognize four mathematical expressions that resemble handwriting. Participants are required to design an algorithm capable of identifying and solving mathematical formulas in 100,000 images filled with noise and interference. During the finals, the challenge becomes more complex, incorporating higher scoring criteria and more intricate calculations, with the final recognition rate being the key performance metric.
This section will detail the various strategies I employed to handle the preliminary round of the competition, specifically focusing on identifying expressions involving four mixed operations.
**Problem Description**
The competition essentially revolves around an OCR (Optical Character Recognition) task—essentially converting image-based text into digital text. The goal is to accurately interpret handwritten mathematical expressions from images.
**Data Set**
The preliminary data set includes 100,000 images, each measuring 180x60 pixels, along with a corresponding label file named `labels.txt`. Each image contains a mathematical expression composed of:
- Three operands: three integers ranging from 0 to 9.
- Two operators: which can be '+', '-', or '*' representing addition, subtraction, and multiplication.
- Zero or one pair of parentheses.
The images are named from `0.png` to `99999.png`. Here is an example of a sample image:
The `labels.txt` file contains 100,000 lines, each line containing the formula and its result, separated by a space. For instance:
```
(3-7)+5 1
5-6+2 1
(6+7)*2 26
(4+2)+7 13
(6*4)*4 96
```
**Evaluation Index**
The primary evaluation metric is accuracy. In the preliminary round, only integer addition, subtraction, and multiplication are considered, and the results must be exact integers. Therefore, both the correct sequence of characters and the accurate result are necessary for a successful match.
In addition to the official accuracy metric, we also used CTC loss as a secondary evaluation method during model training.
**Data Augmentation Using Captcha**
We had access to the official dataset of 100,000 images, which could be directly used for training. However, to improve model robustness, we generated additional synthetic data using a captcha-like generator. The generated data followed the same rules: three numbers, two operators, and either zero or one pair of parentheses. This approach helped increase the diversity of the training data and improved overall accuracy.
**Generator Implementation**
The generator follows simple rules to create valid mathematical expressions. Initially, the code was straightforward, but after further refinement, it became more efficient and flexible.
One important adjustment was made to ensure that the minus sign (`-`) was not distorted during image resizing. By modifying the `_draw_character` function in the Python script, we prevented the minus sign from becoming too thick, which significantly improved the visual quality of the generated images.
After implementing these changes, we tested the generator and observed better results compared to the original version.
**Model Structure**
The model architecture was based on previous work, with some enhancements. We increased the number of convolutional kernels, added Batch Normalization (BN) layers to speed up training, and made minor adjustments to support multi-GPU training. If you're using a single GPU, simply remove the code related to parallel processing.
The BN layers proved effective in accelerating training and improving convergence speed.
**Model Visualization**
Visualizing the base model and the full model gave us insights into how the network processed the input and produced the output.
**Model Training**
After several experiments, I found that the validation function wasn't necessary, as the model achieved 100% accuracy on the validation set. Therefore, I focused on minimizing the `val_loss`.
I trained the model for 50 epochs using Adam optimizer, with 100,000 samples per epoch. The model converged quickly, reaching stable performance within just 10 epochs.
**Training Results**
The model was divided into four parts and trained in parallel across four GPUs. The results were combined, and the final CTC loss was calculated to guide the training process.
**Visualizing Predictions**
The model's predictions were visually verified, and it performed exceptionally well on the generated data, showing near-perfect accuracy.
**Final Submission**
After packaging the model into a Docker container, we submitted it to the competition system. After a few minutes of processing, the system returned a perfect score of 1.0.
**Summary**
The preliminary round was relatively straightforward, allowing us to achieve a high score. However, the official test set was later expanded to 200,000 images, and our model's accuracy dropped slightly to 0.999925. To improve this, we plan to refine the model further, fully train it, and combine predictions from multiple models.
**Challenges in the Extended Dataset**
On the extended dataset, some images were particularly challenging to recognize. For example, image `117422.png` initially appeared unrecognizable, but after applying image preprocessing techniques, we were able to extract the correct formula.
Another problematic image was `142660.png`, which could not be resolved even after preprocessing. This might have been due to a rare bug or error in the image generation process. As a result, we did not achieve a perfect score in the preliminary round, but we still obtained a strong performance.
The `labels.txt` file contains 100,000 lines, each line containing the formula and its result, separated by a space. For instance:
```
(3-7)+5 1
5-6+2 1
(6+7)*2 26
(4+2)+7 13
(6*4)*4 96
```
**Evaluation Index**
The primary evaluation metric is accuracy. In the preliminary round, only integer addition, subtraction, and multiplication are considered, and the results must be exact integers. Therefore, both the correct sequence of characters and the accurate result are necessary for a successful match.
In addition to the official accuracy metric, we also used CTC loss as a secondary evaluation method during model training.
**Data Augmentation Using Captcha**
We had access to the official dataset of 100,000 images, which could be directly used for training. However, to improve model robustness, we generated additional synthetic data using a captcha-like generator. The generated data followed the same rules: three numbers, two operators, and either zero or one pair of parentheses. This approach helped increase the diversity of the training data and improved overall accuracy.
**Generator Implementation**
The generator follows simple rules to create valid mathematical expressions. Initially, the code was straightforward, but after further refinement, it became more efficient and flexible.
One important adjustment was made to ensure that the minus sign (`-`) was not distorted during image resizing. By modifying the `_draw_character` function in the Python script, we prevented the minus sign from becoming too thick, which significantly improved the visual quality of the generated images.
After implementing these changes, we tested the generator and observed better results compared to the original version.
**Model Structure**
The model architecture was based on previous work, with some enhancements. We increased the number of convolutional kernels, added Batch Normalization (BN) layers to speed up training, and made minor adjustments to support multi-GPU training. If you're using a single GPU, simply remove the code related to parallel processing.
The BN layers proved effective in accelerating training and improving convergence speed.
**Model Visualization**
Visualizing the base model and the full model gave us insights into how the network processed the input and produced the output.
**Model Training**
After several experiments, I found that the validation function wasn't necessary, as the model achieved 100% accuracy on the validation set. Therefore, I focused on minimizing the `val_loss`.
I trained the model for 50 epochs using Adam optimizer, with 100,000 samples per epoch. The model converged quickly, reaching stable performance within just 10 epochs.
**Training Results**
The model was divided into four parts and trained in parallel across four GPUs. The results were combined, and the final CTC loss was calculated to guide the training process.
**Visualizing Predictions**
The model's predictions were visually verified, and it performed exceptionally well on the generated data, showing near-perfect accuracy.
**Final Submission**
After packaging the model into a Docker container, we submitted it to the competition system. After a few minutes of processing, the system returned a perfect score of 1.0.
**Summary**
The preliminary round was relatively straightforward, allowing us to achieve a high score. However, the official test set was later expanded to 200,000 images, and our model's accuracy dropped slightly to 0.999925. To improve this, we plan to refine the model further, fully train it, and combine predictions from multiple models.
**Challenges in the Extended Dataset**
On the extended dataset, some images were particularly challenging to recognize. For example, image `117422.png` initially appeared unrecognizable, but after applying image preprocessing techniques, we were able to extract the correct formula.
Another problematic image was `142660.png`, which could not be resolved even after preprocessing. This might have been due to a rare bug or error in the image generation process. As a result, we did not achieve a perfect score in the preliminary round, but we still obtained a strong performance.Bluetooth, LFP batttery,lithium battery
Enershare Tech Company Limited , https://www.enersharepower.com