hsk@infra
/
$ cd ~/projects/tosach

Tosach

Scene text recognition with a CRNN

limbo hand 2026-01-27

Implementation of the CRNN (Convolutional Recurrent Neural Network) architecture from Shi et al., 2015 for end-to-end scene text recognition using TensorFlow/Keras.

Status: limbo — the architecture is implemented and the dataset is prepared, but training has not been run yet. I went to great lengths to read up on the background material and set it up, but the scope of the project became too large. The dataset is huge and the compute required is equally extensive. I deployed the legacy approach of per-letter recognition, but the proper fully fledged CRNN is yet to be trained. I'll probably complete it once I am up for the challenge of getting VertexAI to work. For now I'm pursuing smaller scale projects.

Overview

The model recognises text in natural scene images by combining:

  • CNN (VGG-style) for visual feature extraction
  • Bidirectional LSTM for sequence modelling
  • CTC loss for sequence-to-label transcription without character-level annotations

Project Structure

FileDescription
train_crnn.pyCRNN model architecture + training pipeline
synthtiger_loader.pyData loader for the SynthTiger dataset (tf.data)
extract_synthtiger.pyMemory-efficient extractor for the large SynthTiger zip
study_content.txtExtracted text from the reference paper
study.pdfOriginal paper (Shi et al., 2015)
legacy/Earlier prototype (deprecated)

Dataset

SynthTiger v1.0 — synthetic text images with ground truth labels. The dataset is ~38GB and must be extracted before training.

pip install stream-unzip
python extract_synthtiger.py

Architecture (from the paper)

Input (W × 32 grayscale)
  → 7 Conv layers (64 → 512 maps) + BatchNorm + MaxPool
  → Map-to-Sequence
  → 2× Bidirectional LSTM (256 hidden units)
  → CTC Transcription
  → Predicted text

Legacy

The legacy/ folder contains an earlier prototype that used a simpler per-character classification approach:

  • Simple CNN (Conv2D → MaxPool → Dense) trained on 28×28 grayscale character images from a Kaggle OCR dataset
  • Per-character pipeline: segment individual characters → classify each with the CNN → concatenate results
  • Abandoned in favour of the CRNN approach, which handles full words end-to-end without character segmentation
FilePurpose
model_training.pyCNN classifier training (62 classes: A–Z, a–z, 0–9)
data_process.pySorts character images into class folders
run_inference.pyRuns predictions on validation samples
idea.mdOriginal project plan
$