Writing Ocr Text And Re-Reading Position Differences

[ImMyQuest]

Hi Guys,

I'm getting some differences between when I write out Ocr'd text to a pdf and when I read it back from the pdf, in terms of line positioning. This is causing me to display the text read with extra line feeds. I'm not sure whether it's how I'm writing the text. I'm using a TextWriter and the Append method like this - tw.Append(spn.Origin, spn.Text, font, spn.Size), spn being a Span and font created from New Font(spn.Font) at the start of looping.

Attached is the scanned pdf and a screenshot comparing (part of) the Ocr'd text to the re-read text. Also attached are 3 text files with the data after Ocr, before writing and after re-reading. Paragraph 3) splits after the hyphen on the first line and after "still a" on the second line. The data for the second line shows -

before writing, the Y1 values are consistent:
are Blk:7 Ln:5 Org:Point(293.52878, 345.53262) Siz:13.4508705 Y0:329.45575 Y1:345.5487
still Blk:7 Ln:6 Org:Point(315.02258, 345.53262) Siz:10.6641865 Y0:329.45575 Y1:345.5487
a Blk:7 Ln:7 Org:Point(346.25247, 345.53262) Siz:7.007743 Y0:329.45575 Y1:345.5487
problem, they Blk:7 Ln:8 Org:Point(347.7798, 345.53262) Siz:14.379586 Y0:329.45575 Y1:345.5487

after re-reading, the Y1 values vary:
are Bl:7 Ln:1 Y0:331.15366 Y1:349.47372
still Bl:7 Ln:1 Y0:334.1326 Y1:348.65723
a Bl:7 Ln:2 Y0:338.04135 Y1:347.5859
problem, Bl:7 Ln:2 Y0:330.16086 Y1:349.74585
they Bl:7 Ln:2 Y0:330.16086 Y1:349.74585

Currently, I'm testing whether the difference in Y1 values between words is greater than 2 and if so add a newline.

My question is: is the way I'm writing out the text causing the differences in Y1 values and what can I do about it? Is there a better/easier way to write out the text to get consistent results.

Thanks.

doc.pdf
afterOcr.txt
beforeWrite.txt
reReading.txt

[JorjMcKie]

You do not give us the script that does all this. One major point is that you must use a mono-spaced font when writing the text, because Tesseract's own font (GlyphLessFont) is mono-spaced too.

As an aside, this means that text recognition is precise (in any sense of the word possible at all when talking OCR) only down to single words - not single characters.

When MuPDF interprets the page to deliver us the "dict" hierarchy "block -> line -> span" it will apply some heuristics to identify the respective boundary boxes which will require compromises.

Here is an image after OCR-ing your page with DPI=150 showing the bboxes of the detected spans:

And here are the lines bboxes:

The potentially explains some of your observations.

[ImMyQuest]

Hi Jorj,

Thank you for your reply. Here is my code to write out the Ocr text -

Private Sub WritePageText()
    Dim dic = _textPageOCR.ExtractDict(_Page.GetBound),
        tw As New mupdf.NET.TextWriter(_Page.GetBound),
        font As mupdf.NET.Font = Nothing,
        first As Boolean = True

    For Each blk As Block In dic.Blocks.Where(Function(e) e.Type = BlockTypes.Text)

        For Each ln As Line In blk.Lines

            For Each spn As Span In ln.Spans

                If first Then
                    font = New mupdf.NET.Font(spn.Font)
                    first = False
                End If

                tw.Append(spn.Origin, spn.Text, font, spn.Size)
            Next

        Next

    Next

    tw.WriteText(_Page,,, overlay:=BACKGROUND)
End Sub

Looking into it further, I discovered the font name returned by spn.Font is "GlyphLessFont" and the descender given by spn.Desc is -0.001. However, the font returned by the line "font = New mupdf.NET.Font(spn.Font)" is "Noto Serif Regular" which has a descender of -0.293. I think this causes the Y1 values to change on output.

How can I obtain a GlyphLessFont?

Thank you.

[JorjMcKie]

How can I obtain a GlyphLessFont?

You cannot - and you should not because it is what its name suggests: it has no definitions for appearances, meaning it will remain invisible whatever you would do with it.

If you use a font with proportional (=variable) character widths as you currently do, you will introduce metric mismatches ... beyond the imprecisions inevitably coming with OCR as such.

Use a mono-spaced font like Courier instead. This will still introduce deviations because the Courier descender -0.317 is very different from that of GlyphlessFont, but at least it should deliver a better approximation.

[ImMyQuest]

On further analysis, I realised that when I read back the text, if I use ExtractDict() rather than ExtractWords(), I can test span.Origin.Y instead of word.Y1. As I understand it, span.Origin is the baseline and is therefore consistent along a line, regardless of descenders (with some minor variation).

I notice one other anomaly in my example - on the line "Program B" - "B" is on the start of the following line. Why should that be?

[JorjMcKie]

But in this example (which shows span origins and bboxes) the origin values of text "1 lb. of nitrogen ..." are at the bottom of rectangles with a crazy height:

and in no way similar to the visible start of the line.

As per the "Program B" problem:
OCR at DPI=150 delivers the following span bboxes (blue) in that region:

The corresponding printout is this

pp(spans[143])
{'size': 30.73430061340332,
 'flags': 12,
 'bidi': 0,
 'char_flags': 0,
 'font': 'GlyphLessFont',
 'color': 0,
 'alpha': 0,
 'ascender': 1.0,
 'descender': -0.0010000000474974513,
 'text': 'Program',
 'origin': (3.3573660850524902, 543.893310546875),
 'bbox': (3.3573660850524902,
          507.9215393066406,
          95.2652587890625,
          543.9292602539062)}
pp(spans[144])  # this is for "B Deluxe Program":
{'size': 26.677385330200195,
 'flags': 12,
 'bidi': 0,
 'char_flags': 0,
 'font': 'GlyphLessFont',
 'color': 0,
 'alpha': 0,
 'ascender': 1.0,
 'descender': -0.0010000000474974513,
 'text': ' DetinePragiam',
 'origin': (95.2652587890625, 543.893310546875),
 'bbox': (95.2652587890625,
          507.9215393066406,
          229.32009887695312,
          543.9292602539062)}
pp(spans[145])
{'size': 23.143918991088867,
 'flags': 12,
 'bidi': 0,
 'char_flags': 0,
 'font': 'GlyphLessFont',
 'color': 0,
 'alpha': 0,
 'ascender': 1.0,
 'descender': -0.0010000000474974513,
 'text': '|',
 'origin': (226.3824005126953, 545.811767578125),
 'bbox': (226.3824005126953,
          505.52337646484375,
          233.0299835205078,
          545.85205078125)}

The font sizes differ significantly between span 143 and span 144, so separate lines were created. Also note that the extracted text is complete nonsense because of italics with a gray background color.
Not to talk of other artifacts cause by the paper quality.

[JorjMcKie]

When doing OCR with a larger DPI=300 then the above data change completely. The spans around "Program B ..." are now delivered with a different text ("Program 8 (SSE"), but still front sizes of the 3 spans differ widely, such that again different lines are generated.

[ImMyQuest]

Thanks very much Jorj for your assistance. I certainly understand now more of the issues in using Ocr results.

I am extracting text with 600dpi, because it seemed to generate more accurate results. I have also been working on a method to adjust the contrast of the scan to improve readability, before executing Ocr.

I read about best/fast traineddata and wondering whether using best would help. I'll try next.

[JorjMcKie]

Glad to read that this was helpful!

A final comment:
Endlessly increasing the DPI will not necessarily improve the result. In parallel with the actual text dirt, wrinkles and other artifacts will also increase and more likely be taken as text particles. Underlines like in your case will not go away and continue to confuse the OCR engine.