Bounding Box Labeling

Interface Legend

Project Creation

You can create a Bounding Box Labeling project in Datasaur. It is pretty simple, you need to prepare the image or document file for this project. From your project home page, open "Create a Custom Project." In order to begin, you should upload the supported image or document file.
The type of Image files we accept: .bmp, .gif, .jpeg, .jpg, .png, .svg, .tiff, .tif, .webp
The type of document files we accept: .pdf
Here is an example:
So now our window should look like this,
For the Step 2. Preview please make sure you select None for the Apply OCR Method under the preview settings. Afterward, we can move on to the next steps, under Step 3. Labeler’s tasks, you can add a label set by ticking the Bounding Box labeling checkbox. Here’s what the settings will do for your project:

Text Transcription

The Text Transcription setting allows the labeler to add corresponding text to a Bounding box. Disabling this setting means the labeler could not add the text.

Require caption

By turning on the Text Transcription setting, the labeler can add text to a bounding box. You can choose whether a specific label must have a text by disabling or enabling the Require caption checkbox.
For the rest of the Project Creation Process, you will then be able to take all the steps one takes in other project types: Preview, Labeler's Tasks, Assignment, and Project Settings.

Creating Bounding Box(es)

The first thing you will see inside the Bounding Box labeling project will be the file viewer for the documents or images from the previous steps. You can start creating a Bounding Box by clicking and dragging it inside the viewer area. The Bounding Box will consist of four coordinate points.
You can also create a Multiple Bounding Box that will be considered as one by holding the CTRL key when you try to draw the next Bounding Box. When you are finished drawing the number of Bounding Boxes you desire, you can release the CTRL key and starts doing the next step.

Applying Label and Caption

Every time we finished drawing Bounding Box(es) on a certain area, a modal will be shown. This modal will have 2 fields by default consisting of Label and Caption. You can select the label from the list and will be automatically moved to the Caption field, here you can type anything from the description of the Bounding Box to the character reading inside of it.
Once you finished filling in these 2 fields, you can hit the enter button to submit both values, and then the Bounding Box with the corresponding label and caption will be created. You can try to quickly review the value by hovering on the created Bounding Box, and it will show you a tooltip containing the values. Here’s what it will be looks like for a complete Bounding Box and the tooltip.

Automatically fill the caption using OCR

In addition to manual typing, we are now supporting an option to automatically fill captions using Tesseract! When you draw a bounding box around the desired text in the document, Tesseract will return it as the caption.
Please keep in mind that the processing time may be longer for longer text selections within the bounding box.
Enabling OCR for caption can be done in the Step 3. To do so, click on the Auto-caption provider dropdown and select Tesseract. After that, you can continue setting up your project configuration and launch it.\
Auto-caption provider dropdown is available when “Allow text caption” is enabled.
After your project has been successfully created, you are able to modify the auto-caption setting by clicking the triple-dots in the Bounding Box Labels extension. Disabling or enabling the setting will affect all documents within the project.
Auto-caption setting will not appear if no auto-caption provider has been selected in the Step 3.
There are a couple of things that you need to know about auto-caption.
  1. 1.
    Auto-captioning process only happens on drawing bounding box process, not resizing or repositioning the created bounding box
  2. 2.
    Auto-captioning process will be rerun when you resize or reposition the bounding box in a drawing state
  3. 3.
    Auto-captioning process can be canceled by clicking the (x) inside the Caption text field
  4. 4.
    Auto-captioning process will be canceled if you click outside the label box, and the bounding box will be created
    1. 1.
      It will return an empty value if caption for label is optional
    2. 2.
      It will throw an error if caption for label is required
  5. 5.
    Auto-caption can be used for multiple bounding boxes
    1. 1.
      It will read all the bounding boxes
    2. 2.
      It will be separated by a line break between the bounding box content