Bounding Box Labeling
Bounding Box Project
The following guide will show you how to create a bounding box project and how to draw and label bounding boxes on your PDF/image/document. As a complimentary guide, watch this 2:00 minute Youtube video on how to create Bounding Box project.
Interface Legend
If you have already your project, use this legend for understand the basics of the Bounding Box UI. Also, see this :43 second Youtube video on the basics of drawing and labeling a bounding box!
Project Creation
You can create a Bounding Box Labeling project in Datasaur. It is pretty simple, you need to prepare the image or document file for this project. From your project home page, open "Create a Custom Project." In order to begin, you should upload the supported image or document file.
The type of Image files we accept: .bmp, .gif, .jpeg, .jpg, .png, .svg, .tiff, .tif, .webp
The type of document files we accept: .pdf
Here is an example:
So now our window should look like this,
For the Step 2. Preview please make sure you select None for the Apply OCR Method under the preview settings. Afterward, we can move on to the next steps, under Step 3. Labeler’s tasks, you can add a label set by ticking the Bounding Box labeling checkbox. Here’s what the settings will do for your project:
Label Sets
You can utilize .csv, .tsv, or .json formats for the bounding box label set.
For
.csv/.tsv, we support color names (e.g., red), hex values (e.g., #00FF00), and RGB (e.g., rgb(0,0,255)). You can also utilize the label set with just names, as shown in the Datasaur sample - Bbox only name.csv below. Other values such as captionAllowed and captionRequired will use default settings.For
.json, we support hex and RGB only.
Text Transcription
The Text Transcription setting allows the labeler to add corresponding text to a Bounding box. Disabling this setting means the labeler could not add the text.
By turning on the Text Transcription setting, the labeler can add text to a bounding box. You can choose whether a specific label must have a text by disabling or enabling the Require caption checkbox.
Require caption
For the rest of the Project Creation Process, you will then be able to take all the steps one takes in other project types: Preview, Labeler's Tasks, Assignment, and Project Settings.
Upload Pre-labeled Files
In Bounding Box Labeling Project, you also have the option to upload pre-labeled files. To do this, you can simply upload a JSON file along with the document that you want to label. Ensure that the JSON file follows the Datasaur Schema format. For further details about the Datasaur Schema, refer to this link.
Label Sets Capabilities
If you've created a project but forgot to add a label set, no worries. You can either drag and drop or click the three-dot button to include a new label set.
If you've initially set up a label set while creating a bounding box project but need to make edits, simply click the three-dot button and select 'Edit label set'. This will open a pop-up section where you can make your edits.
If you need to replace the current label set, you can do so by clicking the three dots and choosing 'Replace existing label set'. This will provide a drag-and-drop area where you can upload your desired label set.
By default, both Reviewers and Labelers have access to these capabilities. However, if you prefer to restrict this capability for your Labelers, you can adjust it in File > Settings > Administrator and toggle off the option.
Creating Bounding Box(es)
The first thing you will see inside the Bounding Box labeling project will be the file viewer for the documents or images from the previous steps. You can start creating a Bounding Box by clicking and dragging it inside the viewer area. The Bounding Box will consist of four coordinate points.
You can also create a Multiple Bounding Box that will be considered as one by holding the CTRL key when you try to draw the next Bounding Box. When you are finished drawing the number of Bounding Boxes you desire, you can release the CTRL key and starts doing the next step.
Applying Label and Caption
Every time we finished drawing Bounding Box(es) on a certain area, a modal will be shown. This modal will have 2 fields by default consisting of Label and Caption. You can select the label from the list and will be automatically moved to the Caption field, here you can type anything from the description of the Bounding Box to the character reading inside of it.
Once you finished filling in these 2 fields, you can hit the enter button to submit both values, and then the Bounding Box with the corresponding label and caption will be created. You can try to quickly review the value by hovering on the created Bounding Box, and it will show you a tooltip containing the values. Here’s what it will be looks like for a complete Bounding Box and the tooltip.
Automatically fill the caption using OCR
In addition to manual typing, we are now supporting an option to automatically fill captions using Tesseract! When you draw a bounding box around the desired text in the document, Tesseract will return it as the caption.
Please keep in mind that the processing time may be longer for longer text selections within the bounding box.
Enabling OCR for caption can be done in the Step 3. To do so, click on the Auto-caption provider dropdown and select Tesseract. After that, you can continue setting up your project configuration and launch it.\
Auto-caption provider dropdown is available when “Allow text caption” is enabled.
After your project has been successfully created, you are able to modify the auto-caption setting by clicking the triple-dots in the Bounding Box Labels extension. Disabling or enabling the setting will affect all documents within the project.
Auto-caption setting will not appear if no auto-caption provider has been selected in the Step 3.
There are a couple of things that you need to know about auto-caption.
Auto-captioning process only happens on drawing bounding box process, not resizing or repositioning the created bounding box
Auto-captioning process will be rerun when you resize or reposition the bounding box in a drawing state
Auto-captioning process can be canceled by clicking the (x) inside the Caption text field
Auto-captioning process will be canceled if you click outside the label box, and the bounding box will be created
It will return an empty value if caption for label is optional
It will throw an error if caption for label is required
Auto-caption can be used for multiple bounding boxes
It will read all the bounding boxes
It will be separated by a line break between the bounding box content
Reviewer Capabilities Overview
Capabilities
View all labelers' work.
Note: Bounding box labels applied in similar positions may overlap.
Apply labels.
Update labels applied by labelers.
Limitations
Unable to accept or reject labels.
Unable to view all labelers' work with information on who applied each label.
When updating a label applied by a labeler, there is no information or indicator showing that the label has been changed
Last updated
