Mixed Label Sets
This feature allows you to combine two labeling types.
Datasaur is currently support:
- 1.token-based and document-based types in one interface.
- 2.
More combinations coming soon!
Let's follow the steps below to create a token and document-based project!
As an admin, you can create a mixed label set by selecting multiple labeling types, below are the steps to create it.
- Open team workspace.
- Create a new project.
- Upload file(s).
- In step 3, you can simultaneously select token labeling and document labeling.
- Continue to the next step and launch your project.
- Here is the project page view when the mixed label set project is finished.
Once your mixed label set project has been created, you will have an interface looks like below: a label set, and a question set on the right side.
We don’t have any particular rule for doing which type of labeling first. However, we recommend that you label the tokens first, then classify the document by answering the question.
If you have multiple documents within a project, submitting the answer will help go directly to the next document.
We use dropdown question type for the sample screenshot above. Please kindly refer to this page for other question types.
Conflict resolution is practically similar with our current behavior. Users will have an information which conflict type in the document (labels, sentence editing, or relation) and the conflict counter in each document.
The difference is, that you will need to display the Document Labeling extension by scrolling down the conflict extension and clicking Document Labeling.
Document Labeling extension will show, and you can resolve the conflicts as resolving conflicts in the row-based or document-based projects.
After the reviewing process is done, you can click Mark as Complete button to indicate that the project is completed!
If there are unresolved conflicts in certain documents, a popup modal shows where the conflicts are. You can open the dropdown, then click the document name. It will go directly to the desired document.
Please note that marking the project as complete will make the project as a read-only. You will need to mark the project as in progress to do some modification.
Here is the way to access Project Analytics.
- Project page.
- On Project row, press the three dots icon ( ⋮ ) to show options.
- Press “View Project Details”.
As an Admin, you can filter the project analytics based on project type.
- For a single typed project, the dropdown will automatically choose the project type.
- For multi-typed project, the dropdown will select the default (one of the project types) and you can choose the filter based on the project’s project types.
Interface
- Project Analytics Interface.
- Filter by Project Type option.
As the Admin, you can download every widget based on the filter applied (date and project type).
These widgets are affected by the project type filter:
- Number of tasks
- If filtered by document-based, it will be calculated based on document labels.
- If filtered by token-based, it will be calculated based on token labels.
- Total conflict per labeler
- If filtered by document-based, it will be calculated based on document labels.
- If filtered by token-based, it will be calculated based on token labels.
- Conflict resolved by reviewer
- If filtered by document-based, it will be calculated based on document labels.
- If filtered by token-based, it will be calculated based on token labels
The IAA on project analytics page will not be filtered by project type.
Here is the way to access Member Analytics:
- Members page.
- On Member row, press the three dots icon ( ⋮ ) to show options.
- Press “View Member Details”.
For double typed project, the “labels applied”, “conflicts”, “#of accepted”, and “#of rejected” will be calculated from both project types.
- The ML Assisted Labeling will automatically apply on token-based.
- You can use the Dictionary on mixed label set project in the same way as you would use in token-based project.
- You can use the Search Extension on mixed label set project in the same way as you would use in token-based project.
- You can use the Grammar Checker on mixed label set project in the same way as you would use in token-based project.
- You can now see the information that was available on the Dashboard Extension, on the Project Status Bar and Extension.
- The Project Status extension provides a button to Mark the project as complete.
- The Status Bar provides information that was available on the Dashboard Extension:
- Total Labels Applied:Number of the labels applied on the current file.
- Last Labeled Row:The last labeled row that the labeler/reviewer applied.
- Total Labeled Row:Number of the rows in a File that are labeled by the labeler/reviewer.
- The status bar only shows token labels analytics.
Let's follow the steps below to create a bounding box and document-based project!
As an admin, you can create a mixed label set by selecting multiple labeling types, below are the steps to create it.
- Open team workspace.
- Create a new project.
- Upload file(s).
- In step 3, you can simultaneously select bounding box labeling and document labeling.
.png?alt=media)
- Continue to the next step and launch your project.
- Here is the project page view when the mixed label set project is finished.
.png?alt=media)
Once your mixed label set project has been created, you will have an interface looks like below: a bounding box label set, and a question set on the right side.
.png?alt=media)
We don’t have any particular rule for doing which type of labeling first. However, we recommend that you label the bounding box first, then classify the document by answering the question.
If you have multiple documents within a project, submitting the answer will help go directly to the next document.
Extensions are the same with the bounding box labeling project with additional extension
Document Labeling. More information about extensions can be found here.Following bounding box labeling rules, we currently don't support conflict resolution. It's one consensus settings for now.
Type | Format |
|---|---|
Bounding Box-based + Document-based | |
Document-based only |
mixedLabelSet_BBL-Doc (1).json
5KB
Code
Sample export
Last modified 1mo ago