Comment on page
LLM Lab
LLM Lab is a platform designed for your LLM experimentation. It allows you to integrate your knowledge base and deploy LLM applications easily.
Navigate to Datasaur App and log in with your credentials. Once logged in, go to the Menu and select 'LLM Lab'.
LLM Lab Menu
- 1.Creating a New Application: Click on “Create new application” to begin crafting your LLM application.
Create new Appliication in LLM Lab
- 2.Design your Prompt Template: Utilize the interface to build your prompt template. You can add more templates by clicking “Add prompt template”.
Add Prompt Template in Playground Tab
- 3.Duplicate for Efficiency: Use the “Duplicate prompt template” feature for quicker prompt iterations.
Duplicate Prompt Template
- 4.Model Selection and Comparison: Change the LLM model in Advance Configuration per prompt template to compare results. Currently, OpenAI LLM Provider is available, with more to come.
Model Selection
- 5.Testing Your Prompts: After crafting your prompt, use “Add prompt” to test its effectiveness.
Add Prompt
- 6.Bulk Prompt Upload: Use the upload feature (.CSV file) for bulk prompts. Format your file as described, with each row representing a prompt and each column header as a variable. Please find the example of format file below.
Sample File Upload Prompts LLM Lab Playground (1).csv
250B
Binary
Upload Prompts
- 7.Viewing Results: Click “Run Prompt” to see the outcomes. Monitor the processing time for each prompt and view the average times for each prompt template.
Run Prompt
- 8.Hyperparameter Adjustment: Fine-tune your application by adjusting hyperparameters in Advanced configuration per template.
Advanced Configuration
Tips: You can expand the prompt template for better crafting space, expand the prompt template.
Expand the Prompt
Expanded Prompt Template
- 1.Choosing Your Template: After finalizing your application, go to the “Deployment” tab and select the prompt template for deployment.
Deployment Tab
- 2.Deployment Success: Your LLM Application is now ready, accessible via cURL, Python, and TypeScript.
Deployment Success
Access Vector Stores: In the LLM menu, select 'Vector Stores'.
Accessing Vector Stores
New Vector Store: Click “Create new vector store”.
Create New Vector Store
Configuring Internal Vector Store: Enter the store name, select Datasaur as the provider, choose the embedding model, set chunk size and overlap, then click “Next”.
Vector store settings
File Properties: Facilitating user to give information about the file. We will discuss further regarding file properties on a separate section. This is optional. If you don’t need it, you can continue with click “Create vector store”.
File Properties
Setting Up External Provider: For external providers, input the store name, base URL, and choose the authentication method. Then click “Next”.
External Vector Store
File Properties: Facilitating user to give information about the file. We will discuss further regarding file properties on a separate section. This is optional. If you don’t need it, you can continue with click “Create vector store”.
File Properties
- 1.Building Your Knowledge Base: Add files to your vector store by clicking “Add file”. Supported formats are .urls, .txt, .pdf, .docx (temporarily unavailable). Click “Update vector store” after adding files.
Add File to Knowledge Base
- 2.Monitoring and Updating: Observe the processing status of each file. Update the store as needed, considering file processing times.
Update Vector Store
Update Embedding
Knowledge Base Updated
Testing Your Knowledge Base: Use the search function to validate the effectiveness of your knowledge base in providing context.
Search to Test The Knowledge Base
Log activities of vector store: making it easier to track changes and actions on your vector store.
Activity Log
As mentioned before, file property will enhance your data organization by facilitating user to give information about the file.
There are two easy ways for the user to add file properties during Vector Store creation:
A. The first way is the user may upload their file properties.
You may drag and drop the file into the dropdown area. Our supported formats are as follows: .csv and .json. For JSON files, please find below a sample of our accepted format.
complete-file-property-sample.json
6KB
Code
B. The second way for a user to add a file property is to create it from scratch.
File Property from scratch
Let's take a look at the File property types available below.
1. Text Field
Text Field allows the user to give information by typing in free-form text, up to a single line at a time.
Users also can add some validation by expanding the Advanced Settings.
- You can also allow selecting multiple answers by checking the box to Allow multiple answers.
2. Text Area
Text Area allows the user to give information by typing in free-form text. In contrast to Text Fields, this allows for multiple-line answers.
- You can also allow selecting multiple answers by checking the box to Allow multiple answers.
3. Dropdown
Dropdown requires user to give information by picking one of several multiple-choice answers.
- If you have a .csv with a pre-set list of answers, you can upload the .csv as an option set.
Datasaur sample - Option set (dropdown).csv
178B
Binary
- You can also allow selecting multiple answers by checking the box to Allow multiple answers.
4. Hierarchical Dropdown
Hierarchical dropdown allows the user to give information with hierarchically organized options.
- Just like with the Dropdown type, you can also upload an option set once you have created the hierarchical question. The sample format for hierarchical option sets can be found here.
💡Let's break down the components of this file
1. The header
id,label is the header. This will always be the first row in the .csv. The first label will have 1 as the id, as same as the example above.2. id format
The id format is similar to Microsoft Word's numbering format. In the example above, Characters is a part of Novel name and the id will be 1.1.
- 1.Novel name: the root-level.
- 2.1: id for the root-level
- 3.Characters: the second-level.
- 4.1.1: the second-level id.
5. Date
Date allows the user to give information in two ways. The key benefit of selecting Date is that this format validates that a correct date has been filled in.
- Typing the date in manually.
- Clicking on the calendar symbol, then selecting the date.
If you want to fill date questions with the current timestamp at the time users open the file in knowledge base, you can check the Use current date as default value.
6. Time
Time allows the user to give information in two ways. The key benefit of selecting Time is that this format validates that a correct time has been filled in.
- Typing it manually.
- Clicking on the clock symbol, then selecting the time.
If you want to fill time questions with the current timestamp at the time the user opens the file in knowledge base, you can check the Use current time as default value.
7. Slider
Slider allows the user to give information by moving the sliding bar (ex: from 1 to 10).
You have the flexibility to personalize the slider color according to your preferences. While the default color for “Start at” and “End to” is blue, we provide 11 alternative default color options for you to select from.
When it comes to colors, you have the choice of using hex codes, color names, or RGB values. If you opt for any of these choices, the dropdown will be labeled as “Custom”.
To get a glimpse of how the color will appear, simply drag the slider thumb on the Preview.
Please note that we only allow numbers as the slider value.
8. Checkbox
Checkbox allows the user to give information by checking it. You can also put a description.
9. URL
URL allows you to put the URL links and apply validation on it.
10. Radio Button
Radio Button allows the user to give information by selecting one answer.
You can also insert a hint to give a description of the Radio Button. Here is an example of using the Radio Button in the labeling process:
11. Grouped Attributes
Grouped Attributes allows the user to combine multiple questions that pertain to a single group.
- You can also allow selecting multiple answers by checking the box to Allow multiple answers.
After vector store creation, the file property will appear on each file on knowledge base. Once you’ve already filled the information on the file property, just click “Submit answers”.
- 1.Linking Vector Store to Application: In LLM Lab, connect your vector store to your application.Context Embeeding
- 2.Comparison and Deployment: Compare results with and without the vector store. Redeploy your application with the updated prompt template and vector store.
Deployment Process
- 3.Select the prompt template and click “Choose”.
Choose Prompt Template
Deployment success and show Vectore Store Information
Access Models: In the LLM menu, select 'Models'.
Go to My Models tab and click Manage Providers.
Add Credentials: Select a provider that you want to connect.
Last modified 1d ago