Help Center

Uncategorized

Retrain an Extractor Asset

Retraining is the process of updating an Asset with new data obtained from both new documents and existing documents, along with their annotations. As data distributions change and new patterns emerge, retraining helps the Asset to adapt to the variations in the document sets and maintain its predictive power. This process also helps the Asset to continue performing accurately and effectively over time. 

Users must have any one of the following policies to retrain an Extractor Asset: 

  • Administrator Policy
  • Creator Policy

This guide will walk you through the steps on how to retrain an Extractor Asset.

  1. Upload documents
  2. Select the extractor asset
  3. Initiate retrain
  4. Select documents
  5. Annotate
  6. Train
  7. Review results and validate
  8. Publish the asset

Step 1: Upload documents

If you wish to retrain the extractor Asset with new documents, you must upload the appropriate  documents to the Platform. Otherwise, you can skip this step and proceed to retrain with your existing documents.

For more information about uploading new documents, see Upload Documents page.

Step 2: Select the extractor asset

  1. Head to the Asset Studio module and select the Extractor asset you wish to retrain.



Note: Retrain is applicable only for the Extractor Asset with status as Published.

Step 3: Initiate retrain

Once you select the Published Extractor Asset from the list, you will land on the Review results page.

  1. On the Review Results page, click Retrain.



  2. In the Retrain dialog that appears, enter the Description and then click Proceed retrain.



Step 4: Select documents

This step allows you to select the documents to retrain the Asset. You can either select  the newly uploaded documents or continue with the previously chosen documents used for the Asset’s initial training, along with their respective annotations.

Things to Know before selecting documents

Annotation tag :  The annotation tags are visible to you to identify the documents and files that were previously selected for the annotation. You can choose to include or exclude the files you want. 

Annotation Summary: The annotation summary displays information about all the fields and the number of annotated instances that are associated with the selected Document type.


Use document annotation: In the Annotation Summary, select the Use document annotation check box to continue with the previous annotations on the documents. Otherwise, the platform discards the previous annotations and allows you to perform new annotations. 

Previously annotated files and folder: When you start selecting  documents, you can also access the previously annotated files and folders with annotation tags. By default, the files and folders are selected.

  • To exclude the previously selected/annotated files and folder, unselect the Select All checkbox. 

To select the document, follow the steps below:

  1. In the Select Documents page, search or select the document set.


  2. On the main page,  select the files you wish to annotate.



     
  3. You can select the files using the check boxes listed against the files. 
  4. Use to expand the view of the listed files against a folder. 

Note: Select a minimum of 10 documents to proceed with training. However, we recommend having a volume of 25 documents or more to achieve higher accuracy.

  1. Click Proceed to continue with the annotation process.

Step 5: Annotate

Data annotation is the process of labeling data to show the outcome you want your machine learning model to predict.
You can only annotate the Asset with the existing fields, tables and sections. You cannot add any new fields or tables while retraining the Asset.

For more information on how to annotate fields, tables and sections, see Annotate fields, Annotate a table and Annotate Section.

Step 6: Train

Once you complete the annotation, you can proceed to train the asset.

 For more information on training an asset, see Train.

Step 7: Review results and validate

This step allows you to assess the Asset’s predictions, accuracy, and confidence score. Additionally, you can utilize the Validate feature to evaluate the extraction Asset’s performance on a new set of documents.

For more information about  reviewing results and validation, see Review results and validate.

Step 8: Publish the asset

If the desired accuracy has been achieved, you can proceed to Publish the asset. For more information on how to publish the Asset, see the Publish the asset. 

Note: To identify the retrained Asset, check its version. For example, if the initial Asset version is 1.0, the retrained Asset’s version will be 2.0. You can avail both versions in the Asset Studio. 

Once the retrained Asset is published, you can download the API and its documentation. The API can be invoked independently or used within a specific use case. If you wish to consume this Asset via API, see Consume an Asset via API.

It is recommended to use URL aliases, if you wish to consume multiple versions of an Asset. It allows you to consume its different versions via a single API. For more information, see URL aliases.

You can also consume this Asset in the Asset Monitor module. For more information, see Consume an Asset via Create Transaction.

Manage Users and Permissions

User management is a feature that allows you to create and manage the users and their permissions within the platform. The platform offers predefined policies that can be assigned to users to ensure they have secure and appropriate access to specific operations and resources, which improves efficiency and security.

This guide will walk you through the steps on how to create and manage users, and how to control user access.

Users

The Users section in the Administration module allows you to create, manage and delete users on the platform. The following are the operations that you can perform in the Users section. You must have the Administrator policy to perform these operations:

Create a new user

This section provides instructions on how to create a new user account on the platform.

Note: This feature is not applicable for the Single Sign On (SSO) users.

The platform can be integrated with SSO, allowing you to use your organization’s login credentials to access the platform. You can only access the platform with Viewer Policy during your initial login. Your access Policy will change based on your roles and responsibilities. Contact the Platform Administrator if you wish to modify your policy.

  1. Head to the Administration module and select User management.



  2. In the Users tab, click Create User.


  1. On the Create new user page that appears, enter the First name, Last Name, a unique User name and Email.
  2. Select the appropriate predefined Policies from the drop down list and then click Submit. 

Note: You can assign multiple policies to a user.

Note: The platform offers predefined policies that are configured based on attributes. To customize these policies, contact our support team at mapsupport@intellectdesign.com. For more information about policies, see Policies  section.

Edit user account information

This section provides instructions on how to edit user account information on  the platform. 

Note: For SSO users, only the policies field can be edited.

  1. Head to the Administration module and select User management.



  2. In the Users tab, select the user whose information you wish to modify.



  3. On the User account information page that appears, click Edit account.



  4. Make the desired changes to the user’s information and click Submit.


Note: You can assign multiple policies to a user. 

Remove a user

This section provides instructions on how to delete an existing user from the platform.

Note: SSO users who no longer need access must be removed from the platform and the in-house SSO portal to prevent unauthorized access.

  1. Head to the Administration module and select User management.



  2. In the Users tab, select the user you wish to delete. 


  1. On the User account information page that appears, click and select Delete.

Reset password

This section provides instructions on how to reset the password. 

Note: This feature is not applicable for users who sign in using Single Sign On (SSO).

  1. Head to the Administration module and select User management.



  2. In the Users tab, select the user whose password you wish to reset.



  3. On the User account information page that appears, click Reset password. The respective user will then receive an email to reset their password.



Attribute based access control

Attribute Based Access Control (ABAC) is a method of controlling access for users based on their attributes (characteristics), the resource they are trying to access, and the environment in which the access is being requested. 

According to NIST, ABAC is defined as “an access control method where subject requests to perform operations on objects are granted or denied based on assigned attributes of the subject, assigned attributes of the object, environment conditions, and a set of policies that are specified in terms of those attributes and conditions.”

Access control methods protect your sensitive digital assets. The two most recommended access control methods today are role-based access control (RBAC) and attribute-based access control (ABAC). The main difference between RBAC and ABAC is the method of providing access. RBAC grants access based on roles, while ABAC grants access based on a wider range of factors, such as the user’s attributes, the object’s attributes, and the action being performed.

ABAC is more flexible and granular than RBAC, allowing for fine-grained control and dynamic decision-making. The Platform uses ABAC to control access to its resources. This means that users can only access the modules and permissions that they are authorized to access, based on their attributes, the resource they are trying to access, and the environment in which the access is being requested. The policies define which users have access to which modules and permissions. For more information about the policies, see Policies section.

Policies

A policy is the set of permissions that determines what actions a user can perform on the platform modules and its resources. For example, a policy can permit users to access one or more modules, and to some or all the actions associated with those modules. This approach allows users to only access the information and features that they need, which can improve security and usability.

Currently the platform offers six predefined policies such as Creator, Annotator, Viewer, Manager, Reviewer, and Administrator.

Note: The platform also offers customized policies. Contact our support team at mapsupport@intellectdesign.com to create customized policies.

Predefined policies

The following table displays the predefined policies along with their descriptions:

PolicyDescription
CreatorThe creator policy allows users to have full access to the Asset Studio, Document Library, Asset Monitor and Human-in-the-Loop modules. Creators can create and manage document sets, build, deploy, consume, and monitor assets. They also have limited access to the Administration module to only define and manage document types.
AnnotatorThe annotator policy allows users to only annotate assets in the Asset Studio module. Annotators can also delete annotations, import and export annotations, and view the history of annotations.
ViewerThe viewer policy allows users to access the platform modules such as Asset Studio, Asset Monitor and Human-in-the-Loop. Viewers are read-only users and cannot create, edit, or delete assets.
ManagerThe Manager policy allows users to access only the Asset Monitor module. Managers can monitor the assets, activate and deactivate assets, enable manual review, and set threshold limits.
ReviewerThe reviewer policy allows users to access only the Human-in-the-Loop module. Reviewers can review transactions, verify and correct the fields that have fallen below the threshold settings. 
AdministratorThe administrator policy allows users full access and permissions to all modules in the platform. Administrators have exclusive permissions to add and manage users and their permissions within the platform. They can also establish connectors for document upload.



The following table shows the modules and the permitted actions that users can perform using each policy:

ModulesPermissionsAdministrator
Creator
Viewer
Reviewer
Manager
Annotator
Administration Add, edit and remove users✔
Assign predefined policies✔
Create, edit and delete the document type(s) ✔✔
Add, configure and delete connectors✔
Add, edit and delete the URL aliases and Download the API✔
Document LibraryCreate, update and delete document set(s)✔✔
Asset StudioView all  assets, search for an Asset✔✔✔✔
Create, Annotate, Train, Fine-tune, Validate, Review results, Publish, Retrain, Download Asset API, Upload Asset image, Edit Asset name, and Delete an Asset.✔✔
Annotate documents, import, export annotation and review annotation ✔✔✔
Human-in-the-LoopView the list of transactions flagged for manual review✔✔✔✔
Review, verify and correct the fields that have fallen below threshold limits✔✔✔
Asset MonitorView performance metrics of operational assets, search for an Asset✔✔✔✔
Activate and deactivate assets✔✔✔
Enable manual review and set threshold limits✔✔✔
Consume an asset via Create Transaction✔✔✔

Assign a predefined policy

New User

You can assign a predefined policy to a new user when creating their account. For more information about assigning a predefined policy to a new user, see Create a new user section. 

Existing User

To update the policy for an existing user, follow the instructions below:

Note: If you are an SSO user, you can only access the platform with the Viewer Policy during your initial login. Contact the Platform Administrator if you wish to modify your policy.

  1. Head to the Administration module and select User management. 


  1. In the Users tab, select the user whose policy you wish to update.



  2. On the User account information page that appears, click Edit Account.


  1. In the Policies field, assign the appropriate policies.


     

Note: You can assign multiple policies to a user.

Enable Human in the Loop

Human-in-the -Loop (HITL) combines the power of human intelligence with machine automation, allowing humans to review and correct results when accuracy or confidence scores fall below a threshold. HITL ensures high-quality outcomes and continuous improvement through human intervention and feedback.

This guide will walk you through the steps on how to enable Human-in-the-Loop (HITL) for your asset.

Set accuracy threshold

Users must have any one of the following policies to enable manual review, and set threshold limits:

  • Administrator Policy
  • Creator Policy
  • Manager Policy
  1. In the Asset Monitor page, click next to the specific asset you want to set a threshold on and select Settings.


  2. A window will appear.



  3. Click on an asset and enable Manual Review.
  4. Set the threshold limit for the accuracy of respective fields. This determines the minimum acceptable accuracy level  required for each field to meet quality standards.
  5. Click on Next and ensure that the asset is activated. Then, click Submit.


Human-in-the-loop review

Users must have any one of the following policies to review transactions, verify and correct the fields that have fallen below the threshold settings:

  • Administrator Policy
  • Creator Policy
  • Reviewer Policy
  1. If the accuracy falls below the set threshold, the document will appear in the HITL (Human-in-the-Loop) queue.



  2. In the Human-in-the-Loop page, select the document for review. 



  3. By default, the Sort fields by need review toggle is enabled. This ensures that fields requiring review will appear at the top of the review section.
  4. To review and/or update a field, select the field and draw a bounding box on the document or manually edit the field value. 
  5. Once the field is reviewed, you can tab out and the Review needed icon disappears and is updated with the Reviewed icon. 
  6. Once you have reviewed the fields, click Submit to be directed to the Human in Loop transaction listing page or click Submit and Move Next to review the next transaction in the queue.

Monitor Assets

Monitoring key metrics is essential for assessing the performance and effectiveness of a document AI Asset that is consumed as an API. With the Asset Monitor module, you can actively monitor real-time transactions, view various metrics and Asset activation status.

Users must have any one of the following policies to monitor the performance of Assets:  

  • Administrator Policy
  • Creator Policy
  • Manager Policy
  • Viewer Policy

Monitor Asset

  1. Head to the Asset Monitor module, you can visualize the Asset information.



    • Optional: Click the Failed notification to know the failed reasons. 
    • Optional: Click the Review notification to review the documents. 

Grid columns 

  1. On the Asset Monitor page, click Grid column option.



  2. In the Grid columns that appear, select the required metrics for monitoring.



Note: The column information may differ based on the Assets. 

View Asset Metrics

  1. On the Asset Monitor module, click the respective asset for which  you wish to view the metrics details.



Note: If you are facing difficulty to find the Asset. See Find an Asset.

  1. In the Overview page that appears, you can evaluate the operational efficiency and usage patterns of an Asset from the following metrics.



     
    • Total transactions – Total transactions represent the overall number of API calls made to the asset during the specified period.
    • Documents processed: Indicates the cumulative number of processed documents, showcasing the asset’s effectiveness in handling various document types.
    • Pages processed:   Indicates the cumulative number of processed pages, showcasing the Asset’s effectiveness in handling various document types.
    • Average Duration:  Provides the average duration required to process a transaction, giving an overview of the efficiency and performance of the asset.
    • Transaction by status: Displays the distribution of transactions based on their status, allowing for easy identification of completed, in-progress, failed, or pending transactions.
    • Graph: Average processing time per transaction, Confidence score per transaction.
  1. Use Filter option to view the metrics for the required time frame to evaluate the operational efficiency and usage patterns of an Asset by analyzing the below metrics over a specific time period, ranging from the 24 hours to one year. Also you can customize the time frame.

Transaction

  1. Head to the Transaction tab.



  2. In the Transaction tab that appears, you can view the transaction information. 



Note: Use the Search Bar and Filter to find your transaction. If you wish to create a transaction from this page, see Consume an Asset via Create Transaction.  

  1. You can view transaction details including the Transaction ID, Status, Input tokens (or) No. of documents, completion tokens (or) No. of pages, model cost, initiated on,  Duration for further analysis and monitoring. 

Login to the Platform

Access your account to start leveraging the features of the platform. This guide will walk you through the steps on how to log in.


Login through email

You will receive an email from the platform team with the URL, user id and temporary password which the user can use to login. Open your preferred web browser and enter the platform URL “http://magicplatform.intellectai.com/” in the address bar and press Enter. 

  1. On the login page, enter your registered username or email address and click on Continue.


  1. Enter your password in the password field.  


  2. Click on Login to initiate the login process.
  3. If you have forgotten your password, you can reset it by clicking the Forgot password? link.
  4. If you are still having trouble logging in, you can contact customer support for help.

Login through SSO

We also provide SSO capability. If you have opted for SSO integration and login, you would be able to login with your network ID on your inhouse portals.

Platform modules

  1. Click on and you can view the list of modules in the platform.



  • The Asset Studio module allows you to build and manage the below document AI models which we refer to as “Assets.”
    • Classifier (Classic AI) Asset
    • Extractor (Classic AI) Asset
    • Use case Asset
  • The Document Library module allows you to create and manage document sets required for training and testing the Assets.
  • The Asset Monitor module allows you to monitor Assets in real time, enable human in the loop, activate/deactivate Assets and consume Assets.
  • The Human in Loop module allows you to review transactions and correct fields that have fallen below the threshold settings.
  • The Administration module allows you to define user management policies, establish connectors for document upload and administer the document types.

October 2024

Oct 6, 2024

Version: 1.6.1.2


Overview 

The latest update to our Magic platform introduces significant new performance enhancements, aimed at improving security and expanding the capabilities of our Assets, ensuring a more efficient platform. 

Announcement 

  • Integration of Classifier and Extractor Workflows: We are pleased to announce the integration of two powerful workflows—Classifier and Extractor—into a unified category known as Classic AI. Users can now create a Classic AI asset and select its type, either Classifier or Extractor, during the creation process.

    • User experience: This update simplifies the user experience, making the process more intuitive and user-friendly.

    • Streamline Workflow: This update streamlines the procedure and reduces complexity, allowing for a more efficient workflow.

Updates

  • Asset Visibility and Privacy: Added the ability for asset creators to designate assets as Public or Private while creating it. This provides full control over who can view and manage these assets, resulting in stronger privacy controls and more secure, effective asset management. It is applicable for the following assets.
  • Filter Enhancement: The update introduces significant enhancements to the filtering system, providing users with a more intuitive, efficient, and secure experience when managing and utilizing assets.

    • Improved Filter Selection: Enhanced the filter selection process to offer a more user-friendly and intuitive asset filtering experience.

    • Saved Filters: Introduced the ability to save filter settings, allowing users to quickly apply their preferred filters and display results based on saved preferences.

    • Filter Name Validation: Implemented a validation feature for filter names, ensuring all validation rules are followed to prevent errors or security threats, thereby enhancing the robustness and security of the filtering process.

    • Improved Filter Experience: Asset tags and user list options have been enhanced for a smoother and more efficient user experience.

    • Scope of Enhancement: These filter enhancements have been applied across four key areas, with slight variations based on the specific context:

      • Asset Studio: Filters help find and manage assets efficiently within the Asset Studio. For more information, see Utilizing Filter.
      • Asset Monitor: Filters assist in tracking and monitoring asset status within the Asset Monitor. For more information, see Utilizing Filter.


© ­Intellect Design Arena Ltd,