Video assessment modules now has the ability to support auto-marking of Video assessments and provide a 'Machine analysis score' for Administrators and Learners. This allows learners to review their video assessments before submitting and admins the ability to review the videos submissions alongside the machine analysis scores to decide on final scores, feedback and completion statuses.

How to enable this feature?

An Account owner navigates to Account > Litmos Feature > Video Assessment feature to enable this feature. Admin has the option to turn the feature on or off for individual video assessment module on need basis. 

Admin experience

Administrators will be able to 'enable video analysis' on video assessment modules and and set keywords to include and avoid at the module level.  Admin can now set the max time limit up-to 15 minutes. Keywords should be entered as separate single words and should be separated by commas with no sapce.

NOTE:

• Once the AI setting for a module is enabled or disabled it is not recommended to update the settings as this will affect the learner's experience who are already assigned to that module.
• A sentence will not be considered as a single keyword.

Once learner submits the video the administrators will be able to access submitted videos along with the machine analysis scores for each submission. The Administrator can then review the videos alongside the machine analysis scores to decide on final scores, feedback and completion statuses for the submissions. Once the Admin completes the markings, the machine analysis will not be needed and is no longer available to review for those video assessment attempts

NOTE:

• Machine analysis for Video assessment is only available to the reviewers that are logged into the system. 
• Once reviewer completes the markings, the machine analysis will not be available to Admin for those video assessment attempts.

Learner experience

Learner can record or upload their video assessment module and self- review the video before submitting for Admin review. 

Once learners record or upload their video assessments and click “analyze video”, learners will be prompted to ‘analyze and continue the course’ to initiate and allow machine analysis time to process the transcript.

Note: A 15 minute video will take on average 7-10 mins process but could take longer in some instance.

The learner will have to navigate back to the module listing within the course to view the machine analysis once the analysis is complete. Learner will see the machine analysis progress as 'Video analysis processing' or 'Video analysis complete' which is auto refreshed every few seconds so the learners see the latest update.

AI analyzes the learner’s transcript for emotional sentiment, rate of speech and keywords that the admin has configured.  

• Machine Analysis Score is sum of
1. Keywords to Include : Checks to see if the transcript has all the configured keywords to include.Partial keyword matches will not be included in the Machine analysis score. Scores based on unique keywords to include in transcript divided by keywords to include defined by Admin
2. Keywords to avoid: Checks to see if the transcript has any of the configured keywords to exclude. Partial keyword matches will not be included in the Machine analysis score. Scores based on total keywords to avoid minus unique keywords to avoid in transcript divided by total keywords to avoid
3. Rate of speech: Checks the rate of speech on the video and assigns the scores based on the tier values. Note: The tiers are based on general best practice and considers the conversational speech as the ideal range (120 to 180 WPM).

• Overall Sentiment  - Checks the tone of the Learner’s transcript to determine the Learner's sentiment and reflects on the progress bar.

Note:

• Translations are currently only supported for English language.
• The AI Video Assessment is available to the learner only on the Mobile web app
• Partial keyword matches will not be included in the Machine analysis score