Make AU support of moveOn criteria at runtime mandatory

[MrBillMcDonald]

At the June 12th meeting, the group discovered a conflict with moveOn requirements between the AU and LMS. The current spec language indicates that AU usage of moveOn is optional. This is problematic if the LMS issues a moveOn value inconsistent with the AU's design.

The group proposed the following changes to section 10.0 xAPI State Data Model - moveOn - AU usage :

The AU MUST always respect this value and send appropriate statements (e.g. AU must send a completed statement and a passed statement if moveOn is set to “CompletedAndPassed”) If the AU is unable to act on the moveOn value provided, it MUST gracefully exit and notify the learner that it cannot proceed.

[brianjmiller]

Couple of issues with this (without having given it much thought):

1. The completed and passed statements are intended for a registration level rather than a session level. It is entirely possible that one or more valid cmi5 sessions could happen where a completed and passed aren't issued only to have the learner return in a different session and have those statements issued at that time. There is also the explicit allowance of an LMS to change the moveOn criteria, so presumably it could be re-launched having changed this value and then the AU could reach a satisfied state which itself could be intentional behavior by either the AU or the LMS. (IOW requiring multiple launch sessions to reach a satisfied state.) Therefore suggesting the AU "MUST" exit seems to imply immediately rather than eventually, but the AU can't know whether it will be issuing those statements. If the intention is for the AU to know a pri ori that it won't ever (in any session) be issuing those statements then see the suggestion below.

2. Right now there is never a case where the spec suggests any required communication between the AU and the learner. This opens a massive can of worms for compatibility because of the implied UI/UX, language, system capability, etc. issues with including a requirement of "notify the learner".

Suggestion: the problems above (and without having been on the call, the implicit intention) suggests that the AU can know ahead of time whether it will be able to comply with a particular moveOn criteria value, suggesting that it would be better for the AU itself to inform the LMS at the time of package import which possible moveOn values can be satisfied by the AU. IOW, if the intention is to prevent faulty launching by an LMS system to an AU based on the LMS' decision of what the moveOn criteria will be it would make more sense to front load this error handling by advising the LMS what possible moveOn criteria can be used with a particular AU therefore preventing the majority of faulty launches before the learner could even hit them. This would be a course structure change and would be additive.

This won't solve 100% of the cases where an LMS behaves in an incompatible way, so if the desire remains to have a requirement on the AU that it must check the provided moveOn criteria to see whether it will ever follow it I think the requirements need to be significantly tightened down to include the issuing of a statement to that effect or inclusion of an extension in the terminated statement to distinguish why the AU terminated in such a manner (because it was not based on an action of the learner). And the language around notifying the learner should be removed because it is significantly problematic in the specification itself (it can't be tested), while being pretty obvious to the AU developer that they need to use whatever error/notification system they already have in place for communicating to the learner. (IOW reasonable content will do it anyways, unreasonable content probably doesn't care about following the spec to begin with).

If we get to this point, then I think it should be codified that the exit needs to occur before issuing the "initialized" statement as the cmi5 session was not in fact correctly initialized.

Finally, adding a MUST requirement (or set of requirements as described above) should be considered a breaking change and should only be done within the scope of a "major release" of the specification which carries with it all sorts of additional problems because there are implementations already in use that will not have compatibility with those new requirements.

I'll have to read the meeting minutes to hopefully see more about why there is a conflict, AFAICT the only reason this would be a problem is that the registration would never reach a "satisfied" statement being issued, but I suspect the vast majority of cases would still be functional, and there is already the "waived" potential in the spec. And I suspect the real world handling of this is that content is tested before being issued to a learner and the LMS user+LMS admin will realize that their implementation means they can never reach a satisfied state and correct the behavior (if such a state is needed). I think at best this solution is in search of a problem, or worse causes a set bigger ones.

[bscSCORM]

I agree with what @brianjmiller is saying about this, and will add that while I think it would be nice if the AU properly declared what move on criteria it handled so this sort of issue could be detected, in practice the LMS administrator probably shouldn't 100% trust such a declaration. Where compatibility is really important they should test the content when importing, to check for support of a particular moveOn criteria or anything else is required for content to smoothly work with the LMS.

[MrBillMcDonald]

Ben & Brian,

Thanks so much for your comments !

In the June 19 cmi5 WG meeting, we reviewed them and discussed an alternative approach.

The current thinking is to add a new AU metadata element to the course structure. The new element would be called moveOnAllowed and it would define a list of moveOn criteria that the AU is capable of handling.

This will be a proposed spec change for the next "major release". We will discuss the new wording at the next meeting (June 26th)

See minutes:

https://github.com/AICC/CMI-5_Spec_Current/wiki/CMI-5-Working-Group-Meeting-Minutes-%E2%80%93-June-19th

[MrBillMcDonald]

closed per Feb 9th meeting

[MrBillMcDonald]

Review Feb 9th for new feature consideration