In a Knewton-powered learning application, goals help express the scope and pedagogy used to guide the recommendations delivered to a learner. Product designers and instructors can use goals to tailor a product or course to their needs.
See our goal api specs to build your first goal.
In this section:
Knewton provides near real-time recommendations to guide learners toward completion of their individual learning objectives. Recommendations may consist of instructional and assessment material. But how does the learning application tell Knewton what a particular learner’s objectives are? How does an instructor guide a class, group, or individual toward specific objectives?
In a Knewton-powered learning application, goals help express the scope and pedagogy used to guide the recommendations delivered to a learner. A goal can be used in the context of large assignments (“achieve proficiency on this week’s ten learning objectives”) or more narrowly specified ones (“see initial instruction and a few questions on the two learning objectives we’ll cover tomorrow”). Product designers and instructors can use goals to tailor a product or course to their needs.
Goals are also used as context for some of Knewton’s analytics. Consider as an example an instructor, Mr. Smith. Mr. Smith might create a goal that requires learners to achieve proficiency with a set of learning objectives that will be on a quiz this coming Friday. He can use Knewton-powered analytics to track progress on that goal. “How many students have started or completed this goal? What progress have my students made toward proficiency on the assigned learning objectives?” The analytics used to answer these questions reflect learners’ activity on the goal. Note that this is not the case for all available analytics: Mr. Smith might ask “What score would each student be likely to receive if they took the quiz today?” The Predicted Score metric would answer this question by considering learner activity across potentially many goals.
By varying the number of goals, scope of each goal, and other goal parameters, a learning application can fulfill a wide variety of product visions for any configuration of learners. Goals give instructors, administrators, learners, and other users of the learning application a great deal of flexibility. Any authorized user can use goals to create targeted homework assignments, configure broad review centers, and generally structure the overall learning experience in accordance with the pedagogical and instructional functions of that particular learning application.
Goals can apply to individuals or groups of learners. In either case, recommendations requested for the goal are optimized for each individual learner.
Applications should use Partner Admin credentials to create goals on behalf of instructors or learners. The time it takes for a new goal to be created is a function of the number of target and recommendable modules in the goal and, if it is being assigned at creation, the number of registrations it is being assigned to. While most responses are quite quick, in the worst case, response times can be up to 10 seconds to create and assign very large goals.
The sections below describe the options available to learning applications when creating goals.
The “targets” of a goal indicate what content learners are expected to master by satisfying the goal’s completion criteria. Additionally, learners may see non-target content while working on a goal when deemed appropriate for the goal’s adaptive behavior and remediation depth.
Targets may be specified via the targets parameter in a goal creation or update request as learning objective (LO) IDs (See Content Identifiers).
Targets may alternately be specified as module IDs. If target modules are specified, then Knewton infers a set of target LOs that contains all LOs that are assessed by at least one of the specified target modules. Directly specifying target LOs is preferred.
In some applications, a single adaptive assignment may cover learning objectives (LOs) across
several loosely related topics. In this case, it is usually undesirable for the
recommender to switch focus from one LO to a different LO in another topic. The
experience is more coherent for learners if they can complete all of the LOs in one topic
before moving to the next topic. The goals API enables this behavior by allowing
target LOs to be specified in groups via the topics parameter in the
request body rather than as a flat list via the targets parameter. Target
modules may not be used with topics.
Note that from the Knewton Platform perspective, a goal’s topics are just ad hoc groups:
The “scope” of a goal defines a “recommendable pool,” which is the set of content items that the recommender can draw from when a recommendation is requested. Thus, the scope controls what content learners may encounter when working on the adaptive assignment.
The simplest way to specify the scope of a goal is to leverage the learning objective (LO)
prerequisite relationships encoded in the graph, and to define the recommendable pool
as containing all of the content that is covered by LOs within a
certain number of prerequisite hops away from the goal’s target LOs. This approach can also
be thought of as putting a limit on how far back in a content domain the recommender
will take a struggling student when remediating them. The scope.remediation_depth
parameter in the request body is used to set the desired value. If our LO prerequisite
graph looked like this, with LO A being the goal’s only target LO:
Then the available remediation_depth values will have the following effects:
none: Only content covered by LO A will be recommended.one: Initially content covered by LO A will be recommended. For struggling students, remedial content covered by LOs B and C may be recommended.two: Same as for one, but students who continue to struggle on LOs B or C may be recommended content covered by D and E, or F and G, respectively.three: Same as for two, but remediating all the way back to LOs H, I, and J if necessary.maximum: Same as for three, but remediating all the way back to LOs K and L if neccessary (and beyond, if there were futher prerequisite relationships).Note that there is an entity limit on the number of modules that can be included in a goal’s recommendable pool (see Entity Limits). If this limit is reached, modules closer to the goal’s targets in terms of prerequisite distance will be prioritized for inclusion over those further from the goal’s targets.
In some cases it can be useful to prevent certain content from being recommended
that would otherwise be included in the recommendable pool. Specific modules or groups
of modules tagged with specific taxons can be excluded from the recommendable pool
using the scope.exclude parameter in the request body.
The recommendable pool can also be defined explicitly by including specific modules or
groups of modules tagged with specific taxons using the scope.include parameter in
the request body. The same results can generally be achieved with less effort using a
combination of scope.remediation_depth and scope.exclude, so this method of
specifying the goal scope is deprecated.
The completion_criteria parameter of the request body may be specified to configure various
preconditions for completion of the goal. Three parameters may be used to configure a goal’s completion criteria.
completion_criteria.min_predicted_mastery refers to the knowledge state at which Knewton’s proficiency model will deem the student sufficiently proficient to complete an LO. Setting a lower min_predicted_mastery will allow students to complete a goal with a generally lower rate of accuracy; conversely, a higher min_predicted_mastery will require students to demonstrate a higher level of proficiency with target material before completing an LO.completion_criteria.min_work_per_target refers to the minimum number of unique questions a student must answer before completing an LO, regardless of their knowledge state. Students may require more questions to reach a sufficient level of min_predicted_mastery, but each student will be required to answer at least min_work_per_target questions on each LO.completion_criteria.max_work_on_goal allows a student to complete a goal after a specified number of unique questions, even if the above criteria are not satisfied. If the student satisfies the other completion criteria in fewer than max_work_on_goal interactions, the goal may be completed sooner. This parameter can cause unusual or unbalanced assignment experiences (e.g., working for most the goal on just one of the goal’s several target LOs), and it is only recommended for special circumstances.Each goal target will be considered complete when all of the specified criteria have been satisfied. The entire goal will be considered complete when either:
completion_criteria.
min_predicted_mastery and completion_criteria.
min_work_per_target)completion_criteria.max_work_on_goal questionsThe mastery computation used to check against the completion_criteria.min_predicted_mastery
parameter also underpins the
Status & Progress analytic. Therefore,
it is important for a consistent user experience that integrations utilizing the Status And
Progress analytic to report assignment progress to students and instructors create goals
using flexible goal completion criteria.
A target score between 0 and 1 may be specified via the targets.score parameter of the request
body. Each goal target will be considered complete for a registration when the Expected Score
analytic value for that target exceeds the specified target score for the registration.
The targets.completion_behavior parameter of the request body is used to control when the
entire goal will be considered complete in the context of target score.
all: The goal will be complete when each of the targets is completed.average (DEPRECATED): The goal will be complete when the average Expected Score across all targets
exceeds the specified target score.It is important for a consistent user experience that integrations utilizing the deprecated Expected Score analytic to report assignment progress to students and/or instructors create goals using target score completion criteria.
Aside from specifying what content and completion criteria are relevant to a given adaptive experience, scoped goals can also include an adaptive_behavior field (optional) to choose from a range of recommendation behaviors appropriate to different use cases. The adaptive_behavior for a given goal tells the recommendations engine how to structure the pedagogy for an adaptive experience:
| adaptive_behavior | Example goal use case | Pedagogical overview |
|---|---|---|
| practice (default) | Homework after class | Work on assessment questions until completing the assignment or requiring just-in-time instruction or remediation. |
| introduce | Pre-lecture preparation | Begin with mandatory instructional material on each learning objective and proceed to assessment questions. |
| review | Revisiting material before an exam | Interleave assessment questions on the target learning objectives; if a gap is demonstrated, focus on that learning objective for practice and instruction. Note: For review, remediation_depth should be set to 0, for clarity. (Review does not include remediation, regardless of the remediation_depth set on the goal.) |
For each adaptive_behavior, the recommendations engine consults content parameters associated with the targets’ Knowledge Graph (generated from the partner-provided Content Inventory) to structure the resulting adaptive experience.
Once created, a goal’s adaptive_behavior cannot be changed via a goal update. Further technical details for goal management can be found in the reference guide.
Note: adaptive_behavior can only be used with completion_criteria, not targets.score.
A goal must be assigned to a registration in order for recommendations and analytics to be generated. A single goal can be assigned to many registrations. This assignment can happen in one of three approaches:
When a goal is completed by a learner, Knewton returns a status of “ready” in the recommendation response. Additionally, Knewton by default returns an empty set of modules in the recommendation response when the learner moves to the ready status.
An existing goal can be updated to reflect a change that has occurred. For example, an instructor may choose to give an extension to an assignment in the learning application. The goal would therefore be updated with a new duration or end date.
Changes to an existing goal will be applied to all learners who have the goal assigned. Additionally, any learners who have the goal assigned after the change will work against the updated version.
Recommendations and analytics for this updated goal will refresh when a new event is received for each registration (e.g. graded event, ungraded event, focus event).
In order to check goal completion progress by retrieving Status & Progress analytics metrics, the application must define a goal with the following configuration:
targets.include should be specified with target learning objectivestargets.completion_behavior: “all”completion_criteria should be specified (instead of targets.score)Also, the application must specify a goal_id when submitting a Graded Event for it to impact a goal’s status & progress.
Getting Started
Working with Adaptive Assignments
Predictive Learner Analytics
General API Usage
Brand Guidelines
Glossary