Attribute hierarchy method
≈ 19 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
The attribute hierarchy method (AHM), is a cognitively based psychometric procedure developed by Jacqueline Leighton, Mark Gierl, and Steve Hunka at the Centre for Research in Applied Measurement and Evaluation (CRAME) at the University of Alberta. The AHM is one form of cognitive diagnostic assessment that aims to integrate cognitive psychology with educational measurement for the purposes of enhancing instruction and student learning. A cognitive diagnostic assessment (CDA), is designed to measure specific knowledge states and cognitive processing skills in a given domain. The results of a CDA yield a profile of scores with detailed information about a student’s cognitive strengths and weaknesses. This cognitive diagnostic feedback has the potential to guide instructors, parents and students in their teaching and learning processes. To generate a diagnostic skill profile, examinees’ test item responses are classified into a set of structured attribute patterns that are derived from components of a cognitive model of task performance. The cognitive model contains attributes, which are defined as a description of the procedural or declarative knowledge needed by an examinee to answer a given test item correctly. The inter-relationships among the attributes are represented using a hierarchical structure so the ordering of the cognitive skills is specified.…
The attribute hierarchy method (AHM), is a cognitively based psychometric procedure developed by Jacqueline Leighton, Mark Gierl, and Steve Hunka at the Centre for Research in Applied Measurement and Evaluation (CRAME) at the University of Alberta. The AHM is one form of cognitive diagnostic assessment that aims to integrate cognitive psychology with educational measurement for the purposes of enhancing instruction and student learning. A cognitive diagnostic assessment (CDA), is designed to measure specific knowledge states and cognitive processing skills in a given domain. The results of a CDA yield a profile of scores with detailed information about a student’s cognitive strengths and weaknesses. This cognitive diagnostic feedback has the potential to guide instructors, parents and students in their teaching and learning processes. To generate a diagnostic skill profile, examinees’ test item responses are classified into a set of structured attribute patterns that are derived from components of a cognitive model of task performance. The cognitive model contains attributes, which are defined as a description of the procedural or declarative knowledge needed by an examinee to answer a given test item correctly. The inter-relationships among the attributes are represented using a hierarchical structure so the ordering of the cognitive skills is specified. This model provides a framework for designing diagnostic items based on attributes, which links examinees' test performance to specific inferences about examinees' knowledge and skills. The hierarchy contains two independent branches which share a common prerequisite – attribute A1. Aside from attribute A1, the first branch includes two additional attributes, A2 and A3, and the second branch includes a self-contained sub-hierarchy which includes attributes A4 through A9. Three independent branches compose the sub-hierarchy: attributes A4, A5, A6; attributes A4, A7, A8; and attributes A4, A9. As a prerequisite attribute, attribute A1 includes the most basic arithmetic operation skills, such as addition, subtraction, multiplication, and division of numbers. Attributes A2 and A3 both deal with factors. In attribute A2, the examinee needs to have knowledge about the property of factors. In attribute A3, the examinee not only requires knowledge of factoring (i.e., attribute A2), but also the skills of applying the rules of factoring. Therefore, attribute A3 is considered a more advanced attribute than A2. The self-contained sub-hierarchy contains six attributes. Among these attributes, attribute A4 is the prerequisite for all other attributes in the sub-hierarchy. Attribute A4 has attribute A1 as a prerequisite because A4 not only represents basic skills in arithmetic operations (i.e., attribute A1), but it also involves the substitution of values into algebraic expressions which is more abstract and, therefore, more difficult than attribute A1. The first branch in the sub-hierarchy deals, mainly, with functional graph reading. For attribute A5, the examinee must be able to map the graph of a familiar function with its corresponding function. In an item that requires attribute A5 (e.g., item 4), attribute A4 is typically required because the examinee must find random points in the graph and substitute the points into the equation of the function to find a match between the graph and the function. Attribute A6, on the other hand, deals with the abstract properties of functions, such as recognizing the graphical representation of the relationship between independent and dependent variables. The graphs for less familiar functions, such as a function of higher-power polynomials, may be involved. Therefore, attribute A6 is considered to be more difficult than attribute A5 and placed below attribute A5 in the sub-hierarchy. The second branch in the sub-hierarchy considers the skills associated with advanced substitution. Attribute A7 requires the examinee to substitute numbers into algebraic expressions. The complexity of attribute A7 relative to attribute A4 lies in the concurrent management of multiple pairs of numbers and multiple equations. Attribute A8 also represents the skills of advanced substitution. However, what makes attribute A8 more difficult than attribute A7 is that algebraic expressions, rather than numbers, need to be substituted into another algebraic expression. The last branch in the sub-hierarchy contains only one additional attribute, A9, related to skills associated with rule understanding and application. It is the rule, rather than the numeric value or the algebraic expression that needs to be substituted in the item to reach a solution. \begin{bmatrix} 0&1&0&1&0&0&0&0&0\\ 0&0&1&0&0&0&0&0&0\\ 0&0&0&0&0&0&0&0&0\\ 0&0&0&0&1&0&1&0&1\\ 0&0&0&0&0&1&0&0&0\\ 0&0&0&0&0&0&0&0&0\\ 0&0&0&0&0&0&0&1&0\\ 0&0&0&0&0&0&0&0&0\\ 0&0&0&0&0&0&0&0&0 \end{bmatrix} Each row and column the A matrix represents one attribute; the first row and column represents attribute A1 and the last row and column represents attribute A9. The presence of a 1 in a particular row denotes a direct connection between that attribute and the attribute corresponding to the column position. For example, attribute A1 is directly connected to attribute A2 because of the presence of a 1 in the first row (i.e. attribute A1) and the second column (i.e., attribute A2). The positions of 0 in row 1 indicate that A1 is neither directly connected to itself nor to attributes A3 and A5 to A9. The direct and indirect relationships among attributes are specified by the binary reachability matrix (R) of order (k,k), where k is the number of attributes. To obtain the R matrix from the A matrix, Boolean addition and multiplication operations are performed on the adjacency matrix, meaning R=(A+I)^n where n is the integer required to reach invariance, n = 1,2,\cdots ,m , and I is the identity matrix. The R matrix for the Ratio and Algebra hierarchy is shown next. \begin{bmatrix} 1&1&1&1&1&1&1&1&1\\ 0&1&1&0&0&0&0&0&0\\ 0&0&1&0&0&0&0&0&0\\ 0&0&0&1&1&1&1&1&1\\ 0&0&0&0&1&1&0&0&0\\ 0&0&0&0&0&1&0&0&0\\ 0&0&0&0&0&0&1&1&0\\ 0&0&0&0&0&0&0&1&0\\ 0&0&0&0&0&0&0&0&1 \end{bmatrix} Similar to the A matrix, each row and column in the matrix represents one attribute; the first row and column represents attribute A1 and the last row and column represents attribute A9. The first attribute is either directly or indirectly connected to all attributes A1 to A9. This is represented by the presence of 1's in all columns of row 1 (i.e., representing attribute A1). In the R matrix, an attribute is considered related to itself resulting in 1's along the main diagonal. Referring back to the hierarchy, it is shown that attribute A1 is directly connected to attribute A2 and indirectly to A3 through its connection with A2. Attribute A1 is indirectly connected to attributes A5 to A9 through its connection with A4. The potential pool of items is represented by the incidence matrix (Q) matrix of order (k, p), where k is the number of attributes and p is number of potential items. This pool of items represents all combinations of the attributes when the attributes are independent of each other. However, this pool of items can be reduced to form the reduced incidence matrix (Q r ), by imposing the constraints of the attribute hierarchy as defined by the R matrix. The Q r matrix represents items that capture the dependencies among the attributes defined in the attribute hierarchy. The Q r matrix is formed using Boolean inclusion by determining which columns of the R matrix are logically included in each column of the Q matrix. The Q r matrix is of order (k,) where k is the number of attributes and i is the reduced number of items resulting from the constraints in the hierarchy. For the Ratio and Algebra hierarchy, the Q r matrix is shown next. \begin{bmatrix} 1&1&1&1&1&1&1&1&1\\ 0&1&1&0&0&0&0&0&0\\ 0&0&1&0&0&0&0&0&0\\ 0&0&0&1&1&1&1&1&1\\ 0&0&0&0&1&1&0&0&0\\ 0&0&0&0&0&1&0&0&0\\ 0&0&0&0&0&0&1&1&0\\ 0&0&0&0&0&0&0&1&0\\ 0&0&0&0&0&0&0&0&1 \end{bmatrix} The Q r matrix serves an important test item development blueprint where items can be created to measure each specific combination of attributes. In this way, each component of the cognitive model can be evaluated systematically. In this example, a minimum of 9 items are required to measure all the attribute combinations specified in the Q r matrix. The expected examinee response patterns can now be generated using the Q r matrix. An expected examinee is conceptualized as a hypothetical examinee who correctly answers items that require cognitive attributes that the examinee has mastered. The expected response matrix (E) is created, using Boolean inclusion, by comparing each row of the attribute pattern matrix (which is the transpose of the Q r matrix) to the columns of the Q r matrix. The expected response matrix is of order (j,i), where j is the number of examinees and i is the reduced number of items resulting from the constraints imposed by the hierarchy. The E matrix for the Ratio and Algebra hierarchy is shown below. \begin{bmatrix} 0&0&0&0&0&0&0&0&0\\ 1&0&0&0&0&0&0&0&0\\ 1&1&0&0&0&0&0&0&0\\ 1&1&1&0&0&0&0&0&0\\ 1&0&0&1&0&0&0&0&0\\ 1&1&0&1&0&0&0&0&0\\ 1&1&1&1&0&0&0&0&0\\ 1&0&0&1&1&0&0&0&0\\ 1&1&0&1&1&0&0&0&0\\ 1&1&1&1&1&0&0&0&0\\ 1&0&0&1&1&1&0&0&0\\ 1&1&0&1&1&1&0&0&0\\ 1&1&0&1&1&1&0&0&0\\ 1&0&0&1&0&0&1&0&0\\ 1&1&0&1&0&0&1&0&0\\ 1&1&1&1&0&0&1&0&0\\ 1&0&0&1&1&0&1&0&0\\ 1&1&0&1&1&0&1&0&0\\ 1&1&1&1&1&0&1&0&0\\ 1&0&0&1&1&1&1&0&0\\ 1&1&0&1&1&1&1&0&0\\ 1&1&1&1&1&1&1&0&0\\ 1&0&0&1&0&0&1&1&0\\ 1&1&0&1&0&0&1&1&0\\ 1&1&1&1&1&0&1&0&0\\ 1&0&0&1&1&0&1&1&0\\ 1&1&0&1&1&0&1&1&0\\ 1&1&1&1&1&0&1&1&0\\ 1&0&0&1&1&1&1&1&0\\ 1&1&0&1&1&1&1&1&0\\ 1&1&1&1&1&1&1&1&0\\ 1&0&0&1&0&0&0&0&1\\ 1&1&0&1&0&0&0&0&1\\ 1&1&1&1&0&0&0&0&1\\ 1&0&0&1&1&0&0&0&1\\ 1&1&0&1&1&0&0&0&1\\ 1&1&1&1&1&0&0&0&1\\ 1&0&0&1&1&1&0&0&1\\ 1&1&0&1&1&1&0&0&1\\ 1&1&1&1&1&1&0&0&1\\ 1&0&0&1&0&0&1&0&1\\ 1&1&0&1&0&0&1&0&1\\ 1&1&1&1&0&0&1&0&1\\ 1&0&0&1&1&0&1&0&1\\ 1&1&0&1&1&0&1&0&1\\ 1&1&1&1&1&0&1&0&1\\ 1&0&0&1&1&1&1&0&1\\ 1&1&0&1&1&1&1&0&1\\ 1&1&1&1&1&1&1&0&1\\ 1&0&0&1&0&0&1&1&1\\ 1&1&0&1&0&0&1&1&1\\ 1&1&1&1&0&0&1&1&1\\ 1&0&0&1&1&0&1&1&1\\ 1&1&0&1&1&0&1&1&1\\ 1&1&1&1&1&0&1&1&1\\ 1&0&0&1&1&1&1&1&1\\ 1&1&0&1&1&1&1&1&1\\ 1&1&1&1&1&1&1&1&1 \end{bmatrix} If the cognitive model is true, then 58 unique item response patterns should be produced by examinees who write these cognitively-based items. A row of 0s is usually added to the E matrix which represents an examinee who has not mastered any attributes. To summarize, if the attribute pattern of the examinee contains the attributes required by the item, then the examinee is expected to answer the item correctly. However, if the examinee's attribute pattern is missing one or more of the cognitive attributes required by the item, the examinee is not expected to answer the item correctly. Referring back to the pictorial representation of Ratio and Algebra hierarchy, an item can be constructed to measure the skills described in each of the attributes. For example, attribute A1 includes the most basic arithmetic operation skills, such as addition, subtraction, multiplication, and division of numbers. An item that measures this skill could be the following: examinees are presented with the algebraic expression 4(t + u) + 3 = 19 , and asked to solve for (t + u). For this item, examinees need to subtract 3 from 19 and then divide 16 by 4. Attribute A2 represents knowledge about the property of factors. An example of an item that measures this attribute is "If (p + 1)(t – 3) = 0 and p is positive, what is the value of t?” The examinee must know the property that the value of at least one factor must be zero if the product of multiple factors is zero. Once this property is recognized, the examinee would be able to recognize that because p is positive, (t – 3) must be zero to make the value of the whole expression zero, which would finally yield the value of 3 for t. To answer this item correctly, the examinee should have mastered both attributes A1 and A2. Attribute A3 represents not only knowledge of factoring (i.e., attribute A2), but also the skills of applying the rules of factoring. An example of an item that measures this attribute is “ \mbox{If } \frac{x+y}{a-b} = \frac{2}{3} \mbox{ , then } \frac{9x+9y}{10a-10b} = ? ”. Only after the examinee factors the second expression into the product of the first expression would the calculation of the value of the second expression be apparent. To answer this item correctly, the examinee should have mastered attributes A1, A2, and A3. HCI_i = 1 - \frac{\sum_{j=1}^J\sum_{g\in s_j} X_{i_j}(1-X_{i_g})}{N_{c_i}} where J is the total number of items, X i j is examinee i ‘s score (i.e., 1 or 0) to item j, S j includes items that require the subset of attributes of item j, and N c i is the total number of comparisons for correctly answered items by examinee i. The values of the HCI range from −1 to +1. Values closer to 1 indicate a good fit between the observed response pattern and the expected examinee response patterns generated from the hierarchy. Conversely, low HCI values indicate a large discrepancy between the observed examinee response patterns and the expected examinee response patterns generated from the hierarchy. HCI values above 0.70 indicate good model-data fit.
Text derived from https://en.wikipedia.org/w/index.php?oldid=1309519371 under CC BY-SA 4.0; modifications have been made.