Dissecting the Cognitive Landscape: Literary vs. Scientific Intellect at Cogn-IQ.org

Analyzing the multifaceted nature of human intellect, our recent study casts light on how cognitive abilities distinctly manifest in literary and scientific contexts. Using multidimensional scaling on JCCES and ACT assessment results from 60 participants, we discerned a bifurcated structure: one dimension differentiating literary from scientific tasks, another contrasting JCCES and ACT assessments. This nuanced perspective reinforces the argument for a tailored approach to education, one that respects the divergent cognitive demands of different academic domains. Furthermore, it stresses the necessity for educators and psychologists to be discerning when interpreting test scores, acknowledging the context-dependent nature of cognitive assessments. 

Our findings suggest that the journey to educational excellence must accommodate the intricate dichotomy between literary finesse and scientific acumen. The insights gleaned are not merely academic; they offer a pragmatic blueprint for enhancing educational tactics and refining assessment tools, ensuring they capture the essence of domain-specific competencies. While the study's exploratory nature necessitates further inquiry, particularly into the underlying cognitive and neural substrates, it establishes a foundational framework for future research aimed at optimizing learning and testing strategies. The horizon of cognitive assessment beckons with the promise of greater clarity, urging a pivot toward more nuanced and contextualized understandings of intelligence.

Link to Full Article: Jouve, X. (2016). Multidimensional Structure Of Cognitive Abilities: Differentiating Literary And Scientific Tasks In JCCES And ACT Assessments. https://www.cogn-iq.org/articles/multidimensional-structure-of-cognitive-abilities-differentiating-literary-and-scientific-tasks-in-jcces-and-act-assessments.html

Exploring the Underlying Dimensions of Cognitive Abilities: A Multidimensional Scaling Analysis of JCCES and GAMA Subtests

Abstract

This study aimed to investigate the relationships between tasks of the Jouve Cerebrals Crystallized Educational Scale (JCCES) and General Ability Measure for Adults (GAMA) using multidimensional scaling (MDS) analysis. The JCCES measures Verbal Analogies, Mathematical Problems, and General Knowledge, while the GAMA assesses nonverbal cognitive abilities through Matching, Analogies, Sequences, and Construction tasks. A total of 63 participants completed both assessments. MDS analysis revealed a 2-dimensional solution, illustrating a diagonal separation between nonverbal and verbal abilities, with Mathematical Problems slightly closer to the verbal side. Seven groups were identified, corresponding to distinct cognitive processes. The findings suggest that JCCES and GAMA tasks are not independent and share common underlying dimensions. This study contributes to a more nuanced understanding of cognitive abilities, with potential implications for educational, clinical, and research settings. Future research should address the study's limitations, including the small sample size and potential methodological constraints.

Keywords: cognitive abilities, JCCES, GAMA, multidimensional scaling, verbal abilities, nonverbal abilities, fluid intelligence, crystallized intelligence

Introduction

The study of cognitive abilities has been an area of significant interest in the field of psychometrics, which aims to develop and refine methods for assessing individual differences in mental capabilities (Embretson & Reise, 2000). Among the diverse cognitive abilities, crystallized and fluid intelligence have been particularly influential constructs in the understanding of human cognition (Cattell, 1963). Crystallized intelligence refers to the acquired knowledge and skills, while fluid intelligence reflects the capacity for abstract reasoning and problem-solving, independent of prior knowledge or experience (Cattell, 1963; Horn & Cattell, 1966). Various instruments have been developed to assess these cognitive abilities, including the Jouve Cerebrals Crystallized Educational Scale (JCCES; Jouve, 2010a) and the General Ability Measure for Adults (GAMA; Naglieri & Bardos, 1997).

Although JCCES and GAMA are used as independent measures of crystallized and nonverbal cognitive abilities, respectively, the relationships between the tasks within these instruments remain less explored. Previous research has identified separate factors for JCCES and GAMA subtests (Jouve, 2010b), but a more detailed investigation into the underlying cognitive processes is warranted. Multidimensional scaling (MDS) is a statistical technique that can provide insight into the relationships between tasks by representing them as points in a multidimensional space (Cox & Cox, 2001; Borg & Groenen, 2005). The present study aims to apply MDS to analyze the relationships between the tasks of JCCES and GAMA, in order to identify common underlying dimensions and provide a more nuanced understanding of the cognitive abilities assessed by these instruments.

The literature on cognitive abilities suggests that tasks within JCCES and GAMA may not be entirely independent and could share some common underlying dimensions (Carroll, 1993; Spearman, 1927). For instance, the verbal analogies (VA) and general knowledge (GK) tasks in JCCES tap into language development, a crucial aspect of crystallized intelligence (Horn & Cattell, 1966). Similarly, the matching (MAT), analogies (ANA), sequences (SEQ), and construction (CON) tasks in GAMA are related to fluid intelligence, involving abstract reasoning and problem-solving skills (Naglieri & Bardos, 1997). However, the specific relationships between these tasks and their underlying cognitive processes remain to be further elucidated.

The present study seeks to address this gap in the literature by employing MDS to investigate the relationships between JCCES and GAMA tasks, with the aim of identifying common underlying dimensions. In line with previous research (Jouve, 2010b), we hypothesize that the MDS analysis will reveal a clear distinction between verbal and nonverbal abilities. Furthermore, we expect that the analysis will provide a more detailed classification of the tasks, reflecting the underlying cognitive processes involved in each task. By providing a comprehensive understanding of the relationships between the tasks within JCCES and GAMA, this study will contribute to the psychometric literature and inform the development of more targeted interventions and assessments in educational, clinical, and research settings.

Method

Research Design

The current study employed a correlational research design to investigate the relationships between the tasks from the Jouve Cerebrals Crystallized Educational Scale (JCCES) and the General Ability Measure for Adults (GAMA). This design was chosen because it allowed the researchers to examine the associations between the variables of interest without manipulating any variables or assigning participants to experimental conditions (Creswell, 2014).

Participants

A total of 63 participants were recruited for the study. Demographic information regarding age, gender, and ethnicity was collected but not used in this study. The participants were selected based on their willingness to participate and their ability to complete the JCCES and GAMA assessments. No exclusion criteria were set.

Materials

The JCCES is a measure of crystallized cognitive abilities (Jouve, 2010a), which reflect an individual's acquired knowledge and skills (Cattell, 1971). It consists of three subtests: Verbal Analogies (VA), Mathematical Problems (MP), and General Knowledge (GK).

The GAMA is a standardized measure of nonverbal and figurative general cognitive abilities (Naglieri & Bardos, 1997). It consists of four subtests: Matching (MAT), Analogies (ANA), Sequences (SEQ), and Construction (CON).

Procedures

Data collection was conducted in a quiet and well-lit testing environment. Participants first completed the JCCES, followed by the GAMA. Standardized instructions were provided to ensure that participants understood the requirements of each subtest. The JCCES and GAMA were administered according to their respective guidelines. 

Statistical Analyses

The data were analyzed using Multidimensional Scaling (MDS) with XLSTAT. The choice of MDS was informed by its ability to represent the structure of complex datasets by reducing their dimensionality while preserving the relationships between data points (Borg & Groenen, 2005). Kruskal's stress (1) was used to measure the goodness of fit for the MDS solution (Kruskal, 1964). Analyses were conducted for dimensions ranging from 1 to 7, with random initial configuration, 10 repetitions, and stopping conditions set at convergence = 0.00001 and iterations = 500.

Results

The data included scores from the Jouve Cerebrals Crystallized Educational Scale (JCCES), which measures Verbal Analogies (VA), Mathematical Problems (MP), and General Knowledge (GK), and the General Ability Measure for Adults (GAMA), a nonverbal assessment comprising Matching (MAT), Analogies (ANA), Sequences (SEQ), and Construction (CON) tasks.

Results of the Statistical Analyses

Based on Kruskal's stress values, the best solution was obtained for a 2-dimensional representation space (stress = 0.100), as higher dimensions did not result in significant improvements. The results for the 2-dimensional space (RSQ = .9418) are illustrated in Figure 1.

Interpretation of Results

The results from the MDS analysis indicate that the 2-dimensional solution provides a nuanced representation of the relationships between the JCCES and GAMA tasks. The findings suggest that the tasks from the two measures are not independent, and there are common underlying dimensions that account for their relationships. The MDS configuration reveals a diagonal separation between nonverbal abilities (MAT, ANA, SEQ, and CON) on one side and verbal abilities (GK and VA) on the other, with MP being unrelated to either side but slightly closer to the verbal side.

Based on their proximities, seven groups can be identified (in alphabetical order):

1. Abstract Reasoning and Pattern Recognition: This group, comprising SEQ and CON tasks, reflects abilities related to identifying and extrapolating patterns, as well as spatial visualization and manipulation. Both tasks share the common cognitive processes of abstract reasoning and pattern recognition, making them closely related to fluid intelligence.
2. Nonverbal Analogical Reasoning: Represented solely by the ANA task, this group reflects the ability to identify relationships and draw analogies between seemingly unrelated objects. The unique focus on analogical reasoning in a figurative context sets this task apart from the other tasks within the fluid intelligence group.
3. Crystallized intelligence: Consisting of MP, GK, and VA, this group represents abilities related to accumulated knowledge and experience, as well as the application of learned information in problem-solving situations.
4. Fluid intelligence: Comprised of ANA, SEQ, and CON, this group represents cognitive processes involving reasoning, problem-solving, and abstract thinking, which are not dependent on prior knowledge or experience.
5. Language development: Represented by GK and VA, this group reflects abilities related to language comprehension, vocabulary, and the use of language in various contexts.
6. Quantitative reasoning and knowledge: Represented uniquely by MP, this group reflects abilities related to understanding, interpreting, and applying numerical information and mathematical concepts.
7. Visual-spatial representation: Represented solely by MAT, this group reflects abilities related to visualizing and manipulating spatial information.

Particularities

The MDS analysis suggests that the CON task is more closely related to the fluid intelligence subgroup (ANA, SEQ, and CON) rather than the visual-spatial representation subgroup (MAT). This finding could be attributed to the nature of the CON task, which involves problem-solving and reasoning abilities that extend beyond visual-spatial skills. Although visual-spatial abilities may be necessary for the task, the CON task requires individuals to analyze and identify patterns, think abstractly, and apply their reasoning skills, which are more closely aligned with fluid intelligence processes. As a result, the CON task seems to tap into a broader range of cognitive processes than just visual-spatial representation, making it a better fit for the fluid intelligence subgroup.

In the fluid intelligence group, the MDS analysis reveals an intriguing difference in the proximity between the tasks. SEQ and CON are closely related, while ANA is positioned farther apart. This distinction can be better understood by examining the underlying processes involved in each task.

The SEQ task requires individuals to identify patterns and complete a sequence by deducing the logical progression. This involves abstract reasoning, pattern recognition, and the ability to extrapolate from given information. Similarly, the CON task involves assembling objects or shapes to create a specific configuration. This also requires abstract reasoning and pattern recognition, as well as spatial visualization and manipulation skills. Due to these shared cognitive processes, SEQ and CON tasks form a closely related subgroup within fluid intelligence.

On the other hand, the ANA task involves identifying relationships between pairs of objects or concepts and applying that understanding to a new set of objects or concepts. Although this task also requires abstract reasoning and problem-solving skills, it differs from SEQ and CON tasks in the sense that it demands a higher level of analogical reasoning, which involves identifying similarities and relationships between seemingly unrelated entities. This unique cognitive demand in the ANA task sets it apart from the other tasks in the fluid intelligence group.

Limitations

There are some limitations in the current study that may have affected the results. Firstly, the sample size of 63 participants may not be sufficient to provide robust and generalizable results. A larger sample would improve the reliability of the MDS analysis and potentially lead to more conclusive findings. Secondly, a lack of inclusion criteria could have been introduced in the recruitment process, affecting the representativeness of the sample and the generalizability of the results. Finally, there may be methodological limitations associated with the use of MDS, such as the assumption that the data are interval-scaled and that the relationships between tasks can be represented in a Euclidean space. These assumptions may not be entirely accurate for the current dataset, potentially affecting the interpretation of the results.

Discussion

Interpretation of Results and Comparison with Previous Research

The current study aimed to investigate the relationships between the tasks of the JCCES and GAMA, with the intent of identifying underlying dimensions that account for these relationships. In line with our hypotheses, we found a clear distinction between verbal and nonverbal abilities. The MDS analysis provided a nuanced representation of the relationships between the tasks, revealing a diagonal separation between nonverbal abilities (MAT, ANA, SEQ, and CON) and verbal abilities (GK and VA), with MP being closer to the verbal side. This finding is consistent with previous research (Jouve, 2010b), which identified separate factors for JCCES and GAMA subtests, emphasizing the distinctiveness of the two instruments in assessing crystallized and nonverbal cognitive abilities.

Our analysis identified seven groups, providing a more detailed understanding of the cognitive processes involved in each task. This classification aligns with the theoretical distinction between crystallized and fluid intelligence (Cattell, 1963), with the crystallized intelligence group consisting of MP, GK, and VA, and the fluid intelligence group comprising ANA, SEQ, and CON. However, our analysis also revealed unique relationships between tasks that warrant further discussion.

Detailed Analysis of the Groups

The seven groups identified in our analysis not only align with the distinction between crystallized and fluid intelligence (Cattell, 1963) but also offer a more granular understanding of the cognitive processes involved in each task. The following sections provide a more in-depth discussion of these groups, linking them with relevant literature.

Abstract Reasoning and Pattern Recognition

This group, consisting of SEQ and CON tasks, reflects abilities related to identifying and extrapolating patterns and spatial visualization and manipulation. Both tasks share the common cognitive processes of abstract reasoning and pattern recognition, making them closely related to fluid intelligence (Carroll, 1993). Research has demonstrated that these abilities play a significant role in various cognitive domains, such as problem-solving and decision-making (Sternberg, 1985).

Nonverbal Analogical Reasoning

The ANA task represents a unique group that reflects the ability to identify relationships and draw analogies between seemingly unrelated figurative objects. This ability is closely related to fluid intelligence (Spearman, 1927) and has been associated with higher-order cognitive processes, such as problem-solving, creativity, and critical thinking (Gentner, 1983; Holyoak & Thagard, 1995).

Crystallized Intelligence

The group consisting of MP, GK, and VA represents abilities related to accumulated knowledge and experience, as well as the application of learned information in problem-solving situations (Cattell, 1963). Crystallized intelligence is considered to be a product of both genetic factors and environmental influences, such as education and cultural exposure (Horn & Cattell, 1966).

Fluid Intelligence

Comprising ANA, SEQ, and CON, this group represents cognitive processes involving reasoning, problem-solving, and abstract thinking, which are not dependent on prior knowledge or experience (Cattell, 1963). Fluid intelligence is thought to be primarily determined by genetic factors and is believed to decline with age (Horn & Cattell, 1967).

Language Development

Represented by GK and VA, this group reflects abilities related to language comprehension, vocabulary, and the use of language in various contexts. Language development has been linked to both crystallized intelligence (Horn & Cattell, 1966) and general cognitive ability (Carroll, 1993).

Quantitative Reasoning and Knowledge

The MP task represents a unique group that reflects abilities related to understanding, interpreting, and applying numerical information and mathematical concepts. Quantitative reasoning and knowledge have been associated with both crystallized and fluid intelligence (Horn & Cattell, 1966; McGrew, 2009) and are considered essential components of general cognitive ability (Carroll, 1993).

Visual-Spatial Representation

The MAT task represents a distinct group that reflects abilities related to visualizing and manipulating spatial information. Visual-spatial representation is closely linked to fluid intelligence (Carroll, 1993) and has been shown to play a crucial role in various cognitive domains, such as navigation, mental rotation, and object recognition (Kosslyn, 1994).

By linking the identified groups with relevant literature, our analysis contributes to a more nuanced understanding of the cognitive processes underlying the tasks of the JCCES and GAMA. This detailed classification can inform the development of more targeted interventions and assessments in educational, clinical, and research settings.

Unexpected and Significant Findings

One intriguing finding was the positioning of the CON task within the fluid intelligence group rather than the visual-spatial representation subgroup. The CON task appeared to tap into a broader range of cognitive processes, such as abstract reasoning and pattern recognition, which are more closely aligned with fluid intelligence processes. Another interesting observation was the distinction between the ANA task and the other tasks within the fluid intelligence group. The unique focus on analogical reasoning sets the ANA task apart from the other tasks, emphasizing its distinct cognitive demands.

Implications for Theory, Practice, and Future Research

The present study adds to the growing body of literature on the relationships between cognitive abilities and contributes to our understanding of the cognitive processes involved in various tasks. The findings suggest that employing JCCES and GAMA as complementary tools can provide a more comprehensive assessment of an individual's cognitive profile. This approach has practical implications for educational, clinical, and research settings, where a thorough understanding of cognitive abilities is crucial for making informed decisions.

Future research should address the limitations of the current study by employing larger and more diverse samples, as well as investigating the potential utility of combining JCCES and GAMA to predict cognitive and academic outcomes. Additionally, exploring the relationships between these cognitive abilities and other relevant factors, such as socioeconomic background or educational attainment, would provide valuable insights into the broader context of cognitive functioning.

Limitations

There are several limitations in the current study that may have affected the results or the interpretation of the findings. First, the sample size of 63 participants may limit the generalizability and robustness of the results. Second, the lack of inclusion criteria in the recruitment process could affect the representativeness of the sample. Third, there may be methodological limitations associated with the use of MDS, such as the assumptions regarding interval-scaled data and Euclidean space representation.

Conclusion

In summary, this study provides a nuanced understanding of the relationships between the JCCES and GAMA tasks, revealing a clear distinction between verbal and nonverbal abilities, and further dividing these abilities into seven groups. These findings contribute to the existing literature on cognitive abilities and suggest that using JCCES and GAMA as complementary tools can offer a comprehensive assessment of an individual's cognitive profile. The implications for theory and practice include the potential to develop targeted interventions and assessments in educational, clinical, and research settings.

However, the study is not without limitations, such as a small sample size, lack of inclusion criteria in the recruitment process, and methodological constraints associated with the use of MDS. Future research should address these limitations and explore the potential utility of combining JCCES and GAMA to predict cognitive and academic outcomes, as well as investigate the relationships between cognitive abilities and other relevant factors.

Overall, this study highlights the importance of understanding the complex relationships between various cognitive abilities and offers a solid foundation for future research to build upon in the pursuit of developing more effective assessment tools and interventions.

References

Borg, I., & Groenen, P. J. F. (2005). Modern multidimensional scaling: Theory and applications (2nd ed.). New York: Springer.

Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. New York: Cambridge University Press. https://doi.org/10.1017/CBO9780511571312

Cattell, R. B. (1971). Abilities: Their structure, growth, and action. Boston, MA: Houghton Mifflin.

Cox, T. F., & Cox, M. A. A. (2001). Multidimensional scaling (2nd ed.). New York: Chapman & Hall/CRC. https://doi.org/10.1201/9780367801700

Creswell, J. W. (2014). Research Design: Qualitative, Quantitative and Mixed Methods Approaches (4th ed.). Thousand Oaks, CA: Sage. 

Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Associates. https://doi.org/10.4324/9781410605269

Gentner, D. (1983). Structure-mapping: A theoretical framework for analogy. Cognitive Science, 7(2), 155-170. https://doi.org/10.1016/S0364-0213(83)80009-3

Holyoak, K. J., & Thagard, P. (1995). Mental leaps: Analogy in creative thought. Cambridge, MA: MIT Press.

Horn, J. L., & Cattell, R. B. (1966). Refinement and test of the theory of fluid and crystallized general intelligences. Journal of Educational Psychology, 57(5), 253-270. https://doi.org/10.1037/h0023816

Jouve, X. (2010a). Jouve Cerebrals Crystallized Educational Scale. Retrieved from http://www.cogn-iq.org/tests/jouve-cerebrals-crystallized-educational-scale-jcces

Jouve, X. (2010b). Differentiating Cognitive Abilities: A Factor Analysis of JCCES and GAMA Subtests. Retrieved from https://cogniqblog.blogspot.com/2014/10/differentiating-cognitive-abilities.html

Kosslyn, S. M. (1994). Image and brain: The resolution of the imagery debate. Cambridge, MA: MIT Press.

Kruskal, J. B. (1964). Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, 29(1), 1–27. https://doi.org/10.1007/BF02289565

McGrew, K. S. (2009). CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research. Intelligence, 37(1), 1–10. https://doi.org/10.1016/j.intell.2008.08.004

Naglieri, J. A., & Bardos, A. N. (1997). General Ability Measure for Adults (GAMA). Minneapolis, MN: National Computer Systems.

Spearman, C. (1927). The abilities of man: Their nature and measurement. New York: Macmillan.

Sternberg, R. J. (1985). Implicit theories of intelligence, creativity, and wisdom. Journal of Personality and Social Psychology, 49(3), 607–627. https://doi.org/10.1037/0022-3514.49.3.607

Identifying the Underlying Dimensions of the JCCES Mathematical Problems using Alternating Least Squares Scaling

Abstract

This study aimed to investigate the underlying dimensions of the Jouve Cerebrals Crystallized Educational Scale (JCCES) Mathematical Problems (MP) using Alternating Least Squares Scaling (ALSCAL). The dataset consisted of intercorrelations between 38 MP items, with 588 participants. Various dimensional solutions were assessed for the goodness of fit. A 4-dimensional solution provided a reasonable fit (RSQ = 0.815, stress = 0.155), accounting for 81.5% of the variance in the disparities. The 4-dimensional solution is more parsimonious than the marginally better 5-dimensional solution (RSQ = 0.851, stress = 0.127). The study's limitations include the sample size, selection bias, and the exploratory nature of ALSCAL. The results contribute to understanding the structure of mathematical problem-solving abilities and have implications for theory, practice, and future research in cognitive and educational psychology.

Keywords: Jouve Cerebrals Crystallized Educational Scale, Mathematical Problems, Alternating Least Squares Scaling, multidimensional scaling, cognitive abilities

Introduction

Psychometrics is a scientific discipline focused on the development and evaluation of psychological assessments, including the measurement of cognitive abilities such as mathematical problem-solving skills (Embretson & Reise, 2000). The Jouve Cerebrals Crystallized Educational Scale (JCCES) Mathematical Problems (MP) subtest is an instrument used to assess these skills. To improve our understanding of the structure underlying the JCCES Mathematical Problems, the present study investigates the dimensionality of this instrument using Alternating Least Squares Scaling (ALSCAL).

The JCCES MP is grounded in various psychometric theories, particularly item response theory (IRT) and classical test theory (CTT) (De Ayala, 2009; Nunnally & Bernstein, 1994). Additionally, the JCCES MP aligns with cognitive and educational psychology theories, such as the multiple-component model of mathematical problem-solving (Swanson & Beebe-Frankenberger, 2004), which posits that problem-solving requires a combination of distinct cognitive abilities. Other relevant theories include Geary's (1994) cognitive mechanisms and Hecht's (2001) cognitive strategies that underlie mathematical problem-solving.

The selection of ALSCAL as the analytical method for this study is based on its suitability for exploratory research on multidimensional scaling (MDS) (Kruskal & Wish, 1978; Young, et al., 1978). ALSCAL has been used in various psychometric research to examine the dimensional structure of cognitive assessments (e.g., Gorsuch, 1983; Hambleton & Swaminathan, 1985). The method provides a data-driven approach to derive dimensional solutions and assess their goodness of fit, which can inform the development and interpretation of psychological assessments.

The present study's research question is: What is the lowest dimensionality offering a reasonable fit for the structure of items in the JCCES Mathematical Problems, as assessed by ALSCAL? By answering this question, the study aims to contribute to the literature on the dimensional structure of mathematical problem-solving assessments and inform future research and educational practice. In the context of previous research, the study will examine whether the identified dimensions align with established theories, such as Geary's (1994) cognitive mechanisms, Hecht's (2001) cognitive strategies, and the multiple-component model (Swanson & Beebe-Frankenberger, 2004).

Method

Research Design

The present study employed a correlational research design to investigate the structure of items in the Jouve Cerebrals Crystallized Educational Scale (JCCES) Mathematical Problems (MP) using Alternating Least Squares Scaling (ALSCAL; Young, et al., 1978). This design was chosen as it allowed for the exploration of potential relationships between the items without manipulating any variables.

Participants

The sample consisted of 588 participants, who were recruited through convenience sampling from various online media. Participants' demographic characteristics included age, gender, and educational level, which were collected through self-report measures. No exclusion criteria for the study were set.

Materials

The primary material used in this study was the JCCES Mathematical Problems subtest. The JCCES is a measure of crystallized intelligence, with established reliability and validity (Jouve, 2010a; 2010b). The Mathematical Problems subtest consists of 38 items that assess a range of mathematical problem-solving abilities, such as numerical reasoning, analytical thinking, and computational fluency. Participants completed the subtest in a computerized format.

Procedures

Participants were instructed that they had all the necessary time to complete the subtest, and their responses were scored automatically and then checked manually to insuring the most possible data reliability. The resulting dataset consisted of intercorrelations between the 38 items of the MP subtest.

Statistical Methods

ALSCAL was used to analyze the intercorrelations between the items and derive solutions for different dimensionalities, ranging from two to five dimensions. The analysis involved iterative optimization procedures to minimize stress values, with convergence criteria set at an improvement of less than 0.001 for stress values across consecutive iterations (Young et al., 1978). Kruskal's Stress Formula 1 was used to compute stress values, while RSQ values represented the proportion of variance in the scaled data (disparities) accounted for by their corresponding distances (Kruskal & Wish, 1978). The goodness of fit for various dimensional solutions was assessed, with the aim of identifying the lowest dimensionality offering a reasonable fit, operationalized as an RSQ value greater than 0.80 and a stress value within the range of 0.10 to 0.20.

Results

The primary objective of this study was to investigate the structure in items of the Jouve Cerebrals Crystallized Educational Scale (JCCES) Mathematical Problems (MP) using Alternating Least Squares Scaling (ALSCAL). The dataset consisted of intercorrelations between the 38 items of the MP subtest, with a sample size of 588 participants. The analysis involved assessing the goodness of fit for various dimensional solutions, with the aim of identifying the lowest dimensionality offering a reasonable fit, operationalized as an RSQ value greater than 0.80 and a stress value within the range of 0.10 to 0.20.

Statistical Analyses

ALSCAL (Young, et al., 1978) was employed to derive solutions for different dimensionalities, ranging from two to five dimensions. The analysis entailed iterative optimization procedures to minimize stress values, with convergence criteria set at an improvement of less than 0.001 for stress values across consecutive iterations. Kruskal's Stress Formula 1 was used to compute stress values, while RSQ values represented the proportion of variance in the scaled data (disparities) accounted for by their corresponding distances.

Results of Statistical Analyses

The ALSCAL analysis was conducted to derive solutions for different dimensionalities, and the results are presented below in a comprehensive manner.

5-dimensional solution

The 5-dimensional solution showed the lowest stress value (0.127) among all the solutions, indicating a relatively better fit to the data. The RSQ value, which represents the proportion of variance in the scaled data accounted for by the corresponding distances, was 0.851. This value suggests that 85.1% of the variance in the disparities could be explained by the distances in the 5-dimensional solution. However, the additional dimension compared to the 4-dimensional solution may increase model complexity without providing a substantial improvement in fit.

Iteration history:

• Iteration 1: SSTRESS = 0.23325
• Iteration 2: SSTRESS = 0.18514, Improvement = 0.04811
• Iteration 3: SSTRESS = 0.18247, Improvement = 0.00267
• Iteration 4: SSTRESS = 0.18223, Improvement = 0.00024

4-dimensional solution

The 4-dimensional solution was identified as the lowest dimensionality offering a reasonable fit based on the predefined criteria (RSQ > 0.80, stress within 0.10 to 0.20). The stress value for this solution was 0.155, while the RSQ value was 0.815, indicating that 81.5% of the variance in the disparities was accounted for by the distances in the 4-dimensional solution.

Iteration history:

• Iteration 1: SSTRESS = 0.26892
• Iteration 2: SSTRESS = 0.22219, Improvement = 0.04673
• Iteration 3: SSTRESS = 0.21927, Improvement = 0.00292
• Iteration 4: SSTRESS = 0.21902, Improvement = 0.00025

3-dimensional solution

The 3-dimensional solution showed a stress value of 0.210 and an RSQ value of 0.739. This solution did not meet the predefined criteria for a reasonable fit, as the RSQ value was below the threshold of 0.80.

Iteration history:

• Iteration 1: SSTRESS = 0.32326
• Iteration 2: SSTRESS = 0.28030, Improvement = 0.04295
• Iteration 3: SSTRESS = 0.27771, Improvement = 0.00259
• Iteration 4: SSTRESS = 0.27727, Improvement = 0.00045

2-dimensional solution

The 2-dimensional solution had the highest stress value (0.305) among all the solutions, indicating a relatively poor fit to the data. The RSQ value was 0.647, suggesting that only 64.7% of the variance in the disparities was accounted for by the distances in the 2-dimensional solution.

Iteration history:

• Iteration 1: SSTRESS = 0.42841
• Iteration 2: SSTRESS = 0.37073, Improvement = 0.05768
• Iteration 3: SSTRESS = 0.36893, Improvement = 0.00180
• Iteration 4: SSTRESS = 0.36702, Improvement = 0.00191
• Iteration 5: SSTRESS = 0.36696, Improvement = 0.00006

Interpretation of Results

The 4-dimensional solution, with a stress value of 0.155 and an RSQ value of 0.815, suggests that the structure of the Jouve Cerebrals Crystallized Educational Scale (JCCES) Mathematical Problems (MP) can be adequately represented in a 4-dimensional space. This solution accounts for 81.5% of the variance in the disparities, and it represents a balance between model complexity and goodness of fit.

The ALSCAL analysis results imply that there are four underlying dimensions or constructs in the JCCES Mathematical Problems items that contribute significantly to the structure of the data. These dimensions may represent distinct cognitive abilities, problem-solving strategies, or other factors that influence an individual's performance on the JCCES Mathematical Problems items.

It is essential to note that while the 5-dimensional solution provided marginally better-fit statistics (RSQ = 0.851 and stress = 0.127), the additional dimension would increase the model complexity without a substantial improvement in the goodness of fit. As a result, the 4-dimensional solution is more parsimonious and appropriate for this study.

To further interpret and understand the meaning of these dimensions, it is necessary to examine the item content and characteristics of the JCCES Mathematical Problems items, as well as any relevant theoretical frameworks in the field of cognitive and educational psychology. This examination would help researchers identify and label the dimensions, providing a better understanding of the underlying structure of the JCCES Mathematical Problems and informing future research and educational practice.

Limitations

Despite the successful identification of a 4-dimensional solution that met the predefined criteria, there are certain limitations to this study. First, the sample size of 588 participants may not be sufficient to generalize the findings to a broader population. Additionally, selection bias may be present, as the participants may not be representative of the entire population of interest. Finally, the study is limited by the methodological approach, as ALSCAL is an exploratory technique and may not provide definitive conclusions about the underlying structure of the data.

In conclusion, the 4-dimensional solution provided a reasonable fit for the structure in items of the JCCES Mathematical Problems, with an RSQ value of 0.815 and a stress value of 0.155. This solution offers a basis for further investigation and interpretation of the underlying dimensions in the JCCES Mathematical Problems dataset. However, it is important to consider the limitations of this study when interpreting and generalizing these findings.

Discussion

Interpretation of Results and Comparison with Previous Research

The results of the present study indicate that a 4-dimensional solution best represents the structure of the JCCES Mathematical Problems items. This finding is consistent with previous research suggesting that mathematical problem-solving involves multiple dimensions or cognitive abilities (e.g., Geary, 1994; Hecht, 2001; Swanson & Beebe-Frankenberger, 2004). These dimensions may represent distinct skills or strategies, such as numerical reasoning, spatial visualization, analytical thinking, and computational fluency. The identification of these dimensions provides a deeper understanding of the underlying structure of the JCCES Mathematical Problems and can inform both theoretical and practical applications in the field of cognitive and educational psychology.

Unexpected Findings and Their Importance

One notable finding in the present study was that the 5-dimensional solution, although offering slightly better-fit statistics, did not provide a substantial improvement in the goodness of fit compared to the 4-dimensional solution. This finding suggests that the additional dimension in the 5-dimensional solution may not be necessary or meaningful, and the 4-dimensional solution is more parsimonious and appropriate. This result highlights the importance of considering model complexity and parsimony in addition to fitting statistics when selecting the best solution in multidimensional scaling analyses.

Implications for Theory, Practice, and Future Research

The present study's findings contribute to the understanding of the structure of mathematical problem-solving abilities as assessed by the JCCES Mathematical Problems. By identifying four underlying dimensions, researchers can further explore these dimensions' nature and implications for cognitive and educational psychology theories. For instance, the findings can inform the development of more targeted interventions and instructional strategies to improve specific dimensions of mathematical problem-solving abilities.

Moreover, the results can help practitioners, such as educators and clinicians, to better interpret and use the JCCES Mathematical Problems in various settings, such as educational assessment, cognitive assessment, and intervention planning. By understanding the underlying dimensions, practitioners can more accurately identify students' strengths and weaknesses and provide targeted support to enhance their mathematical problem-solving skills.

Limitations and Alternative Explanations

As mentioned earlier, several limitations should be considered when interpreting the findings of the present study. The sample size and potential selection bias may limit the generalizability of the results to a broader population. Additionally, the exploratory nature of the ALSCAL analysis does not allow for definitive conclusions about the underlying structure of the data. Future studies may employ confirmatory techniques, such as confirmatory factor analysis or structural equation modeling, to validate the 4-dimensional solution identified in the present study.

Another limitation is the potential influence of other factors, such as individual differences in motivation, attention, or working memory capacity, which may have affected participants' performance on the JCCES Mathematical Problems and, consequently, the identified dimensions. Future research could examine these potential influences and incorporate them into the analysis to gain a more comprehensive understanding of the underlying structure of mathematical problem-solving abilities.

Directions for Future Research

Future research should aim to replicate and extend the present study using larger and more diverse samples to increase generalizability. Furthermore, researchers could examine the content and characteristics of the JCCES Mathematical Problems items to better understand and label the identified dimensions. This analysis could involve examining the items in relation to relevant theoretical frameworks, such as the multiple-component model of mathematical problem-solving (Swanson & Beebe-Frankenberger, 2004), to provide more meaningful interpretations of the dimensions.

Additionally, longitudinal studies could investigate the development of the identified dimensions across different age groups and educational levels to explore their potential implications for educational practice and cognitive development. Finally, future research could examine the relationship between the identified dimensions and other cognitive abilities or academic achievement measures to explore the practical significance and predictive validity of the JCCES Mathematical Problems.

Conclusion

In conclusion, this study found that a 4-dimensional solution best represents the structure of the JCCES Mathematical Problems items, accounting for 81.5% of the variance in the disparities. These dimensions may represent distinct cognitive abilities or problem-solving strategies, providing valuable insights into the structure of mathematical problem-solving abilities. The findings have significant implications for both theory and practice, informing future research and educational interventions targeting specific dimensions of mathematical problem-solving.

However, it is important to acknowledge the study's limitations, including the sample size, potential selection bias, and the exploratory nature of the ALSCAL analysis. Future research should focus on validating the 4-dimensional solution using confirmatory techniques, examining the content of the items, and investigating potential influences of individual differences. Longitudinal studies and investigations of the relationship between the identified dimensions and other cognitive abilities or academic achievement measures are also recommended to further our understanding of the underlying structure of mathematical problem-solving abilities.

References

De Ayala, R. J. (2009). The theory and practice of item response theory. New York, NY: Guilford Press.

Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Associates. https://doi.org/10.4324/9781410605269

Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston, MA: Kluwer-Nijhoff.

Hecht, S. A., & Vagi, K. J. (2010). Sources of Group and Individual Differences in Emerging Fraction Skills. Journal of educational psychology, 102(4), 843–859. https://doi.org/10.1037/a0019824

Geary, D. C. (1994). Children's mathematical development: Research and practical applications. Washington, DC: American Psychological Association. https://doi.org/10.1037/10163-000

Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Erlbaum.

Jouve, X. (2010a). Investigating the Relationship Between JCCES and RIAS Verbal Scale: A Principal Component Analysis Approach. Retrieved from https://cogniqblog.blogspot.com/2010/02/on-relationship-between-jcces-and.html

Jouve, X. (2010b). Relationship between Jouve Cerebrals Crystallized Educational Scale (JCCES) Crystallized Educational Index (CEI) and Cognitive and Academic Measures. Retrieved from https://cogniqblog.blogspot.com/2010/02/correlations-between-jcces-and-other.html

Jouve, X. (2010c). Jouve Cerebrals Crystallized Educational Scale. Retrieved from http://www.cogn-iq.org/tests/jouve-cerebrals-crystallized-educational-scale-jcces

Kruskal, J. B., & Wish, M. (1978). Multidimensional scaling. Beverly Hills, CA: Sage. https://doi.org/10.4135/9781412985130

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.

Swanson, H. L., & Beebe-Frankenberger, M. (2004). The Relationship Between Working Memory and Mathematical Problem Solving in Children at Risk and Not at Risk for Serious Math Difficulties. Journal of Educational Psychology, 96(3), 471–491. https://doi.org/10.1037/0022-0663.96.3.471

Young, F. W., Takane, Y., & Lewyckyj, R. (1978). ALSCAL: A nonmetric multidimensional scaling program with several individual-differences options. Behavior Research Methods & Instrumentation, 10(3), 451–453. https://doi.org/10.3758/BF03205177

Analyzing the Item Structure of the General Knowledge Subtest in the Jouve-Cerebrals Crystallized Educational Scale (JCCES) Using Multidimensional Scaling

Abstract

The purpose of this study was to analyze the item structure of the General Knowledge Subtest in the Jouve-Cerebrals Crystallized Educational Scale (JCCES) using multidimensional scaling (MDS) analyses. The JCCES was developed as a more efficient assessment of cognitive abilities by implementing a stopping rule based on consecutive errors. The MDS analyses revealed a horseshoe-shaped scaling of items in the General Knowledge Subtest, indicating a continuum wherein the constraints for dissimilarities have all been supported. The two-dimensional scaling solution for the General Knowledge Subtest indicates that the items are well-aligned with the construct being assessed. Limitations of the study, including the sample size and assumptions made in the MDS analyses, are discussed.

Keywords: Jouve-Cerebrals Crystallized Educational Scale, General Knowledge Subtest, multidimensional scaling, stopping rule, cognitive abilities, item structure

Introduction

Psychometric tests have been used for decades to assess cognitive abilities in various domains (Bors & Stokes, 1998; Deary, 2000). However, lengthy tests have been associated with several issues, including fatigue, boredom, and inaccuracy in results (Sundre & Kitsantas, 2004). To address these issues, the Cerebrals Cognitive Ability Tests (CCAT) were revised, resulting in the development of the Jouve-Cerebrals Crystallized Educational Scale (JCCES). One modification made to the JCCES was implementing a stopping rule after a certain number of consecutive errors, a technique used in some Wechsler subtests and the Reynolds Intellectual Assessment Scale (RIAS) (Wechsler, 2008; Reynolds & Kamphaus, 2003). The purpose of this study was to analyze the item structure of the General Knowledge Subtest in the JCCES, specifically examining the two-dimensional scaling solution using multidimensional scaling (MDS) analyses.

Method

The use of Rasch analysis to estimate item difficulty parameters is a well-established technique in psychometrics (Wright & Stone, 1979). Similarly, the adoption of a stopping criterion based on consecutive errors is a technique used in other cognitive ability tests, such as the Wechsler Adult Intelligence Scale (WAIS) and the Kaufman Assessment Battery for Children (KABC) (Wechsler, 2008; Kaufman & Kaufman, 1983). The present study administered the JCCES General Knowledge Subtest to 588 participants and implemented a stopping criterion of five consecutive errors after determining that three consecutive errors were inappropriate. The rearrangement of items based on Rasch estimates allowed for the examination of the item structure in a more systematic and objective manner. MDS analyses were then conducted to explore the underlying structure of the item response data.

Results

As shown in Figure 1, the present study's MDS analyses produced a two-dimensional scaling solution for the General Knowledge Subtest with a Kruskal's Stress of .18 and a squared correlation (RSQ) of .87. The horseshoe-shaped scaling pattern of the items indicates a continuum of difficulty levels, with the constraints for dissimilarities supported. It is called Guttman's effect (Guttman, 1950; Collins & Cliff, 1990). This pattern is consistent with the concept of item difficulty in psychometric testing (Lord & Novick, 1968) and supports the validity of the test in measuring cognitive abilities. These findings also suggest that the implementation of a stopping rule based on consecutive errors is an effective way to improve the efficiency of the cognitive ability test.

Figure 1. Multidimensional Scaling (MDS) of the General Knowledge subtest items.

Note. N = 588.

Discussion

The results of this study show the benefits of implementing a stopping rule to improve the efficiency of cognitive ability tests. The horseshoe-shaped scaling pattern observed in the General Knowledge Subtest aligns well with the concept of item difficulty in psychometric testing. However, the limitations of this study should be acknowledged. The sample size of 588 is relatively small for this type of analysis, and caution should be taken when generalizing the findings to other populations (Hair et al., 1998). Additionally, the selection of the stopping criterion at five consecutive errors was determined based on the current sample and may not be optimal for all populations. Methodological limitations, such as the assumptions of linearity and homoscedasticity in the MDS analyses, may have influenced the results.

Conclusion

In conclusion, the JCCES provides a more efficient assessment of cognitive abilities, with the General Knowledge Subtest demonstrating a horseshoe-shaped scaling pattern indicative of a continuum of difficulty levels. The two-dimensional scaling solution indicates that the items are well-aligned with the construct being assessed. Although there are limitations to the study, these findings provide valuable insights into the item structure of the JCCES General Knowledge Subtest and support the use of a stopping rule based on consecutive errors to improve the efficiency of the test. Future research could explore the generalizability of the findings to larger and more diverse samples, as well as investigate the optimal stopping criterion for different populations.

References

Bors, D. A., & Stokes, T. L. (1998). Raven's Advanced Progressive Matrices: Norms for first-year university students and the development of a short form. Educational and Psychological Measurement, 58(3), 382–398. https://doi.org/10.1177/0013164498058003002

Collins, L. M., & Cliff, N. (1990). Using the longitudinal Guttman simplex as a basis for measuring growth. Psychological Bulletin, 108(1), 128–134. https://doi.org/10.1037/0033-2909.108.1.128

Deary, I. J. (2000). Looking down on human intelligence: From psychometrics to the brain. Oxford, UK: Oxford University Press. https://doi.org/10.1093/acprof:oso/9780198524175.001.0001

Guttman, L. (1950). The basis for scalogram analysis. In S. A. Stouffer, L. Guttman, E. A. Suchman, P. F. Lazarsfield, S. A. Star, & J. A. Clausen (Eds.), Measurement and prediction (pp. 60 – 90). Princeton, NJ: Princeton University Press.

Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (1998). Multivariate data analysis (Vol. 5). Upper Saddle River, NJ: Prentice Hall.

Kaufman, A. S., & Kaufman, N. L. (1983). Kaufman Assessment Battery for Children. Circle Pines, MN: American Guidance Service.

Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.

Reynolds, C. R., & Kamphaus, R. W. (2003). Reynolds Intellectual Assessment Scales (RIAS) and the Reynolds Intellectual Screening Test (RIST), Professional Manual. Lutz, FL: Psychological Assessment Resources.

Sundre, D. L., & Kitsantas, A. (2004). An exploration of the psychology of the examinee: Can examinee self-regulation and test-taking motivation predict consequential and non-consequential test performance? Contemporary Educational Psychology, 29(1), 6–26. https://doi.org/10.1016/S0361-476X(02)00063-2

Wechsler, D. (2008). Wechsler Adult Intelligence Scale–Fourth Edition (WAIS–IV). San Antonio, TX: Pearson. https://doi.org/10.1037/t15169-000

Wright, B. D., & Stone, M. H. (1979). Best test design: Rasch measurement. Chicago, IL: MESA Press.