Abstract
This study aimed to investigate the underlying dimensions of the Jouve Cerebrals Crystallized Educational Scale (JCCES) Mathematical Problems (MP) using Alternating Least Squares Scaling (ALSCAL). The dataset consisted of intercorrelations between 38 MP items, with 588 participants. Various dimensional solutions were assessed for the goodness of fit. A 4-dimensional solution provided a reasonable fit (RSQ = 0.815, stress = 0.155), accounting for 81.5% of the variance in the disparities. The 4-dimensional solution is more parsimonious than the marginally better 5-dimensional solution (RSQ = 0.851, stress = 0.127). The study's limitations include the sample size, selection bias, and the exploratory nature of ALSCAL. The results contribute to understanding the structure of mathematical problem-solving abilities and have implications for theory, practice, and future research in cognitive and educational psychology.
Keywords: Jouve Cerebrals Crystallized Educational Scale, Mathematical Problems, Alternating Least Squares Scaling, multidimensional scaling, cognitive abilities
Introduction
Psychometrics is a scientific discipline focused on the development and evaluation of psychological assessments, including the measurement of cognitive abilities such as mathematical problem-solving skills (Embretson & Reise, 2000). The Jouve Cerebrals Crystallized Educational Scale (JCCES) Mathematical Problems (MP) subtest is an instrument used to assess these skills. To improve our understanding of the structure underlying the JCCES Mathematical Problems, the present study investigates the dimensionality of this instrument using Alternating Least Squares Scaling (ALSCAL).
The JCCES MP is grounded in various psychometric theories, particularly item response theory (IRT) and classical test theory (CTT) (De Ayala, 2009; Nunnally & Bernstein, 1994). Additionally, the JCCES MP aligns with cognitive and educational psychology theories, such as the multiple-component model of mathematical problem-solving (Swanson & Beebe-Frankenberger, 2004), which posits that problem-solving requires a combination of distinct cognitive abilities. Other relevant theories include Geary's (1994) cognitive mechanisms and Hecht's (2001) cognitive strategies that underlie mathematical problem-solving.
The selection of ALSCAL as the analytical method for this study is based on its suitability for exploratory research on multidimensional scaling (MDS) (Kruskal & Wish, 1978; Young, et al., 1978). ALSCAL has been used in various psychometric research to examine the dimensional structure of cognitive assessments (e.g., Gorsuch, 1983; Hambleton & Swaminathan, 1985). The method provides a data-driven approach to derive dimensional solutions and assess their goodness of fit, which can inform the development and interpretation of psychological assessments.
The present study's research question is: What is the lowest dimensionality offering a reasonable fit for the structure of items in the JCCES Mathematical Problems, as assessed by ALSCAL? By answering this question, the study aims to contribute to the literature on the dimensional structure of mathematical problem-solving assessments and inform future research and educational practice. In the context of previous research, the study will examine whether the identified dimensions align with established theories, such as Geary's (1994) cognitive mechanisms, Hecht's (2001) cognitive strategies, and the multiple-component model (Swanson & Beebe-Frankenberger, 2004).
Method
Research Design
The present study employed a correlational research design to investigate the structure of items in the Jouve Cerebrals Crystallized Educational Scale (JCCES) Mathematical Problems (MP) using Alternating Least Squares Scaling (ALSCAL; Young, et al., 1978). This design was chosen as it allowed for the exploration of potential relationships between the items without manipulating any variables.
Participants
The sample consisted of 588 participants, who were recruited through convenience sampling from various online media. Participants' demographic characteristics included age, gender, and educational level, which were collected through self-report measures. No exclusion criteria for the study were set.
Materials
The primary material used in this study was the JCCES Mathematical Problems subtest. The JCCES is a measure of crystallized intelligence, with established reliability and validity (Jouve, 2010a; 2010b). The Mathematical Problems subtest consists of 38 items that assess a range of mathematical problem-solving abilities, such as numerical reasoning, analytical thinking, and computational fluency. Participants completed the subtest in a computerized format.
Procedures
Participants were instructed that they had all the necessary time to complete the subtest, and their responses were scored automatically and then checked manually to insuring the most possible data reliability. The resulting dataset consisted of intercorrelations between the 38 items of the MP subtest.
Statistical Methods
ALSCAL was used to analyze the intercorrelations between the items and derive solutions for different dimensionalities, ranging from two to five dimensions. The analysis involved iterative optimization procedures to minimize stress values, with convergence criteria set at an improvement of less than 0.001 for stress values across consecutive iterations (Young et al., 1978). Kruskal's Stress Formula 1 was used to compute stress values, while RSQ values represented the proportion of variance in the scaled data (disparities) accounted for by their corresponding distances (Kruskal & Wish, 1978). The goodness of fit for various dimensional solutions was assessed, with the aim of identifying the lowest dimensionality offering a reasonable fit, operationalized as an RSQ value greater than 0.80 and a stress value within the range of 0.10 to 0.20.
Results
The primary objective of this study was to investigate the structure in items of the Jouve Cerebrals Crystallized Educational Scale (JCCES) Mathematical Problems (MP) using Alternating Least Squares Scaling (ALSCAL). The dataset consisted of intercorrelations between the 38 items of the MP subtest, with a sample size of 588 participants. The analysis involved assessing the goodness of fit for various dimensional solutions, with the aim of identifying the lowest dimensionality offering a reasonable fit, operationalized as an RSQ value greater than 0.80 and a stress value within the range of 0.10 to 0.20.
Statistical Analyses
ALSCAL (Young, et al., 1978) was employed to derive solutions for different dimensionalities, ranging from two to five dimensions. The analysis entailed iterative optimization procedures to minimize stress values, with convergence criteria set at an improvement of less than 0.001 for stress values across consecutive iterations. Kruskal's Stress Formula 1 was used to compute stress values, while RSQ values represented the proportion of variance in the scaled data (disparities) accounted for by their corresponding distances.
Results of Statistical Analyses
The ALSCAL analysis was conducted to derive solutions for different dimensionalities, and the results are presented below in a comprehensive manner.
5-dimensional solution
The 5-dimensional solution showed the lowest stress value (0.127) among all the solutions, indicating a relatively better fit to the data. The RSQ value, which represents the proportion of variance in the scaled data accounted for by the corresponding distances, was 0.851. This value suggests that 85.1% of the variance in the disparities could be explained by the distances in the 5-dimensional solution. However, the additional dimension compared to the 4-dimensional solution may increase model complexity without providing a substantial improvement in fit.
Iteration history:
- Iteration 1: SSTRESS = 0.23325
- Iteration 2: SSTRESS = 0.18514, Improvement = 0.04811
- Iteration 3: SSTRESS = 0.18247, Improvement = 0.00267
- Iteration 4: SSTRESS = 0.18223, Improvement = 0.00024
4-dimensional solution
The 4-dimensional solution was identified as the lowest dimensionality offering a reasonable fit based on the predefined criteria (RSQ > 0.80, stress within 0.10 to 0.20). The stress value for this solution was 0.155, while the RSQ value was 0.815, indicating that 81.5% of the variance in the disparities was accounted for by the distances in the 4-dimensional solution.
Iteration history:
- Iteration 1: SSTRESS = 0.26892
- Iteration 2: SSTRESS = 0.22219, Improvement = 0.04673
- Iteration 3: SSTRESS = 0.21927, Improvement = 0.00292
- Iteration 4: SSTRESS = 0.21902, Improvement = 0.00025
3-dimensional solution
The 3-dimensional solution showed a stress value of 0.210 and an RSQ value of 0.739. This solution did not meet the predefined criteria for a reasonable fit, as the RSQ value was below the threshold of 0.80.
Iteration history:
- Iteration 1: SSTRESS = 0.32326
- Iteration 2: SSTRESS = 0.28030, Improvement = 0.04295
- Iteration 3: SSTRESS = 0.27771, Improvement = 0.00259
- Iteration 4: SSTRESS = 0.27727, Improvement = 0.00045
2-dimensional solution
The 2-dimensional solution had the highest stress value (0.305) among all the solutions, indicating a relatively poor fit to the data. The RSQ value was 0.647, suggesting that only 64.7% of the variance in the disparities was accounted for by the distances in the 2-dimensional solution.
Iteration history:
- Iteration 1: SSTRESS = 0.42841
- Iteration 2: SSTRESS = 0.37073, Improvement = 0.05768
- Iteration 3: SSTRESS = 0.36893, Improvement = 0.00180
- Iteration 4: SSTRESS = 0.36702, Improvement = 0.00191
- Iteration 5: SSTRESS = 0.36696, Improvement = 0.00006
Interpretation of Results
The 4-dimensional solution, with a stress value of 0.155 and an RSQ value of 0.815, suggests that the structure of the Jouve Cerebrals Crystallized Educational Scale (JCCES) Mathematical Problems (MP) can be adequately represented in a 4-dimensional space. This solution accounts for 81.5% of the variance in the disparities, and it represents a balance between model complexity and goodness of fit.
The ALSCAL analysis results imply that there are four underlying dimensions or constructs in the JCCES Mathematical Problems items that contribute significantly to the structure of the data. These dimensions may represent distinct cognitive abilities, problem-solving strategies, or other factors that influence an individual's performance on the JCCES Mathematical Problems items.
It is essential to note that while the 5-dimensional solution provided marginally better-fit statistics (RSQ = 0.851 and stress = 0.127), the additional dimension would increase the model complexity without a substantial improvement in the goodness of fit. As a result, the 4-dimensional solution is more parsimonious and appropriate for this study.
To further interpret and understand the meaning of these dimensions, it is necessary to examine the item content and characteristics of the JCCES Mathematical Problems items, as well as any relevant theoretical frameworks in the field of cognitive and educational psychology. This examination would help researchers identify and label the dimensions, providing a better understanding of the underlying structure of the JCCES Mathematical Problems and informing future research and educational practice.
Limitations
Despite the successful identification of a 4-dimensional solution that met the predefined criteria, there are certain limitations to this study. First, the sample size of 588 participants may not be sufficient to generalize the findings to a broader population. Additionally, selection bias may be present, as the participants may not be representative of the entire population of interest. Finally, the study is limited by the methodological approach, as ALSCAL is an exploratory technique and may not provide definitive conclusions about the underlying structure of the data.
In conclusion, the 4-dimensional solution provided a reasonable fit for the structure in items of the JCCES Mathematical Problems, with an RSQ value of 0.815 and a stress value of 0.155. This solution offers a basis for further investigation and interpretation of the underlying dimensions in the JCCES Mathematical Problems dataset. However, it is important to consider the limitations of this study when interpreting and generalizing these findings.
Discussion
Interpretation of Results and Comparison with Previous Research
The results of the present study indicate that a 4-dimensional solution best represents the structure of the JCCES Mathematical Problems items. This finding is consistent with previous research suggesting that mathematical problem-solving involves multiple dimensions or cognitive abilities (e.g., Geary, 1994; Hecht, 2001; Swanson & Beebe-Frankenberger, 2004). These dimensions may represent distinct skills or strategies, such as numerical reasoning, spatial visualization, analytical thinking, and computational fluency. The identification of these dimensions provides a deeper understanding of the underlying structure of the JCCES Mathematical Problems and can inform both theoretical and practical applications in the field of cognitive and educational psychology.
Unexpected Findings and Their Importance
One notable finding in the present study was that the 5-dimensional solution, although offering slightly better-fit statistics, did not provide a substantial improvement in the goodness of fit compared to the 4-dimensional solution. This finding suggests that the additional dimension in the 5-dimensional solution may not be necessary or meaningful, and the 4-dimensional solution is more parsimonious and appropriate. This result highlights the importance of considering model complexity and parsimony in addition to fitting statistics when selecting the best solution in multidimensional scaling analyses.
Implications for Theory, Practice, and Future Research
The present study's findings contribute to the understanding of the structure of mathematical problem-solving abilities as assessed by the JCCES Mathematical Problems. By identifying four underlying dimensions, researchers can further explore these dimensions' nature and implications for cognitive and educational psychology theories. For instance, the findings can inform the development of more targeted interventions and instructional strategies to improve specific dimensions of mathematical problem-solving abilities.
Moreover, the results can help practitioners, such as educators and clinicians, to better interpret and use the JCCES Mathematical Problems in various settings, such as educational assessment, cognitive assessment, and intervention planning. By understanding the underlying dimensions, practitioners can more accurately identify students' strengths and weaknesses and provide targeted support to enhance their mathematical problem-solving skills.
Limitations and Alternative Explanations
As mentioned earlier, several limitations should be considered when interpreting the findings of the present study. The sample size and potential selection bias may limit the generalizability of the results to a broader population. Additionally, the exploratory nature of the ALSCAL analysis does not allow for definitive conclusions about the underlying structure of the data. Future studies may employ confirmatory techniques, such as confirmatory factor analysis or structural equation modeling, to validate the 4-dimensional solution identified in the present study.
Another limitation is the potential influence of other factors, such as individual differences in motivation, attention, or working memory capacity, which may have affected participants' performance on the JCCES Mathematical Problems and, consequently, the identified dimensions. Future research could examine these potential influences and incorporate them into the analysis to gain a more comprehensive understanding of the underlying structure of mathematical problem-solving abilities.
Directions for Future Research
Future research should aim to replicate and extend the present study using larger and more diverse samples to increase generalizability. Furthermore, researchers could examine the content and characteristics of the JCCES Mathematical Problems items to better understand and label the identified dimensions. This analysis could involve examining the items in relation to relevant theoretical frameworks, such as the multiple-component model of mathematical problem-solving (Swanson & Beebe-Frankenberger, 2004), to provide more meaningful interpretations of the dimensions.
Additionally, longitudinal studies could investigate the development of the identified dimensions across different age groups and educational levels to explore their potential implications for educational practice and cognitive development. Finally, future research could examine the relationship between the identified dimensions and other cognitive abilities or academic achievement measures to explore the practical significance and predictive validity of the JCCES Mathematical Problems.
Conclusion
In conclusion, this study found that a 4-dimensional solution best represents the structure of the JCCES Mathematical Problems items, accounting for 81.5% of the variance in the disparities. These dimensions may represent distinct cognitive abilities or problem-solving strategies, providing valuable insights into the structure of mathematical problem-solving abilities. The findings have significant implications for both theory and practice, informing future research and educational interventions targeting specific dimensions of mathematical problem-solving.
However, it is important to acknowledge the study's limitations, including the sample size, potential selection bias, and the exploratory nature of the ALSCAL analysis. Future research should focus on validating the 4-dimensional solution using confirmatory techniques, examining the content of the items, and investigating potential influences of individual differences. Longitudinal studies and investigations of the relationship between the identified dimensions and other cognitive abilities or academic achievement measures are also recommended to further our understanding of the underlying structure of mathematical problem-solving abilities.
References
De Ayala, R. J. (2009). The theory and practice of item response theory. New York, NY: Guilford Press.
Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Associates. https://doi.org/10.4324/9781410605269
Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston, MA: Kluwer-Nijhoff.
Hecht, S. A., & Vagi, K. J. (2010). Sources of Group and Individual Differences in Emerging Fraction Skills. Journal of educational psychology, 102(4), 843–859. https://doi.org/10.1037/a0019824
Geary, D. C. (1994). Children's mathematical development: Research and practical applications. Washington, DC: American Psychological Association. https://doi.org/10.1037/10163-000
Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Hillsdale, NJ: Erlbaum.
Jouve, X. (2010a). Investigating the Relationship Between JCCES and RIAS Verbal Scale: A Principal Component Analysis Approach. Retrieved from https://cogniqblog.blogspot.com/2010/02/on-relationship-between-jcces-and.html
Jouve, X. (2010b). Relationship between Jouve Cerebrals Crystallized Educational Scale (JCCES) Crystallized Educational Index (CEI) and Cognitive and Academic Measures. Retrieved from https://cogniqblog.blogspot.com/2010/02/correlations-between-jcces-and-other.html
Jouve, X. (2010c). Jouve Cerebrals Crystallized Educational Scale. Retrieved from http://www.cogn-iq.org/tests/jouve-cerebrals-crystallized-educational-scale-jcces
Kruskal, J. B., & Wish, M. (1978). Multidimensional scaling. Beverly Hills, CA: Sage. https://doi.org/10.4135/9781412985130
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.
Swanson, H. L., & Beebe-Frankenberger, M. (2004). The Relationship Between Working Memory and Mathematical Problem Solving in Children at Risk and Not at Risk for Serious Math Difficulties. Journal of Educational Psychology, 96(3), 471–491. https://doi.org/10.1037/0022-0663.96.3.471
Young, F. W., Takane, Y., & Lewyckyj, R. (1978). ALSCAL: A nonmetric multidimensional scaling program with several individual-differences options. Behavior Research Methods & Instrumentation, 10(3), 451–453. https://doi.org/10.3758/BF03205177
