Journal of Technology and Science Education

• Home
• About
• Login
• Archives
• Submissions
• Publication Fee
• Indexing & Statistics

Home > Vol 8, No 4 (2018) > Martínez Romera

STERNBERG-WAGNER THINKING STYLES:
A RESEARCH TOOL IN SOCIAL SCIENCE DIDACTICS

Universidad de Málaga (Spain)

Received March 2018

Accepted June 2018

Abstract

Due to its nature, the teaching of social sciences and the way of thinking of teachers and students maintain a complex, if not delicate relationship. Personal viewpoints and affinities on issues like gender, how things must be done or arranged in class could have a strong influence on the development of a subject. To understand this in a given context, we propose Sternberg-Wagner test analysis and data mining as class tool to identify student behavior patterns and its influence on selected variables (scores, gender, choice of small group for class tasks). As the study shows, significant differences have been detected between sexes as to the form they prefer to do a task, and the way they conceive it. Knowing this can help to better class planning, insofar identifies strengths and weaknesses that otherwise might not have been detected.

 

Keywords – Didactics, Social sciences, Pre-service teacher training, Social behavior, Educational research.

To cite this article:

Martínez-Romera, D.D. (2018). Sternberg-Wagner thinking styles: A research tool in social science didactics. Journal of Technology and Science Education, 8(4), 398-407. https://doi.org/10.3926/jotse.422

 

----------

1. 1. 1. Introduction

This article offers a hybrid technological tool designed to collect useful data for teachers looking for assistance to take decisions about how to plan or improve their didactic strategies (Niehaus, Reading & García, 2018) in Social Science contexts. In few areas of education, the interest in knowing the impact of the teachers’ and students’ way of thinking is relevant in Social Sciences (Jamil-Asghar, 2016; Smetanová, Drbalová & Vitáková, 2015; Selesho, 2014; Valli, Perkkilä & Valli, 2013; Keirn & Luhr, 2012).

There are two large groups of reasons lie at the basis of this fact. First of all, the plural nature of the epistemology of this discipline (Bautista, 2015; Gerring, 2012; Beltran, 1985) should be accommodated to extensive subjects as History, Sociology, Anthropology, and even Geography in countries like Spain or France. Thus, classical approaches as quantitative or qualitative analysis are accompanied by methodologies including critical-rational, historical, or comparative analysis, to name just a few. Here, the difficulty is about quality considerations on taught science.

Secondly, social thought covers rational and irrational thinking dispositions, following Perkins’ theory (Bouhnik & Carmi, 2012; Perkins, Jay & Tishman, 1993). The latter implies beliefs, feelings, and collective/self‑identity questions (Taylor, 2017; Christiansen & Koeman, 2015; Hocutt, 2005). Therefore, picked ways of teaching could be considered, even if not alleged, indoctrination. Here, the difficulty lies in the ability to not project teachers’ viewpoint as unquestionable standard. It is important, then, to distinguish between facts and personal valuations, beliefs, or affinities.

Pre-service teachers need to be aware of their future responsibilities and challenges, especially nowadays, where learning, technological, and sociological dynamics are faster than in past decades (Cartelli, 2012). In Spain, a didactic background is compulsory to graduates who wish to work in secondary education or vocational training, through one-year master’s degree, since 2011. During that course, they acquire basic knowledge about Pedagogy, teaching methods, and educational legislation among other contents. However, the brevity of the formative period not always grants they take on appropriately the second fact: as far as own ideology is a decision to be taken in academic contexts, it is considered part of the intellectual debate. This explains how teachers see, sometimes, themselves as owners of the ultimate truth for their students, instead focusing on their autonomous mind development (De Robertis, 2017; Midhatovna-Galimullina & Nikolaevna-Korshunova, 2016; Rockenbach, Mayhew, Wolniak, Terenzini, Seifert, Bowman, et al., 2016).

Sternberg-Wagner questionnaire is among most reputed tools to explore and classify thinking styles in education (Yuan, Zhang & Fu, 2017; Canbolat, Erdogan & Yazlik, 2016; Fan, 2016; Zhu, 2013; Fink & Garner, 2008). It covers five categories, disaggregated in thirteen styles, each of them checked through eight questions in a mini-test. In exchange for the time required to gather, to process and to analyze it, we obtain a pretty accurate image of the intellectual structure of people’s mind. This is the reason why the present work has two main objectives: to identify the behavior patterns present in class, as learning and analysis tool, insofar each way of thinking has its strengths and weaknesses; and to explore if they have any influence in the academic performance, social or gender behavior within the class.

2. Methodology

The target population was 29 students of the subject Initiation to Research and Educational Innovation in Social Sciences within the Master Degree in Secondary Education, during 2016-2017 course in University of Málaga (Spain). The students did several individual tasks, and one in small group that included a practical exhibition of it. The latter was assessed through 360º methodology (Martins & Martins, 2006; Brutus & Gorriti, 2005) using CoRubric, a federated online rubric tool (Martínez-Romera, Cebrián-Robles & Cebrián-de-la-Serna, 2016; Stevens & Levi, 2013). Seven students left the course before it was finished. To avoid personal influences, the project aims were explained only after data gathering.

Firstly, we used the Sternberg-Wagner test about thinking styles (Sternberg, 1999). It was needed to detect and to classify the patterns of self-concept about teaching dimensions present in class. Its 104 questions are arranged in octets, one for each style that implies a specific psychological tendency, as summarized in Table 1.

Then, we analysed if those patterns had any influence with:

1. 1.Group election (social behavior).

2. 2.Peer review marks (academic behavior).

3. 3.Teacher final marks (as external reference).

4. 4.Gender (sex influence).

Finally, we briefly explore some interdependencies among selected variables through associative data mining (Algarni, 2016; Koedinger, D’Mello, McLaughlin, Pardos & Rosé, 2015; Papamitsiou & Economides, 2014; Ahmadi & Ahmad, 2013) with WEKA (University of Waikato Environment for Knowledge Analysis). We chose Apriori, a very well know algorithm for boolean association rules (Tanna & Ghodasara, 2014) whose key concepts matches the behavior of our population: it requires itemsets with internal frequencies in its variables (denoted as Lk), its subsets must be also frequent (downward closure property) and it implies a join operation between candidates in order to generate the rules.

Category	Style	Likes to...
Functions	Legislative	Create, discover, design; use own method; less structure.
	Executive	Follow instructions; do what is requested; structure must be given.
	Judicial	Criticise and evaluate people and things.
Forms	Monarchic	Do one thing at a time; spend almost all energy and resources on it.
	Hierarchic	Do many things at one; prioritize what, when, how much time and energy to do a thing.
	Oligarchic	Do many things at once; have problems with prioritizing.
	Anarchic	Follow an extraordinary approach to problems; hate systems, guidelines, and any restrictions.
Levels	Global	Work with the bigger picture, generalizations, and abstracts.
	Local	Work with detail, specifications, and specific examples.
Scopes	Internal	Work alone; focus on the inside and to be independent.
	External	Work with other people; focuses on the outside and to be interdependent.
Leaning	Liberal	Do things in a new manner and deviate from traditions.
	Conservative	Do things in a proven and real manner and follow traditions.

Table 1. Thinking styles dimensions and meanings according Sternberg-Wagner test

The algorithm uses a two-steps logic to find association rules: the join step, were is set the k-itemsets of candidates (denoted Ck); the prune step, determines if Ck is a superset of Lk. Ck tends to growth fast, so Apriori uses two properties to refine its results: itemsets that are not frequent cannot be subset of frequent k-item sets; every subset of candidate k item not present in Lk-1 is removed as it cannot be frequent. Results are ranked by its support, percentage of itemsets covered by each rule, starting at 100% and decreasing it in steps of 5% until confidence is lower than 0.9 or 10% bound is reached.

Due to the amount of potential rules derived from de analysis, Apriori establishes four alternative ways for ranking rules (Frank, Hall, Witten, 2016): Confidence, determines the proportion of examples covered by the premise also covered by the consequent; Lift, result of dividing confidence by the support; Leverage, proportion of new examples covered by both premise and consequent, considering them statistically independent; and Conviction, criterion defined by Brin, Motwani, Ullman and Tsur, (1997) but not specifically documented by WEKA.

3. Analysis

For the first step, the data gathering of the questionnaire gave us a total of 3,016 items to work with. We considered maximum frequency as modal style, and we used ‘Mixed’ if tie happened. Summarized in this way, we obtained the first approximation about thinking dynamics in class, as shown in Figure 1.

Despite statistical highest frequency on each style defines the class as Mixed, Hierarchic, Local, External and Liberal (MHLEL for short) in its way of thinking, the minimum and maximum absolutes within each category are relatively close, which implies high internal diversity. Differences were less than 10% in Functions & Levels, and 25% in Scopes & Leaning; leaving Forms as the less diverse one, with a gap of 52%. This makes clear the need for more detailed analysis.

In order to do so, we went through each student’s questionnaire. The result, as shown in Table 2, were joined with sex and group variables, and the three main scores considered here, for further analysis: FM as final mark assessed by teacher; PInd as peer assessment on each member of the group exposition and PGrp as peer evaluation of the group, done only by audience students. The last two variables are expressed as arithmetic mean.

Figure 1. Class thinking styles, global results

Student ID 26, with a JHGEC profile, scores the best result on FM and PGrp, and ID 13, JHLEL profile, does the same on PInd. On the other side ID 16, with a JMMEL profile, receives the lowest mark on FM, ID 22, with a LHLIL profile, on PInd; and ID 6 and 7, MHLIC and EHLIC profiles respectively, tie as the lowest on PGrp.

All members of the group A are Hierarchical, three are Global and External. All members of the group B share the same styles in four of the five categories; they are Hierarchic, Local, Internal and Conservative. All members of the group C are Judicial and Liberal, and three are Hierarchic and External. Two members of the group D share Local and External tendencies. Three members of the group E share Hierarchic and Liberal styles, a different trio share an Internal scope, and another trio share Local levels. The Group F shares Hierarchic, Local and Conservative styles. Finally, the group G have a couple sharing Mixed functions and Liberal tendencies, and another couple shares External scopes.

Peer assessment reveals that best individual average is that of the group G, best group average is for the group A, and best FM point out to F. On the opposite side, we found the group B in all three cases. Square of Pearson product-moment correlation coefficient shows a strong link between PInd and PGrp (79.40%), but not with FM (35.29% and 36.70% respectively). As shown in the Figure 2, the gap between teacher and students criteria ranges from almost 1.5 points in D to barely 0.2 points in F:

ID	Sex	GrOUP	Functions	Forms	Levels	Scopes	Leaning	FM	PInd	PGrp
1	M	E	Executive	Hierarchic	Local	External	Liberal	7.52	9.01	8.96
2	M	–	Mixed	Hierarchic	Global	External	Liberal	–	–	–
3	H	–	Mixed	Hierarchic	Global	Internal	Liberal	–	–	–
4	H	A	Mixed	Hierarchic	Global	External	Liberal	8.75	8.63	9.58
5	M	–	Executive	Anarchic	Mixed	External	Liberal	–	–	–
6	H	B	Mixed	Hierarchic	Local	Internal	Conservative	7.59	8.58	8.13
7	H	B	Executive	Hierarchic	Local	Internal	Conservative	7.4	8.38	8.13
8	M	–	Mixed	Mixed	Mixed	External	Liberal	–	–	–
9	H	D	Mixed	Hierarchic	Local	External	Conservative	8.68	9.34	9.34
10	M	D	Judicial	Mixed	Local	External	Liberal	7.64	9.31	9.34
11	H	E	Mixed	Mixed	Local	External	Conservative	7.81	8.94	8.96
12	H	–	Mixed	Hierarchic	Global	External	Liberal	–	–	–
13	H	A	Judicial	Hierarchic	Local	External	Liberal	9.19	9.56	9.58
14	M	G	Mixed	Oligarchic	Global	External	Liberal	7.98	9.17	9.18
15	M	F	Legislative	Hierarchic	Local	Internal	Conservative	9.07	8.48	9.07
16	M	C	Judicial	Mixed	Mixed	External	Liberal	7.15	9.04	8.67
17	H	F	Mixed	Hierarchic	Local	External	Conservative	9.21	9.44	9.07
18	H	C	Judicial	Hierarchic	Local	External	Liberal	9.16	9.02	8.67
19	H	C	Judicial	Hierarchic	Global	Internal	Liberal	8.17	8.37	8.67
20	H	D	Legislative	Anarchic	Global	Mixed	Mixed	7.23	8.61	9.34
21	H	C	Judicial	Hierarchic	Local	External	Liberal	8.92	9.12	8.67
22	M	E	Legislative	Hierarchic	Local	Internal	Liberal	8.15	7.80	8.96
23	H	G	Mixed	Anarchic	Mixed	Mixed	Liberal	8.7	9.02	9.18
24	H	A	Mixed	Hierarchic	Global	Internal	Conservative	8.4	8.74	9.58
25	H	G	Executive	Monarchic	Local	External	Conservative	8.74	9.43	9.18
26	M	A	Judicial	Hierarchic	Global	External	Conservative	9.4	9.54	9.58
27	M	–	Mixed	Hierarchic	Global	Internal	Conservative	–	–	–
28	H	E	Mixed	Hierarchic	Global	External	Liberal	8.89	9.39	8.96
29	M	–	Mixed	Hierarchic	Global	External	Liberal	–	–	–

Table 2. Individual thinking styles. Paired with sex, picked group and marks

Figure 2. Marks, ordered by FM results

Considering Sex, males tend to be Mixed (53.3%) or Judicial (27.7%) on category Functions, meanwhile females are mostly Judicial (42.85%) or Legislative (28.57%). On category Forms, both are clearly Hierarchic, 73.3% for men and 57.14% for women, but with cluster exclusions: Oligarchic is not present on men as it happens with Anarchic on women. Something similar happens in category Levels, with both males and females choosing Local over Global (60% and 57.14% respectively). Also, with category Scopes, were External (60% and 71.42% respectively) is the dominant option. Finally, we found another clear choosing difference in category Leaning, as men chose almost fifty-fifty between Conservative and Liberal (46.66% each), but women were clearly Liberal over Conservative (71.42% and 28.57% respectively).

If we repeat this analysis, focusing this time on the influence of each thinking style category inside group elections, we gather 40 packs of 2 or more people, as shown in Table 3:

Category	Clusters by number of people inside
	2	3	4	∑Clusters	∑Students
Functions	5	0	1	6	14
Forms	5	2	1	8	20
Levels	6	2	0	8	16
Scopes	6	3	0	9	21
Leaning	7	1	1	9	21
∑Cluster-Category	29	8	3	40

Table 3. Group election by category and its thinking styles cluster size

Counting internal group coincidences aggregations (∑Clusters), and the total number of individuals included (∑Students), the best categories for clustering are, in decreasing order: Leaning, Scopes, Forms, Levels and Functions. As far as large aggregations ( 3) are concerned, the best options are, in decreasing order: Forms, Scopes, Leaning, Levels and Functions.

Data mining was focused on association rules (Scheffer, 2001; Liu, Hsu & Ma, 1998; Agrawal & Srikant, 1994). In order to do so, discretization of numeric variables (step needed to convert them to a qualitative form) was done on PInd, PGrp and FM. Three score intervals were defined on them: low, medium, and high. The Table 4 summarizes best rules found:

Group (rules found)	Best rule (number of members affected)
A (24)	Form = Hierarchic & PGrp = High (4) Form = Hierarchic & Level = Global & PGrp = High (3) Scope = External & Leaning = Liberal & PInd = High & PGrp = High (2)
B (10)	Form = Hierarchic & PInd = Low (2) Form = Hierarchic & Leaning = Conservative & PInd = Low (2)
C (26)	Function = Judicial & Leaning = Liberal & PGrp = Med (4) Function = Judicial & Levels = Local & Scope = External & PInd = High (2) Level = Local & Leaning = Liberal & PGrp = Med & PInd = Med (2)
D (1)	PInd = Low & PGrp = High (2)
E (13)	Function = Mixed & PGrp = Med (2) Level = Local & PInd = Low & PGrp = Med (2)
F (4)	Form = Hierarchic & Level = Local & Leaning = Conservative & PInd = High (2) Form = Hierarchic & Leaning = Conservative & PGrp = Med (2)
G (1)	Function = Mixed & Leaning = Liberal & PInd = High & PGrp = High (2)

Table 4. Association rules selection in each group

On the one side, Apriori reveals that D and G are the less coherent groups, as far as only one rule could be established within them. On the other side, A and C cover 71.4% of the all rules found, pointing out them as groups with strong and diverse internal bonds. Finally, B and F had two members each so, despite their size, must be considered the strongest one too. If fact, those groups are the only ones with perfect rules, affecting all members. For that analysis the two main criteria considered were standard support and confidence > 0.8, as they were explained in Methodology. Further settings were discarded due to the small size of the itemsets and our interest in discovering general rules (common, or very common subsets) and not marginal ones (what would only be of interest in samples of greater size).

4. Discussion

Social behaviour is foundational in this classroom. Group C shares two thinking styles among all its members (Function & Leaning); Group A, one (Form); Groups B and F, small ones, shares 2 and 3 styles respectively; E has one couple with strong bonds (sharing 2 thinking styles); Group G, despite having only one rule, has two members sharing Function and Leaning styles; and only Group D seems to be a group with no social patterns, as far their only sharing items are mark ranges. Being acquaintances from previous degree or time shared on the actual one become very relevant and emphasized intellectual thinking as academic affiliation strategy.

On the one hand, peer review does not show clear patterns. Internal gap between PInd and PGrp affects almost all teams, with D and B as the bigger ones. In fact, only Group F shows a convergence criterion among PInd, PGrd, and even FM. So, in this case, possible antagonistic or dissenting ways of teacher thinking do not seem to influence the criteria as assessors. The only verifiable point here is that teacher marks in all cases, except Group F, is the lowest, by far. The common strategies could explain that a clear goal of the students, insofar, it to pass the subject.

On the other hand, Sex has been shown by itself as a very interesting criterion to classify thinking styles in class, for two reasons: what they do choose and what they do not. Women picked massively Judicial or Legislative forms, but no one of them picked Anarchic. Also, women in class were clearly Liberal in their leaning. Men did not show a clear tendency in forms, with frequent ties (mixed forms); and no one identify himself as Oligarchic and leaning almost split them in half. The rest of styles, despite internal diversity, does not reflect sex patterns.

Internal group coherence tends to be strong in all team groups, showing that pre-service teacher students rely on their concept of teacher to choose their work group. Even if they lack of knowledge about it, as long as they created the groups before the theoretical explanation and the Sternberg-Wagner test.

A big study carried out in China (Yuan et al., 2017) over larger samples confirm thinking styles as powerful tool to predict stress-coping and participation in classroom. But, as we observed here, further studies (Song, 2018) have shown there is no clear relationship between thinking styles and learning outcomes. Other independent variables, like gender and age still remain without clear relationship explanation. With that in mind, our results about gender thinking styles must be taken prudently.

5. Conclusion

Data collected and analysed this way could be very useful for teachers in the long term. It allows better didactics strategies, regarding both theoretical and practical tasks, as far as it points out students’ strengths and weaknesses.

For example, in this case, it can be foreseen that women probably will not show interest on activities that could seem chaotic or unstructured at first (Anarchic form), meanwhile men will not be attracted to proposals that imply doing several things at once (Oligarchic form). Also, both sexes will tend to prefer collaborative tasks (External scope) over individual ones, so teacher could think about leveraging this, working on the inherent benefits of the Internal approach.

Sternberg-Wagner test is a well-known instrument, data gathering can be easily done in one hour and, even, it is possible to do that through online-forms. So, its impact on a course schedule is minimal. Also, its statistical analysis is reasonably fast, with free or open source tools, as shown above, and grants its access to every teacher with a computer and internet access. Thus, its results can be applied to class plan one or two weeks after data gathering.

However, we can detect several weaknesses like the need of a team-work subject oriented, as it makes much easier identify patterns. In the same way, it is an uncommon approach due to its strong mathematical component, so it requires a different way of doing things as teachers. A clear threat to this is the teacher background on statistics and ITC, as far as it is therefore essential that to have certain knowledge for those methods and tools.

Finally, we think that this class-research tool requires more research to clarify and optimize, also bigger study groups and coordinated researches in different subjects and education levels. But even without it, this approach has proven its usefulness in a practical class context. Current lines of work about thinking styles as predictor for student’s performance (Matošková & Kovářík, 2016) or creative thinking (Chegeni, Darabi & Niroomandi, 2016) are good examples of it and of its utility to continue exploring the benefits of this approach for the improvement of teaching in the field of Social Sciences, and represent the evolution of this scientific approach since the first systematic international studies on Sternberg’s intelligence theories (Sternberg, Castejón, Prieto, Hautamäki & Grigorenko, 2001).

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This work was supported by the Spanish National Plan for Excellence Programmes on Research, Development & Innovation under Grant EDU2013-41974- P.

References

Agrawal, R., & Srikant, R. (1994). Fast algorithms for mining association rules in large databases. Procedings of the 20th International Conference on Very Large Data Bases, 478-499.

Ahmadi, F., & Ahmad, S. (2013). Data mining in teacher evaluation system using WEKA. International Journal of Computer Applications, 63(10), 14-18. https://doi.org/10.5120/10501-5268

Algarni, A. (2016). Data mining in education. International Journal of Advanced Computer Science and Applications, 7(6), 456-461. https://doi.org/10.14569/IJACSA.2016.070659

Bautista, J.J. (2015). The evolution of social science methodology. The International Journal of Interdisciplinary Civic and Political Studies, 10(3), 35-42. https://doi.org/10.18848/2327-0071/CGP/v10i03/59365

Beltrán, M. (1985). Cinco vías de acceso a la realidad social. Reis, 29(29), 7-41. Available at:  http://www.jstor.org/stable/40183084 https://doi.org/10.2307/40183084

Bouhnik, D., & Carmi, G. (2012). E-learning Environments in Academy: Technology, Pedagogy, and Thinking Dispositions. JITE: Research, 11.

Brin, S., Motwani, R., Ullman, J.D., & Tsur, S. (1997). Dynamic Itemset Counting and Implication Rules for Market Basket Data. Proceedings of the ACM SIGMOD, 265-276. https://doi.org/10.1145/253260.253325

Brutus, S., & Gorriti, M. (2005). La Evaluación Multifuente Feedback 360º. Revista de Psicología del Trabajo y de las Organizaciones, 21(3), 235-252. Available at: https://doaj.org/article/b99d22c270f1495bafb77853b8bc2d96

Canbolat, N., Erdogan, A., & Yazlik, D.O. (2016). Examining the relationship between thinking styles and technological pedagogical content knowledge of the candidate mathematics teachers. Journal of Education and Training Studies, 4(11), 39-48. https://doi.org/10.11114/jets.v4i11.1819

Cartelli, A. (2012). Digital Technologies, Informing Science, and Transformations in Teaching-Learning Processes. Proceedings of the Informing Science & IT Education Conference (In SITE) (367-378). Montreal, Canada. https://doi.org/10.28945/1660

Chegeni, S., Darabi, R., & Niroomandi, M. (2016). Predicting creative thinking of students based on Sternberg thinking styles. Academic Journal of Psychological Studies, 5(3), 228-240.

Christiansen, B., & Koeman, J. (2015). Nationalism, cultural indoctrination, and economic prosperity in the digital age (1st ed.). Hershey: IGI Global. https://doi.org/10.4018/978-1-4666-7492-9

De Robertis, E.M. (2017). The phenomenology of learning and becoming: Enthusiasm, creativity, and self-development. New York: Palgrave Macmillan. https://doi.org/10.1057/978-1-349-95204-5

Fan, J. (2016). The role of thinking styles in career decision-making self-efficacy among university students. Thinking Skills and Creativity, 20, 63-73. https://doi.org/10.1016/j.tsc.2016.03.001

Fink, D., & Garner, S. (2008). Wisdom in Student Assignments: Its Operationalization and Manifestation. Proceedings of the Informing Science & IT Education Conference (In SITE). https://doi.org/10.28945/3222

Frank, E., Hall, M.A. & Witten, I.H. (2016). The WEKA Workbech. Online appendix. In Frank, E., Hall, M.A., & Witten, I,H. (Eds.), Data Mining: Practical Machine Learning Tools and Techniques. Cambridge: Morgan Kaufmann.

Gerring, J. (2012). Social science methodology (2. ed.). Cambridge: Cambridge University Press.

Hocutt, M. (2005). Indoctrination vs. education. Academic Questions, 18(3), 35-43. https://doi.org/10.1007/s12129-005-1016-y

Jamil-Asghar, J. (2016). Teacher, text and thinking. Journal of Research in Social Sciences, 4(2), 40. Available at: https://search.proquest.com/docview/1826443825

Keirn, T., & Luhr, E. (2012). Subject matter counts: The pre-service teaching and learning of historical thinking. The History Teacher, 45(4), 493-511. Available at: http://www.jstor.org/stable/23265942

Koedinger, K.R., D’Mello, S., McLaughlin, E.A., Pardos, Z.A., & Rosé, C.P. (2015). Data mining and education. Wiley Interdisciplinary Reviews: Cognitive Science, 6(4), 333-353. https://doi.org/10.1002/wcs.1350

Liu, B., Hsu, W., & Ma, Y. (1998). Integrating classification and association rule mining. Fourth International Conference on Knowledge Discovery and Data Mining, 80-86.

Martínez-Romera, D.D., Cebrián-Robles, D., & Cebrián-de-la-Serna, M. (2016). Assessment of teaching skills with e-rubrics in master of teacher training. Jett, 1(Extra), 120-141. Available at: https://goo.gl/xBFHgR

Martins, N., & Martins, E.C. (2006). Increased value of 360° assessments. HR Future, (September), 52-53. Available at: http://hdl.handle.net/10500/14438

Matošková, J., & Kovářík, K. (2016). Development of a Situatinal Judgment Test as a Predictor of College Student Performance. Journal of Psycoeducational Assessment, 35(8), 768-784. https://doi.org/10.1177/0734282916661663

Midhatovna-Galimullina, N., & Nikolaevna-Korshunova, O. (2016). Cultural competences in the educational process of higher school. Sovremennye Issledovaniâ Socialʹnyh Problem, 11, 646-667. https://doi.org/10.12731/2218-7405-2015-11-53

Niehaus, E.K., Reading, J., & García, C.E. (2018). Road to Researcher: The Development of Research Self-Efficacy in Higher Education Scholars. Journal for the Study of Postsecondary and Tertiary Education, 3, 1‑20. https://doi.org/10.28945/3950

Papamitsiou, Z., & Economides, A.A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Journal of Educational Technology & Society, 17(4), 49‑64. Available at: http://search.proquest.com/docview/1660157007

Perkins, D., Jay, E., & Tishman, S. (1993). Beyond abilities: A dispositional theory of thinking. Merrill Palmer Quarterly, 39(1), 1-21.

Rockenbach, A.N., Mayhew, M.J., Wolniak, G.C., Terenzini, P.T., Seifert, T.A., Bowman, N.A. et al. (2016). How college affects students (1st ed.). US: Jossey Bass Ltd.

Scheffer, T. (2001). Finding association rules that trade support optimally against confidence. 5th European Conference on Principles of Data Mining and Knowledge Discovery, 424-435. https://doi.org/10.1007/3-540-44794-6_35

Selesho, J.M. (2014). Reflection on the quality of teacher education professional thinking as reflected by pre-service teachers. Mediterranean Journal of Social Sciences, 5(2). https://doi.org/10.5901/mjss.2014.v5n2p57

Smetanová, V., Drbalová, A., & Vitáková, D. (2015). Implicit theories of critical thinking in
teachers and future teachers. Procedia - Social and Behavioral Sciences, 171, 724-732.https://doi.org/10.1016/j.sbspro.2015.01.184

Sternberg, R., Castejón, J.L., Prieto, M.D., Hautamäki, J., & Grigorenko, E.L. (2001). Confirmatory factor analysis of the Sternberg Triarchic Abilities Test in three international samples: An empirical test of the triarchic theory of intelligence. European Journal of Psychological Assessment, 17(1), 1-16. https://doi.org/10.1027//1015-5759.17.1.1

Sternberg, R.J. (1999). Thinking styles. Cambridge: Cambridge Univ. Press.

Stevens, D.D., & Levi, A.J. (2013). Introduction to rubrics: An assessment tool to save grading time, convey effective feedback, and promote student learning (2nd ed.). Virginia: Stylus Publishing.

Song, Y. (2018). An Investigation of the Relationships between Thinking Style, Participation in Classroom Dialogue and Learning Outcomes – A Study based in Mainland China (Doctoral thesis). https://doi.org/10.17863/CAM.17558

Tanna, P., & Ghodasara, Y. (2014). Using Apriori with WEKA for Frequent Pattern Mining. International Journal of Engineering Trends and Technology (IJETT), 12(3), 127-131. https://doi.org/10.14445/22315381/IJETT-V12P223

Taylor, R.M. (2017). Indoctrination and social context: A system‐based approach to identifying the threat of indoctrination and the responsibilities of educators. Journal of Philosophy of Education, 51(1), 38-58. https://doi.org/10.1111/1467-9752.12180

Valli, P., Perkkilä, P., & Valli, R. (2013). How can i be a better teacher? development of finnish adult pre‑ service teachers’ pedagogical thinking. Procedia - Social and Behavioral Sciences, 106, 1306-1320. https://doi.org/10.1016/j.sbspro.2013.12.146

Yuan, W., Zhang, L-F., & Fu, M. (2017) Thinking styles and academic stress coping among
Chinese secondary school students. Educational Psychology, 37(8), 1015-1025. https://doi.org/10.1080/01443410.2017.1287343

Zhu, C. (2013). Students’ and teachers’ thinking styles and preferred teacher interpersonal behavior. The Journal of Educational Research, 106(5), 399-407. https://doi.org/10.1080/00220671.2012.736431

---