Third International Mathematics and Science Study-Repeat (TIMSS-R): What has changed in South African pupils’ performance in mathematics between 1995-1998?

SARAH J. HOWIE
Education and Training Systemic Studies
Human Sciences Research Council
Private Bag X41, Pretoria, 0001
South Africa

Tel: +27 12 302 2481
Fax: +27 12 302 2532
Email:
sjhowie@hsrc.ac.za

Note:
Also see the selected items and results below.
Click here for more information on TIMMS and TIMSS-R.

1. WHAT IS TIMSS-R?

The Human Sciences Research Council conducted TIMSS, under the auspices of the International Association for the Evaluation of Educational Achievement, among 15 000 South African students from more than 400 primary and secondary schools. Of the 63 countries that started the study, only 41 completed it. South Africa was the only country in Africa to do so. South Africa’s results from TIMSS 1995 can be found in Howie (1997) and in Howie and Hughes (1998).

In 1998, TIMSS was repeated (and is designated as TIMSS-Repeat or TIMSS-R), with tests and questionnaires administered in 38 countries. TIMSS-R was managed internationally by Boston College in the USA in collaboration with Statistics Canada in Ottawa (Canada), the IEA Data Processing Centre in Hamburg (Germany) and the Educational Testing Service in Princeton (USA).

The following countries participated in TIMSS-R:

Australia
Belgium (Flemish)
Bulgaria
Canada
Chile
Chinese Taipei
Cyprus
Czech Republic
England
Finland

Hong Kong
Hungary
Indonesia
Iran
Israel
Italy
Japan
Jordan
Korea
Latvia

Lithuania
Macedonia
Malaysia
Moldova
Morocco
Netherlands
New Zealand
Philippines
Romania
Russian Federation

Singapore
Slovak Republic
Slovenia
South Africa
Thailand
Tunisia
Turkey
United States

TIMSS-R was conducted in South Africa by researchers from the Group: Education at the Human Sciences Research Council. More than 8 000 Grade 8 learners were assessed in 200 schools, and more than 350 teachers and 190 principals of those schools participated.


2. HOW WAS THE INFORMATION COLLECTED?

The TIMSS-R research included tests and questionnaires to collect information about pupils’ achievement and background information about schools, teachers and pupils.

The mathematics and science tests were developed internationally in a collaborative manner. Three different types of questions were included in the pool of TIMSS-R questions: multiple-choice questions, short answer questions and extended answer questions. Some of the questions from 1995 were incorporated into the new tests as trend items, in order to measure the difference in achievement between 1995 and 1999.

Two questionnaires seeking information on national level curriculum plans, reforms, issues and policies with respect to mathematics and science curricula were administered to curriculum specialists. Questionnaires were also administered to the principals, mathematics teachers, science teachers and pupils.

A total of 225 schools were selected at random from all 9 provinces. One intact mathematics class per school was selected. Schools from all former departments of education are represented and both government and private schools are included in this sample. Ultimately, 194 schools and 8147 pupils were included in the international dataset for analysis. South Africa achieved an 85% response rate and the national sample is representative for the country.


3. HOW DID SOUTH AFRICAN PUPILS PERFORM IN MATHEMATICS COMPARED TO OTHER COUNTRIES?

South African pupils performed poorly when compared to other participating countries (see Appendix A - a pdf file. Also see the selected items and results below). The average score of 275 points out of 800 points is well below the international average of 487 points. The result is significantly below the average scores of all other participating countries, including the two other African countries of Morocco and Tunisia as well as that of other developing or newly developed countries such as Malaysia, the Philippines, Indonesia and Chile. As in 1995, pupils from the Asian countries of Singapore (at the top with 604 scale points), Korea, Chinese Taipei, Hong Kong and Japan demonstrated the best achievement in mathematics.

As can be seen in Appendix A, only the most proficient pupils in South Africa (and the same holds for Chile, Morocco and the Philippines) attained the level of the average pupils from Singapore. South African pupils scoring around the country’s average score fell below the least proficient pupils from almost all other countries with the exception of Morocco, the Philippines, Chile and Indonesia.


4. HOW DID PUPILS FROM DIFFERENT PROVINCES PERFORM IN MATHEMATICS?

The province with the highest average scale score for mathematics was Western Cape with 381 scale points, but this was still significantly below the international mean score of 487. Northern Cape and Gauteng achieved the second highest score with 318. Northern Province was below all the other provinces with a score of 226.

Table 1 Inter-provincial results for mathematics

Province

Number of pupils

Mean Scale Score
Out of 800 points

Minimum
score

Maximum
score

1 Eastern Cape

932

256

15

594

2 Free State

901

276

5

574

3 Gauteng

605

318

51

647

4 KwaZulu-Natal

1228

292

5

612

5 Mpumalanga

963

253

5

601

6 North West

690

267

18

594

7 Northern Cape

728

318

52

608

8 Northern Province

1166

226

6.5

458

9 Western Cape

933

381

78

699

South Africa

8146

275

5

699

An important finding is that there was no (statistical) difference in the scores of the girls compared to the boys. However, only in Western Cape were the girls’ scores better in mathematics than the boys. The largest differences between boys and girls were found in Free State (34 points) and Mpumalanga (38 points) where the boys perform on average 4-5% higher on the achievement test than the girls.


5. HOW DID SOUTH AFRICAN PUPILS PERFORM ACROSS THE DIFFERENT MATHEMATICS TOPICS?

South African pupils’ performance was relatively low in every mathematics topic (from 37% for algebra to 45% for data representation, analysis and probability). For South Africa, the average score for data representation, analysis and probability is the highest in that the score of 356 points (out of 800 points) makes this the scale with the smallest difference to the international average. Ironically, this is also the area not included in the intended curriculum at Grade 8 level. The lowest achievement was in the area of algebra with a score of 293 points out of 800 points.


6. HOW HAS THE PERFORMANCE IN MATHEMATICS AND SCIENCE CHANGED FROM 1995 TO 1998?

There are 26 countries in TIMSS-R that also participated in the original TIMSS study, including South Africa. Overall, the international average achievement in mathematics did not change, as there is only a small increase by 2 scale points (from 519 in 1995 to 521 in 1999 across the 26 countries). There was no real difference in performance between the pupils in 1999 and those in 1995. Less than 0.5% of pupils in South Africa reached the average score of the top 10% of pupils internationally.

Internationally, there were no significant differences in achievement for mathematics for either girls or boys between 1995 and 1999. In South Africa there was an increase in the score for girls (from 264 in 1995 to 267 in 1999). There was a decrease in the score for boys, who on average scored 10 points less in 1999 (from 293 in 1995 to 283 in 1999). Neither of these results is statistically significant.

In terms of gender differences between girls and boys, there were significant differences in some countries in 1999, as in 1995. An important finding, however, is that fewer countries had these differences (3 countries compared to 6) in 1999. In 1995, the Czech Republic, Iran, Japan, Korea, the Netherlands and Israel all had significant differences in achievement that favoured the boys. In 1999, only three countries still have these significant differences favouring boys; the gap has decreased in others. There were no significant differences found in 1995 or in 1999 for South Africa pupils. Although in 1995, there was a 29-point difference between girls and boys, this gap has narrowed to a 16-point difference in 1999.

The trend data reveal very few changes among the content areas in mathematics, as changes on this level are difficult to effect and take a number of years to show themselves. It is expected that TIMSS 2003 will be able to show more differences in achievement on this level

The only significant difference in achievement (between 1995 and 1999) internationally in specific content areas was found in data representation, analysis and probability. The average percent correct of South African pupils decreased by 2% for fractions and number sense, measurement and data representation, analysis and probability, and 1% for geometry and algebra.


7. WHAT KINDS OF SCHOOLS PARTICIPATED IN TIMSS-R?

The principals reported that:

  • Of teachers at South African schools, 68% are at the same school for 5 years or longer.
  • The average class size in Grade 8 is 46 pupils.
  • Urban schools are larger than rural schools. Average enrolment in urban schools is 1 028 while rural schools had 622 pupils on average.
  • Pupils are required to live in the area by 66% of schools.
  • Principals in South African schools spent much less time on instructional leadership and much more on teaching than their international peers.
  • The average number of school days was 195. However, the number of instructional school days reported by the principals varied from 120 to 280, and principals of 15% of pupils reported that there were one or more days of less than four hours at school.
  • The shortest school lesson time reported was 15 minutes.
  • Principals of 45% of pupils reported that instruction was hampered by the shortage of instructional materials and 31% by inadequate space.
  • Principals of 69% of pupils reported that Grade 8 pupils are absent from school daily and/or weekly and 75% report that pupils arrive late for school.
  • Principals of 20% of schools report serious drug problems at Grade 8 level.
  • Vandalism of school property is a serious problem reported by principals of 34% of pupils.

8. WHO WERE THE GRADE 8 MATHEMATICS TEACHERS THAT PARTICIPATED AND WHAT DO THEY DO AT SCHOOL?

  • Only 39% of mathematics teachers in Grade 8 are female.
  • Generally, pupils who have mathematics and science teachers older than 40 years achieved better scores in mathematics.
  • 27% of pupils were taught mathematics by teachers with no formal qualifications in mathematics.
  • The average class size for mathematics was 50 pupils.
  • South African teachers spend more time on administrative tasks than teachers in other countries.
  • South African mathematics teachers placed a higher emphasis on using a calculator than teachers internationally. Pupils of South African teachers who placed a low emphasis on calculators in class attained higher scores in mathematics.

9. CONCLUSION

The TIMSS-R results are disappointing. However, the performance of South African pupils’ has remained constant despite an increase in the number of pupils enroling at school. Nonetheless, there is a lack of comparability of South African pupils’ results with those of their peers in other developing countries, a number of which participated in 1998/1999.

Now that access to education and the right to learn has been established for the majority in the country, it is time to set key priorities for the country’s future. If South Africa is to succeed in a rapidly changing and competitive technological world, it will need to develop and protect its capacity to produce well-qualified teachers. In order to improve the lack of change detected by TIMSS-R from developing further, resources have to be put into a variety of well-designed, planned and effective programmes promoting and implementing mathematics and science. Greater collaboration within and between government and the private sector will be required to optimise energies and resources.

 

SELECTED TEST ITEMS FROM THE TIMSS-R STUDY:

1.

Subtract:
   7 003
4 078

 

A. 2 035
B. 2 925
C. 3 005
D. 3 925

   

2.

The sum 691 + 208 is closest to the sum

 

A. 600 + 200
B. 700 + 200
C. 700 + 300
D. 900 + 200

   

3.

Subtract: 4,722 – 1,935 =

 

A. 2,787
B. 2,797
C. 2,887
D. 2,897

   

4.

Find the value of x if 12x – 10 = 6x +32

 

Answer: ________________________

   

5.

There are 68 rows of cars in a parking lot. Each row has 92 cars. Which of these would give the closest estimate of the total number of cars in the parking lot?

 

A. 60 ´ 90 = 5 400
B. 60 ´ 100 = 6 000
C. 70 ´ 90 = 6 300
D. 70 ´ 100 = 7 000

   

6.

John sold 60 magazines and Mark sold 80 magazines. The magazines were all sold for the same price. The total amount of money received for the magazines was R 700. How much money did Mark receive?

 

Answer: ________________________

   

7.

Chris plans to order 24 issues of a magazine. He reads the following advertisements for two magazines. Ceds are units of currency (money) in Chris’ country.

 

 

Which magazine is less expensive for 24 issues? How much less expensive? Show your work.

   

8.

 

How many of the shaded right triangles shown above are needed to cover the surface of the rectangle (exactly)?

 

A. Four
B. Six
C. Eight
D. Ten

   

9.

The figure shows a shaded rectangle inside a parallelogram

 

What is the area of the shaded rectangle?

 

Answer: _______________________

   

10.

A rectangular garden that is next to a building has a path around the other three sides as shown.

 

What is the area of the path?

 

A. 144 m2
B. 64 m2
C. 44 m2
D. 16 m2

 

Percentage of correct responses by South African participants by question with the international percentage in brackets:

1.

37 (74)

 

2.

37 (80)

 

3.

42 (77)

 

4.

5 (44)

 

5.

30 (65)

6.

9 (44)

 

7.

1 (24)

 

8.

12 (46)

 

9.

3 (43)

 

10.

15 (42)

Home About Membership Council Activities Publications Conferences News Contact us Links