The University Entrance Exam Dilemma

I had been told and had long held the idea that the English test at the
university entrance exams were focused on “esoteric grammatical details” and
was somewhat relieved to read that that isn’t the case anymore from the
article below. Perhaps someone else might find today’s article in the
Language Connection column of the Daily Yomiuri helpful too. Source: Daily Yomiuri

***

Mike Guest Special to The Daily Yomiuri

Ed is a friend of mine who works at a nearby high school. A conscientious
and hard-working teacher, Ed is frustrated by the way university entrance
exams affect the content and style of teaching at high schools, particularly
those of high academic standing. Recently, over beers, knowing that I have
researched and been involved with such exams, Ed had several questions for
me. Many of these questions have been asked by English teachers nationwide
so I thought it might be beneficial for readers to “overhear” our
conversation.

Ed: We are told by the government to create a curriculum that will lead to
greater skills in using English in real-life situations. Yet we also have to
help prepare students for university entrance exams that work against the
government’s directive since the exams don’t really invite real-life
scenarios or interactions.

Mike: That really is a dilemma. Of course, the university entrance exams
have no pretense of being a measure of real-life, practical English skills
because the main purpose of these exams is to measure suitability for
further academic English study. Remember that a test is considered valid if
it meets its purpose, and since the purpose is to see if students are ready
or able to adapt to a higher academic level of English, having a focus on
real-life practical English would make the test less valid.

Ed: But couldn’t some productive, meaningful, interactive writing or
speaking, interactive quality be included? This would be a more rounded
measure of skill, right?

Mike: Well, it depends on which test you are referring to. If you mean the
Center Shiken, the answer is no.

Ed: Why?

Mike: Every year 500,000 to 600,000 candidates take the Center Shiken [as
the admissions test run by the National Center for University Entrance
Examinations is popularly known]. Time is an important factor because there
are four other subjects to be tested in addition to English. And, if writing
and speaking were included, not only would it demand more time to administer
the test but it would also increase the number of skills the examinees have
to master, making it even more stressful for them. Not only that, but the
Center Shiken is considered to be a strict meritocracy. To measure speaking
or productive writing skills would mean a lot of subjective, scoring by a
variety of evaluators with varying standards. Objectivity would go out the
window.

Ed: I’ve heard that many of the tests focus on esoteric grammatical detail.

Mike: Actually, I have analyzed some 20-year-old Center Shiken and
individual university exams, and while one could well argue that the tests
20 years ago were inordinately focused upon minute grammatical detail, the
tests now measure a wide variety of comprehensive skills using a wide
variety of texts and problems. I could identify less than 10 percent that
focused on sentence-level grammatical manipulations–and none that I would
think of as esoteric. I can’t say that the claim is valid anymore.

Ed: What about the second stage, individual university entrance exams?

Mike: Research has shown that over the years more and more of these exams
are testing a wider variety of skills such as expository writing in English,
paraphrasing, interpreting and so on. Yes, that introduces an element of
subjectivity in the grading but with only a few teachers involved in the
marking–and fewer examinees–standards can be monitored and maintained.
Second-stage exams also often require an English interview or essay outside
of the normal test time. And listening is included in many of these tests
and of course is now a part of the Center Shiken, too. So you could focus
upon these practical, productive skills and still help prepare students for
the exams.

Ed: I’m concerned about the reliability of these tests. Many of them look
extremely difficult.

Mike: Well, reliability can be determined in any number of ways and, yes,
one is if the test is too easy or too hard. If it is too hard there will be
large sections where almost all students will get zero or, if it is too
easy, 100 percent. In such a case, the test would not be considered reliable
because it hasn’t done its job, which is to stratify or rank students for
the purpose of eventual placement. Of course, the content of the tests
change every year so any such unreliable pattern would likely not continue.
Anyway many universities do analyze the results and take care to avoid using
any features that made a section too easy or too difficult for future exams.
The Center Shiken has made several such adjustments over the years.

Ed: Don’t you think that having the questions written in Japanese is strange
for an English test?

Mike: Not really. It actually increases test reliability. For example, an
examinee might have been able to answer a certain problem on the test, but
misunderstanding the question due its being in English would result in a
zero. Yet that zero would not reflect an inability to do the task that the
question require. which is what you are trying to measure.

Ed: An old testing maxim for teachers is, “Test what you’ve taught in the
way that you taught it.” That doesn’t seem to be the case here.

Mike: That maxim is fine if you are doing an achievement test, something
that sums up a course. And it makes sense if the test involves individual
teachers testing their own class. But university entrance exams are
ultimately forward-looking, for the purpose of student placement, and there
are over 50,000 teachers involved who have all taught slightly different
content and in slightly different ways.

Ed: I worry that just the slightest fault in these tests could lead to a
student not getting into university.

Mike: With the current demographics that’s not really a problem anymore.
These days, any student who has graduated from high school can enter some
university, as there are more places than applicants nationwide. Not only
that, but universities have become much more flexible about allowing credit
transfers from junior colleges or vocational schools, as well as
recommendations, so even if one struck out on the entrance exams one may
still eventually enter one’s hoped-for university. All in all, it means that
the tests are not as monolithic and mighty as is often thought.

Ed: Doesn’t this “recommendation system” seem a bit subjective and open to
abuse?

Mike: I suppose that’s possible. But it’s also arguable that it makes entry
more fair. Some examinees, such as those already working, those in other
universities or nonstandard high-schools, may not have access to, or time
for, the kind of exam preparation that intensive high schools or cram
schools offer. Recommendations allow such able candidates, who might not
score well on a standard university entrance exam, to gain entry.

Ed: It seems that you are satisfied with the current university entrance
exam system then.

Mike: Actually, no. I don’t have big problems with their reliability or
validity but I do question whether English should be a required subject on
the Center Shiken at all, especially given the government’s call for
increased performance skills in English. But that’s another column, another
conversation, another beer…

Guest is an associate professor of English at the Medical College of
Miyazaki University. He can be reached at mikeguest59atyahoo.ca.

(Mar. 17, 2006)