The Interface Between Interlanguage, Pragmatics and Assessment:
Proceedings of the 3rd Annual JALT Pan-SIG Conference


The eight articles in these proceedings are from a conference co-sponsored by the Pragmatics SIG, Testing & Evaluation SIG, CUE SIG, and the Tokyo and West Tokyo Chapters of the Japan Association for Language Teaching (JALT). This event was held at Tokyo Keizai University on May 22 - 23, 2004. Dr. Andrew D. Cohen of the University of Minnesota and Dr. Jenny Thomas of the University of Wales were plenary speakers and about 40 others presentations and poster sessions were featured during this event.
The relationship between testing, pragmatics, and fluency has been an ongoing research topic for many language teachers and researchers across the world. The 2002 and 2003 Conference Proceedings reflect some of this research. The proceedings of those conferences offered a rich source of information about various ways testing and pragmatics and university education interlink. This year's conference proceedings continues to explore themes raised in earlier conferences.
The opening essay by Dr. Andrew D. Cohen considers some facets of fluency. After describing early approaches to speech act performance assessment, some recent efforts to assess speech act ability were mentioned. Cohen also highlighted some factors often used when assessing speech acts and described his own work in developing online pragmatics materials.
Kazumi Kimura and Masako Kondo next explore how Japanese danraku and English paragraphs vary. Kimura and Kondo note that many Japanese EFL students use danraku-like structures in their essays. This results in a lack of focus and overall organization. The authors suggest that the essential features of an English paragraph need to been explained to EFL writers, and clearly distinguished from Japanese danraku.
H. P. L. Molloy and Mika Shimura then describe an interlanguage pragmatics research study which explores how sensitive L2 English users are to situational differences when making scripted complaints. After analyzing data via G- and D-studies and some ad hoc scatterplots, they found that situational variance accounted for less than 3% of the total variance when making scripted complaints. Their data suggests that participant idiosyncrasies were likely the major reason for variation in L2 pragmatic performance. The authors concluded that situational sensitivity may have a less high-profile role in determining pragmatic behavior than previously suggested.
Keita Kikuchi describes how a needs analysis was conducted at a university in Tokyo. Since learners, teachers, and program coordinators tend to have differing views regarding appropriate curricular targets, Kikuchi underscored the value of obtaining information from each of these groups via interviews, questionnaires, and observational data. Rather than basing curriculum development ontop-down management ultimatums, Kikuchi argues for combining multiple sources of information. Specifically, Kikuchi attempted to assess six components of Aoyama Gakuin's English program: (1) curricular goals, (2) salient problems, (3) learners' preferences concerning materials, (4) the overall English level of most students, (5) the attitudes towards English among most students, and (6) suggestions for improving the program. Responses were varied and gave program designers some valuable data with which to plan the future curriculum.
Joseph Falout then explores ways to teach the TOEIC® communicatively. A number of practical activities designed to help learners progress from mere declarative knowledge of linguistic forms towards a more practical "procedural knowledge" in which linguistic tasks are performed in real time without too much deliberation are described. Examples of focused, communicative tasks designed to foster automaticity are highlighted in his article.
Holistic and analytic writing assessment methods are then constrasted by Yuji Nakamura, using Weigle's adaptation of Bachman and Palmer's framework for test usefulness. The author suggests that "the best practice is to have multiple raters and multiple rating items." The least recommended practise is to have just one rater with a single-item holistic scale. In between these two options, one compromise might be to have multiple raters use a holistic scale. Nakamura echoed Weigle in pointing out how construct validity and reliability issues should be of primary concern in testing, and practicality and test impact issues should be of lesser weight.
The article by Matthew Apple and Etsuko Shimo describes learner attitudes towards portfolio assessment writing projects. The authors contrast traditional assessment methods with portfolio assessment. They point out that though portfolio assessment is generally more time-consuming and difficult than traditional assessment procedures, it is also more rewarding and seems to promote greater awareness of meta-cognitive skills.
Miyoko Kobayashi's investigation of reading comprehension test performance examines two of the test method facets proposed by Bachman, namely the nature of the input and the nature of the expected response. Meyer's model of prose analysis was used to manipulate the nature of the input to create four different types of text organization. Three different types of response format were used: cloze test items, open-ended questions and summary writing. Kobayashi concludes that these two variables influence reading comprehension test performance. Furthermore, it was shown that the influence of the text organization and response format varies between the different proficiency levels. A follow-up study examined the texts and response formats used for the Center Examination and shows that some students may have been disadvantaged by these tests.
We would like to conclude by offering special thanks to the following people for their help in organizing this conference: Chiyo Hayashi, Jeff Hubbell, Tadashi Ishida, Megumi Kawate-Mierzejewska, Marish Mackowiak, Yoko Mochizuki, Peter Ross, Stan Pederson, Steve Ross, Chizuko Sato, Chris Sullivan, Sayoko Yamashita, and Junko Fujio. Without their kind assistence, this event would never have taken place.

- T Newfields, Yvonne Ishida, Mark Chapman, & Mayumi Fujioka
2004 Conference Proceedings Co-editors

2004 Pan SIG-Proceedings: Topic Index Author Index Page Index Title Index Main Index
Complete Pan SIG-Proceedings: Topic Index Author Index Page Index Title Index Main Index

[ p. i ]