GuidelinesforEvaluatingaResearchArticle.pdf

257

Speaking of research

Guidelines for evaluating research articles

Phillip Rumrill∗, Shawn Fitzgerald andMegen WareKent State University, Department of EducationalFoundations and Special Services Center forDisability Studies, 405 White Hall, P.O. Box 5190,Kent, OH 44242-0001, USA

The article describes the components and composition ofjournal articles that report empirical research findings in thefield of rehabilitation. The authors delineate technical writingstrategies and discuss the contents of research manuscripts,including the Title, Abstract, Introduction, Method, Results,Discussion, and References. The article concludes with ascale that practitioners, manuscript reviewers, educators, andstudents can use in critically analyzing the content and scien-tific merits of published rehabilitation research.

Keywords: Evaluation, research articles, guidelines for cri-tique

1. Introduction

The purpose of this article is to examine the com-ponents of a research article and provide guidelinesfor conducting critical analyses of published works.Distilled from the American Psychological Associa-tion’s [1] Publication Manual and related descriptionsin several research design texts [4,8,9,12,15], descrip-tions of how authors in rehabilitation and disabilitystudies address each section of a research article arefeatured. The article concludes with a framework thatrehabilitation educators, graduate students, practition-ers, and other Work readers can use in critiquing re-search articles on the basis of their scientific merits andpractical utility.

∗Corresponding author: Tel.: +1 330 672 2294; Fax: +1 330 6722512; E-mail: [email protected].

2. Anatomy of a research article

For nearly 50 years, the American Psychological As-sociation has presented guidelines for authors to followin composing manuscripts for publication in profes-sional journals [1]. Most journals in disability studiesand rehabilitation adhere to those style and formattingguidelines. In the paragraphs to follow, descriptionsof each section of a standard research article are pre-sented: Title, Abstract, Introduction, Method, Results,Discussion, and References.

2.1. Title

As with other kinds of literature, the title of a scien-tific or scholarly journal article is a very important fea-ture. At the risk of contravening the age-old adage “Youcan’t judge a book by its cover,” Bellini and Rumrill [4]speculated that most articles in rehabilitation journalsare either read or not read based upon the prospectivereader’s perusal of the title. Therefore, developing aclear, concise title that conveys the article’s key con-cepts, hypotheses, methods, and variables under studyis critical for researchers wishing to share their findingswith a large, professional audience. A standard-lengthtitle for a journal article in the social sciences is 12–15words, including a sub-title if appropriate. Because so-cial science and medical indexing systems rely heavilyon titles in their codification schemes to track and cat-egorize journal articles by topic, providing a title thatclearly delineates a general research domain or topicarea is of utmost importance. If the title is vague orambiguous, chances are that the prospective reader willnot continue to read through the document to establishwhere it might fit in terms of a specific research domainor topic area. Examples of clearly descriptive titlesthat can be found in the contemporary rehabilitationliterature include:

“Rehabilitation Counselors’ Assessments of Appli-cants’ Functional Limitations as Predictors of Rehabil-itation Services Provided” [3].

Work 14 (2000) 257–263ISSN 1051-9815 / $8.00  2000, IOS Press. All rights reserved

258 P. Rumrill et al. / Guidelines for evaluating research articles

“Employer Concerns About Hiring Persons withPsychiatric Disabilities: Results of the Employer Atti-tude Questionnaire” [6].

“Self-Perceived Reasons for Unemployment Citedby Persons with Spinal Cord Injury: Relationship toGender, Race, Age, and Level of Injury” [13].

“Vocational Rehabilitation Counselors’ AttitudesToward Self-Employment Outcomes” [18].

“Surveying the Employment Concerns of Peoplewith Multiple Sclerosis: A Participatory Action Re-search Approach” [20].

“Effect of Graduate Research Instruction on Per-ceived Research Anxiety, Research Utility, and Confi-dence in Research Skills” [21].

Before we move into descriptions of the content sec-tions of a research article, we want to briefly addressthe concept of technical writing as it applies to the com-position of academic manuscripts. Journals adheringto the American Psychological Association’s [1] pub-lication guidelines favor manuscripts that are writtenin direct, uncomplicated sentences. Editors prefer thattext be written in the “active voice”; whenever possible,sentences should begin with their subjects and followwith verbs and objects (e.g., “The researcher conductedan experiment” rather than “An experiment was con-ducted by the researcher”). Technical writing is markedby the “less is more” maxim; extraneous phrases andclauses that add words to the sentence without enhanc-ing the overall statement should be avoided (e.g., “Inorder to. . . ”, “For purposes of. . . ”, “As far as. . . isconcerned. . .”). Another element of sound technicalwriting is the sparing use of adverbs (e.g., very, some-what, strikingly) and adjectives that do not serve to fur-ther define or specify the terms that they are modifying(e.g., interesting, important, good, noteworthy).

In addition to the American Psychological Associa-tion’s guidelines for technical writing, authors shouldconsider these six criteria for effective compositionprovided by George Orwell (1946) in Politics and theEnglish Language:

1. Never use a metaphor, simile, or other figure ofspeech which you are used to seeing in print.

2. Never use a long word where a short one will do.3. If it is possible to cut a word out, always cut it

out.4. Never use the passive (voice) where you can use

the active.5. Never use a foreign phrase, a scientific word, or

jargon word if you can think of an everyday En-glish equivalent.

6. Break any of these rules sooner than say anythingoutright barbarous (p. 170).

Organization is also key in preparing an effectivelycomposed journal manuscript, with multi-level head-ings serving to guide the flow of text and keep thereader on track. For authoritative information regard-ing the style and formatting guidelines for submittingmanuscripts to most journals in social science fields,readers should consult the American Psychological As-sociation’s [1] Publication Manual. For informationconcerning the style and formatting requirements ofWork and other journals published by IOS Press, seethe Guidelines for Authors section included in the be-ginning of this edition.

2.2. Abstract

Next to the title, the abstract is the most widely readsection of a journal article. In an empirical article, theabstract should be a succinct, 100–150 word summaryof the investigation’s key features, including purpose,objectives, research questions/hypotheses, sample, sci-entific procedures, independent and dependent vari-ables, and salient results. Results of the study shouldbe summarized in full in the abstract; authors shoulddescribe both significant and non-significant findings,not only those which upheld their hypotheses or expec-tations. The abstract serves as an advance organizerfor the article, and it should include every importantpremise, method, and result of the investigation. Likethe Preface that commonly orients readers to full-lengthtextbooks, the abstract provides a complete, albeit sum-mary, preview of the article. Some journals, includ-ing Work and the Journal of Vocational Rehabilitation,ask authors to list key descriptors on the abstract page,which are then used for purposes of indexing. In mostcases, the title is what determines whether a reader willread the abstract; the abstract determines whether thereader will read the body of the article.

2.3. Introduction

Immediately following the abstract, the introductorysection of the article sets the stage for the study uponwhich the article was based. It orients the reader to theproblem or issue being addressed, develops the logicand rationale for conducting the investigation, and al-most always expresses the empirical hypotheses or re-search questions. Heppner et al. [9] suggested thatthe introduction should answer questions such as whythe topic is an important one to study, what previous

P. Rumrill et al. / Guidelines for evaluating research articles 259

work bears on the topic, how existing work logicallyconnects to the author’s research questions and/or hy-potheses, how the question will be researched,and whatpredictions can be made.

To answer these questions, authors typically addressthree major elements in the introductory section of anarticle: (1) The Research Problem, (2) The Frameworkfor the Study, and (3) The Research Questions and Hy-potheses [8,15]. We will describe each of these intro-ductory elements in linear fashion, but we do not meanto imply an order in terms of how they should be ad-dressed. Many (if not most) authors blend these con-siderations to fit the flow and logic of their respectivemanuscripts.

The research problem. Usually in the very first sen-tences of an empirical journal article, the author drawsthe reader’s attention to the scope, impact, and currentstatus of the problem or issue being investigated. Thisorientation is most effectively achieved by applying thebroadest-possible perspective to the concern. A studyof success rates among participants in a stress inocula-tion program for people with diabetes mellitus might beintroduced by citing national statistics concerning theincidence and prevalence of this very common disease.An article describing the effects of a model job place-ment program for women with breast cancer might be-gin with a review of existing literature concerning em-ployment and breast cancer, with a particular focus onthe difficulties that women have in re-entering the la-bor force following diagnosis and treatment. Authorsreporting a longitudinal study of the post- school em-ployment outcomes of secondary students with devel-opmental disabilities would likely introduce their arti-cle with a review of the disappointing adult outcomeswhich that population has experienced since the incep-tion of formalized transition services in the mid–1980s.

The framework for the study. The specific theoret-ical and empirical framework for the particular inves-tigation is another important part of the Introduction.Authors summarize existing literature related to theidentified problem, then build a logical rationale for astudy that addresses gaps or inconsistencies in the lit-erature. The author should present the theoretical orconceptual model that informs the inquiry and providesenough background to enable the reader to appreciatethe rationale of the current study. This framework elu-cidates the purpose of the current study (e.g., to eval-uate the effectiveness of a job placement program forwomen with breast cancer), which is then operational-ized in the research questions or hypotheses. Socialscientific theories which have figured pominently in

the frameworks of recent rehabilitation investigationsinclude Hershenson’s [10] model of work adjustment,Bandura’s [2] concept of situational self-efficacy, andBolton and Brookings’ [5] integrated model of empow-erment.

The research questions and hypotheses. The Intro-duction section of a research article typically includesa statement of the research questions and/or hypothe-ses that served to guide the study. A more specula-tive research question tends to be used in descriptiveresearch designs (e.g., surveys, program evaluations,empirical literature reviews) or in qualitative studies.Examples of research questions could include: “Whatconcerns do college students with disabilities have re-garding their future career prospects?”; “What themesare evident in the psycholinguistic development of deafwomen?”; and “What steps are Fortune 500 employ-ers taking to provide on-the-job accommodations forworkers with disabilities?”.

The hypothesis, on the other hand, is predictive bydesign. Its specificity is dependent upon the theory un-derlying it or previous, relevant research, but it shouldinclude the direction of the expected results when-ever possible. Independent and dependent variablesneed not be operationalized in theory-based hypotheses(because this is done in the Method section), but theexpected relationship among study variables must beclearly articulated. Examples of directional hypothesescould include: “Participation in a cognitive-behavioralstress inoculation program will decrease symptom on-set and magnification”; “Anxiety, depression, and lowself-esteem will be collectively, positively, and signif-icantly related to work interference”; and “Rehabilita-tion counselors will rate people with severe disabili-ties as less favorable candidates for employment thansimilarly qualified people with mild or no disabilities”.

2.4. Method

The Method section delineates how the researchquestions were addressed and/or how the hypotheseswere tested. It should provide the reader with sufficientinformation so that one could replicate the investiga-tion, and it should leave no question as to what was“done” to the participants. Because the Method sectionis the primary source for determining the validity of thestudy [4], the quality and clarity of this section are gen-erally regarded as the strongest determinants of whetheran empirically-based manuscript will be accepted forpublication [9,16].

260 P. Rumrill et al. / Guidelines for evaluating research articles

Although the type and order of sub-sections foundin the Method section of a research article vary de-pending upon the design of the study and the author’sjudgement related to the flow of text, most articles in-clude descriptions of the study’s subjects/participants,instruments/measures/variables, materials, design, andprocedures.

Subjects/participants. According to Heppner etal. [8,9], the Method section should include (a) the totalnumber of subjects and numbers assigned to groups, ifapplicable; (b) how subjects were selected and/or as-signed; and (c) demographic and other characteristicsof the sample relevant to the study’s purpose. Some au-thors also include a description of the population fromwhich the study sample was drawn, a description of thespecific sampling procedure used (e.g., simple random,stratified, cluster; [4]), an indication of the represen-tativeness of the sample vis a vis the broader popula-tion, the circumstances under which subjects partici-pated (e.g., whether they were compensated, what risksthey assumed), statistical power analyses, and responserates (if applicable).

Instruments/measures/variables. The Method sec-tion must include a detailed description of how all studyvariables were operationalized, measured, scored, andinterpreted. All instruments or measures that were usedin sampling, conducting the study, and evaluating re-sults must be specified in terms of content (e.g., num-ber of items, response sets), how measures were ad-ministered, scoring procedures, relationship to studyvariables, and psychometric properties (e.g., standard-ization, reliability, validity). Authors should also in-clude a rationale for selecting each instrument, that is,why that instrument was the best choice for measuringa particular construct.

Materials. Researchers should also include a de-scription of any materials that were used to carry outthe investigation. Written guides for participants, in-structional manuals, media or technology, and scien-tific apparatus or equipment should be described in de-tail. Some authors include a description of the settingin which the study was executed or data were collected.

Design. One of the most important features of theMethod section is a clear description of the design ofthe study. This is essential because the design serves asthe link between (a) the research questions/hypothesesand the scientific procedures used in carrying out thestudy and (b) the findings of the study and how theseare interpreted. Authors typically label their designsin terms of how variables were manipulated, observed,and analyzed. Thereby, the design is the unifying force

in connecting the research objectives to both the resultsand the knowledge claim that is made. To every extentpossible, a direct reference to the hypotheses shouldbe made when authors identify the design of a particu-lar investigation. For example, Rumrill, Roessler, andDenny [19] described their design as follows: “The re-searchers selected a three-group, posttest-only (exper-imental) design to assess the intervention’s univariateand multivariate effects on (a) self-reported attitudes(situational self-efficacy and acceptance of disability)and (b) participation in the accommodation request pro-cess.”

Procedures. The most important component of theMethod section is the easiest to describe. In chrono-logical order, authors simply list every step they tookin developing, administering, and evaluating the study.Beginning with the recruitment of participants, follow-ing the study through collection of the last datum, andincluding everything in-between – the Procedures sub-section should provide the reader with a step-by-stepprotocol that could serve as a guide for replicating thestudy. Descriptions of any interventions should be pro-vided in detail, along with summaries of the qualifi-cations of project personnel who were instrumental inexecuting the investigation. Procedures should also in-clude how the investigation ended, along with a state-ment of any debriefing or follow-up services providedto participants.

2.5. Results

The Results section of a research article should in-clude a complete inventory of all relevant findings ob-tained by the investigators. In articles that report quan-titative studies, results are typically presented in twoparts – (a) summary, or descriptive, statistics relatedto participants’ performance on the measures that weretaken (e.g., means, standard deviations, frequencies,percentages) and (b) statistical analyses related to thespecific hypotheses of the study (e.g., analysis of vari-ance, multiple regression, factor analysis). We believethat all analyses conducted as part of the investigationshould be reported in full, not only those which yieldedstatistically significant results. The Publication Man-ual of the American Psychological Association [1] pro-vides considerable guidance related to how statisticsshould be presented in the Results section, but it doesnot always provide adequate guidelines regarding whatstatistical information should be included. Heppner etal. [9] identified a pattern in recent social science lit-erature whereby researchers tend to err on the side of

262 P. Rumrill et al. / Guidelines for evaluating research articles

providing too little statistical information: “The trendhas been to report less; for example, one rarely seesanalysis of variance source tables anymore. More dis-turbing is the tendency not to report important informa-tion (such as size of test statistic and probability levels)when results are non-significant. This minimalist pointof view puts the emphasis on statistical significance andignores concepts such as effect size, estimation, andpower.”

In recent years, the “minimalist” perspective (interms of reporting statisitical findings) has been chal-lenged by numerous researchers and statisticians [11,14,22]. The most serious argument against this per-spective relates to the influence that sample size hasin determining the significance of any statistical test.Hayes [7], for example, pointed out that virtually anystudy can be made to yield statistically significant re-sults if the researcher includes enough subjects. Toavoid the possibility of misleading research consumers,the latest edition of the Publication Manual [1] suggeststhat all authors provide estimates of practical or clinicalsignificance along with all statistical significance testsreported in the Results section.

A quantitative Results section should be limited tothe findings obtained by the researcher(s) in the cur-rent investigation. Speculation concerning what thosefindings mean in a larger context is reserved for theDiscussion section.

The Results sections of qualitatively oriented articlesdisplay much more variety in the content and manner ofpresentation than is found in quantitative studies. Be-cause the researcher’s subjective interpretations help toshape the processes and outcomes of qualitative inves-tigations, results are often framed in broad, interpretivecontexts. In that regard, the lines between the Resultsand Discussion sections are often blurred in qualitativeresearch.

Researchers (qualitative and quantitative) commonlyuse tables and figures to summarize and/or graphicallypresent their results. There is wide variability in thecontent and presentation of tables and figures, withthe most important universal requirement being easyinterpretability for the reader.

2.6. Discussion

The Discussion section serves as the researcher’s fo-rum to go beyond the current investigation and discussthe contributions of study findings to existing litera-ture, theory, and professional practices. The first partof a thoughtful Discussion is often an analysis of the

study’s results vis a vis the research questions and hy-potheses. Researchers should begin with a discussionof whether the hypotheses were upheld, posit possibleexplanations for those outcomes,and draw implicationsfrom the findings back to the research problem that wasidentified in the Introduction. If the results providea warrant for modifying or re-testing the conceptualframework upon which the investigation was based, theDiscussion section is the place to suggest a reformula-tion of the underlying theory. Researchers should alsoinclude a statement of the scientific limitations of thecurrent study, along with specific recommendations forfuture research. Finally, the researcher ends the arti-cle with a cogent summary of the conclusions, in themost general sense, that can be drawn from the methodsand findings of the current study. Some authors use aseparate Conclusion section for this purpose.

2.7. References

The final section of a research article is always alisting of the references that were cited in the body ofthe text. References are listed in alphabetical order,according to authors’ last names. Most rehabilitationjournals require adherence to the American Psycholog-ical Association’s [1] guidelines regarding the compo-sition of the References section.

3. A scale for critiquing research manuscript andarticles

Understanding the components, organization, andcomposition of a research article will help make Worksubscribers better informed consumers as they read em-pirically based publications. As readers digest the con-tents of research articles and apply them to their prac-tices, the “anatomy” of research reports can serve as auseful rubric for critically analyzing the quality, con-tent, and practical significance of published articles.Table 1 presents specific questions for conducting asection-by-section critique of a rehabilitation researcharticle.

4. Conclusion

This article examined the components of a researcharticle and provided guidelines for conducting a criticalanalysis of published research. Although the descrip-tions of the components of a research article provide

P. Rumrill et al. / Guidelines for evaluating research articles 263

only a skeletal summary of what should be includedin a published research article, they should provide thereader enough information to both prepare manuscriptsfor publication and evaluate the empirical research thatappears in Work and other rehabilitation journals.

References

[1] Washington, D.C., American Psychological Association, Pub-lication manual of the American Psychological Association,(Fourth Edition), 1994.

[2] Bandura, A., Social foundations of thought and action: Asocial cognitive theory, Englewood Cliffs, NJ: Prentice-Hall,1986.

[3] Bellini, J., Bolton, B. and Neath, J., Rehabilitation counselorsassessments of applicants functional limitations as predictorsof rehabilitation services provided, Rehabilitation CounselingBulletin 41(4) (1998), 242–258.

[4] Bellini, J. and Rumrill, P., Research in rehabilitation counsel-ing: A guide to design, methodology, and utilization, Spring-field, IL: Charles C. Thomas Publishers, 1999.

[5] Bolton, B. and Brookings, J., Development of a multifaceteddefinition of empowerment, Rehabilitation Counseling Bul-letin 39(4) (1996), 256–264.

[6] Diksa, E. and Rogers, E., Employer concerns about hiring per-sons with psychiatric disability: Results of the employer atti-tude questionnaire, Rehabilitation Counseling Bulletin 40(1)(1996), 31–44.

[7] Hayes, W., Statistics for psychologists, New York: Holt, Rine-hart, and Winston, 1981.

[8] Heppner, P., Kivlighan, D. and Wampold, B., Research designin counseling, Pacific Grove, CA: Brooks/Cole, 1992.

[9] Heppner, P., Kivlighan, D. and Wampold, B., Research designin counseling, (2nd Edition), Pacific Grove, CA: Brooks/Cole,1999.

[10] Hershenson, D., A systems reformulation of a developmentalmodel of work adjustment, Rehabilitation Counseling Bulletin

40(1) (1996), 2–10.[11] Hunter, J., Needed: A ban on the significance test, Psycholog-

ical Science 8 (1997), 3–7.[12] Kazdin, A., Research design in clinical psychology, (2nd Edi-

tion), Needham Heights, MA: Allyn and Bacon, 1998.[13] Krause, J. and Anson, C., Self-perceived reasons for unem-

ployment cited by persons with spinal cord injury: Relation-ship to gender, race, age, and level of injury, RehabilitationCounseling Bulletin 39(3) (1996), 217–227.

[14] McClure, P., Determining the significance of significance: P-values, effect size, and clinical judgement, Journal of HandTherapy 12 (1999), 40–41.

[15] McMillan, J. and Schumacher, S., Research in education: Aconceptual introduction, (Fourth Edition), New York: Long-man, 1997.

[16] Munley, P., Sharkin, B. and Gelso, C., Reviewer ratingsand agreement on manuscripts reviewed for the Journal ofCounseling Psychology, Journal of Counseling Psychology 35(1988), 198–202.

[17] Orwell, G., Politics and the English language, in: A collectionof essays, G. Orwell ed., San Diego: Harcourt, Brace, andJovanovich, 1946, pp. 156–171.

[18] Ravesloot, C. and Seekins, T., Vocational rehabilitation coun-selors’ attitudes toward self-employment outcomes, Rehabili-tation Counseling Bulletin 39(3) (1996), 189–201.

[19] Rumrill, P., Roessler, R. and Denny, G., Increasing confidencein the accommodation request process among persons withmultiple sclerosis: A career maintenance self-efficacy inter-vention, Journal of Job Placement 13(1) (1997), 5–9.

[20] Rumrill, P., Roessler, R. and Koch, L., Surveying the employ-ment concerns of people with multiple sclerosis: A participa-tory action research approach, Journal of Vocational Rehabil-itation 12(2) (1999), 75–82.

[21] Schaller, J. and Parker, R., Effect of graduate research in-struction on perceived research anxiety, research utility, andconfidence in research skills, Rehabilitation Education 11(4)(1997), 273–287.

[22] Thompson, B., AERA editorial policies regarding statisticalsignificance testing: Three suggested reforms, EducationalResearcher 25(2) (1996), 26–30.

P. Rumrill et al. / Guidelines for evaluating research articles 261

Table 1A scale for critiquing research articles

Instructions: Answer the following questions regarding the article, “ ”. Use examplesfrom the article to support your analyses.

A. Title

1. Does the title describe the study?2. Do the key words of the title serve as key elements of the article?3. Is the title concise, i.e., free of distracting or extraneous phrases?

B. Abstract

4. Does the abstract summarize the study’s purpose, methods, and findings?5. Does the abstract reveal the independent and dependent variables under study?6. Are there any major premises or findings presented in the article that are not mentioned in the abstract?7. Does the abstract provide you with sufficient information to determine whether you would be interested in reading

the entire article?

C. Introduction

8. Is the research problem clearly identified?9. Is the problem significant enough to warrant the study that was conducted?

10. Do the authors present a theoretical rationale for the study?11. Is the conceptual framework of the study appropriate in light of the research problem?12. Do the author’s hypotheses and/or research questions seem logical in light of the conceptual framework and research

problem?13. Are hypotheses and research questions clearly stated? Are they directional?14. Overall, does the literature review lead logically into the Method section?

D. Method

15. Is the sample clearly described, in terms of size, relevant characteristics, selection and assignment procedures, andwhether any inducements were used to solicit subjects?

16. Do the instruments described seem appropriate as meausres of the variables under study?17. Have the authors included sufficient information about the psychometric properties (e.g., reliability and validity) of

the instruments?18. Are the materials used in conducting the study or in collecting data clearly described?19. Are the study’s scientific procedures thoroughly described in chronological order?20. Is the design of the study identified (or made evident)?21. Do the design and procedures seem appropriate in light of the research problem, conceptual framework, and research

questions/hypotheses?22. Overall, does the method section provide sufficient information to replicate the study?

E. Results

23. Is the Results section clearly written and well organized?24. Are data coding and analysis appropriate in light of the study’s design and hypotheses?25. Are salient results connected directly to hypotheses?26. Are tables and figures clearly labeled? Well organized? Necessary (non-duplicative of text)?

F. Discussion and Conclusion

27. Are the limitations of the study delineated?28. Are findings discussed in terms of the research problem, conceptual framework, and hypotheses?29. Are implications for future research and/or rehabilitation counseling practice identified?30. Are the author’s general conclusions warranted in light of the results?

G. References

31. Is the reference list sufficiently current?32. Do works cited reflect the breadth of existing literature regarding the topic of the study?33. Are bibliographic citations used appropriately in the text?

H. General Impressions

34. Is the article well written and organized?35. Does the study address an important problem in the lives of people with disabilities?36. What are the most important things you learned from this article?37. What do you see as the most compelling strengths of this study?38. How might this study be improved?

Copyright of Work is the property of IOS Press and its content may not be copied or emailedto multiple sites or posted to a listserv without the copyright holder's express writtenpermission. However, users may print, download, or email articles for individual use.

Our customer support team is here to answer your questions. Ask us anything!