Centre for Positive Psychology and Education

Upali Jayasinghe completed his PhD at the SELF Research Centre in 2003. His thesis formed an in-depth analysis of the peer review process that determines the outcomes of funding allocations at the Australian Research Council. This project involved applied quantitative methods that aimed at improving the reliability of the peer review process. Upali’s research expertise is in the application of multilevel modelling and other statistical models, with a particular focus on health and medical research. Dr Jayasinghe is currently employed as a Senior Research Fellow at the Centre for Primary Health and Equity, located at the University of New South Wales.

Qualifications

MSc Maths (Moscow), Grad Dip Stats (Sri Jayawardenapura), Grad Dip App Stats (Colombo), PhD (UWS)

Thesis title

Peer review in the assessment and funding of research by the Australian Research Council

Abstract

In higher education settings the peer review process is highly valued and used for evaluating the academic merits of grant proposals, journal submissions, academic promotions, monographs, text books, PhD thesis and a variety of other academic products. The purpose of this thesis was to evaluate the peer review process for awarding research grants used by the Australian Research Council (ARC) Large Grants Program and to propose strategies to address potential shortcomings of the system. This study also evaluated psychometric properties such as the reliabilities of various ratings that are part of the assessment process of the ARC Large grants Program. Data for the all grant applications submitted for the 1996 round of the Large Grants Program were provided by the ARC. In a variation to the typical peer review process, applicants were given an opportunity to nominate assessors to review their proposals. The results indicated that global ratings given by the researcher-nominated assessors were systematically higher and less reliable than those by panel-nominated external reviewers chosen by the ARC. The reliability of peer reviews is not adequate by most standards. A critical direction for future research is considering what strategies need to be put in place to improve the quality of the reviews. To improve the reliability it is recommended that researcher-nominated reviewers should not be used; that there should be more reviews per proposal and a smaller more highly selected core of reviewers should perform most of the reviews within each sub-discipline providing a greater control over error associated with individual reviewers.

Thesis submission date

2003

Supervisors

Professor Herb Marsh & Professor Nigel Bond

Publications

Harris, M.F., Jayasinghe, U.W., Chan, B., Proudfoot, J., Crookes, P., Zwar, N. & Powell Davies, G. (2011) Patient and practice characteristics predict the frequency of general practice multidisciplinary referrals of patients with chronic diseases: A multilevel study. Health Policy, 101(2), 140-145.

Taggart, J., Chan, B., Jayasinghe, U.W., Christl, B., Proudfoot, J., Crookes, P., Beilby, J., Black, D. & Harris, M.F. (2011) Patients’ assessment of chronic illness care (PACIC) in two Australian studies: structure and utility. Journal of Evaluation in Clinical Practice, 17(2), 215-221.

Holton, C.H., Proudfoot, J., Jayasinghe, U.W., Grimm, J., Bubner, T., Winstanley, J., Harris, M.F. & Beilby, J. (2010) A tool to measure whether business management capacity in general practice impacts on the quality of chronic illness care. Health Serv Manage Res, 23(4), 147—153.

Christl, B., Harris, M.F., Jayasinghe, U.W., Proudfoot, J., Taggart, J. & Tan, J. (2010) Readiness for organisational change among general practice staff. Quality & Safety in Health Care, 19(5).

Boyages, J., Jayasinghe, U.W. & Coombs, N. (2010) Multifocal breast cancer and survival: each focus does matter particularly for larger tumors. European Journal of Cancer, 46, 1990–1996.

Kirby, S.E., Dennis, S.M., Jayasinghe, U.W. & Harris, M.F. (2010) Patient related factors in frequent readmissions: the influence of condition, access to services and patient choice, BMC Health Services Research, 10, 216.

Rose, V., Harris, M., Ho, M.T. & Jayasinghe, U.W. (2009)  A better model of diabetes self-management? Interactions between GP communication and patient self-efficacy in self-monitoring of blood glucose. Patient Education and Counseling, 77, 260–265.

Jayasinghe, U.W. & Boyages, J. (2009) Tumour location is not an independent prognostic factor for survival following a diagnosis of breast cancer. The Breast, 18, 41–46

Jayasinghe, U.W., Proudfoot, J.,  Barton, C.A.,  Amoroso, C., Holton, C., Powell Davies, G., Beilby, J. & Harris, M.F. (2009) Quality of life of Australian chronically-ill adults: patient and practice characteristics matter. Health and Quality of Life Outcomes, 7(50).

Laws, R.A., Jayasinghe, U.W., Harris, M.F., Williams, A.M., Powell Davies, G.P. & Kemp, L.A. (2009) Explaining the variation in the management of lifestyle risk factors in primary health care: A multilevel cross sectional study. BMC Public Health, 9(165).

Marsh, H.W., Jayasinghe, U.W. & Bond. N.W (2008) Improving the Peer-review Process: Reliability, Validity, Bias and Generalizability. American Psychologist, 63(3), 160–168.

Jayasinghe, U.W., Proudfoot, J., Holton, C., Powell Davies, G., Amoroso, C., Bubner, T., Beilby, J. & Harris, M. (2008) Chronic ill Australian’ satisfaction with Accessibility and Patient-centredness. International Journal for Quality in Health Care, 20, 105-114.

Jayasinghe, U.W., Bilous, A.M. & Boyages, J. (2007) Is survival from infiltrating lobular carcinoma of the breast different from that of infiltrating ductal carcinoma? The Breast Journal, 13, 470-485.

Harris, M.F., Proudfoot, J.G., Jayasinghe, U.W., Holton, C.H., Powell Davies, G.P., Amoroso, C.L., Bubner, T.K. & Beilby, J.J. (2007) Job satisfaction of staff and the team environment in Australian general practice. MJA. 186, 570-573.

Proudfoot, J., Jayasinghe, U.W., Fernando, I., Beilby, J., Amoroso, C., Powell Davies, G., Grimm, J., Holton, H., Bubner, T. & Harris, M. (2007) Quality of chronic disease care in general practice: the development and validation of a provider interview tool. BMC Family Practice, 8(21).

Proudfoot, J., Jayasinghe, U.W., Holton, H., Grimm, J., Bubner, T., Amoroso, C., Beilby, J. & Harris, M. (2007) Team climate for innovation in Australian general practices. International Journal for Quality in Health Care, 19, 164–169.

Jayasinghe, U.W., Marsh, H.W. & Bond N.A (2006) New Reader Trial Approach to Peer Review in Funding Research Grants: An Australian Experiment. Scientometics, 69, 591–606.

Taylor, R, Jayasinghe, U.W., Koelmyer, L., Ung, O. & Boyages, J. (2006) Reliability and validity of arm volume measurements for assessment of lymphoedema. Physical Therapy, 86(2), 205–214.

Wan, Q., Harris, M.F., Jayasinghe, U.W., Flack, J., Georgiou, A., Penn, D.L. & Burns, J.R. (2006) Quality of diabetes care and coronary heart disease absolute risk in patients with type 2 diabetes mellitus in Australian general practice. Quality and Safety in Health Care 15, 131–135.

Jayasinghe, U.W., Taylor, R. & Boyages, J. (2005) Is age at diagnosis an independent prognostic factor for breast cancer survival? ANZ Journal of Surgery, 75, 762–767.

Leong, C., Boyages, J., Jayasinghe, U.W., Bilous, M., Ung, O., Chua, B., Salisbury, E. & Wong, A.Y. (2004) The effect of margins on ipsilateral breast tumour recurrence after breast conservation therapy for node-negative breast cancer. CANCER, 100, 1823-32.

Jayasinghe, U.W., Marsh, H.W. & Bond, N.A (2003) Multilevel cross-classified modelling approach to peer review of grant proposals: The effects of assessor and researcher attributes on assessor ratings. Journal of the Royal Statistical Society, 166(3), 279-300.

Jayasinghe, U.W., Marsh, H.W. & Bond, N. (2001) Peer review in funding of research in higher education: The Australian experience. Education Evaluation and Policy Analysis, 23, 343-364.

Craven, R.G., Marsh, H.W., Debus, R.L. & Jayasinghe, U.W. (2001) Diffusion effects: control group contamination threats to the validity of teacher-administered interventions. Journal of Educational Psychology, 93, 639-645.