We develop bespoke interest, ability and personality assessments for clients which are reliable, valid and legally defensible. Custom psychometric test design and construction represents a significant time and financial investment. It is likely to be suitable for those organisations who have an ongoing selection or development requirement. It is worth remembering that custom test development takes months, not days, therefore most companies do prefer to choose a ready made solution.
A properly designed and constructed test must have certain technical properties, and pass through a number of stringent quality control processes. It is certainly easy to write a poor quality test – one which ‘looks’ like a test – and to the untrained eye it may seem plausible enough. However, the quality of a test is determined by its psychometric and scaling properties, and not what the test items look like.
Unfortunately, many organisations only find out that the tests they are using have been produced by unqualified developers, and do not reach the required standard, when called upon to justify their use at an employment tribunal.
Commissioning us to develop your own in-house test has a number of advantages:
- It allows you to use a psychometric test which just focuses on those traits which are relevant to you.
- The administration, interpretation and reporting procedures can be customised to streamline with your wider HR processes and functions.
- You own the copyright.
A Typical Development Process Might Involve
- Defining the trait or characteristic to be measured in psychological terms.
- Item writing – writing good test items is exceptionally difficult and perhaps only 10% of the initial items will pass the various quantitative and qualitative quality control processes and make it into the final test.
- Response format – this needs to be chosen on the basis of the test function, and must avoid range restriction and allow the analysis of data at the Scalar level – in practice we deal with Interval data rather than Ratio data, as there is no absolute zero value in ability/personality assessment.
- Trial group choice – this generally needs to be screened and stratified.
- First trial – Standard Item Analysis is carried out (see any good textbook on the subject for P = * value cut-offs, mean and SD parameters) to reduce the number of items.
- Second Trial – item analysis repeated, and a First Order Factor Analysis carried out. Oblique or Orthogonal Exploratory Factor Analysis is used depending on the characteristics being measured. Second Order Factor Analysis is also carried out at this stage to establish the macro structure of the characteristic being measured and to ensure internal scale coherence.
- Finally Standardisation and Reliability Analysis are carried out. The Alpha Coefficient, or Cronbach’s Alpha, is the most widely used reliability analysis method. The standardisation and normative data production is straightforward, and done according to accepted methods. In calculating reliability, further items may be moved, or removed from certain scales so that all of the items in a particular scale contribute to that scale’s reliability – in other words, the degree to which it is free of error.
- Finally, the administration, scoring and reporting functions are finalised, and organisational users trained in its use.
- One more important activity also occurs at this stage – we decide upon success criteria, or the standards against which the predictive value (validity) of the test will be subsequently measured, and the analysis method to be used.
- Now your test is ready to use – but the development of the test will be an ongoing task with norms being added and updated as data from test takers is amassed.
If you would like to discuss bespoke psychometric test design and construction please contact us.