ASSESSING CREATIVITY VIA PROBLEM SOLVING TASK ON A VIDEO-BASED MEDIA IN HIGH STAKES RECRUITING PROCESS
1 Insper (BRAZIL)
2 Universidade Federal de Mato Grosso (BRAZIL)
About this paper:
Conference name: 14th International Technology, Education and Development Conference
Dates: 2-4 March, 2020
Location: Valencia, Spain
Abstract:
Creativity has become a determinant of performance and long-term success in organizations over the last 30 years. As a complex phenomenon, there are different definitions and ways to measure it. A general definition would be the process, outcomes, and products of attempts to develop and introduce new and improved ways of doing things. This process requires specific attributes at the individual level such as knowledge and divergent thinking. The present work reports the results of a strategy to assess creativity in candidates in a high stakes large-scale recruiting process of a Brazilian multinational organization, for the first placement of undergraduate and college egress. Self-reported questionnaires are commonly used, but for the purposes of this research, a problem-solving task was chosen. The model of "the investment theory of creativity" of Sternberg was adopted to build the assessment framework. The candidates were presented to two contextualized problems that don't have a specific solution and therefore demand new ideas and were invited to state possible solutions in a 3 min video.
The videos would be assessed in the following dimensions:
(1) Design: customer-centered, supply centered, marketing centered, point of sale, other.
(2) Applied cognitive flexibility: non-implementation, implementation;
(3) Learning agility: non-testing condition, testing condition;
(4) Originality: low frequency (1-2 ideas), medium frequency (3-4 ideas); high frequency (more than 5).
(5) Innovation: in the box, out of the box;
(6) Communication skills: voice, voice/text, voice/text/scheme, integrated multimedia and
(7) Inclusivity: current customer, new customer.
The task was proposed to almost 4,000 people, previously approved in cognitive tests among more than 50,000 applicants. 3,003 candidates recorded and submitted a video with their answers. The participants were from 402 different cities spread all over Brazil, ages varying from 20 to 29 years old and 44% of them were female. All of them had just concluded higher education in 453 different majors. A panel of seven specialists was trained to code according to the established protocol. Each video was independently coded by two graders based on a structured rubric and the overall interobserver agreement was 82%, meaning a high level of internal consistency reliability. 86% of the videos presented just voice, 7% voice and text and 7% also brought schemas, using integrated media strategies to communicate their ideas. 66% of the candidates proposed 1-2 ideas, 27% proposed 3-4 and 7% proposed 5 or more. The ideas characterized as “out of the box” were proposed by 28% of the candidates. 78% of them described an implementation process or pilot project to put their ideas in practice, but only 8% came up with ways of testing if their ideas would solve the original problem. Only 4% of proposals aim to include new customers in the context of the problems. These results suggest the adequacy of the strategy, but further research must be made. The criterion validity must be assessed by the comparison to traditional tests and to external evaluations of the candidate’s performance. In conclusion, the development of standardized protocols to assess videos in selection processes could increase the efficacy and reduce biases in the selection, favoring both organizations and candidates.Keywords:
Creativity, recruiting process, video-based assessment, career readiness.