The Age of Analytics
Adoption of new technology is often a failure. Here are some steps for designing and using new data-driven tools.
By Aditya Johri
Each year, universities spend millions of dollars on their IT infrastructure. This infrastructure supports teaching, through learning management systems (LMS) such as BlackBoard or Moodle, and research such as experiments and analyses that require high-performance computing. In the past decade, analytics systems that leverage big data have become a standard across different industries. From a recent study of more than 400 large firms, Bain & Company concluded that early adopters of big data analytics had a significant lead over the rest of the corporate world. Higher education institutions are also cognizant of the potential value of analytics to improve organizational processes, workflows, resource allocation, and institutional measurement. In recent years, a slew of for-profit companies have developed and sold analytical tools that promise great benefits to higher education institutions by leveraging the data generated by different systems and activities. How useful are such tools, and what are the barriers to their usefulness? These are among the questions that my collaborators and I have been pursuing in a research project funded by the National Science Foundation’s BIGDATA program.
It is no secret that most technology adoption projects are unsuccessful. In our research, we have identified a few steps that can be taken to mitigate the risks of adopting new technology related to learning analytics (LA). First, as is common for most technology adoption processes, a crucial requirement to ensure that LA can be integrated into existing organizational routines is to design technology with a user-centered approach. Across a range of technological products, there is growing evidence that such an approach is essential for user adoption. A user-centered design coupled with ways that technology can be tailored by the user leads to higher adoption. For instance, it is essential that a LMS provide a way for faculty to monitor class participation. But an instructor may need to adapt the interface or dashboard depending on how a course is structured and how technologically proficient the instructor is.
Second, given the diversity of departments and offices within an academic institution, it is important to share practices that work both for specific roles and across roles and units. Advisers who use LA proficiently can shed light on what works and how some elements of what they do might benefit faculty. These sharing sessions can be face-to-face but can also consist of e-mails, mailing lists, and other elements that fit within existing routines. Sharing of practices has to be both bottom-up and top-down and will itself involve creating new work practices. The brainstorming that results from this kind of sharing will help representatives of the various departments and offices reach a common evaluation of new technologies.
Finally, it is important to think of technology and organizational development together, so that planning for the adoption of new technology and for organizational change can be linked. This notion captures the two ideas advanced above that users should be able to tailor new technology and that organizational practices need to be created to support technology adoption. These two aspects need to work together at multiple levels—practices of individuals, groups, and the organization as a whole. As an instructor, if I want to better understand students’ use of LMS, I need new dashboards and analysis, but I also need to change my own practices to ensure that I am taking what the system is telling me into account. If all I do is monitor but don’t respond with any changes, the LA is not going to be very effective. Overall, for any kind of IT adoption, systems-thinking is essential to develop an ecosystem where tools can be adopted by a range of users, functions, and roles.
Aditya Johri is an associate professor in the Department of Information Sciences and Technology at George Mason University.