Preliminary Analysis Of IMLS-Funded Maker Literacies Pilot

Over the last year I've been busy with our Maker Literacies IMLS-funded pilot program Maker Competencies and the Undergraduate Curriculum and unfortunately have had little time to post to our blog. Now that we've come to the end of the pilot, I truly plan to remedy this with exciting new content!

The grant team will be spending the summer conducting our analysis and writing our report to the IMLS. I've got some preliminary analysis to share in this post, and will post more over the summer as our results begin to gel. By the end of September 2018 we will have launched an official Maker Literacies website where we'll post our full report and analysis, plus curriculum materials for each of the courses that participated in the pilot study.

In total, over 350 students, enrolled in 17 different courses spread across five campuses, representing 10 distinct disciplines* all completed a project in their academic library makerspaces during the Spring 2018 semester. 328 of these students consented to participate in our study and completed a pre- and post-self-assessment combination, providing valuable data about their comfort and knowledge with eleven defined competencies. From participating faculty, we’ve secured 13 unique curriculum packets licensed for re-use by anyone in the world, and invaluable feedback for improving our list of competencies and their integration into curriculum as student learning outcomes (SLOs).  Our first-pass review of all this data revealed the following four broad conclusions.

First, we received rich responses from participating faculty about the competencies themselves. Feedback included suggestions for making them more inclusive and transdisciplinary, suggestions for improving or building upon the existing competencies, and suggestions for adding competencies not in the beta list.

Second, in-class assessments done by faculty members seemed to be more successful when incorporating a self-reflection aspect. Faculty who required students to keep journals, or added a self-reflection component to their presentations were able to directly observe growth in their students. They were also able to provide deeper analysis of the learning that took place in their courses than instructors who did not implement a self-reflection component.

Third, some librarians did not feel comfortable assisting with curriculum development, and those who did were not consulted as true partners in curriculum development by participating faculty. Both of these aspects point to the need to develop librarians as experts and leaders in curriculum planning in order to foster better partnerships with their faculty members.

Lastly, there is much room to improve our pre- and post-self-assessment data collection methods and validating the student data for reliability.  

Each of these points will be expounded upon in our final report to the IMLS, and I plan to provide regular updates here on this blog as new conclusions emerge.

* Art, Computer Science, Education, Engineering, English, Geology, History, Mathematics, Philosophy, and Public Administration.