For this post, I am evaluating Mini Course Generator as part of EDUO 652 coursework to see how it might perform as a tool for generating course content. Mini Course Generator is a tool that allows the user to enter a description of the course and generate content that can be directly uploaded to a learning management system.
Figure 1. Screen Grab from Mini Course Generator Tool.
Figure 2. Screen Grab from Mini Course Generator Tool shows the completed course.
Evaluation
Interestingly, many resources exist on responsible and inclusive AI model building (Li & Gu, 2023). World Economic Forum, 2022), evaluating the output of an AI application seems to be left up to the individual user; we are reminded, for example, to check our biases, without having a systematic checklist to compare against. For this post, I evaluated the AI application according to the following elements:
Does it match my purpose for using AI tools?
My purpose is to be able to generate more common content quickly, using information already available on the internet, so that I can focus more time on the content that requires more effort to develop, such as content around emerging technologies with limited access to Subject Matter Experts in the field.
Is it intuitive and easy to use?
This tool was straightforward to use, but the trade-off is realized in the quality of the output. The content generated is not completely relevant or course-specific, as it does not ask for more information at the onset of using the tool. Additionally, while the output is meant to be a completed course that you can upload directly to your learning management system, there are several things about this content that I would need to modify and edit to be aligned with best practices in instructional design, such as meaningful learning objectives and interactive and experiential elements.
What is the tools approach to handling data and security?
This tool did not include much information about what happens to any information I fed it and did require a login. To log n, I needed to provide an email address and my name. I gave the prompt very little information for this assignment to avoid sharing anything proprietary. This aspect would be more deeply examined and evaluated if I used this tool for workforce development
.
Is the generated content accurate, relevant, and of good quality?
Although I have an engineering background, my areas of expertise do not include battery manufacturing, so while a lot of content was generated quickly, it took me much longer to have a Subject Matter Expert colleague assess the content for accuracy. Some content was technically correct for this tool, but a lot wasn’t exactly right or industry-specific.
Does it match my values for equity and accessibility?
Zhang and Deng (2022) point to two main facets of ethical issues in the educational use of AI: technology design and development and actual use in academic practice. Since this tool would be used by instructors to generate content for students, but students would not be interacting with the too interface, I looked at accessibility and equity through the lens of what students would be exposed to, which is the generated content as the output of the technology as designed.
The content generated did not come across as being biased towards gender. This trial version generated two accompanying images, and I was pleased to see that the second image (Figure 3) featured a non-white person and a female. It would require further analysis to see if other bases might exist (Odubela, 2022).
Figure 3. Screen Grab from Mini Course Generator Tool editable course content
The tool doesn’t seem to have any sort of accessibility features. For example, no alternative text for the images was included, and no way to check how Ryan (learner persona) might perceive the image.
In addition to the challenges with using AI raised by Zhang and Deng (2022), Addy et al. (2023) bring awareness around who benefits and who is excluded from using AI. This tool could make learning more accessible to Maria if the content generated in alternative languages is accurate. An area of further exploration might be to see if you can feed the tool accessibility and equity best practices when writing the initial prompt that can be used to generate content.
Pros and cons of using the selected AI
The pro from using this tool was not that it generated a lot of content, but that it also generated typical sections of content It helped with seeing how the content around a particular subject could be broken up into smaller chunks, made relevant to a specific job role, and packaged so that it might work better for adult learning.
This tool asked me for a prompt that described the training I was hoping to generate but didn’t explicitly call out for any learning objectives. If I were to continue to explore this tool, I would next try to be more specific in my course description. Additionally, I would play with including verbs from Bloom’s taxonomy to see if that influenced the generated content. Moreover, it was clear that this was simply a ChatGPT-type application plopped into an authoring tool. For example, these images show the content generated is more conversational than a finished learning or training product (see Figure 4).
Figure 4. Screen Grab from Mini Course Generator Tool showing conversational quality.
Would I use this tool?
As of today, I will not use this tool. However, as these tools evolve, I will continue to check back to see what improvements and best practices have been implemented. For now, I plan to continue using traditional Instructional design models like ADDIE or SAM for identifying learning objectives, creating a course framework, and addressing equity and accessibility, and then use ChatGPT or Google NotebookLM to have better control over generating accurate and relevant content.
References:
Addy, T., Kang, T., Laquintano, T., & Dietrich, V. (2023). Who benefits and who is excluded? Transformative learning, equity, and generative artificial intelligence. Journal of Transformative Learning, 10(2). https://jotl.uco.edu/index.php/jotl/article/view/518/388
Li, S., & Gu, X. (2023). A risk framework for human-centered artificial intelligence in education. Educational Technology & Society, 26(1), 187-202.
Odubela, A. (2022). Foundations of responsible ai [Video file]. LinkedIn. https://www.linkedin.com/learning/foundations-of-responsible-ai/understanding-responsible-ai
World Economic Forum. (2022, June). A Blueprint for equity and inclusion in artificial intelligence [White paper]. https://www.weforum.org/publications/a-blueprint-for-equity-and-inclusion-in-artificial-intelligence/
Zhang, K., & Deng, P. (2022, May). Exploring the technology and problems of artificial intelligence education applications. In 2022 4th International Conference on Computer Science and Technologies in Education (CSTE) (pp. 265-268). IEEE.
Comentarios