I'll quickly highlight another aspect of the significant potential issues with privacy and data integrity surrounding connecting a Large Language Model (LLM) to Canvas. GDPR is built on a dignity concept of privacy that the US lacks, and the major LLMs are US-based [1], so one can argue that an LLM having access to a course Canvas page is counter to the fundamental European value of the dignity of everyone (students, staff, administrators) who are involved in the course.
What I think is more important to think about as a teacher is whether students should be allowed to have a single way of working (even if it 'works for them'). Post-degree a student simply cannot expect to work within their preferred system (even if freelance or in research). There are always systems that one simply has to adapt to, and to which one would never be allowed to grant a LLM access (e.g., what about doing one's tax return?). Thus a student wanting to only use their preferred approach to studying seems to be betraying that they have a shallow understanding of what it means to learn and study at university: their goal seems to be content mastery and nothing else (and their experience has been that an LLM makes that 'easier'). For example, they are not learning how to be in the world, how to work with different systems, how to navigate formal structures.
Quite frankly, it's disturbing that these aspects, that I wouldn't even consider part of the 'hidden curriculum' [2], because these are skills for life, not just about succeeding at university, may need to be made part of the formal (explicit) curriculum of university studies. Imagine, every course that is taken predominantly by first year students at the start of their first term has an intended learning outcome 'students will learn to navigate formal university systems (e.g., the learning management system) and appreciate that organisations use a range of specialised systems with specific purposes'? That feels more than a little patronising to me, but maybe that's what generative 'AI' is going to force academia into: catering to the lowest common denominator of possible understanding while teachers have to assume sufficient individual intelligence for university studies. (I cannot help but think this would also force teachers into a horribly uncomfortable state of hypocrisy, and make it much harder to teach with a 'growth mindset'.)
[1] Whitman, J. Q. (2004). The two western cultures of privacy: Dignity versus liberty. The Yale Law Journal, 113(6), 1151–1221. https://doi.org/
10.2307/4135723
[2] Snyder, B. R. (1971). The Hidden Curriculum. MIT Press.
---
P.S.: An argument might be that the student must obtain informed consent from everyone who is a member of the course Canvas page. Because access to the personal data is not 'legitimate interest'; a LLM does not have any legitimate interest in the personal data of students taking a course; it it could be argued, then there would be no reason for course pages to require a log in. One missing consent form, even of a student who decided to not take the course, prevents the whole. The teacher who has their doubts may also refuse to give their consent, and even advise (with reason) students against giving theirs. Moreover, consent can be retracted at any time, and the retraction of a single person's consent should be followed immediately by the removal of the LLM's access (arguably, managing the consent should be the teacher or course administrator's responsibility, though the student should make the request unsupported). And this must be repeated with every single course.