Learning+Analytics

flat

Attribution (April 8, 2012) : https://docs.google.com/document/d/1bBRCyldBoS0ywDhNIgB9f4k7-hLUzviu3HUX7JBUDzQ/edit
 *  Learning Analytics: Guidelines for ethical use **

(please add questions, comments, resources...we’ll edit and clean up at the end of the week. For now, brainstorm and share resources. What are your thoughts/concerns/visuals for ethics and privacy in LA?).

=Analytics Principles =

 The collection of all data, whether formal course activities, supporting/related activity streams or additional profile data is analyzed for the purpose of better understanding the learner’s needs and their performance level in order to better support their learning process to drive success (at the individual learner level, which then builds to course, instructor(s), programs, departments, schools, districts...). All analytics should be done in the spirit of this goal and other uses of the data in it’s collection, sharing, or analysis is not in compliance with the principles of the analysis program of the institution. Any party participating in uses or practices not aligned with these goals, or contrary to them are subject to disciplinary review and action, up to and including criminal persecution. --Don’t assume that your users “don’t care about privacy,” no matter what they do on Facebook or Twitter that seems to indicate so. “ People appear to be unconcerned about privacy until it is actually breached [48],[49] and they do not necessarily act according to the privacy preferences they claim to have.” (This is from a really good article on how to engineer privacy into software: Spiekermann and Cranor’s “Engineering privacy” h [|__ttp://kulino.ninehub.com/file.php/1/Jurnal_dan_Artikel_Ilmiah/Rekayasa_Perangkat_Lunak/Vol_35_Issue_1_2009/05_Engineering_Privacy.pdf__] )

--No matter who owns the data (the student, the institution, or the system vendor), it should not be released to a third party for marketing purposes. --Transparency so that the learner (subject) has the right to know who is collecting the data, what data is collected, for what purpose(s) it is being collected (why), how the data will be used, and how to access data about themselves. --Transparency so the learner (subject) has a right to easily understandable and accessible information about privacy and security practices. The White House (2012). Consumer data privacy in a networked world: A framework for protecting privacy and promoting innovation in the global economy [|__http://www.whitehouse.gov/sites/default/files/privacy-final.pdf__] <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;"> Just read the executive summary section. <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">--Data must be kept secure at many levels- such as with HIPPA where certain “levels” of administration and specific assignment give very specific access to information (in a learner context, for instructors guiding students, specific student names and progress information need to be shared; however, for district-level analytics, specific student names and individual data need not be shared to analyze and communicate metrics regarding performance) <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">--Data must be reviewable by the learner (subject) to ensure accuracy. <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">-- Should the student have the ability to opt-out from having their data collected? <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;"> * If yes, how would it impact in the analytic system? <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;"> * If no, aren’t we recreating the software business model of “don’t agree, don’t install” (if a student doesn’t want his/her data to be collected, but doesn’t the institution doesn’t grant this right, his/her only option is to withdraw from the course)? <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">- What is the recourse for any individual who has had their data misused or inappropriately shared? What is the recourse against any party misusing the data? What if the harm to a learner or instructor was an unintended consequence (ref: Target outting a teen’s pregnancy by attempting to offer discounts- although it is unlikely something of this nature would occur, it illustrates a case on unintended potential harm caused by analytics even when a specific individual’s identity was not disclosed or revealed)? <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">-- As a design principle, the ultimate purpose of collecting and analyzing data is to make learners more intelligent, not to make courses more intelligent per se. Therefore, if there is a trade-off between making data available to learners so that they can better program their course work, or embedding it in the courseware that learners are obliged to use (i.e. where learners are more programmed than programmer), then priority should be given to the former.

<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">In some cases, universities my be required to collect and hold certain info about students,eg in the UK data s collected about socio-economic background to be able to demonstrate that they are not discouraging certain groups of students from having an opportunity to go to uni.

<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">Methods ought to be convivial - as in, tools for conviviality (Illich) <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">Methods and tools for analysing people’s records of learning, should be within the grasp of people with ‘everyday resources’, especially those being measured, and able to return information that is useful to them long before being useful to an abstract body like an institution.
 * <span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">[|__Early discussion thread on the ethics of learning analytics__]
 * <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;"><span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; vertical-align: baseline;">[|__An ethical framework for ubiquitous learning__] <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;"> (work in progress)
 * <span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">[|__Researching with my bare hands__]

=<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">Data ownership =

<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">-- Who owns the data mined from a learning process? The person who generated it, the institution who mined it, the LMS’ vendor, the government, ...?

<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">-- Who has the responsibility to provide the security measures to the data collected?

=<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">Responsibility of the Institution =

<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">-- If an institution collects data on a learner and predicts that the student will be unsuccessful, does the institution have a moral obligation to act? So, if the institution can’t assist/remediate, should it tell the student “I don’t think you’re a good fit...I can’t help you...I think you should disenroll from this institution instead of wasting more time/money here.” =<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">Transparency of the output =

<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">-- Who sees the outcome of analytics and to what level of detail?

<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">the obvious answer is ‘everyone’ maybe, but what if the analytics indicate that a student gets no extras compared to a student who does? How does a university justify providing what may be a differential service?

<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">-- What about the power relations that can be created/fueled with Learning Analytics? The data collected from Learning Analytics tools can be used, nowadays, to predict students’ outcomes. In some moment of the future, as we create an even bigger warehouse of data from students, we can be able to identify and predict behaviors that may be seen as “not adequate” for some institutions. Other point of this same issue can the instructors’ performance assessment through the Learning Analytics tools.

<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">In the UK there are some existing algorithms that calculate expected performance based on accredited certificates (e.g. ALIS predictions for A-Level predictions based on GCSE <span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; vertical-align: baseline;">[|__http://www.tes.co.uk/article.aspx?storycode=17159__] <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">). It is interesting to find discussions like this: <span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; vertical-align: baseline;">[|__http://www.thestudentroom.co.uk/showthread.php?t=153903&page=2__] <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;"> where some students suggest that ‘ALIS’ is a test rather than an analytic tool - some even perceive they have taken the test. They also tend to predict end performance at the end of a two-year period based on grades received before the start of that programme. Without transparency there can be a strong determinism in this presentation. As Einstein famously said <span style="background-color: #ffffff; color: #181818; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">“Everybody is a genius. But if you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid.” To me transparency needs to extend beyond just the data or algorithm to ensure students understand the purpose (good blog post on this by Doug Clow from #lak11: <span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; vertical-align: baseline;">[|__http://dougclow.wordpress.com/2011/03/11/marx-and-learning-analytics-towards-a-praxis-of-educational-improvement/__] <span style="background-color: #ffffff; color: #181818; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">)

=<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">Algorithms =

<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">-- Who owns? who can tweak? How are they tested and validated? Depending on how “prescriptive/predictive/punitive” these may be, what is the learners’ ability to challenge issues that inevitably will result (we have all seen poor recommendations by Amazon, and that has little consequence).

<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">-- Does the student have a right to see and understand the basis for interventions? Algorithms, by their nature, will be approximations of a historical set of observations used to predict possible future behaviours. There was a useful reference to cleaning data in ‘the promise and perils of big data’ ( <span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; vertical-align: baseline;">[|__http://www.thinkbiganalytics.com/uploads/Aspen-Big_Data.pdf__] <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">) which suggested that choices re which data to exclude will lead to different results.

<span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">Examples <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 15px; text-decoration: none; vertical-align: baseline;">Any fields or examples of privacy/ethics that we can use as a template?

=<span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 13px; vertical-align: baseline;">[|__OECD Guidelines on the Protection of Privacy and Transborder Flows of Personal Data__] <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 13px; text-decoration: none; vertical-align: baseline;"> (see Part Two, basic principles such as informed consent, use limitations, etc.) =


 * <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 13px; text-decoration: none; vertical-align: baseline;"><span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 13px; text-decoration: none; vertical-align: baseline;">Attention trust: <span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 13px; vertical-align: baseline;">[|__http://p2pfoundation.net/Attention_Trust__]
 * <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 13px; text-decoration: none; vertical-align: baseline;"><span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 13px; text-decoration: none; vertical-align: baseline;">Privacy Rights Clearinghouse <span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 13px; vertical-align: baseline;">[|__http://www.privacyrights.org/__]
 * <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 13px; text-decoration: none; vertical-align: baseline;"><span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 13px; text-decoration: none; vertical-align: baseline;">Freedom of Information and Protection of Privacy Act (Alberta, Canada) <span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 15px; vertical-align: baseline;">[|__http://www.servicealberta.ca/foip/__]
 * <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 13px; text-decoration: none; vertical-align: baseline;"><span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 13px; text-decoration: none; vertical-align: baseline;"> A Consumer Privacy Bill of Right (The White House 2012 Washington D.C. . USA) <span style="background-color: transparent; color: #1155cc; font-family: Arial; font-size: 13px; vertical-align: baseline;">[|__http://www.whitehouse.gov/sites/default/files/privacy-final.pdf__] <span style="background-color: transparent; color: #000000; font-family: Arial; font-size: 13px; text-decoration: none; vertical-align: baseline;"> Just read the executive summary section.