many new changes for the new quarter

- switched to using a configuration.json file
- reworked the download_student_info.py to fix bugs
- substantial change to the scripts to use the new config structure
- other small changes
- wrote a new README file based on the old readme and material I sent
  to Matt McGarrity

README_daily

@@ -0,0 +1,127 @@
+I keep my entire data directory in git and I'd recommend that you do
+too. Just make sure you don't commit and publish student records into
+the public git repository. I usually just keep a separate branch for
+classes.
+
+Daily Process
+================================
+
+1. Open your terminal (on Windows, this will likely be powershell in anaconda)
+
+2. Change into the directory with the coldcall scripts.
+
+3. Download new data with: `python download_student_info.py`
+
+   This will download the latest version of absence data into `data/optout_poll_data.tsv` as well as th student information into `data/student_information.tsv`.
+
+   If you noticed any changes you need to make (e.g., the same preferred names, incorrectly entered absences, etc) you should edit the Google sheets and then running the download again with the same script.
+
+4. When you're ready, fun the main script in the same directory: python coldcallbot-manual.py
+   
+   This will both:
+   
+   - output a paper list in terminal. I often redirect this to a file like: `python coldcallbot-manual.py > data/paper_call_list-2024-09-26.txt` or similar.
+   - Create the computed call list in the `data/` folder
+
+During case, I take notes on student answers on paper during class (typically I
+only note down non "GOOD" answers) and then add these to the sheet
+immediately after class.
+
+After class each day, you need to open up "call_list-YYYY-MM-DD.tsv"
+and edit the two columns in which you store the results of the
+case. The first columns `answered` means that the person responded and
+answered the question (i.e., they were present in the room but away
+from their computer and unresponsive). This is almost always TRUE but
+would be FALSE if the student were missing.
+
+The assessment column should be is "GOOD", "SATISFACTORY", "POOR", "NO
+MEANINGFUL ANSWER" or "ABSENT" but you can do whatever makes sense in
+this and we can work with it when it comes to grading. Just make sure
+you are consistent!
+
+Details on my rubric is here:
+
+https://wiki.communitydata.science/User:Benjamin_Mako_Hill/Assessment#Rubric_for_case_discussion_answers
+
+
+Assessment and Tracking
+======================================
+
+These scripts rely on a file in this repository called
+`data/student_information.csv` which I have set to be downloaded
+automatically from a Google form using the download script.
+
+I don't expect that these will necessary work without
+modification. It's a good idea to go line-by-line through these to
+make sure they are doing what *you* want and that you agree with the
+assessment logic built into this.
+
+For reference, that file has the following column labels (this is the
+full header, in order):
+
+    Timestamp
+    Your UW student number
+    Name you'd like to go by in class
+    Your Wikipedia username
+    Your username on the class Discord server
+    Preferred pronouns
+    Anything else you'd like me to know?
+
+The scripts in this directory are meant to be run or sourced *from*
+the data directory. As in:
+
+    $ cd ../data
+    $ R --no-save < ../assessment_and_tracking/track_participation.R
+
+There are three files in that directory:
+
+track_enrolled.R:
+
+    This file keeps track of who is in Discord, who is enrolled for
+    the class, etc. This helps me remove people from the
+    student_informaiton.csv spreadsheet who are have dropped the
+    class, deal with users who change their Discord name, and other
+    things that the scripts can't deal with automatically.
+
+    This all need to be dealt with manually, one way or
+    another. Sometimes by modifying the script, sometimes by modifying
+    the files in the data/ directory.
+
+    This requires an additional file called
+    `myuw-COM_482_A_autumn_2020_students.csv` which is just the saved
+    CSV from https://my.uw.edu which includes the full class list. I
+    download this one manually.
+
+track_participation.R:
+
+    This file generates histograms and other basic information about
+    the distribution of participation and absences. I've typically run
+    this weekly after a few weeks of the class and share these images
+    with students at least once or twice in the quarter.
+
+    This file is also sourced by compute_final_case_grades.R.
+
+compute_final_case_grades.R:
+
+    You can find a narrative summary of my assessment process here:
+
+    https://wiki.communitydata.science/User:Benjamin_Mako_Hill/Assessment#Overall_case_discussion_grade
+
+    This also requires the registration file (something like
+    `myuw-COM_482_A_autumn_2020_students.csv`) which is described
+    above.
+
+    To run this script, you will need to create the following subdirectories:
+
+    data/case_grades
+    data/case_grades/student_reports
+
+
+One final note: A bunch of things in these scripts assumes a UW 4.0
+grade scale. I don't think it should be hard to map these onto some
+other scale, but that's an exercise I'll leave up to those that want
+to do this.
+
+   
+ 5. after class, update the call list in the data folder to remove lines for any call that didn't happen (or you don't want to count) and update the assessments:
+ 

coldcall.py

@@ -16,6 +16,7 @@ class ColdCall():
             config = json.loads(config_file.read())
 
         self.today = str(datetime.date(datetime.now()))
+
         # how much less likely should it be that a student is called upon?
         self.weight = 2
         self.record_attendance = record_attendance
@@ -43,7 +44,7 @@ class ColdCall():
 
         return previous_questions
 
-    def __get_preferred_names(self):
+    def get_preferred_names(self):
         # translate the unique name into the preferred students name,
         # if possible, otherwise return the unique name
 
@@ -55,8 +56,9 @@ class ColdCall():
         return(preferred_names)
         
     def __get_preferred_name(self, selected_student):
-        if selected_student in self.preferred_names:
+        preferred_names = self.get_preferred_names()
-            return self.preferred_names[selected_student]
+        if selected_student in preferred_names:
+            return preferred_names[selected_student]
         else:
             return None
 
@@ -94,6 +96,8 @@ class ColdCall():
                 print("\t".join([self.unique_row, self.preferred_row, "answered", "assessment", "timestamp"]), file=f)
 
         preferred_name = self.__get_preferred_name(selected_student)
+        if preferred_name == None:
+            preferred_name = ""
 
         # open for appending the student
         with open(self.__fn_daily_calllist, "a") as f:

coldcallbot-manual.py

@@ -3,39 +3,39 @@
 from coldcall import ColdCall
 from datetime import datetime
 from csv import DictReader
+import json
 
 current_time = datetime.today()
+with open("configuration.json") as config_file:
+    config = json.loads(config_file.read())
 
 ## create the coldcall object
-cc = ColdCall(record_attendance=False, preferred_name_field="Name you'd like to go by in class")
+cc = ColdCall(record_attendance=False)
 
 def get_missing(d=current_time):
     date_string = f'{d.month}/{d.day}/{d.year}'
-    with open("data/absence_poll_data.tsv", 'r') as f:
+    with open(config["optout_file"], 'r') as f:
         for row in DictReader(f, delimiter="\t"):
             if row["Date of class session you will be absent"] == date_string:
-                yield(row["Your UW student number"])
+                yield(row[config["unique_name_rowname"]])
 
 full_names = {}
 registered_students = []
-with open("data/2022_winter_COM_481_A_students.csv", 'r') as f:
+with open(config["roster_file"], 'r') as f:
     for row in DictReader(f, delimiter=","):
         student_no = row["StudentNo"].strip()
         registered_students.append(student_no)
-        full_names[student_no] = f"{row['FirstName']} {row['LastName']}"
+        full_names[student_no] = f"{row[config['roster_firstname_rowname']]} {row[config['roster_lastname_rowname']]}"
-## print("Registered:", registered_students)
+# print("Registered:", registered_students) # useful for debug
 
 missing_today = [x for x in get_missing(current_time)]
-## print("Missing Today: ", missing_today)
+# print("Missing Today: ", missing_today)  # useful for debug
 
-preferred_names = {}
+preferred_names = cc.get_preferred_names()
-with open("data/student_information.tsv", 'r') as f:
+# print("Preferred names:", preferred_names)  # useful for debug
-    for row in DictReader(f, delimiter="\t"):
-        preferred_names[row["Your UW student number"]] = row["Name you'd like to go by in class"]
-## print("Preferred names:", preferred_names)
 
 students_present = [s for s in registered_students if s not in missing_today]
-## print("Students present:", students_present)
+# print("Students present:", students_present)  # useful for debug
 
 for i in range(100):
     selected_student = cc.select_student_from_list(students_present)

configuration.json

@@ -0,0 +1,16 @@
+{ 
+    "roster_file" : "data/FIXME.csv",
+    "roster_unique_rowname" : "StudentNo",
+    "roster_firstname_rowname" : "FirstName",
+    "roster_lastname_rowname" : "LastName",
+    "student_info_file" : "data/student_information.tsv",
+    "student_info_gsheet_id" : "FIXME",
+    "student_info_gsheet_gid" : 99999999,
+    "optout_file" : "data/optout_poll_data.tsv",
+    "optout_gsheet_id" : "FIXME",
+    "optout_gsheet_gid" : 99999999,
+    "daily_calllist_file" : "data/call_list-{date}.tsv",
+    "daily_attendance" : "data/attendance-{date}.tsv",
+    "unique_name_rowname" : "Your UW student number", 
+    "preferred_name_rowname" : "Name you'd like to go by in class"
+}

download_student_info.py

@@ -9,7 +9,7 @@ with open("configuration.json", 'r') as config_file:
 base_url = 'https://docs.google.com/spreadsheets/d/{id}/export?gid={gid}&format=tsv'
 
 student_info_url = base_url.format(id=config["student_info_gsheet_id"], gid=config["student_info_gsheet_gid"])
-subprocess.run(["wget", url, "-O", config["student_info_file"]], check=True)
+subprocess.run(["wget", student_info_url, "-O", config["student_info_file"]], check=True)
 
-optout_url = base_url.format(id=config["optout_info_gsheet_id"], gid=config["output_info_gsheet_gid"])
+optout_url = base_url.format(id=config["optout_gsheet_id"], gid=config["optout_gsheet_gid"])
 subprocess.run(["wget", optout_url, "-O", config["optout_file"]], check=True)