Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
astro300_f17:day4 [2017/08/31 19:28] – ccheng | astro300_f17:day4 [2017/09/06 22:37] (current) – [General Takeaways] tzick | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ======AY 375 - Fall 2016: Fourth Day Lesson Plan====== | ||
+ | |||
+ | =====General Takeaways===== | ||
+ | |||
+ | - Deeper conceptual knowledge CAN be probed with multiple choice questions, but writing effective questions takes time and practice. | ||
+ | - Especially when writing free response questions, it can be useful to develop a grading rubric for each question as a way of ensuring that your questions are specific, clear, and not testing the same concept over and over again. | ||
+ | |||
+ | |||
+ | |||
+ | =====Section Recap (15 minutes)===== | ||
+ | |||
+ | Remind them that this is something we intend to do every week and that everyone should come prepared to share about how their previous sections went. | ||
+ | |||
+ | Remind them what to think about for section recap: | ||
+ | * What did you do? | ||
+ | * How did you implement your activities? | ||
+ | * What worked? | ||
+ | * What didn't work? | ||
+ | * What would you do differently? | ||
+ | * How did you assess learning? | ||
+ | * Did you receive any unexpected questions/ | ||
+ | * Did anything unexpected happen? | ||
+ | * What were you thinking about while you were running section? Any moments of panic? | ||
+ | |||
+ | Open the floor up for general questions and sharing about how sections are going. | ||
+ | |||
+ | ===== Multiple Choice and Free Response Questions (40 min) ====== | ||
+ | |||
+ | - (5 min) Come up with either a multiple choice or free response question for the class you're teaching for and write it down on a piece of paper. Also think about what learning objectives it tests and what level of learning it probes. | ||
+ | - (6 min) Trade your questions with a partner. Have them attempt to answer the questions or at least determine what learning objectives were being probed, what level of learning it targeted, and what would have constituted as an " | ||
+ | - For both: | ||
+ | * Is the wording clear? | ||
+ | * What concepts are being tested? | ||
+ | - For the multiple choice: | ||
+ | * Is there clearly only one correct answer? | ||
+ | * Are there any obvious throw-away answers? | ||
+ | * Are you able to rule out any response because of the wording alone (i.e., are there hints in the structure)? | ||
+ | - For the free response: | ||
+ | * Does part B test the small conceptual/ | ||
+ | * What if students cannot solve part A? What does that imply for part B? | ||
+ | * What sort of responses might students give under the pressures of an exam setting? | ||
+ | - (6 min) Come back as a class and discuss. | ||
+ | * Did you learn anything surprising? | ||
+ | * Is this easy? (Unfortunately, | ||
+ | * What part of question writing did you find the most difficult? | ||
+ | - (20 min) Go through question example slides as a class exercise. | ||
+ | * MC Summary | ||
+ | * Test what you teach and teach what you test! | ||
+ | * Write short, clear questions and solutions. | ||
+ | * All answers should be of a similar tone and length. | ||
+ | * Avoid throw-aways, | ||
+ | * Be sure to not suggestively word your responses. | ||
+ | * Exams should have a variety of difficult and easy questions. Some easier questions at the start of the exam can enhance motivation. | ||
+ | * FR Summary | ||
+ | * Test what you teach and teach what you test! | ||
+ | * Write clear prompts. Be explicit about what you want students to provide (no core dumps). | ||
+ | * Multiple parts should test multiple ideas, not the same idea again and again. | ||
+ | * Solutions should require novel ideas, not a summary of material in the prompt. | ||
+ | * Reminder about timing: always take your own quiz/ | ||
+ | |||
+ | |||
+ | **Some notes on multiple choice questions: | ||
+ | |||
+ | Despite their outward appearance, these questions are actually inherently nonobjective. Grading an essay exam is subjective to the personal feelings of the grader, compared to running a Scantron through a machine. Grading written problems falls somewhere between the two. This is only partially correct: " | ||
+ | |||
+ | The ultimate goal of testing is to measure what the students actually understand, and the process of interpreting the meaning of a student' | ||
+ | - the physical format and layout of the question | ||
+ | - the conceptual hierarchy of the questions | ||
+ | - the statistical item analysis | ||
+ | Below are some guidelines for each of these items. | ||
+ | |||
+ | == Layout == | ||
+ | |||
+ | You are testing the students' | ||
+ | |||
+ | >> You forget that the star Betelgeuse is a red giant and apply the method of magnitudes to determine its distance. The true distance to Betelgeuse is actually... (a) shorter than you calculated, (b) the same distance that you previously calculated, (c) farther than you calculated. | ||
+ | |||
+ | Is short, to the point, and clear. You might be tempted to elaborate on small points that are not the main conceptual item that is being tested, but care must be taken. For example, | ||
+ | |||
+ | >> You forget that the star Betelgeuse is a red giant (a very luminous star in the top right of the HR diagram with relatively low surface temperature) and apply the method of spectroscopic parallax---a comparison of the star's apparent magnitude, estimated from the HR diagram, and its absolute magnitude---to determine its distance from Earth, which can be considered the same as its distance to the Sun because the Earth-Sun distance is negligible given the scales involved. The true distance from Earth to Betelgeuse is actually... | ||
+ | |||
+ | In an attempt to be completely clear, the stem has become more difficult for most students to understand! | ||
+ | |||
+ | Over the years, students have learned that when novice faculty include choices such as "all of the above", | ||
+ | |||
+ | == Concepts == | ||
+ | |||
+ | Consider | ||
+ | |||
+ | >> The thermo nuclear reactions in a stellar core are the result of (A) fission, (B) fusion. | ||
+ | |||
+ | This tests what? Unfortunately, | ||
+ | |||
+ | >> The Monotillation of Traxoline (attributed to Judy Lanier) It is very important that you learn about traxoline. Traxoline is a new form of zionter. It is monotilled in Ceristanna. The Ceristannians gristerlate large amounts of fevon and then bracter it to quasel traxoline. Traxoline may well be one of our most lukized snezlaus in the future because of our zionter lescelidge. | ||
+ | |||
+ | >> Directions: Answer the following questions in complete sentences. Be sure to use your best handwriting. (1.) What is traxoline? (2.) Where is traxoline monotilled? | ||
+ | |||
+ | Notice how easy it is to get 100% without understanding a single thing about the passage! Pay attention to wording. | ||
+ | |||
+ | == Analysis == | ||
+ | |||
+ | Less applicable for GSIs, but if particular questions are frequently missed, it is necessary to probe whether they are missed because of a lack of clarity, or because the question is testing difficult concepts. If the former, revise! | ||
+ | |||
+ | |||
+ | See below for some general bullet points on what to look out for. | ||
+ | |||
+ | **Some notes on free response questions: | ||
+ | |||
+ | These notes are written with "Astro C10 quizzes" | ||
+ | |||
+ | * These points are valid for both quizzes and exams: | ||
+ | * Test the material emphasized - Exams should reflect the fact that students should know the big concepts really well, as opposed to knowing a bunch of smaller concepts only peripherally. | ||
+ | * Keep questions short and to the point - Students should spend the majority of their test time thinking and answering/ | ||
+ | * Edit questions for clarity - Clear questions tend to be shorter. If anything is ambiguous, it confuses and slows down students and makes it harder for you to grade it fairly. | ||
+ | * Don't write a long test - Keep it concise, to the point, and clear! | ||
+ | * Quizzes vs. Homeworks | ||
+ | * They' | ||
+ | * The main difference is that quizzes should be shorter and have easier questions, since students have much less time to work on quizzes and must work on them alone. | ||
+ | * Quizzes are: | ||
+ | * short | ||
+ | * usually given in section | ||
+ | * questions are of exam difficulty (//i.e.// easier than homework questions) | ||
+ | * Not too in-depth or calculation-based. Some light calculation might be OK, but take care. (Not everyone will remember a calculator no matter how many times you remind them, and many people will be petrified of the idea of computing things on their own.) | ||
+ | * usually only cover recent material (//i.e.// the past 2-3 weeks) | ||
+ | * Quizzes are used: | ||
+ | * by the GSI to gauge each students' | ||
+ | * by the student to gauge their own understanding of recent material **and** get a feel for what a college level intro science course non-Scantron exam will look like and what level of understanding they are expected to have for the exams | ||
+ | * by the prof to get a grade early in the semester that's more important than a single homework, but isn't the big production that an exam is (you might not have covered enough material for a full exam) | ||
+ | * What makes good ones? | ||
+ | * not too long (both in length of individual questions and number of questions) | ||
+ | * not too hard | ||
+ | * not too easy (shoot for a variety of difficulties in questions) | ||
+ | * relevant to recent material | ||
+ | * varied in the //types// of questions (multiple choice, fill in the blank, calculation/ | ||
+ | * unambiguous with easy-to-read questions | ||
+ | * not mathematically demanding - questions should probably not require a calculator and should definitely not include extensive tedious calculations | ||
+ | * representative of same knowledge required for exams | ||
+ | * gradable for partial credit (not simply binary right/wrong like Scantron exams) | ||
+ | * specific about what you're looking for in free response type questions: Don't give students the opportunity to ' | ||
+ | * quick to grade (this makes your life much easier and helps the grading be more fair for all of your students) | ||
+ | * Quizzes are meant to be // | ||
+ | |||
+ | |||
+ | |||
+ | **General notes from previous years in convenient bullet-point form:** | ||
+ | |||
+ | * Exams exist to: | ||
+ | * Evaluate student learning for University-required grading | ||
+ | * Motivate students to study and understand the material | ||
+ | * Allow the instructor to evaluate his/her progress educating students about the material | ||
+ | * Provide feedback to students about their understanding and study habits and illustrate specific gaps in their understanding of the material | ||
+ | |||
+ | * What makes good multiple choice exam questions? | ||
+ | * Not too long/wordy (neither the question nor the possible answers) | ||
+ | * Not too much calculation | ||
+ | * Not //too// tricky (//i.e.// there shouldn' | ||
+ | * Relevant to important material (as opposed to really obscure/ | ||
+ | * **Very clearly written**, precise wording in both question and answers | ||
+ | * //E.g.//, 'Which //best// describes...' | ||
+ | * Pedagogical as well as evaluative (//e.g.//, some questions should probe common misconceptions) | ||
+ | * Questions that are very easy or very difficult are **OK** as long as the test has questions with a variety of difficulties | ||
+ | |||
+ | * What makes bad ones? | ||
+ | * Long answers! | ||
+ | * Excessive use of 'all of the above' (some people say **any** use of ' | ||
+ | * Questions that can be solved without knowledge of the material (usually because of the use of too many blatantly wrong or " | ||
+ | * Multiple potentially correct answers (usually from vague questions or possible answers) | ||
+ | |||
+ | * Non-multiple choice questions | ||
+ | * Types of questions: | ||
+ | * Matching | ||
+ | * Fill-in-the-table/ | ||
+ | * Simple calculations | ||
+ | * Short answers and paragraph/ | ||
+ | * Diagrams, plots, graphs | ||
+ | * Most of the same points discussed above apply here: | ||
+ | * Questions should be clear, easy to read, and unambiguous | ||
+ | * Questions should be relevant to the material presented and emphasized (do not test on obscure passages of the textbook) | ||
+ | * For high-value questions, allow for partial credit | ||
+ | * Make the questions easy to grade! | ||
+ | | ||
+ | |||
+ | |||
+ | =====Break (5 minutes)===== | ||
+ | |||
+ | |||
+ | =====Rubrics and Grading (30 min)===== | ||
+ | |||
+ | ====Activity==== | ||
+ | |||
+ | * Activity in groups of 3. Each group receives the same free response question. | ||
+ | * (5 minutes) Have each group develop a key and rubric for the question. | ||
+ | * (5 minutes) Give each group one student response to that question (three different responses, one for each group). Have each student grade the quiz based on that rubric individually. | ||
+ | * (10 minutes) Have students compare your grade with others in the group and discuss. | ||
+ | * (10 minutes) Discuss as a class and recap the main ideas of grading as a class: | ||
+ | * Reading some responses first is important | ||
+ | * How to handle erroneous info | ||
+ | * The most important part of grading: **Grade fairly and consistently for ALL students**. | ||
+ | * Try not to look at student names while grading anything | ||
+ | * Grade in blue or green (not red!) | ||
+ | * GRADE WHAT YOU TEACH AND TEACH WHAT YOU GRADE | ||
+ | |||
+ | ==== Notes from Past Years==== | ||
+ | |||
+ | === Free-Response Quizzes and Exams=== | ||
+ | * Most questions should have 1 and only 1 correct answer (matching, fill in the blank, put in order, //etc.//). | ||
+ | * Paragraph or few sentence responses or plotting can be uglier. | ||
+ | * Try to give partial credit where you can. **Always** give points for correct steps even if the final answer' | ||
+ | * Obviously if they screw up part (a) by a factor of 2, but carry that extra factor through parts (b) through (f) and get everything else right (while including the factor of 2), they should **only** lose points on part (a). Also, stress this fact to your students so they don't get frustrated if they can't do (a), but the rest are doable (maybe even tell them to make up an answer to use for later parts, or in the question say 'use 5km for the rest of this question if you don't get part (a)'). | ||
+ | * In longer answers, you should usually reward for correct information more than you punish for incorrect information. | ||
+ | * Hopefully on your quizzes and exams you stress to students that they must write clearly and explain their steps and logic clearly. | ||
+ | * Be suspicious: If you see similar, very wrong answers, flag the tests and compare their answers to other questions. Hopefully you can look out for cheating while the quiz/exam is actually going on, but you won't be able to see everything. | ||
+ | * Talk (probably through e-mail) to students in your section(s) who performed very poorly (grades of less than 40% or 50%). They may be too shy to ask for help even if they know they need it! | ||
+ | |||
+ | |||
+ | |||
+ | =====Homework===== | ||
+ | |||
+ | - Peer visits are assigned. Meet up for discussion with both the person who visited you and the person you visited by 9/20. Bring a completed {{: | ||
+ | - Draft a full length quiz and detailed grading rubric for the quiz. Bring TWO copies to class next week. |