The rationale behind this design project was to implement an online recording of a student’s level of understanding and knowledge of a topic prior to learning and post learning. The idea was to feed this information back into the students Individual Learning Plan (iLP) for Sixth Form students studying for A levels. From this we could design individual feedback and guidance on the next stage of progression, whether this being to move on, review learning or re-exam the topic area before moving on. It was based on the practice of using ‘Knowledge Surveys’ as outlined by On the Cutting Edge Professional Development Program for Geoscience Faculty.
In their methodology they use a paper based question and answer system which works well and is sufficient to complete the objects. However I envisaged a method that could combine a ‘Knowledge Survey’ with a little known plugin for the Moodle LMS called “Feedback”. Within this module was the ability to create “survey” like questions, with the added facility of being able to quantify the data being produced in text form, graphical or numerically and also get qualitative feedback from the students. On this basis the information gathered could give the educator an overview of all class understanding and knowledge or “confidence” level. So for example if there was a particular area causing concern or more importantly the ability to give individual feedback.
What is a knowledge survey?
“A standard Knowledge Surveys consists of many questions that cover the entire content of a course. Questions cover all levels of Bloom’s scale of thinking. (From low-level to high-level cognition, the scale goes from knowledge, comprehension, application, analysis, to synthesis.)” (On the Cutting Edge, 2007)
How do you use a Knowledge Survey?
Students at the start of the course or topic are given a series of question up to 200 in some cases; the students do not answer the question, but give a score of how confident they would be at answering the question. This provides a baseline information about their base knowledge preparation needs.
How do you create one?
Generally a tutor would refer to a bank of exam papers going back over several years. To ensure all parts of the topic/subject are covered a range of questions are sorted in to topics. To ensure all levels of thinking are assessed, the tutor scores the question from one to six based on Blooms Taxonomy.
A useful resource/website divides Blooms Taxonomy in to tabular form with exemplars of terminology and use which are outline in Figure 1 and Table 1 below. http://www.teachers.ash.org.au/researchskills/Dalton.htm
Higher Order Thinking
- Creating – designing, constructing, planning, producing, inventing, devising, making
- Evaluating – Checking, hypothesising, critiquing, Experimenting, judging, testing, Detecting, Monitoring
- Analysing – Comparing, organising, deconstructing, Attributing, outlining, finding, structuring, integrating
- Applying – Implementing, carrying out, using, executing
- Understanding – Interpreting, Summarising, inferring, paraphrasing, classifying, comparing, explaining, exemplifying
- Remembering – Recognising, listing, describing, identifying, retrieving, naming, locating, finding
Lower Order Thinking
Sample Knowledge Survey Questions.
|Blooms Taxonomy Level||Question|
|Blooms Taxonomy Level 1||What is the definition of a flood plain?|
|Blooms Taxonomy Level 2||Outline the basic characteristics of a meandering channel.|
|Blooms Taxonomy Level 3||Explain why the outer bend of a meander has faster stream flow.|
|Blooms Taxonomy Level 4||Compare the river regime of a temperate climate with that of an Alpine regime.|
|Blooms Taxonomy Level 5||Judge the success of a river management scheme using an example from the British Isles.|
|Blooms Taxonomy Level 6||Design and flood management scheme for your local river system assuming a 1 in 100 year flood event.|
Step by step procedure
Using Moodle Feedback with a Knowledge Survey
The Feedback module can be downloaded from http://moodle.org/mod/data/view.php?rid=95
Settings required to use feedback as a knowledge survey tool
- Give your Feedback a title.
- This text will be displayed to users before they enter the feedback. In this case the instructions associated with the survey.
- Set open and close feedback, this allows you to set a time period during which the Feedback will be available to responses.
- In the case of a Knowledge Survey you will need to know who is giving the answers so set record users name to “Users’ Names Will Be Logged and Shown With Answers”. Important in terms of knowledge survey is to “Show analysis to students” is set to yes. The summary results can be shown to respondents, or shown only to teachers.
- The other option available are depend on what you wish your students to do after completing the survey.
Creating the questions
- There are a number of options available but for the purpose of the “Knowledge Survey” we will use Multiple-Choice (Rated). This is similar to the multiple choice option, except that each option has a numerical value associated with it. From this it is possible to get an average grade if the multiple-choice (rated) option is used, numerical values are associated with each option, allowing an average or other measurements of responses.
- Save the question and then continue to add the next question as before.
- Pick the type of question, type in the question and (tip) copy and paste answer criteria form previous question.
- Once you have a bank of questions return to update and save changes.
- To check your question you will need to change role to student
- This is the screen the students will be presented with.
- Once the student has completed the survey they will be shown this screen, however you could set up one to direct the students to an alternative page.
Analysis of Results and Responses
- As a teacher again you have additional option to analysis or review student response, thus allowing you to gauge knowledge and review your curriculum.
- Within the Analysis tab you will be able to view a list of all responses submitted for each of the questions within a Feedback activity. These results can be exported to EXCEL. Analysis gives you the ability to look at student response and gives an overview for all of your students, link this with responses and you can look at individual students.
- Analysis of Results
- By clicking on show responses you can see an individuals response alone by clicking on that student, their response are revealed beneath.
Knowledge surveys maybe formative (at the start of a course) or summative (at the end of the course).
Why use knowledge Surveys?
They help students learn, help faculty improve their classrooms, and aid departments and programs as they explore new curricula or pedagogic content (Wirth and Perkins 2005).
In the power point presentation given by Perkins and Wirth they state the following reasons for supporting use of knowledge surveys
- Knowledge surveys are indispensable tools for instructors and for students.
- They aid instructors as they design courses.
- Allow for mid course corrections
- The surveys provide students with full disclosure of the course objectives and serve as study guides.
- Help students develop self-assessment skills
- Evaluate the effectiveness of alternative pedagogies.
Understanding What Our Geoscience Students Are Learning: Observing and Assessing
Applying Blooms Taxonomy