Posts Tagged ‘MSc’

Rationale

The rationale behind this design project was to implement an online recording of a student’s level of understanding and knowledge of a topic prior to learning and post learning. The idea was to feed this information back into the students Individual Learning Plan (iLP) for Sixth Form students studying for A levels. From this we could design individual feedback and guidance on the next stage of progression,  whether this being to move on, review learning or re-exam the topic area before moving on. It was based on the practice of using ‘Knowledge Surveys’  as outlined by On the Cutting Edge Professional Development Program for Geoscience Faculty.

In their methodology they use a paper based question and answer system which works well and is sufficient to complete the objects. However I envisaged a method that could combine a ‘Knowledge Survey’ with a little known plugin for the Moodle LMS called “Feedback”. Within this module was the ability to create “survey” like questions, with the added facility of being able to quantify the data being produced in text form, graphical or numerically and also get qualitative feedback from the students. On this basis the information gathered could give the educator an overview of all class understanding  and knowledge or “confidence” level.  So for example  if there was a particular area causing concern or more importantly the ability to give individual feedback.

What is a knowledge survey?

“A standard Knowledge Surveys consists of many questions that cover the entire content of a course. Questions cover all levels of Bloom’s scale of thinking. (From low-level to high-level cognition, the scale goes from knowledge, comprehension, application, analysis, to synthesis.)” (On the Cutting Edge, 2007)

How do you use a Knowledge Survey?

Students at the start of the course or topic are given a series of question up to 200 in some cases; the students do not answer the question, but give a score of how confident they would be at answering the question. This provides a baseline information about their base knowledge preparation needs.

How do you create one?

Generally a tutor would refer to a bank of exam papers going back over several years. To ensure all parts of the topic/subject are covered a range of questions are sorted in to topics. To ensure all levels of thinking are assessed, the tutor scores the question from one to six based on Blooms Taxonomy.

A useful resource/website divides Blooms Taxonomy in to tabular form with exemplars of terminology and use which are outline in Figure 1 and Table 1 below. http://www.teachers.ash.org.au/researchskills/Dalton.htm

 Blooms Taxonomy

Higher Order Thinking

  • Creatingdesigning, constructing, planning, producing, inventing, devising, making
  • EvaluatingChecking, hypothesising, critiquing, Experimenting, judging, testing, Detecting, Monitoring
  • AnalysingComparing, organising, deconstructing, Attributing, outlining, finding, structuring, integrating
  • ApplyingImplementing, carrying out, using, executing
  • UnderstandingInterpreting, Summarising, inferring, paraphrasing, classifying, comparing, explaining, exemplifying
  • RememberingRecognising, listing, describing, identifying, retrieving, naming, locating, finding

Lower Order Thinking

Figure 1

Sample Knowledge Survey  Questions.

Blooms Taxonomy Level Question
 Blooms Taxonomy Level 1  What is the definition of a flood plain?
Blooms Taxonomy Level 2 Outline the basic characteristics of a meandering channel.
 Blooms Taxonomy Level 3  Explain why the outer bend of a meander has faster stream flow.
Blooms Taxonomy Level 4 Compare the river regime of a temperate climate with that of an Alpine regime.
Blooms Taxonomy Level 5 Judge the success of a river management scheme using an example from the British Isles.
Blooms Taxonomy Level 6 Design and flood management scheme for your local river system assuming a 1 in 100 year  flood event.

Table 1

Step by step procedure

Design Project Screencast

Using Moodle Feedback with a Knowledge Survey

The Feedback module can be downloaded from http://moodle.org/mod/data/view.php?rid=95

Settings required to use feedback as a knowledge survey tool

  1. Give your Feedback a title.
  2. This text will be displayed to users before they enter the feedback. In this case the instructions associated with the survey.
  3. Set open and close feedback, this allows you to set a time period during which the Feedback will be available to responses.
  4. In the case of a Knowledge Survey you will need to know who is giving the answers so set record users name to  “Users’ Names Will Be Logged and Shown With Answers”. Important in terms of knowledge survey is to “Show analysis to students”  is set to yes. The summary results can be shown to respondents, or shown only to teachers.
  5. The other option available are depend on what you wish your students to do after completing the survey.

Creating the questions

  1. There are a number of options available but for the purpose of the “Knowledge Survey” we will use Multiple-Choice (Rated). This is similar to the multiple choice option, except that each option has a numerical value associated with it. From this it is possible to get an average grade if  the multiple-choice (rated) option is used, numerical values are associated with each option, allowing an average or other measurements of responses. 
  2. Save the question and then continue to add the next question as before.
  3. Pick the type of question, type in the question and (tip) copy and paste answer criteria form previous question.
  4. Once you have a  bank of questions return to update and save changes.
  5. To check your question you will need to change role to student
  6. This is the screen the students will be presented with.
  7. Once the student has completed the survey they will be shown this screen, however you could set up one to direct the students to an alternative page.

Analysis of Results and  Responses

  1. As a teacher again you have additional option to analysis or review student response, thus allowing you to gauge knowledge and review your curriculum.
  2. Within the Analysis tab you will be able to view a list of all responses submitted for each of the questions within a Feedback activity. These results can be exported to EXCEL.  Analysis gives you the ability to look at student response and gives an overview for all of your students, link this with responses and you can look at individual students.
  3. Analysis of Results

Showing Responses

  1. By clicking on show responses you can see an individuals response alone by clicking on that student, their response are revealed beneath.

Rationale

Knowledge surveys maybe formative (at the start of a course) or summative (at the end of the course).

Why use knowledge Surveys?

They help students learn, help faculty improve their classrooms, and aid departments and programs as they explore new curricula or pedagogic content  (Wirth and Perkins 2005).

In the power point presentation given by Perkins and Wirth they state the following reasons for supporting use of knowledge surveys

  • Knowledge surveys are indispensable tools for instructors and for students.
  • They aid instructors as they design courses.
  • Allow for mid course corrections
  • The surveys provide students with full disclosure of the course objectives and serve as study guides.
  • Help students develop self-assessment skills
  • Evaluate the effectiveness of alternative pedagogies.

Bibliography

http://serc.carleton.edu/files/NAGTWorkshops/assess/knowledgesurvey/small_perkins_p.pdf

Understanding What Our Geoscience Students Are Learning: Observing and Assessing

http://serc.carleton.edu/NAGTWorkshops/assess/knowledgesurvey.html

Applying Blooms Taxonomy

http://www.teachers.ash.org.au/researchskills/Dalton.htm

Moodle Documents

http://moodle.org/

Understanding more about the close relationship between assessment, feedback and effective learning is the first step towards assessment practices that empower rather than inhibit learning. Technology offers a new perspective through which this relationship can be explored.  Effective Assessment in a Digital Age (2010)

JISC opening statement cites the work of a number of educationists for the justified use of assessment because of the “relationship between learning, assessment and feedback” becoming “more widely understood over the last decade”. (JISC e-Learning Team, 2010)

Recognising that a learning program that encompasses the use of self monitoring and self regulation  is the way forward,  giving a greater depth and more effective learning and that the use of technology will play an important role in that. The latter being facilitated by the greater ownership of laptops, tablet and handhelds. (JISC e-Learning Team, 2010)

Justification for the enhancement of assessment and feedback was initiated by a study carried out by the National Student Survey  in which their finding stated “consistently lower levels of satisfaction with assessment and feedback than with other aspects of the higher education experience” (JISC e-Learning Team, 2010)

What is assessment?

The term ‘assessment’, (Brown, et al., 1997)comes from the Latin ‘ad sedere’, which means to sit down beside.  Assessment is formally defined as a measure of performance (Gagne, et al., 2005)

Assessment can be subdivided in to Formative and Summative. Summative is often used as a tool at the end of a course and used to grade a student and often referred to as the “assessment of learning”. Whilst formative is carried out throughout the course and is seen as a tool used to aid students and tutors by providing constructive feedback and often referred to as “assessment for learning”.

What is Effective Assessment

“Effective assessment and feedback can be defined as a practice that equips learners to study and perform to their best advantage in the complex disciplinary fields of their choice, and to progress with confidence and skill as lifelong learners, without adding to the assessment burden on academic staff. “ (JISC e-Learning Team, 2010)

Technology, although still under-utilised in assessment and feedback practices, offers considerable potential for the achievement of these aims (JISC e-Learning Team, 2010). However Computer aided learning (CAL) can help with effective assessment, take for example the use of multi-choice questions embedded within a LMS such as Moodle. Assessments can be taken, marked, feedback given and at the same time enhancing learning and reducing the work load of the tutor fulfilling the criteria outline above. It is effective in the fact that feedback given is immediate, direct and can quickly dispel misconceptions or common mistakes, therefore used in a formative sense.

Students’ answers can be analyses quickly by the tutor, thus enabling the tutor to identify intervention strategies or modify curriculum content at an early stage, thus improving course content and level of achievement.

Technology and examples of use.

Implementation of ‘Knowledge Surveys’ using Moodle (LMS) and Feedback Survey, across part of the Sixth Form in a Further Education College in the North West of the UK.

The rationale behind this project was to implement an online recording of a student’s level of understanding and knowledge of a topic prior to learning and post learning. From individual feedback, design and guidance on the next stage of progression could be formulated.  It was based on the practice of using ‘Knowledge Surveys’ both in a formative and summative mode but predominantly as a formative assessment tool.

(Wirth and Perkins 2005) stated their belief that they help students learn, help faculties improve their classrooms, and aid departments and programs as they explore new curricula or pedagogic content.

Conclusion

JISC summarised  their report by referring to the ESCAPE project carried out by the University of Hertfordshire, concluding with the following:-

Good assessment for learning…

  • Engages students with the assessment criteria
  • Supports personalised learning
  • Ensures feedback leads to improvement
  • Focuses on student development
  • Stimulates dialogue
  • Considers student and staff effort

A summary that I believe fits well in the implementation of an online aided Knowledge Survey.

References

Brown, G., Bull, J. & Pendlebury, M., 1997. Assessing Student Learning in Higher Education. London: Routledge.

Gagne, R. M., Wager, W. W., Golas, K. C. & Keller, J., 2005. Principles of Instructional Design.. 5th ed. Belmont , Ca: Thomson Wadsworth.

JISC e-Learning Team, 2010. Effective Assessment in a Digital Age, London: JISC.

Perkins, D. & Writh, K., 2005. Knowledge Surveys: Applications and Results. [Online]
Available at: http://serc.carleton.edu/files/NAGTWorkshops/assess/knowledgesurvey/small_perkins_p.pdf
[Accessed 26 November 2012].