evaluation

=Evaluation=

In the ILDF, Evaluation is the third step in the lifetime circle. In this phase the design uses "Evaluation has been characterized as a process used to judge the "worthwhileness" of something in order to make decisions (Baker cited in R.E.Clark, 2000, p.4)". This means that the evaluation process´s result is a basis for judging the online learning intervention. The importance of evaluation in design is highlighted by Tessmer, who stated "Evaluation is a crucial part of every design and development effort because it can determine the worth or value of the instruction or training as well as its strengths and weaknesses (Tessmer, 2003)"

The process of evaluation itself has to be aligned to the nature of the learning task. Dabbagh (Dabbagh, 2005, p.235) mentioned four steps for a systematic evaluation of a learning task.
 * 1) "Clearly determine the purpose, desired results and methods of evaluation."
 * 2) "Formatively evaluate the design and development prior to launching the online course."
 * 3) "Revise the online materials according to the results of the online evaluation."
 * 4) "Implement the online learning experience and evaluate the results according to the identified goals."

Dabbagh distinguishes between summative and formative evaluation, where the summative evaluation focuses on the impact of the online material and the formative evaluation is revising the process after implementing. In this work, we concentrate on the formative evaluation of the learning task by interviewing two students and one teacher.

In the case of constructivist and open learning, his evaluation is more complex, than in other learning environments. The learning outcomes are constructed and more complicated to measure. Traditional evaluation methods do not support the opened nature of online learning. Therefore, the evaluation process has to be aligned with the complexity of the task (Dabbagh, 2005, p.238). Jonassen recommends a design of the evaluation, which is in the same context as the learning task itself and the incorporation of multiple perspectives. This means, that the evaluation process should include more than one evaluator or more than one media for evaluating. This makes the evaluation process more complex.

Definition of purpose, desired results and methods of evaluation
The purpose of the evaluation is to measure the impact of the intervention to the learner and relating this impact to the learning outcomes. The student should have the feeling, that his abilities to take better photographic images are improving. The method of evaluating this is by interview two peers and the tutor, who observed the students. This helps to get another perspective of the learning process.

Formative evaluation.
The formative evaluation will be conducted by interviewing two peers. The peers can be reached. The interview questions for the student is as follows
 * Accessability to the LMS.
 * Useability of the LMS.
 * Useability of the course.
 * Useability of the specific task.
 * Understanding the learning goals and outcomes.
 * Understanding the process of the task.
 * Progress of their knowledge in terms of photographic composition.

Peer one interview
The peer had no problem to access the LMS. She had already some experience with the LMS Moodle and the video tutorial for registration helped her a lot. Having some pre experience, she had no troubles at all with the LMS. The course itself was clear and well instructed. She noted, that with a weak instruction, the learning task will not be as effective and students won´t be able to respond in a structured way. The peer was commenting on the learning goals, that these might be complicated for the student to understand. Inexperienced students might get confused by the terms "analysis" and "discussion". She says, that this task is enhancing her view on the features of what a photo can have, but it needs a lot more practice to measure a real enhancement of the knowledge in composition. The peer said, that the learning task should be embedded in a bigger context, like a whole course and be used it there as a tool throughout the course.

Peer two interview
This peer had no troubles in registering and enrolling in the course. He was experienced with IT, but not with the LMS Moodle. He also didn´t know freemoodle before. He found the LMS comfortable to use, but not entirely intuitive. He complained that the user interface was a little bit messy and there is a lot of scrolling. The instruction of the course is good; he was able to resolve the task without problems. The learning outcomes were clear for him. But as peer one mentioned, it can be a little bit confusing for inexperienced users. He meant, that not everybody agrees on the importance of image analysis and so the motivation was not as high as expected. The process of the task was well understood by the given instructions. He was highlighting that video could be used more often for for instruction in tasks like this. The real progress of knowledge in terms of photographic composition was impossible to measure in this short period of time. The peer mentioned, that he considers the learned composition features more when taking photos and in the post processing phase.

Revision of online materials
The material of the course is video and manuals. This material should be evaluated in terms of their usablility and alignment to the learning outcomes.

Implementation of the online experience
The experience of the learning goals are expected in the use of the appropriate tools and images to analize. It is important, to choose appropriate

Value of the task
Image discussion is much desired among the amateur photographers. In photographic forums, most users desire a critique of their images. But a constructive critique is focused on facts and not on emotions. Unfortunately, very often the discussion is done on emotion and not on facts. The aim of the image discussion is to help the student to discuss images critically and not on emotions. He or she should know, why he likes an image or why not. There is a high demand for image discussion in the amateur photography group. The main problem here is the lack of positive criticism, as in most cases a discussion is based on pure emotions rather than a professional and profound input on where and what to improve. The main objective of the image discussion is to show and emphasize on how to give a critical input and how to do that in a constructive way. So instead of stating that a picture is not good enough, give all the details on how to improve it can be considered a good photo.

Reflection
The idea of writing this e-Portfolio and designing the online image discussion came from two main experiences, that I experienced myself. Firstly, the weak image discussion on www.fotoschule.at. These had very little learning effects to me and also other students complained about the same issue. Secondly, the desire of many users for image discussion in online forums, which are not working properly, because of no precise instruction on how to discuss an image. But, like all online learning tasks, the focus must be on the pedagogy and not on the technology, which must follow the needs of the learners. Unfortunately, this is not always the case and even here, one of the major drivers of implementing the learning task at the institution is the use of technology. Therefore I decided to develop an instructed image discussion as a prototype learning task for the institution www.fotoschule.at. The biggest challenge was the uniqueness of this task. There are no experiences, which I could rely on during my implementation and the assumptions were based on different observations around the image discussion, but not one assumpion was taken by experience with online image discussion. I took some elements of other platforms, which are offering some kind of structured image analysis, like [|www.1x.com] or photographic forums, like www.fotocommunity.de. It was a difficult task to find suitable testers for the prototype and it took me a while to convince some students to collaborate with me and try the learning task. The online image discussion can be considered as a learning tool in a wider context. It can be used in almost every lesson, where images are involved and the instruction can be adapted very easily.