D4.1: Report on the pilot workshops and LD enactment

Transcription

METIS - Meeting teachers co-design needs by means of
Integrated Learning Environments
D4.1: Report on the pilot workshops
and LD enactment
WP4: Implementation
WP Leader: <ULEIC>
Authors: Rudman PD (ULEIC), Conole G (ULEIC)
Project Number: 531262-LLP-2012-ES-KA3-KA3MP
Project information
Project acronym:
METIS
Project title:
Meeting teachers' co-design needs by means of
Integrated Learning Environments
Project number:
531262-LLP-1-2012-1-ES-KA3-KA3MP
Sub-programme or KA:
KA3 Multilateral projects
Project website:
http://www.metis-project.org
Reporting period:
From
01/06/2013
To
28/02/2014
Report version:
3.2
Date of preparation:
02/06/2014
Beneficiary organisation:
University of Valladolid (UVa), Spain
Project coordinator:
Prof. Yannis Dimitriadis
Project coordinator organisation:
University of Valladolid (UVa), Spain
Project coordinator telephone number:
+34 983 423696
Project coordinator email address:
info@metis-project.org
WP Leader:
Gráinne Conole (ULEIC)
WP Leader email address:
gcc7@leicester.ac.uk
Document history
Date
Version Author(s)
Description
14/02/2014
V1.0
Rudman P., Conole G.
12/04/2014
V2.0
28/04/2014
20/05/2014
02/06/2014
V3.0
V3.1
V3.2
First draft for internal review (without
Enactment)
Second draft for internal review (with
Enactment)
Final draft for internal review
Ready for Steering Committee
Finished
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0
Unported License.
This project has been funded with support from the European Commission. This
publication reflects the views only of the author(s), and the Commission cannot be held
responsible for any use which may be made of the information contained therein.
Executive Summary
Despite the technology revolution in the field of education, there remains relatively little
technological support for educators in the area of Learning Design – the selection of
activities that learners will engage with and materials to be made available. The METIS
project seeks to provide support for these choices by providing educators with an online
design tool – an Integrated Learning Design Environment (ILDE) – and a training package in
Learning Design and the ILDE, in the form of face-to-face workshops. The overall aim of this
support is the widespread promotion of good practice in Learning Design.
Work Package 4 (WP4), led by ULEIC, is concerned with the practicalities of running these
training workshops – ensuring that all is in place for these to run successfully and that the
activities, actions and events are recorded to allow the recommendation of improvements in
the workshop design. In addition, a small proportion of workshop participants are
encouraged to volunteer to put into action their learning by creating a Learning Design and
trialling it with their students – a process referred to as “enactment”. Again, WP4 is
concerned with ensuring that all is in place for these to run smoothly, and that the process is
recorded to allow future improvements.
Three METIS partners ran workshops, representing the three educational sectors addressed
by METIS: adult education (Agora), vocational training (KEK Eurotraining) and university
education (Open University, UK). Each partner ran one workshop (some split over several
days) and encouraged some participants to continue with enactment.
All workshops ran successfully, with a total of 47 participants. The main recommendations
for workshops are to reconsider the times allowed for each activity; in particular, many
participants needed more time to assimilate some of the new concepts, and more time to
familiarise themselves with the new software (the ILDE). This varied across sectors,
according to participants’ experience and existing knowledge. Therefore timings should be
reconsidered separately for each workshop. A number of modifications are listed to help
workshops run more smoothly, such as allocating participants into groups prior to the
workshop and not asking participants to capture and upload images of paper documents
themselves.
Five people took part in four enactments (two people jointly), with one enactment
undertaken by a workshop non-participant. Two of the enactments were completed
successfully; one encountered problems using Moodle and the other time restraints. OU
participants were unable to enact due to their not working in a directly teaching role, but it
is expected they will facilitate enactment by others at a later date. Recommendations are to
recruit participants specifically for enactment, with supporting workshop – in addition to
recruiting for the workshops and encouraging enactment – and to be clearer about the
necessity for Moodle access and amount of time the process may take.
Acknowledgements
The authors would like to thank Yannis Dimitriadis and Francesca Pozzi for their invaluable
assistance in reviewing and commenting on each draft of this document.
We would also like to thank the three trialling partners – Open University, UK, KEK
Eurotraining, Greece, and Agora, Spain.
Finally, we would like to thank the workshop participants for allowing their work to be
included in METIS, especially those who went on to enact their designs.
Table of content
1.
Introduction ........................................................................................................................7
1.1 The METIS project.............................................................................................................7
1.2 Scope of METIS Work Package 4 ......................................................................................7
1.3 Scope and purpose of this document...............................................................................8
2.
Preparing to run the workshops .........................................................................................8
2.1 Workshop designs ............................................................................................................9
2.2 Workshop materials ...................................................................................................... 10
2.3 Integrated Learning Design Environment (ILDE) preparation ....................................... 10
2.4 Preparation checklists ................................................................................................... 11
3.
Workshop data acquisition.............................................................................................. 12
4.
Workshops ....................................................................................................................... 13
4.1 Workshop (OU-UK) ........................................................................................................ 13
4.1.1. Participants ............................................................................................................ 13
4.1.2. Workshop observations......................................................................................... 15
4.1.3. Learning Designs created ...................................................................................... 18
4.2 Workshop (KEK-Greece) ................................................................................................ 18
4.2.1. Participants ............................................................................................................ 18
4.2.4. Other observations ................................................................................................ 23
4.3 Workshop (Agora-Spain) ............................................................................................... 23
4.3.1. Participants ............................................................................................................ 23
5.
Preparation for enactment .............................................................................................. 28
6.
Enactment data acquisition ............................................................................................. 29
7.
Enactments ...................................................................................................................... 30
7.1 Enactment (OU-UK) ....................................................................................................... 30
7.1.1. Participants ............................................................................................................ 30
7.1.2. Schedule ................................................................................................................ 30
7.1.4. Delivery context..................................................................................................... 31
D4.1 Report on the pilot workshops and LD enactment
5
7.2 Enactment (KEK-Greece) ............................................................................................... 31
7.2.1. Participants ............................................................................................................ 32
7.2.2. Schedule ................................................................................................................ 32
7.3 Reflective diaries & Interview (KEK-Greece) ................................................................. 34
7.3.1. Were there any difficulties in completing the enactment? .................................. 34
7.3.2. Did enactors need to seek clarification or help with anything? ............................ 34
7.3.3. What were the advantages of undertaking the enactment? ................................ 35
7.3.4. Did it improve their teaching practice and if so in what ways? ............................ 35
7.3.5. How much time did they spend on it? .................................................................. 35
7.4 Enactment (Agora-Spain) .............................................................................................. 36
7.4.1. Participants ............................................................................................................ 36
7.4.2. Schedule ................................................................................................................ 36
7.5 Reflective diaries (Agora-Spain) .................................................................................... 37
7.5.1. Were there any difficulties in completing the enactment? .................................. 37
7.5.2. Did enactors need to seek clarification or help with anything? ............................ 37
7.5.3. What were the advantages of undertaking the enactment? ................................ 37
7.5.4. Did it improve their teaching practice and if so in what ways? ............................ 38
7.5.5. How much time did they spend on it? .................................................................. 38
8.
Discussion and Recommendations for the second round of Workshops (Cycle 3) ......... 38
9.
Recommendations for Cycle Three Enactment ............................................................... 43
10.
Conclusions................................................................................................................. 44
11.
References .................................................................................................................. 45
12.
Appendix A – ILDE content created during the workshops and for the enactments. 47
12.1 ILDE content created during the OU workshop .......................................................... 47
12.2 ILDE content created during the KEK workshop.......................................................... 47
12.3 ILDE content created for the KEK enactments ............................................................ 47
12.4 ILDE content created during the Agora workshop ...................................................... 47
12.5 ILDE content created for the Agora enactments ......................................................... 48
13.
Appendix B – Example checklist (KEK) ........................................................................ 49
D4.1 Report on the pilot workshops and LD enactment
6
METIS PROJECT
1. Introduction
1.1 The METIS project
The METIS project seeks to disseminate good practice in Learning Design (the design of
effective learning interventions (Conole, G. (2012))). The METIS description of work
describes three parts to achieving this goal (METIS Consortium, 2012):
1. “To design and develop an Integrated Learning Design Environment (ILDE). The ILDE
will be an integrated set of tools with which to design learning interventions. These
tools will consist of existing free and open source software, and will include: codesign support for communities of practitioners; learning design authoring tools
following different pedagogical approaches and authoring experiences; interface for
deployment of learning designs on mainstream Virtual Learning Environments
(VLEs).
2. To run a series of Learning Design workshops for teachers at partner institutions.
These workshops will have three main aims:
a. to foster interest in and adoption of learning design methods and innovative
pedagogical approaches among teachers
b. to advance teachers’ skills in the orchestration of ICT-based learning
environments
c. to support teachers’ familiarisation with the ILDE and to promote the usage
of the authoring tools integrated in it
3. To disseminate the project’s outcomes and maintain a community of teachers
engaged with learning design and its tools.”
METIS employs a user-centred design approach, with four distinct cycles:
1.
2.
3.
4.
Creation of workshop package based on existing tools
First delivery of workshops and enactment
Second delivery of workshops and enactment
Final workshop and ILDE package available
There is a detailed description and formative evaluation of the workshop and enactments
after cycle 2 and cycle 3; recommendations from these are used to modify and improve the
workshop package for the following cycle.
1.2 Scope of METIS Work Package 4
The purpose of Work Package 4 (WP4) in the METIS project is to action the second objective
listed above – to ensure the successful running of a series of Learning Design workshops and
enactments. These will apply and evaluate the products of Work Package 2 (Design and
development of the Integrated Learning Design Environment – ILDE) and Work Package 3
(Design and development of teacher training workshops), in authentic settings.
D4.1 Pilot workshops and enactments
7
METIS PROJECT
Typically, METIS workshops are held face-to-face, with a minimum of two facilitators and an
observer and between 10 and 20 participants. From each workshop, one or two participants
are followed by METIS as they enact (implement) one or more Learning Designs (created
either during or after the workshop, or both – see “Workshop design recommendation 8” in
Section 8).
1.3 Scope and purpose of this document
This document describes the first round of METIS workshops in terms of their organisation
and events. It does not seek to evaluate the workshops and enactments – evaluation will be
the subject of the forthcoming deliverable D5.2 (Pozzi F., Persico D., Sarti L., 2014). As an
overview, sections 2 to 4 of this document describe the planning, organisation and running
of the first round of METIS workshops (November-December 2013), while sections 5 to 7
describe participants’ subsequent enactments (January-April 2014). Recommendations are
then given for the second round of workshops.
The individual sections are now listed; specific content for this report, specified by the
Description of Work (METIS Consortium, 2012), is shown in italics. Section 2 describes
preparations for the workshops – their designs, the ILDE and materials required, and support
provided – and section 3, the data to be acquired during the workshops. Section 4 gives a
detailed description of each workshop – participants’ profiles, relevant events of each
workshop activity highlighting the deviations from the plan, and outputs.
Section 5 describes preparations for the enactments – the enactment process and guidelines
followed by partners – and section 6, the data to be acquired over the period of each
enactment. Section 7 gives a detailed description of each enactment – setting description,
including the teacher’s and students’ profiles, schedule and delivery (describing relevant
events). This section concludes with feedback from the teachers, consisting of the teacher’s
reflective diary and a final interview (described in detail in D5.2); these constitute a
discussion with the teacher about her assessment of the learning design, and a comparison
of its effects with the teacher’s expectations.
Recommendations are then given for the second round of Workshops (section 8) and
Enactments (section 9) (to run in Cycle Three of the project during the summer and autumn
of 2014). The final section (10) summarises conclusions drawn throughout the report.
In addition, Appendix A contains the learning designs produced by the participants, both
during the workshops and for the enactments.
2. Preparing to run the workshops
This section describes the project’s preparations for the three Learning Design workshops,
broken down as follows:
1. Workshop designs – activities participants will work on, based upon the generic
design, such as “Form groups”
8
METIS PROJECT
2. Workshop materials – objects necessary to support participants’ activities, such as
A4 paper
3. Integrated Learning Design Environment (ILDE) – the digital platform participants
use to support their activities
4. Preparation checklists – a comprehensive planning aid provided to trialling partners
2.1 Workshop designs
The individual workshop designs were based upon a generic “meta-design” created by the
METIS team. This generic design is detailed in deliverable D3.2 (Brasher, Walsh, McAndrew,
& Mor, 2013).
The generic design was then customised for the needs of each user group, to create three
individual workshop designs (for Agora, KEK and OU). While each partner could have
customised the generic workshop to their own topic, it was considered more practical, for
the first set of workshops, to customise all three using the same topic. Several topics were
considered, examples including collaborative learning, project based learning and formative
assessment. It was agreed that collaborative learning was a common topic amongst all three
trial partners; all three workshop designs therefore took the topic of collaborative learning.
To create the three individual designs, three working groups were established, each
comprising:






Team leader: Author; responsible for co-ordinating work within the team
Co-author: Assists the team leader
User group representative: Provide on-going feedback on design and
implementation’s suitability for their context of use and learning objectives
Critical Friend: Provide constructive feedback
Evaluation manager: WP5 representative – ensure suitable evaluation tools are
embedded in the design
Production manager: WP4 representative – ensure all implementation / production
issues are covered
The three completed designs are detailed in D3.3 (McAndrew, Brasher, Prieto, & Rudman,
2013). (See also example flowchart shown in Figure 1.)
Two of these designs (for Agora and KEK) were tested in practice at the METIS project
meeting in Barcelona (6-8 July 2013), and subsequently refined.
9
METIS PROJECT
Figure 1 – Example flow chart of a workshop design (KEK)
2.2 Workshop materials
The primary material for each workshop was a set of PowerPoint slides, based on those
from the generic workshop. (See example in Figure 2.) Workshop materials are also
described in detail in deliverable D3.3.
Figure 2 – Example opening slide from Agora workshop
2.3 Integrated Learning Design Environment (ILDE) preparation
10
METIS PROJECT
Three copies of the Integrated Learning Design Environment (ILDE) were set up by UPF for
each trial partner to have their own copy. Prior to the workshops, participants were sent
access details and encouraged to log into the relevant ILDE to familiarise themselves with
the software.
At the time of writing, this data is available to view; the Agora and KEK implementations are
open for self-registration, while the OU implementation requires access to be requested.
The web addresses are:



http://ilde.upf.edu/agora
http://ilde.upf.edu/kek
http://ilde.upf.edu/ou
2.4 Preparation checklists
A comprehensive checklist was created for each of the workshops to aid in its organisation
and running. Checklists were arranged by major and minor headings, as shown below.
Actions associated with each checklist item are described in the corresponding sections of
this report. (See also an example checklist (part) in Figure 3, and a complete checklist in
Appendix B.)







Participant recruitment
General workshop considerations
o Logistics
o Design
Customising the generic design
o Participants
o Pre-workshop activities
o Workshop activities
Delivering the workshop
o Media
o The environment
o During the workshop
Recording the workshop
o Before the workshop
o During the workshop
o After the workshop
Enactment
Evaluation
11
METIS PROJECT
Workshop activities:
Decide how many laptops / group are necessary
Define what media / resources are to be used for each activity
Define a backup method / media for each ILDE-related activity
Decide on the points where groups share designs / provide feedback
Include VLE integration and deployment?
Incorporate enactment planning into the workshop
End workshop with overview of the day
Update glossary from the workshop design and the ILDE
If necessary, define any activities for between workshop sessions
Figure 3 – Example of part of a checklist
Regular communications took place between representatives from WP4 and WP1, and the
three trial partners to monitor preparations for the workshops, offer assistance and circulate
ideas between the trial partners.
For example, one partner raised the question of how best to set up equipment so that one
person in a workshop group could work on the ILDE and other members of the group could
easily see their work. This was discussed and various solutions suggested (desktop PC, data
projector, external monitor) and the best solution agreed upon (regular laptop for the user
with an external flat screen monitor mirroring the screen, positioned for the group to see).
This solution was then used by all trial partners.
3. Workshop data acquisition
One person was designated “observer” for each workshop. The observer was provided with
a spreadsheet recording chart, containing:



one page for recording meta-information, such as date, number of facilitators and
participants, group names etc.
an observation sheet for each activity (see example Figure 4)
one page for additional notes, comments and reflections of the observer
For each activity, the start and end time was noted (elapsed time was calculated by the
spreadsheet) and any unanticipated success, difficulty, problem or event was recorded,
along with any comments the observer may have.
The recording charts worked well, with observers finding it suitable for all their observations
and thoughts.
12
METIS PROJECT
4e. Wrap-up (0.17h)
Start / End / Elapsed Time
12:52
13:15
00:23
Recommended length adjustment?
It took a little more time because there were a lot of
comments from the participants.
Resource availability (general)
n/a
Observed difficulties
none
Deviation from the plan
none
Unexpected successes
The participants request another workshop to consolidate
their knowledge.
Enablers and barriers (causes of
difficulties / deviations / successes)
n/a
Quality of interaction
Very good. The participants discuss about the experience.
Quality of outputs
Events (interruptions, person leaves,
tech failure, etc.)
Any other comments
n/a
none
Figure 4 – Example page from the Agora workshop observation sheet
Other data from the workshops used for this report are:



Items created by workshop participants in the ILDE
Photographs taken during the workshop
Photographs of the paper artefacts created
4. Workshops
4.1 Workshop (OU-UK)
The OU-UK pilot workshop was held on 24 October 2013 at the OU-UK. The workshop lasted
7.5 hours, with one hour for breaks and lunch.
4.1.1. Participants
80 people completed the online application to take part in a METIS workshop, from which 20
people accepted a place on the workshop, with 17 people actually attending the workshop.
Two participants left during the lunch break (reasons given relate to other commitments).
Workshop participants were employees of the OU-UK from course creation teams, all with
prior experience of various aspects of Learning Design, i.e. had been involved in Learning
design and enactment prior to the workshop. However, at the OU, each course is created by
a team of people who do not subsequently enact the course themselves (see Section 7.1).
During the workshop, participants worked in four groups, with group selection taking place
before the workshop. Groups contained a range of expertise, and were determined as far as
possible by the faculty the participants worked for:
13




METIS PROJECT
Science – 4 participants: 1 from Library services, 3 from LTS (Learning and
Teaching Solutions). LTS is the OU unit that runs the OU’s publishing process, to
produce teaching and learning material in a variety of print, online and other
electronic media. It includes teams of media specialists who work closely with
course teams, and developers who enhance and sustain the University’s virtual
learning environment
Business and law – 6 participants: one regional manager with Associate Lecturer
experience, one module chair, one academic, 2 LTS people and one academic
from Health and Social Care came to the workshop despite only being on the
waiting list for the event
Health and Social Care – 4 participants: One academic, 2 LTS people and one
from library services.
Mathematics, Computing and Technology – 3 participants: one academic, one
from LTS and a manager of learning and teaching.
Participants were assigned roles (printed on cards that were handed out), such as “Media
developer” or “Accessibility specialist” and asked to look at the ILDE with particular
emphasis on that role. The intention was to maximise constructive feedback on the
software.
Six facilitators were present for the course before lunch and five after. One facilitator sat
with and assisted each group.
•
•
•
•
Three of the facilitators were OU staff from the Institute of Educational
Technology’s Learning and Teaching Development Team (all 3 have extensive
learning design experience e.g. through involvement in the OULDI project),
One was an experienced workshop facilitator from the OU’s Faculty of Education
and Language Studies
one facilitator was a Research Fellow in educational technology with in-depth
knowledge of WebCollage
one facilitators was a Professor within the Institute of Educational Technology
with responsibility for Quality Enhancement (this facilitator left the workshop at
lunch time)
The observer researches Technology-enhanced Learning at the University of Leicester.
14
METIS PROJECT
Figure 5 - Photograph of OU workshop
4.1.2. Workshop observations
Figure 6 (below) is taken from the Observer’s spreadsheet. It shows the planned activities,
planned vs actual time spent per activity, and any significant notes made.
#
Detail
1a
Introduction
20
Two of the plugin screens for the ILDE were not working; they were fixed by a
technician before being required
The consent forms and signup sheet were not distributed
One participant arrived during the session
1b
How to ruin a course
20
13
Output-Arranged post-its, photographed & uploaded to the ILDE
Created successfully on paper; although the ppt asked participants to photograph their
paper and upload to the ILDE, they were not encouraged to do this, and nobody did so
1c
Barriers and challenges to collaborative learning
40
46
Output-Arranged post-its, photographed & uploaded to the ILDE
Created successfully on paper; participants had difficulty with the photograph/upload
task – see discussion.
This activity overran by 13%, and was stopped early; recommend that this activity be
modified or allocated more time
2a
Conceptualise: Learning outcomes
Plan
mins
40
Obsv
mins
23
42
15
METIS PROJECT
Output-Arranged post-its
Successful
Although the activity ran effectively, it was stopped before participants were ready in
order to stay on schedule. Recommend allowing more time for this activity.
2b
3a
Conceptualise: Evidence, examples and patterns of Collab. Learning
45
40
Output-Notes on potentially useful design features added to the ILDE
This was a 30 min talk (Case study) and 10 mins questions
Lunch
Conceptualise: Storyboard
45
51
Output-The teams’ storyboards, using OULDI post-its
Created successfully
The activity began with a 12 minute general discussion that was not indicated in the
workshop design; this may have been to make up for cutting some activities short
before lunch, but it caused this activity to overrun by 6 minutes.
The slide showing a sample use of Compendium post-its was understandable mainly by
the different coloured icons on the post-its. However, one participant asked for further
explanation, stating that they were colour-blind.
3b
Author: using WebCollage
45
73
Output-WebCollage prototype in the ILDE
All four groups created their entry in Web Collage, showing that they were able to
understand and use the software. However, only a proportion of the storyboards have
been transferred to the ILDE.
The activity overran by 47%. While the groups did not appear to have serious problems
that prevented them from working, it seems that the process of learning to use and to
apply the software brings numerous small challenges that together made for slower
progress than anticipated.
The activity began with two presentations.
20 minutes in: began a short description of what the participants should do, followed
by time working in groups..
59 minutes in: began a presentation by one of the groups and a discussion.
Participants particularly commented on difficulties in selecting an appropriate
template.
3c
Implement
80
16
Output-Runnable activity in Moodle
This activity was shortened from a hands-on activity to a demonstration – presumably
to return to the time schedule
At one point an error-style message appeared on the screen in Spanish, followed by a
long wait, during which it was not clear to the observer what was happening.
The transfer to Moodle demonstration did work ok, but it seemed to take longer than
expected.
The activity began 40 minutes behind schedule; shortening this activity (as per the
contingency plan) caused the workshop to be 24 minutes ahead of schedule, rather
than 40 minutes behind.
4a
Evaluate
45
40
16
METIS PROJECT
Output-Evaluation notes in the ILDE
In the ILDE, the number of items listed from the heuristic evaluation from two groups is
5 and 3, while the other two groups do not have entries.
Each group needed to access three pieces of data held electronically: 1) The design of
another team (in their ILDE), 2) Instructions for conducting heuristic evaluations (web
page), and 3) The heuristic evaluation form (on the ILDE). Although groups were asked
to use an iPad for the evaluation form (and iPads were supplied if necessary), all groups
exhibited confusion and difficulty in assembling the three pieces of electronic data for
easy simultaneous use. Groups resolved the issue in different ways, with facilitators
having to encourage the use of an iPad for at least one group. Overall, the distribution
and use of electronic resources for this activity needs further consideration.
The movement of participants between group tables could have been smoother.
Recommend clearer spoken instructions immediately prior to the movement.
4b
Wrap up
15
27
Output-Completed evaluation questionnaires
Completed successfully
It was not entirely clear when the activity finished, since participants left once they had
completed the evaluation form. Thus, the end time was different for each participant.
Overall, 15 minutes was not long enough for the number of questions and discussions
participants wished to have plus the evaluation forms. It would probably be better to
list them as separate events, Wrapup+questions 15 mins, eval forms 15 mins.
The plan did not allow time for questions plus 20 minutes for completing the
questionnaire.
Participants were enthusiastic about asking questions and offering positive suggestions
about the workshop content and running.
Figure 6 - Summary of Observer's notes from OU workshop
In addition, the observer noted the following:
1. Overall, the workshop ran well.
2. There were 17 people split into 4 groups, and 5 facilitators (plus a photographer who
came in from time-to-time, and myself [the observer]). Two participants left at
lunchtime. (I'm told they had 20 down to attend, out of 80 applicants).
3. In conjunction with noise from the air conditioning, the observer had to concentrate
hard to follow some of the presenters. Recommend considering amplification for the
presenters.
4. All participants were employees of the OU with experience in one or other aspects
of creating courses (pedagogy, design, writing, IT, etc.). While the workshop went
smoothly, it's not possible to say to what extent, if at all, that expertise was
necessary for a successful workshop.
5. Each group had a facilitator, and for the most part the facilitator led their group.
Therefore, it is not easy to identify difficult areas, or to assess how the groups would
have fared on their own, especially if they did not have the expertise described
above.
17
METIS PROJECT
6. While participants seemed to be able to do the exercises well, it's not clear how
effectively they learned the necessary skills and whether a longer period of practice
is necessary.
7. Very little was spoken or discussed about enactment.
4.1.3. Learning Designs created
The OU ILDE contains 19 documents worked on during the workshop:
2 x Factors & Concerns
4 x Heuristic Evaluation
7 x Web Collage
3 x Image
3 x Unclassified documents1
Actual content in the OU ILDE, created during the workshop, is shown in Appendix A of this
report, while the ILDE itself is accessible at http://ilde.upf.edu/ou
4.2 Workshop (KEK-Greece)
The KEK pilot workshop was held on 9th November 2013 at KEK’s offices. The workshop
lasted 6.5 hours, plus one hour for breaks and lunch.
4.2.1. Participants
25 people accepted a place on the workshop, with 18 people actually attending the
workshop. Four of the participants worked for KEK, and many participants had a background
in vocational training, e-learning and Moodle.
Participants were offered a Certificate of Completion for attending the workshop.
Participants worked in four groups, with group selection taking place before the workshop,
determined as far as possible by the faculty the participants work for:
•
•
•
•
Primary School & ICT Teacher Trainers: 4 participants
Secondary School & Geologist Teacher Trainers: 5 participants
Physicists Teacher Trainers: 4 participants
Biologists-Chemists Teacher Trainers: 5 participants
1
While most documents in the ILDE are shown under a category (e.g. “Heuristic Evaluation”), a few
documents do not. Although it may be possible to surmise in which category they belong, the data
reported here relate to that held on the ILDE. This will be considered as part of “ILDE design
recommendation 2” (see Section 8).
18
METIS PROJECT
Three facilitators were present for the course, supporting the four groups as required. Of
these, one was an Assistant Professor and one researches Computer Supported Learning,
both at the University of Piraeus.
The observer was an eLearning manager at the University of Piraeus.
Figure 7 - Photograph of KEK workshop
planned vs actual time spent per activity, and any significant notes made.
#
Detail
Plan
mins
Obsv
mins
1a
Welcome; Introduction to METIS and workshop goals; completion of
consent form
Output-completed consent forms / Signup sheet
Consent forms and signup sheets completed
Two participants arrived during the session
Each participant made a short self-presentation
15
27
1b
Introduction to Learning Design
10
29
Participants were confused about terminology which connected with LD. Questions
raised about: (a) the differences of terms Educational Design, Instructional Design and
Learning design, (b) the differences about learning models, learning strategies and
19
METIS PROJECT
learning patterns and (c) the differences about learning design, lesson plan and
learning scenario
Many questions were asked
1c
Introduction to Collaborative Learning
15
16
It may be possible to allocate less time to this activity, because there is another activity
for this reason (Understanding pedagogical patterns for collaborative learning).
Questions for examples about cooperation and collaboration, in order to clarify the
differences
1d
Orientation within ILDE; individual registration
20
21
Output-registered in ILDE
It may be possible to allocate less time to this activity, because participants had already
registered in ILDE.
One participant asked if facilitators could provide her the username and password,
because she had forgotten them and she hadn't received email.
Facilitators asked from participants to make groups within the ILDE, but there was a
problem in the ILDE and they could not create/define the groups so they just shared
everything with each user of their group.
1e
Form groups
5
4
It may be possible to allocate less time to this activity, because there were a first
categorization of participants
Because of non-attendance of some participants we had to rearrange the groups
1f
15
29
Participants took pictures with the camera, which provide the facilitator, and one
facilitator send them by e-mail in order to upload them to ILDE.
2a
Introduction to session
2b
Define course context and learning objectives
25
25
Output-LOs in ILDE
[Unexpected success} Because of the common background participants was able to
define course context and learning objectives easier
It may be possible to allocate less time to this activity, because the activity seemed to
be adequately complete
Facilitators shared with a member of each group, a document within ILDE (for other
conceptualization) in order to use it instead write on paper
Lunch
Learners’ personas activity
20
13
Output-Personas in ILDE
This activity was shortened, presumably to return to the time schedule
Facilitators asked from participants to complete the persona card after the workshop
2c
2d
Think about barriers and challenges
Output-spoken discussion; one list in ILDE
5
25
9
12
20
METIS PROJECT
There was only a spoken discussion between facilitators and participants
This activity was shortened presumably to return to the time schedule
3a
Introduction to session (0.08h)
5
7
Participants asked from facilitators if it is possible to decrease the duration of the
workshop
3b
Understanding pedagogical patterns for collaborative learning
50
36
Output-Decision on pattern to use
Unexpected success: Participants provide examples of collaborative patterns from their
work.
It was given the opportunity to each group to present their work to all participants
One participant mistook the brainstorming pattern with problem solving
3c
Course Map
30
6
Output-course map in ILDE
This activity bypass presumably to return to the time schedule
Participants complained about so many approaches/tools for the same reason. For this
reason and in order to remain on timing plan facilitators only presented the course
map.
3d
Story Board
30
24
Output-story board in ILDE
Participants saw all the time the colour matching and they confused with A) the
differences of resource and tool and B) Learner's work and Learning outcome
Facilitators provided colourful post-its and crayons instead of OULDI post-its
4a
Introduction to Authoring (WebCollage)
15
7
It may be possible to allocate less time to this activity, because participants wanted to
see practical things in order to understand
The facilitators present the philosophy, possibilities and functions of WebCollage
4b
Authoring with WebCollage
50
23
Output-design in ILDE
Participants used WebCollage only in order to check their functionalities
Resource availability was good, but participants underlined that they wanted more
explanatory material
This activity comprised an extended demonstration of an example of WebCollage.
Participants asked questions about their misunderstandings when they try to use
functionalities of ILDE. Facilitators committed that they will sent extra material (aka
tutorial) for WebCollage.
21
4c
METIS PROJECT
Demonstration of Moodle Implementation (Glue!-PS)
20
9
It may be possible to allocate less time to this activity, because participants wanted to
see practical things in order to understand
This activity consisted a step by step demonstration on how participants can transfer
their WebCollage LDs to Moodle through Glue!-PS.
Three groups had completed WebCollage and tried to deploy in Moodle, with one
succeeding (the other two encountered technical difficulties and by the time they were
resolved motivation was lost (they had already received the certificate)).
4d
Enactment discussion
Output-enactment plan in ILDE
No enactment plans were created
15
8
4e
Wrap-up
10
9
The facilitators presented the participants' obligations (aka deliverables) in order to
receive attendance certificate. In addition, facilitators gave a presentation on how they
will support them in this procedure.
4f
Workshop Evaluation
20
16
Output-evaluation forms
Participants underlined the fact that they should use more ILDE and its tools in order to
evaluate them.
Figure 8 - Summary of Observer's notes from the KEK workshop
1. Overall, the workshop ran well.
2. 18 participants (from the 25 of the attendance list) in 4 groups with 3 facilitators.
Two participants left before the end of the workshop.
3. It was not easy for the observer to follow all the activities fully, as this would have
necessitated following all of a group's discussions or ILDE activities.
4. Most of the participants were active teachers in primary and secondary school and
also work as teacher trainers (some of them were employees of KEK), so had the
necessary experience creating vocational training courses.
5. Facilitators helped groups in order to resolve technical issues or better understand
the activities. Facilitators allowed groups to take the initiative, guiding them only
when needed or when groups asked for support.
The KEK ILDE contains 7 documents worked on during the workshop:
22
METIS PROJECT
2 x Web Collage
5 x Unclassified documents
Actual content in the KEK ILDE, created during the workshop, is shown in Appendix A of this
report, while the ILDE itself is accessible at http://ilde.upf.edu/kek
It is of note that during the workshop one participant did not use a Learning Design idea
from her own teaching in order not to share her ideas with the group (see Section 4.2.3).
Another participant deleted her Learning Designs from the ILDE at the end of the workshop,
again to retain privacy of the designs.
4.2.4. Other observations
As mentioned in the Observer’s Notes (1d), there was a problem in the ILDE and participants
could not create / define their groups, so they just shared everything with each user of their
group instead.
It was later observed that uploaded images took a long time to load in the ILDE. This was
partially ameliorated by emailing the images to participants in addition to uploading to the
ILDE.
4.3 Workshop (Agora-Spain)
The Agora workshop was split over three separate sessions on consecutive Saturdays – 16th,
23rd and 30th of November 2013, each from 10am to 1.30pm.
4.3.1. Participants
Agora was conceived as an “open space” for learning, where anyone can improve their basic
skills (such as in English) and improve their employment prospects or apply for University.
The organisation uses a public building in a less advantaged part of Barcelona. Many of the
teachers are volunteers, often retired, and learners could.
METIS participants were a mixture of Agora teachers and collaborators with a range of
teaching experience, from beginners to experienced teachers, and learners from different
Agora courses. Some participants had low levels of ICT and Learning Design skills. None of
the participants were familiar with the concepts of formal learning design. For these reasons
it was decided to design the workshop as longer than one day, allowing more time for
participants to understand the workshop contents.
Therefore the Agora pilot workshop was split into three separate sessions on consecutive
Saturday mornings. They were held on 16, 23 and 30 November 2013 at Agora’s training
centre. Each workshop lasted four hours (12 hours total participation).
Many of the participants rely on public transport. However, following the general strike of 14
November, there was significant public transport disruption on the day of the first workshop
(16 November), meaning that 12 people attended on that day instead of the expected 17.
23
METIS PROJECT
Over the three sessions, 14 different people attended, as shown in Figure 9 (some did not
attend all three sessions).
Session
16 Nov
23 Nov
30 Nov
Participants
12
13
12
Facilitators
3
4
3
Figure 9 - Attendee numbers at the Agora workshop session
Participants worked in three groups, with groups created early in the first workshop session.
Groups were based on a pre-created list of participants’ experience in order to combine
participants from different roles.
One of the facilitators manages the online courses at Agora, one researches Learning Design
at the Universitat Pompeu Fabra.
The observer researches Technology-enhanced Learning at the Universitat Pompeu Fabra.
Figure 10 - Photograph of Agora workshop
planned vs actual time spent per activity, and any significant notes made. Note that this
workshop is split into three separate sessions.
24
METIS PROJECT
Plan Obsv
mins mins
#
Detail
1a
Welcome; Introduction to METIS and workshop goals; completion of
consent form
Output-completed consent forms / Signup sheet
Consent forms and signup sheets completed
One participant was absent for some minutes
15
18
1b
Introduction to Learning Design
Facilitator explained some concepts about Learning design
10
16
1c
Introduction to Collaborative Learning
15
54
The group activity needs more time
People focussed on discussing on the workflow of a classroom session
Participants are grouped into three groups of three or four. Each group had access to a
facilitator, including one group that moved to another room
Groups were asked to write three advantages and three disadvantages of
learning/collaborative design. There are three groups, [a facilitator] also moved with
them
One participant gave the view that to participate in the METIS workshop you need to
be fluent in information technologies
Participants clarified the concepts learned so far; each group listed on paper the
different advantages/disadvantages
1d
Orientation within ILDE; individual registration
Output-registered in ILDE
Success
One participant not remembering their email address or password
Very slow network – unable to connect with some email providers
20
21
1e
Form groups
Skipped
5
0
1f
15
55
It can be shorter than planned
The groups interacted fluently with the facilitators. Each group explained their findings
Facilitators explained the Tps, Jigsaw and Brainstorming patterns. Participants returned
to their groups to perform brainstorming about a concrete B39 scenario
2a
Introduction to session
5
8
There was a new participant this week
The sessions started late because the keyboard on the projector's computer didn't
work
Everyone resumed the groups from the last session
2b
Define course context and learning objectives
Session 1 – 16-November
25
63
25
METIS PROJECT
Output-LOs in ILDE
Participants filled the posters and at the end explained their conceptualization
Groups spent 43 minutes conceptualizing on the given scenario, after which each group
explained their findings
Significant interaction was observed between facilitators and groups
A new completely participant arrived at the workshop during this activity
2c
Learners’ personas activity
Output-Personas in ILDE
Skipped
20
0
2d
Think about barriers and challenges
Output-spoken discussion; one list in ILDE
Skipped
25
0
3a
Introduction to session (0.08h)
Skipped
5
0
3b
Understanding pedagogical patterns for collaborative learning
50
93
Output-Decision on pattern to use
Participants filled the patterns with the content they previously discussed
The first 13 minutes were spent getting into groups
It is required to translate the template into Spanish
There were problems accessing the ILDE:
 no clear information if a user account is required to have access
 template in English; participants ask for clarifications with regards to English
terms
 questions with regards to "A,B,C" steps (the "HOW" in the table)
 error clicking "guardar" in one of the groups
 clarification is needed to differentiate if it is required to copy participants one
by one or it is a profile for the whole group
 problem to complete the questionnaire as it is required UPF account
Interest from participants to differentiate between public and private information
A new completely participant arrived at the workshop during this activity
One of the attendees left leave the workshop at 13h
3c
Course Map
Output-course map in ILDE
Skipped
30
0
3d
Story Board
Output-story board in ILDE
Skipped
30
0
4a
Introduction to Authoring (WebCollage)
15
Third session, 10 participants.
The participants need more time to understand all the concepts involved in the
67
26
METIS PROJECT
WebCollage tool; it was a steep learning curve
Some groups didn't have a mouse (which is useful for WebCollage), the ones available
were PS/2 and the ports were USB
There were concerns from the participants about how the pupils will see their output;
these were addressed by the facilitators
Participants thought that WebCollage must be simplified, even if it is targeted only to
instructors. (They differentiate teachers from instructors (the latter don't have a
teaching degree). e.g. they thought that terms like "implement" are too hard, they
prefer the term "convert"
Participants registered their VLE
4b
Authoring with WebCollage
50
66
Output-design in ILDE
Participants created their design according with the patterns used in the
conceptualizations
The length was correct
Although some participants thought the tool was too complex, they answered that as
with every new tool it takes some time to learn it; however, the initial learning curve is
too steep
At the end of the event even the most sceptic participant agreed that the tool as a
whole was useful to them and they were very enthusiastic about it
In the menu arrows, the users instead of single clicking perform a double click. They'd
like more screen space. A participant thinks that this tool can be useful to justify the
hours of training of a class. Implement at 12:17 with WebCollage, [a facilitator]
explains how to implement the design with WebCollage. A participant tries to follow [a
facilitator], at the end the one group that need help is assisted by [two facilitators]. In
the implementation, the Jigsaw pattern doesn't assign correctly the pupils after being
assignar in each group(duplicates alummni and other issues).
4c
Demonstration of Moodle Implementation (Glue!-PS)
20
10
The output in moodle was correct as expected
It was short because we only deployed a course without any further editing.
A new participant arrived at the workshop during this activity
The browser showed a warning about the long time the Glue!-PS scripts were taking to
complete; however the deployment completed successfully
4d
Enactment discussion
Output-enactment plan in ILDE
Skipped
4e
Wrap-up
10
23
It took a little more time because there were a lot of comments from the participants
The participants requested another workshop to consolidate their knowledge
Interaction was very good; the participants discussed their experience.
4f
Workshop Evaluation
Output-evaluation forms
15
20
0
14
27
METIS PROJECT
The participants went to another room where are computers ready to fill the forms
Figure 11 - Summary of Observer's notes from the Agora workshop


People also learned English vocabulary
1x sheets pertain to session 1, 2x and 3x pertain to session 2 and 4x pertain to
session 3
The Agora ILDE contains 15 documents worked on during the workshop:
4 x Design Narrative
4 x Persona Cards
7 x Web Collage
Actual content in the Agora ILDE, created during the workshop, is shown in Appendix A of
this report, while the ILDE itself is accessible at http://ilde.upf.edu/agora
5. Preparation for enactment
At the beginning of the enactment, a named contact was defined as the contact for
participants at each of the three sites to provide support and guidance as needed during the
enactment.
In addition to discussing the enactment at regular project Flashmeetings, WP4
representatives organised Skype meetings with each of the three trial partners to discuss
their part in enactment and finalise timescales. (These were held with AGORA and KEK in
January 2014. OU Enactment has a longer timescale and is described in Section 7.1.)
There are four elements to enactment:
1. Complete the Learning Design in ILDE based on the documents created during the
METIS workshop – giving participants the opportunity to improve and expand on the
materials produced during the workshop. Materials include: the Course Features,
the Course Map, the Activity Profile and the overarching Storyboard. Participants
check that each of these is consistent with each other. Finally, complete the Learning
Design in WebCollage.
2. Implement (also referred to as “Deploy”) the Learning Design in their VLE. This
encompasses all tasks from finishing the ILDE content to students use of the
materials). Participants prepare all the necessary learning materials - i.e. the
resources and tools associated with the Learning Design.
3. Deliver the Learning Design with students.
28
METIS PROJECT
4. Evaluate the effectiveness of the Learning Design with students - activities, quality of
performance, learning outcomes. In addition, participants are asked to keep records
and retain any digital artefacts the students produce for the formal METIS evaluation
(to be published in D5.2).
A set of guidelines were created to provide guidance for the METIS trial partners:
1. Remind enacting participants of the METIS project, by providing a clear and concise
overview of METIS and the purpose of the evaluation
2. Indicate when the enactment will start and finish (i.e. between January and the end
of February), along with an indication of the amount of time involved (we anticipate
that participants will spend around ten hours over the two months on the
implementation, delivery will be additional, as part of participants normal teaching
workload)
3. Participants are asked to keep an electronic reflective diary of their activities (see
Section 6 of this document)
4. Aim to contact participants once a week (e.g. Face-to-face / via email) to get an
update on progress and their reflective diary updates.
5. Undertake a telephone, Skype or Face-to-face interview at the end of the enactment
(details of this will be published in D5.2).
6. Ask participants to provide digital copies of any artefacts they create
6. Enactment data acquisition
The following data were collected from workshop participants whose enactment was
followed up by the METIS team. These data inform both this document and the subsequent
evaluation document (D5.2).
1. Reflective diaries
The reflective diaries contain a brief description of the enactment work as it
progressed, along the person’s thoughts and perceptions, giving particular
consideration to the following questions:
a. Were there any difficulties in completing the enactment?
b. Did enactors need to seek clarification or help with anything?
c. What were the advantages of undertaking the enactment?
d. Did it improve their teaching practice and if so in what ways?
e. How much time did they spend on it?
Enactors either noted this information themselves, or discussed this with the
facilitator who noted the diary information. It was suggested that enactors spend
around four hours over the two months completing this reflective diary.
2. ILDE documents created (the ILDE documents are shown in Appendix A)
3. Post-enactment interviews (interviews are described in detail in D5.1, since their
primary purpose is one of evaluating the enactments).
29
METIS PROJECT
7. Enactments
7.1 Enactment (OU-UK)
At the OU-UK all undergraduates are distance learners. A course may have thousands of
students studying it at one time (Open University, 2014). It is therefore necessary to employ
a well-controlled process of course creation and delivery to ensure that the process runs
smoothly.
As part of this, there is a clearly defined separation between course creation and course
delivery. Courses are created by a team of people, comprising subject specialists, media
developers, accessibility specialists, administrators and other specialists. The team creates a
variety of interactive learning resources that comprise the course.
Associate lecturers are employed for course delivery; each associate lecturer is responsible
for supporting 15 students. Thus, there will be tens, if not hundreds of Associate Lecturers
engaged at once in supporting one course.
Thus, each course is created by a team of people, who do not subsequently enact the course
themselves. Instead, enactors are team leaders who will train their teams in METIS
principles; their teams will then use the ILDE to create a Learning Design and trial the design
as part of a large online course, as described below.
7.1.1. Participants
Two workshop participants have agreed to follow up on the workshop by creating Learning
Designs that may be enacted. Both participants are team leaders, and so are in a position to
organise their team into using METIS processes and the ILDE.
Health & Social Care team
One of the workshop participants is the Module Chair for a new 'Perspectives in health and
social care' module that is currently under development within the faculty of Health and
Social Care. (In OU module development, the role of the Module Chair is to act as chairperson of the team that will design a module and create its materials).
Business & Law team
One of the workshop participants is a Regional Manager from the Faculty of Business and
Law. (OU Regional Managers have the responsibility for managing the Associate Lecturers
who actually teach the OU modules.)
7.1.2. Schedule
Following the OU workshop, the Module Chair of 'Perspectives in health and social care'
invited the OU METIS team to support the consideration of, and if deemed appropriate,
design and development of collaborative activities within the module. In February 2014 the
OU METIS team ran a 90 minute workshop for 3 academics from the module team. This new
30
METIS PROJECT
module will consist of 3 blocks, each on a different theme, and each of the 3 academics who
attended was the lead author of one block. In the first 20 minutes of this workshop, the
METIS project and the METIS tools relevant to design of collaborative learning were
described by the facilitator, who was then briefed the three participants about the current
status of the module. During the remaining the 70 minutes, two workshop activities drawn
from the METIS workshop structure were run: 'How to ruin a collaborative learning activity'
and 'Conceptualise learning outcomes' (see D3.3, McAndrew et al., 2013).
The module team will undertake the remainder of the design and implementation using the
OU’s regular methodology and tools (this is described further in deliverable D5.2).
Business & Law team
The Regional Manager from the Faculty of Business and Law who participated in the OU
workshop is planning the same professional training for Associate Lecturers, to be delivered
at a face-to-face event in June as well as online. A half-day METIS workshop for the Business
and Law team developing this event and the online material was scheduled for 1st May
2014. However, just prior to completion of this report, this workshop was postponed
indefinitely as the Regional Manager is now unable to be involved in the development of the
face-to-face event, and there is no one else available to take reponsibility for the
combination of face-to-face and online development.
At the time of writing, no documents are reported as being created in the OU ILDE
specifically for enactment. The OU ILDE is accessible at http://ilde.upf.edu/ou
7.1.4. Delivery context
While these course teams will create Learning Designs using the METIS approach, enactment
will be delivered to students by Associate Tutors (as described above).
Approximately 800 students are expected to take part. Since 800 students requires some 53
Associate Tutors, courses have to use existing tools for delivery – new tools would require a
large training all 53 Associate Lecturers (spread out around the UK). To this end, forums are
likely to be the main tool used.
Business & Law team
It is unlikely that the Business and Law team will now be able to participate in METIS
enactment at present, but it may be possible around the time of the second round of METIS
enactments.
7.2 Enactment (KEK-Greece)
31
METIS PROJECT
7.2.1. Participants
All workshop participants were encouraged to take part in enactment. One participant
decided to do so as well as one teacher who had not taken part in the workshop.
Enactor #1
Enactor #1 is a training and coaching mentor in adult education at KEK. She had not
participated in the ILDE workshop, but wanted to participate in the enactment. She was
briefed on METIS and given a short course on the ILDE environment prior to the enactment
phase by the facilitator. Thus, no material was available beforehand and everything was
created from scratch.
Enactor #2
Enactor #2 is a primary school teacher. She had participated in the first (KEK) pilot METIS
workshop where her team created 2 Learning Designs (1 Persona Card and 1 WebCollage).
At the workshop Enactor #2’s team had completed successfully these LDs but they were not
able to deploy their WebCollage LD due to a technical problem with the WebCollage/ILDE
(see Section 7.5.1).
Enactor #2 had not wished to share her Learning Designs with the group, so she created a
different design during the workshop; for the Enactment she created a new learning
scenario from scratch with the “For other conceptualisations” tool for conceptualising and
“WebCollage” tool for authoring.
7.2.2. Schedule
A facilitator met with each enactor once a week to discuss progress and provide assistance
and guidance. The proposed schedule (based on the given guidelines described in Section 5)
and suggested durations for each task are shown below.
Enactor #1
1. Complete the Learning Design in ILDE (8 hours total), comprising:
a. introduction to the environment (3 hours one-to-one and 1 hour homework
on creating the first LD)
b. creating a scenario based on the educational needs and using the design
narrative (2 hours)
c. authoring the LD in WebCollage (2 hours)
2. Implementing/deploying the LD in Moodle (2 hours)
3. Deliver the LD to students (2 hours x 2)
4. Evaluate the effectiveness of the LD (2 hours interview; 2 hours discussion)
Enactor #2
1. Complete the Learning Design in ILDE
2. ILDE/WebCollage recap (1 hour)
3. Conceptual Design (2 hours)
4. Resources and tools (4 hours)
32
5.
6.
5.
7.
METIS PROJECT
Authoring the LD in WebCollage (3 hours)
Implementing/deploying the LD in Moodle (1 hour)
Deliver the LD to students (2 hours x 2)
Evaluate the effectiveness of the LD (1 hour interview/discussion)
The KEK ILDE contains 2 documents created in relation to the KEK enactment:
1 x Web Collage
1 x Unclassified documents
Actual content created in the KEK ILDE in relation to enactment is shown in Appendix A of
this report, while the ILDE itself is accessible at http://ilde.upf.edu/kek
Enactor #1
The delivery of the Learning Designs took place on 19th and 21st February 2014. The class
consisted of 15 students who were high school graduates that had not continued their
education and are currently employed. The topic was “Introduction to Marketing Principles”
– a vocational course, to help participants in their work.
Two LDs were created in the ILDE, a conceptualization LD using a template from design
narratives and a WebCollage LD based on the narrative. The web collage LD consisted of two
presentations, one brainstorming activity and one simulation activity and were delivered in
the classroom over two 2 hour classes.
The implementation phase within the ILDE was not reached. This was mostly due to the fact
that students were not at all familiar with the use of Moodle and there was little time for
getting them to subscribe and use the medium. The teacher got a preview of the ILDE
implementation process on a theoretical level while examples of similar implementation
examples were also shown to her by the facilitator. The Learning Design was trialled, but not
using Moodle. Instead, the teacher used the Moodle Graphical User Interface and created
the activities, resources, groups, etc. by hand, based on the design authored in Web Collage.
This is the manual alternative to the automatic deployment used within the ILDE; it is much
more costly in terms of time and it may not reflect completely the design authored through
Web Collage.
Enactor #2
The delivery of the Learning Designs took place within the week of 10th to 14th March 2014.
The class consisted of 23 primary school students of the 5th grade. The topic was Principles
of Electricity.
Two LDs were created in the ILDE:
33
METIS PROJECT
a. a conceptualisation LD using the “Other Conceptualisation” tool, which includes a
custom template with the basic elements of a learning scenario (title, learning
objectives, learners’ needs and characteristics, instructional model and CLFPs, roles,
resources and tools, activity flow, assessment methods) in flowing text
b. a WebCollage LD based on the other conceptualisation LD. The WebCollage LD used
Inquiry-based learning (5E Instructional Model) and included two CLFPs (Jigsaw and
Role Play). The estimated time for this learning scenario is four credit hours (2 hours
x 2)
At the time of writing, the implementation phase is still ongoing.
7.3 Reflective diaries & Interview (KEK-Greece)
Participants were in regular contact with a facilitator and jointly compiled the reflective
diary.
7.3.1. Were there any difficulties in completing the enactment?
Enactor #1
There was difficulty in deploying the LD in Moodle. There was little time in the course and
students were not at all familiar with using Moodle so this step was omitted.
Enactor #2
Deploying and utilising the LDs were delayed by a combination of technical problems:
ILDE – various dysfunctions occurred (our work in “other conceptualization” was erased
without a reason, slow loading and crashing)
e-mail – many students did not have email addresses, so they could not be added to Moodle
Glue!-PS – the credentials for automatically accessing Moodle were not used correctly by the
ILDE deployment procedure2
7.3.2. Did enactors need to seek clarification or help with anything?
Enactor #1
The teacher was not very familiar with learning design and the introduction to the
environment was not enough – more time was needed in order for her to familiarise herself
with the concept and practices of learning design. Due to time limitations this step was very
2
The user name and password credentials for accessing the ILDE is different from the one used for
accessing Moodle. At the deployment phase the two credentials don’t match and by default the
credentials of the ILDE are used leading to an error. The solution of changing the Moodle credentials
was investigated but Moodle only accepts lower case and it is very common that users have used at
least one upper case letter in their ILDE credentials. The option of changing the ILDE credentials was
also investigated but after imputing the change in the ILDE the credentials seem to remain
unchanged. [Note that this problem has since been resolved.]
34
METIS PROJECT
brief and thus many relevant questions on LD principles/best practices arose during the
phase of creating the LD in the ILDE.
Enactor #2
No comments given.
7.3.3. What were the advantages of undertaking the enactment?
Enactor #1
Hands-on experience on writing a LD in the ILDE and delivering it to a class. Impressions after
the course was taught were very interesting because both the teacher’s and students’
opinions were recorded.
Enactor #2
 Hands-on experience on authoring with WebCollage
 Resolving real challenges on deployment (e.g. email problem)
 Understanding of CLFPs
 Collaborating with facilitator
7.3.4. Did it improve their teaching practice and if so in what ways?
Both enactors liked being able to easily share Learning Designs.
Enactor #1
No further comments given.
Enactor #2
Enactor #2 liked being able to easily connect learning objectives with learning activities.
7.3.5. How much time did they spend on it?
The times below show the overall time the teacher dedicated to enactment, taking into
account time spent on discussions/email correspondence with the facilitator and question
answering.
Enactor #1
Approximately 20 hours.
Enactor #2
Approximately 18-20 hours.
These times are in line with those expected from the schedule (see Section 7.2.2). None-theless, enactment took more time than the enactors had expected, especially:



In duration of authoring a learning scenario from scratch
In orchestration of learning activities
In definition of assessment
35

METIS PROJECT
In connection of learning objectives with learning activities
7.4 Enactment (Agora-Spain)
7.4.1. Participants
All workshop participants were encouraged to take part in enactment. Two participants
decided to do so, while a third participant assisted with an enactment.
One more participant had decided to take part in enactment, but this proved impossible to
arrange during the METIS time period (and is not described here).
Enactor #3
A volunteer at Agora, taking digital literacy classes.
Enactor #4
A volunteer at Agora, taking digital literacy classes.
Enactor #5
A volunteer at Agora who assists Enactor #4 with digital literacy classes. Enactor #5 assisted
with Enactor #4’s enactment. She had not participated in the ILDE workshop, but wanted to
participate in the enactment as she usually works with Enactor #4. She intends to attend the
next round of METIS workshops.
7.4.2. Schedule
A facilitator met with each enactor once a week to discuss progress and provide assistance
and guidance.
The proposed schedule (based on the given guidelines described in Section 5) by task for
each enactor are shown below.
1.
2.
3.
4.
5.
6.
6.
Discuss and decide on learner requirements
Use Narrative tool in ILDE
WebCollage recap
Authoring the LD in WebCollage based on the Narrative
Implementing/deploying the LD in Moodle
Deliver the LD to students
Evaluate the effectiveness of the LD
The Agora ILDE contains 8 documents created in relation to the KEK enactment:
4 x Design Narrative
4 x Web Collage
36
METIS PROJECT
Actual content created in the Agora ILDE in relation to enactment is shown in Appendix A of
this report, while the ILDE itself is accessible at http://ilde.upf.edu/agora
Enactor #4 & Enactor #5
The delivery of the Learning Designs took place during March and April. The design was used
with two classes of 18 students per class. The topic was Photoshop3 tools.
Enactor #3
The delivery of the Learning Designs took place during March and April. The design was used
with two classes of 14 students per class. The topic was PowerPoint and Word4.
7.5 Reflective diaries (Agora-Spain)
Enactors were given the following questions as guides, but were not specifically requested to
answer the questions in their reflections.
7.5.1. Were there any difficulties in completing the enactment?
Enactors at Agora were all volunteer teachers who were originally learners there. For them,
the Learning Design course and the ILDE were significant challenges. Despite this, both
succeeded with their enactment.
Two of the ILDE tools were used, Narrative and WebCollage. It was not possible to use Glue!PS to transfer the WebCollage details into Moodle due to a technical problem5. Therefore,
Moodle content was generated manually.
7.5.2. Did enactors need to seek clarification or help with anything?
The facilitator and the enactors considered the support provided during enactment (weekly
face-to-face meetings) to be critical in supporting their enactment. These meetings focussed
on ascertaining what the enactors wanted to do, and providing appropriate supporting
materials and advice. In particular, the ILDE was seen as complicated, with the workshop
being insufficient to learn its use adequately.
7.5.3. What were the advantages of undertaking the enactment?
Using the ILDE as part of their enactment changed the participants’ views of the tool, being
more positive as their understanding increased – after the workshop, the Enactors felt the
ILDE to be very difficult, but with support during enactment this view changed.
3
Photoshop is a registered trademark of Adobe Ltd.
PowerPoint and Word are registered trademarks of Microsoft Ltd.
5
The college had the most recent version of Moodle that, at the time of the enactment, was not
supported by Glue!-PS While this problem was addressed quickly, it prevented Glue!-PS being used
for this enactment.
4
37
METIS PROJECT
Enactors very much liked the fact that their content could be reused at a future time by
modifying the WebCollage content and exporting to Moodle. This was an important feature.
Enactor #3
The workshop acted as a “first overview” of the ILDE; there were with a lot of new concepts
and the enactment was necessary in order to understand how the theory could be applied in
practice.
Enactor #5
The enactment is a good educative process and a good opportunity to plan and create better
organised classes. Storing the classes in the ILDE platform allowed Enactor #5 to take
Enactor #4’s classes if necessary, knowing the plan for each lesson as well as what has
previously been taught.
7.5.4. Did it improve their teaching practice and if so in what ways?
Enactor #3
Using the pyramid collaborative learning pattern with the class improved class motivation.
7.5.5. How much time did they spend on it?
Although each enactment session was 60-90 minutes, the enactors felt that enactment was
a long process.
8. Discussion and Recommendations for the second round of
Workshops (Cycle 3)
The preceding description of the workshops was examined with a view to finding
improvements that may be incorporated into the second round of METIS workshops. These
recommendations are listed below – the highest priority shown first – separated into
recommendations for design of future workshops, recommendations for running future
workshops and recommendations for enactments. Note that these recommendations are
based upon observations and records of the first round of workshops and enactments;
further recommendations will appear in deliverable D5.2 based upon participant feedback
and related data.
ILDE design recommendation 1: Consider adjusting nomenclature to match
participants’ vernacular
For many of the participants, the workshop was a steep learning curve. Since most
participants had some background in teaching, the basic idea of designing learning
interventions was not unknown, but the Learning Design workshop took these concepts
further. In cases where a concept was familiar, the participants’ work was increased if a
different name was used for the concept to the name they were familiar with.
38
METIS PROJECT
One way to ease participants’ workload would be to adapt each workshop and ILDE
implementation to use names already in use by that group for understood concepts.
ILDE design recommendation 2: Implement an automated recording mechanism
The process of extracting learning designs from the ILDE to use in this report proved very
time consuming. In some cases, it was also not possible to isolate versions created during
the workshop. Therefore, an automated process of storing ILDE entries and related
metadata from specified time periods, and recovering them easily in readable format, is
recommended.
Although this recording is necessary to assess ILDE use within METIS, it should be noted that
one workshop participant deleted her Learning Designs from the ILDE at the end of the
workshop to retain privacy of the designs (see Section 4.2.3).
Workshop design recommendation 1: Allow more time to learn WebCollage
Of the two trial partners that asked participants to use Web Collage, these sessions overran
by 50% and 100%. Although observers sat close to participants, it was not possible to isolate
any specific problems that would cause this. The best explanation is that it was a very “steep
learning curve” for participants. It is clear that Web Collage requires a longer and gentler
explanation in future workshops.
This is likely to be a difficult problem to address. Essentially, time allocated to a workshop (1
session of 1 day, 7-8 hours) does not allow participants to understand all workshop contents,
become familiar with the ILDE software (especially WebCollage), and author usable Learning
Designs for subsequent enactment.
The only practical way to provide sufficient time for all aspects of the workshop to succeed
would seem to be to hold a longer workshop over two (or more) days (as has been shown to
work for the Agora workshop).
Workshop design recommendation 2: Carefully assess participants’ prior
knowledge and adjust activity times to allow for assimilation of new concepts
Generally, it looks as though participants took more time than planned in understanding
new concepts introduced in the workshops, especially for Agora and KEK where participants
were not primarily experienced learning designers. Therefore, clearer and more
comprehensive introductions to concepts should be considered for future workshops. It may
be beneficial to enquire of participants prior to the workshop their understanding of
concepts and tailor each workshop accordingly.
Workshop design recommendation 3: Reassess activity timings after any
workshop where an activity over/underran significantly (e.g. by more than 25%)
39
METIS PROJECT
One trial partner skipped a significant number of activities. While this did bring the
workshop back to schedule (and was presumably part of the contingency plan for earlier
overruns) it is worth considering the effect on participants’ learning of such large changes.
All workshops contained activities that overran some point (in addition to the Web Collage
overruns noted above). However, the contingency plans worked, in that activities were
adjusted successfully to ensure the wrap-up activity was given sufficient time and the
workshop ended on time. It appears that activities overrun (and underrun) where
participants find them difficult /easy / complex / straightforward / different to their initial
understandings, compared to the trainers’ expectations. Since participants vary, it is
probably important that the length of activities is adjusted according to participants’ needs.
Therefore, while overruns should be carefully looked at, they may not necessarily be a
problem requiring change of design.
Workshop design recommendation 4: Consider using an alternative topic to
“Collaborative learning”
The first round of workshops was on the topic of “Collaborative learning”. Using a different
topic for the second round of workshops would expand the generalisability of the METIS
workshop method.
Workshop design recommendation 5: Where possible, organise participants into
groups and create groups on the ILDE before the workshop
Organising participants into groups seemed problematic for two of the trial partners. This
process would be worth discussing in advance of the second round of workshops.
Workshop design recommendation 6: Provide a method for photographing and
uploading images that does not rely on the participants’ own devices
The procedure of uploading paper-based outputs onto the ILDE did not run smoothly; while
the OU ILDE contains 6 documents with images of paper work, there are no such images in
either the KEK or Agora ILDEs. Asking participants to capture and upload images themselves
produced a number of problems:




It was not clear just who was responsible for taking and uploading photographs
Participants’ photography equipment tended to comprise iPads and phones which
were mostly around five mega-pixels resolution; this is not sufficient definition to
read small detail on A1 paper
Transferring photographs from participants’ devices to the ILDE was not always
straightforward
Uploading photographs to the ILDE often required assistance
It is worth considering putting a more formal process in place (perhaps varying across
partners). For example, the facilitator could provide a suitable camera for participants to
40
METIS PROJECT
photograph their paper outputs; the facilitator could email the image files to participants to
upload or upload for the images for them.
Workshop design recommendation 7: In future workshop designs, use the ILDE as
the main or only tool for activities where possible
If the ILDE is to be fully utilised, the earlier stages of design (“How to ruin a course” and
“Barriers and Challenges”) need to be created around the platform, rather than on paper
(since uploading photographs does not constitute a test of the ILDE); overall, it is worth
considering whether the amount of content created in the ILDE during the workshops was
sufficient to allow participants adequate exposure to the software. In addition, practical
problems were encountered in copying and uploading paper-based work to the ILDE.
The benefits of using the ILDE must be balanced against benefits of paper tools, such as
tangibility, ease of manipulation, practicality for collaborative work and participants’
familiarity. There are also the questions of what benefits were gained by uploading the
paper-based work to the ILDE, and, if a paper-based approach works well for some
workshop tasks, what purpose the ILDE serves for those tasks.
It is recommended that this question is considered in depth and a decision made that
addresses these and related questions.
Workshop design recommendation 8: Question the importance of using real
learning designs during the workshop
The METIS project’s Description of Work states that "During the workshops, the trainees
(i.e., participating teachers) will create a number of learning designs susceptible of being
applied to their own teaching practice." However, it is clear from the first round of
workshops that one day is, at best, only enough to understand the concepts involved with
learning design, to become familiar with the ILDE software and to begin its tentative use.
Enactors need to work further on their learning designs before using them with students.
Indeed, sometimes it was preferable to start a new design than to modify one created
during the workshop.
It would probably be better to support this approach, rather than to expect participants
simply to create their designs during the workshop. It may be better to ask participants to
use a simple, fictitious Learning Design example during the workshop rather than actual class
requirements. Observed reasons include:



All documents used in the enactments were created during the enactment time
period, i.e. documents created during the workshops were not later used directly for
enactment.
One participant deleted their workshop Learning Designs to ensure privacy
All the Web Collage sessions overran significantly
Workshop design recommendation 9: Carefully consider the role of the facilitator
41
METIS PROJECT
While all workshops ran well, both the OU and Agora had one facilitator per group; the
observer at the OU reported that “for the most part the facilitator led their group” There is
the potential for a very different experience, depending on whether the facilitator is there to
assist when asked, or when needed. It would be worth considering the difference as a
general question for the next set of workshops.
Practical recommendation 1: Treat enactment discussion and planning as a highpriority workshop activity
While all workshops planned an “Enactment discussion”, two of the workshops did not do
this, and one reduced the time for this from 15 to 8 minutes. To ensure that enactment
remains a major part of METIS, workshops need to allow significant time for participants to
consider their enactment during the workshop. (There may be the exception if enactors and
non-enactors were to have separate workshops (see Section 9)).
Practical recommendation 2: If a major tool (such as WebCollage) is part of a
workshop design, ensure time is allowed for that tool
One trial partner demonstrated Web Collage, but did not ask participants to use it
themselves in any significant way; participants were to be sent a “tutorial”. The plan had
been for participants to use WebCollage, but it appears there was a shortage of time. This
was a significant deviation from the plan; it is recommended to ensure time issues do not
detract from major tasks.
Practical recommendation 3: Be prepared to look further afield for participants in
future workshops
It is worth noting that all three trial partners advertised the pilot workshops to people with
an existing connection to the organisation. Depending on the size of this pool, it may be
more challenging to find sufficient participants for the second round of workshops from the
same groups. The project is committed to running “at least 12 design workshops [with
follow-up enactments] ... along the two cycles, with a participation rate of about 10-20
teachers each.” (METIS Consortium, 2012, pg. 39). It is therefore important to expand
recruitment for future workshops.
42
METIS PROJECT
Other notes
Two trial partners noted that the Glue!-PS transfer took longer than expected during the
workshop (over a minute) during which an error message was shown (in Spanish for the UK
partner). It would look better to display a progress bar, or a notice (in the appropriate
language) that explained the delay, rather than declaring an error. [It is noted that the ILDE
has now been modified to resolve this issue.]
One workshop did not distribute Consent forms at the start. While this was a minor
oversight, it does emphasise the importance of the checklist as a tool. [This is seen as a oneoff event that is unlikely to recur.]
9. Recommendations for Cycle Three Enactment
The preceding description of the enactments was examined with a view to finding
improvements that may be incorporated into the second set of enactments in Cycle 3 of
METIS. These potential improvements are listed below as recommendations, in order of
likely importance.
Enactment recommendation 1: Actively recruit workshop participants who are
willing and able to enact their designs
The plan for Cycle 2 of METIS was for a minimum of two enactors per trial partner. Although
this was achieved for KEK and Agora, and will ultimately be achieved by OU-UK, it had been
hoped that this minimum would be exceeded. This is especially important if the METIS
project is to successfully develop communities of practitioners (UVa, 2012, pg.37). A critical
mass of users is required to start a community; Cycle 3 of METIS needs more than the
minimum number of enactors (two per partner) if this is to be achieved (this is also
recommended by reviewers).
Enactors need to be able and willing to enact their designs. This means specifically recruiting
at least some participants that have appropriate work circumstances (organisational,
pedagogic and student-facing) and will commit at least 20 hours to their enactment. The
project needs specifically to recruit enactors, rather than assume that a proportion of
workshop participants will enact.
It is recognised that this will increase the difficulty of recruitment, and it may be necessary
to recruit also non-enactors, or have some workshops for enactors and some for nonenactors. None-the-less, actively recruiting enactors will ensure sufficient use of the
platform to allow the development of a community.
Enactment recommendation 2: Give potential enactors more information from the
start on the amount of time and work necessary for successful enactment
All enactors reported that their enactment took up significantly more time than they had
expected. While this did not prevent them from completing their enactment, it did in some
43
METIS PROJECT
cases delay enactment. It would seem better to be more realistic in briefing enactors on the
anticipated effort involved. This also relates to the first recommendation, in recruiting
workshop participants who are willing and able to put in the necessary effort for enactment.
Enactment recommendation 3: Implement a defined process for checking each
enactor’s VLE early on, e.g. a questionnaire asking for VLE details (type, version,
institutional use and support)
Some enactors encountered difficulties related to the transferring of Learning Designs to
Moodle. In one case the students were not familiar with Moodle, in another there was a
technical problem with the Glue!-PS and Moodle interface. It would seem possible that
connecting between the ILDE and the VLE is a weak point in the Learning Design process.
Enactment recommendation 4: Consider asking enactors to write a weekly blog
post or similar reflective writing
Enactors were asked to keep a reflective diary. However, the trialling partners felt this would
be too great a workload for individual enactors and suggested a joint recording of enactors’
experience by the enactor and the facilitator. Although this was effective, the project could
benefit more from a more detailed view from the enactor’s perspective.
Enactment recommendation 5: Consider having one publicly available ILDE per
sector
Returning to the building of communities of practitioners, METIS is running additional
dissemination workshops of various kinds, and generally promoting the METIS approach and
the ILDE. If learning designers reached by this publicity are encouraged to use a common
ILDE communities may form.
At present there is one ILDE instantiation for each of the trialling partners (see Section 2.3),
plus one generally available instantiation at http://ilde.upf.edu. This equates to one ILDE per
sector (adult education, vocational training and university education) plus a general ILDE. If
the separate ILDEs were opened up as sector-specific, for public use, rather than promoting
one general ILDE, users may feel more “among friends” with other designers’ work being
more relevant. It may be necessary to add additional privacy options to achieve this.
10. Conclusions
This deliverable has described and discussed the workshops and enactments undertaken for
Cycle three of METIS. The document should be read in conjunction with deliverable D5.2
(Pozzi F., Persico D., Sarti L., 2014) that details evaluation of these workshops and
enactments. Together, these two reports document the first round of METIS workshops.
Observations show that the three workshops ran well, with participants creating Learning
Designs using the ILDE. Those participants that went on to enactment worked hard on
creating and trialling new Learning Designs, with help from committed facilitators.
44
METIS PROJECT
Within that, a number of elements can now be identified as ready for improvement before
the next round of workshops and enactments, the three most significant being:
1) Enactment needs to have greater focus generally. Enactors need to be able and
willing to enact their designs. This means specifically recruiting at least some
participants that have work circumstances (organisational, pedagogic and studentfacing) and will commit at least 20 hours to their enactment. The project needs
specifically to recruit enactors, not assume that a proportion of workshop
participants will enact. In addition, the workshop needs to lead clearly to enactment.
2) Time allocated to workshop activities needs to be reconsidered. Workshop
participants found the concepts and practice of Learning Design, and the interface
and use of the ILDE to be a steep learning curve. This led to activities either
overrunning or being skipped. In particular, the session to use Web Collage overran
greatly. It seems that that one day is not long enough for what is being attempted.
Either the workshop needs to be longer or the contents need to be reduced.
3) The use of paper-based activities has been questioned. Their advantage is one of
familiarity and simplicity, but where an ILDE facility exists for the activities, using
paper prevents learning and testing of the ILDE, and in fact calls into question the
utility of an ILDE-based version. The present compromise, of uploading a photo of
the paper to the ILDE, does not answer these questions.
Recommendations in these reports will be used as a basis for examining all aspects of this
first round of workshops and enactments. In particular, deliverable D3.3 (McAndrew,
Brasher, Prieto & Rudman, 2013) will be reconsidered to identify any changes necessary for
the second round of workshops where recommendations are to be implemented. Such
changes will be reflected in deliverable D3.4 (Final workshops packages), due month 28.
Changes may also be made to the ILDE, which will be reflected in deliverable D2.2 (ILDE
software) and D2.3 (ILDE documentation), both due month 28.
The second round of workshops (Cycle 3) will also undergo description and formative
evaluation, leading to similar deliverables – D4.2 (Report on the workshops and LD
enactment) and D5.3 (Report on second formative evaluation round). These will also
generate recommendations for improvement of the workshop package (including the ILDE),
again, reflected in deliverables D2.2, D2.3 and D3.4.
11. References
Brasher, Walsh, McAndrew, Mor (2013), D3.2: Draft of pilot workshop, METIS Consortium
McAndrew, Brasher, Prieto, Rudman, (2013), D3.3: Pilot workshops: workshops for different
educational levels, METIS Consortium
Conole, G. (2012) Designing for learning in an open world, Heidelberg: Springer
METIS Consortium (2012) METIS Description of Work
45
METIS PROJECT
Open University (2014) Management Information Portal 2014, Unpublished internal
document
Pozzi F., Persico D., Sarti L. (in press), D5.2: Report on first formative evaluation round,
METIS Consortium
UVa (2012) Meeting teachers’ co-design needs by means of Integrated Learning
Environments (METIS), European Commission
46
METIS PROJECT
12. Appendix A – ILDE content created during the workshops and for the
enactments
Below is a list of documents created during the workshops and for Enactment - see separate
file D4.1_pilot_workshops_and_enactment-Design_package-Final for full details.
Note these are documents present on the ILDE at the time of writing that were created /
modified during the workshop / enactment period. The content may differ from that actually
present during these periods. (See ILDE design recommendation 2 in Section 8.)
12.1 ILDE content created during the OU workshop
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
Brainstorm 2: learning outcomes and outputs (Science) (Factors and concerns)
Morning brainstorming: barriers and solutions (Science) (Factors and concerns)
FBL heuristics (Heuristic Evaluation)
HSC activity - heuristic evaulation (Heuristic Evaluation)
MCT heuristics (Heuristic Evaluation)
MTC heuristic evaluation (Heuristic Evaluation)
Christmas card (Web Collage)
FBL collaborative activity (Web Collage)
FBL collaborative activity revised (Web Collage)
HSC WebCollage prototype (simulation) (Web Collage)
Intervention and Reframing: Using Diagramatic Thinking (Web Collage)
Science storyboard (Web Collage)
Untitled LdS (simulation) (Web Collage)
Barriers and heuristics (Image)
HSC A4: conceptualize learning outcomes (Image)
What ruins (Image)
HSC Project A
Untitled LdS
Untitled LdS
12.2 ILDE content created during the KEK workshop
1.
2.
3.
4.
5.
6.
7.
Ηλεκτρισμός Ε' τάξης Δημοτικού (Web Collage)
Σχολικό Τσουνάμι... (Web Collage)
Πρότυπο Διδακτικού Σεναρίου (by CoSyLlab)
Πρότυπο Διδακτικού Σεναρίου Ομάδα 1(by CoSyLlab)
Σχολικό Τσουνάμι...
12.3 ILDE content created for the KEK enactments
8. Marketing Strategies (Web Collage)
9. ΜΑΡΚΕΤΙΝΓΚ ΚΑΙ ΣΤΡΑΤΗΓΙΚΗ ΠΩΛΗΣΕΩΝ ΤΗΣ ΕΠΙΧΕΙΡΗΣΗΣ
12.4 ILDE content created during the Agora workshop
47
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
METIS PROJECT
inicio hojas de calculo (Design Narrative)
Narrativa de la actividad (Design Narrative)
Narrativa de la salida de ajedrez (Design Narrative)
Narrativa de Tegucigalpa (Design Narrative)
Nivel medio de ajedrez (Persona Card)
participant tic jove atur (Persona Card)
participante tic inicial (Persona Card)
test (Persona Card)
Ajedrez (Web Collage)
Curs de prova A (Web Collage)
Curs de Prova Agora (Web Collage)
CURS DE PROVA B (Web Collage)
curso de prueva c (Web Collage)
Españoles en el mundo (Web Collage)
inicio hojas calculo (Web Collage)
12.5 ILDE content created for the Agora enactments
1.
2.
3.
4.
5.
6.
7.
8.
Capas en photoshop (Design Narrative)
Herramientas de corrección en Photoshop (Design Narrative)
Power point. Sesión 1 (Design Narrative)
Word. Sesión 1 (Design Narrative)
Capas en photoshop (Web Collage)
Herramientas de corrección en Photoshop (Web Collage)
Power point. Sesión 1 (Web Collage)
Word. sesión 1 (Web Collage)
48
METIS PROJECT
13. Appendix B – Example checklist (KEK)
Task
Check
Participant recruitment
What is / are the target audience?
In-service teachers; Vocational trainers
within KEK
How will the workshop be
promoted? (Include any URLs)
Announcement in KEK's web page
(www.eurotraining.gr), in elearningnews.gr
and in diorismos.gr - Trainers affiliated with
KEK will be contacted via email/phone
How will students register? (Include
any URLs)
Similar on line application form as:
http://www.eurotraining.gr/index.php/eapplications/web-design-form
Create a general profile of your
participants, as a group
Participants should be familiar with Moodle,
LAMS etc and e-learning MScs or teacher
training will be a plus
General workshop
considerations
Logistics
Date of first workshop
Saturday 9th November 2013
How long will the workshop last
(hours / days)? Split into parts?
5 hours workshop, 5 hours own time
How many workshops?
one workshop
Where will the workshop take
place?
KEK facilities in the centre of Athens
How many participants are
anticipated?
20
How many facilitators will be
needed?
2
Who will facilitate?
*** Two KEK personel will facilitate (aka
"Instructors") - Simos Retalis (Main);
Stathis Mitropoulos (Assistant)
49
METIS PROJECT
Who will observe?
*** One person will observe (aka "Logger")
What is the plan for training the
facilitators?
Facilitators will be briefed / familiarised with
the tools and workshop requirements by
Lori
Design
Describe the theme of the
workshop(s)
*** Collaborative learning
What will participants create that
they can take home?
*** ILDE artefacts they created; Enactment
plan
Define any other aims for the
workshop
*** Understanding the basics of Learning
Design and ability to utilise the ILDE
Customising the generic design
Participants
Consider the key concepts and
rationale appropriate for
participants
Describe the workshop contents,
purpose and benefits
How will participants be assigned
into groups?
What is the intended group size?
5
How will each group's
challenge/project be selected?
Based on member preferences
Pre-workshop activities:
Circulate pre-workshop
questionnaire
No pre-workshop activities
Consider asking participants to
view an “Intro. to LD” podcast/video
They will already be familiar with LD
Consider a pre-workshop task
(recommended)
No pre-workshop activities
Pre-workshop technical support (if
ILDE to be used pre-workshop)
*** n/a
50
METIS PROJECT
Should participants arrange to
bring a device with them?
No
What, if any, preparation should
participants do?
*** Register on the ILDE
Workshop activities:
Decide how many laptops / group
are necessary
The room hosting the workshop will be fully
equipped with an adequate number of PCs
- laptops will be optional
Define what media / resources are
to be used for each activity
***This is defined in the workshop design
Define a backup method / media
for each ILDE-related activity
Decide on the points where groups
share designs / provide feedback
Include VLE integration and
deployment?
*** This process is demonstrated during the
workshop; participants leave with
information to assist them accomplish this
themselves
Incorporate enactment planning
into the workshop
Enactment will be included in the
"homework" participants will be asked to
submit within a week's time/ linked with
certification
End workshop with overview of the
day
*** Wrapup activity incorporated into the
workshop design
Update glossary from the workshop
design and the ILDE
If necessary, define any activities
for between workshop sessions
*** n/a
Timing:
Review the time allocations for
tasks; do they need adjustment?
Is the workshop long enough to
cover a full LD cycle? If not-----list the relevant activities that
could not be included
---define how will these activities be
51
METIS PROJECT
replaced
Prioritise workshop tasks
Create a plan to change or skip
tasks if the workshop falls behind
Delivering the workshop
Media
Prepare PPT (Facilitator slides)
*** V1 ready
ILDE / VLE content
Prepare in-workshop hand-outs
Prepare post-workshop hand-outs
Ensure that any other content is
deployed to selected media
*** n/a
What is the URL for the VLE?
What is the URL for your local
ILDE?
http://ilde.upf.edu/kek/
Ensure technical support for
facilitators during the workshops
Define tagging protocol for ILDE
artefacts created in the workshop
*** All created ILDE artefacts to be tagged
by group name
Allow the use of Greek characters
for user input
Specified in the workshop design
document
Ensure the ILDE can handle the
expected number of users
How do participants take home
their productions?
What is the URL for the WP5
Questionnaire (if online)
The environment
Select and book a suitable room
(and post photos)
ok
ok
Arrange one table per group +
52
METIS PROJECT
space around for all to congregate
Consider using breakout rooms
no
Ensure a user account per
participant on ILDE
*** Participants will be asked to register for
an ILDE account in advance, and may also
do so in the specific workshop activity
Create generic guest accounts for
ILDE and VLE
*** This should not be necessary
Create groups on ILDE in advance
and add users
Ensure user accounts on VLE used
for supporting the workshop
*** Participants will not use the VLE during
the workshop
Ensure user accounts on VLE used
by participants for deployment
*** Participants will not use the VLE during
the workshop
Arrange and test camera +
connection
Working computer and projector
Sufficient laptops (they may bring
their own), plus a few spare
*** The room contains sufficient desktop
PCs for all the participants
Ensure appropriate wifi access /
logins and power sockets
If possible, and not already done,
test wifi
Consider providing one projector /
large monitor per group
*** ???????????
Power strips & extension cords
Physical artefacts
Print handouts (for duringworkshop and post-workshop)
Attendance (signature) sheet
Research consent forms (one /
participant)
JISC post-its
Plain post-its (several colours /
53
METIS PROJECT
group)
A1 (flipchart) paper (one block)
A4 scrap paper
Pens & pencils
Blutac / sellotape etc. to hang up
the A1 paper
Black marker pens (one / group)
Thick coloured writing pens &
highlighters (several / group)
Any additional artefacts necessary
(specify)
Do we need to print the questionnaire?
Pattern documentation
ILDE worksheet
WP4 Instructions for facilitators &
templates for observers
WP5 Questionnaire (if printed)
During the workshop:
Provide technical support for
registration during coffee breaks
Decide what part of the conceptual
design you focus on
Ensure all group members are able
to participate throughout
Allow time for participants to
familiarise with each ILDE function
Stop participants at the end in time
to discuss and plan Enactment
*** This is a specific, high-priority,
workshop activity
Recording the event
Before the workshop
Nominate and brief an observer, to
take notes during the workshop
yes - mentioned previously as a logger
Take a photo of the room
54
METIS PROJECT
During the workshop:
Introduce the observer / assistants,
and explain their role
Circulate attendance sheet to
collect participants' signatures
Circulate and collect signatures on
research consent forms
Record the start and end time of
the workshop and of each activity
*** This is part of the observer's template
Take a photo of each group
working (if permission is given by
all)
Define responsibility for the above
After the workshop:
Photograph (if not already) and
keep the A1 sheets
Collect any other artefacts
produced by participants
Secure any other recordings
Enactment
Define successful enactment for
these participants
Homework on creating an LD according to
their needs and deploying it in the existing
local moodle test server
Define mechanisms to record
enactment activity
Information on each activity to be
referred to post-workshop
Provide support for participants
during enactment phase-----LD support in creating their
designs
yes via email mostly
---Technical support on ILDE and
Moodle integration
yes via email mostly
55
METIS PROJECT
---Peer social networking (the start
of a “teachers’ community”?)
yes
---Consider organising Face-toface meeting(s) during enactment
period
no
Evaluation
Decide whether the questionnaire
needs to be split in two
*** There will only be one questionnaire
Translate the questionnaire
pending for answer
(if necessary) provide the
questionnaire to ITD
Allow 30 minutes at the end of the
workshop for questionnaire
*** There is a specific, high-priority,
workshop activity at the end for completing
the questionnaire
Before participants fill it in explain
the questionnaire aim
Show the questionnaire and
explain the main structure
Ask participants to write their ILDE
username on the questionnaire
Explain that another questionnaire
will be delivered at the wrap up
56

D4.1: Report on the pilot workshops and LD enactment

Transcription

Similar documents

flyer - MyGrandmontRosedale.org

Untitled

Lieutenant Kelly Cardoza (916) 843

Campiano Ontaneta Spedale

Vikatan 2x

Response to Intervention (RTI)

BCWWA`s Capacity Building Program for Small Water Systems

December 2015 - Anne Griffiths :: Contemporary Textile Art