KENNISBANK - ONDERZOEK
Transcription
KENNISBANK - ONDERZOEK
EK O Z R E D N O K KENNISBAN NE COMMUNITY EDUCATION PROJECT EVALUATION SHOW RACISM THE RED CARD SHOW RACISM THE RED CARD Samenvatting Deze rapportage geeft niet alleen inzicht in de projectresultaten van het Show Racism the Red Card-project in Engeland, maar ook is het een goed voorbeeld van een meet & evaluatietraject van een project. De projectrapportage laat bijvoorbeeld zien met welke stakeholders en aspecten allemaal rekening wordt gehouden in het M&E traject en hoe de resultaten intern en extern gepresenteerd worden. Publicatiedatum: 2009 Meer informatie Show Racism the Red Card +44 (0) 191 257 8519 info@theredcard.org 1 INTRODUCTION Show Racism the Red Card is an anti-racism education charity established in 1996. Its campaign focus is the use of high profile, famous footballers to educate young people against racism. The work undertaken includes holding events with football clubs all over the country, running competitions for schools, producing promotional materials aimed at young people including DVDs where players and young people speak about their experiences and views of racism. In addition educational materials have been developed to support teachers and others in using the DVDs as educational resources within both formal and informal educational settings. In May 2004 the organisation established the Community Education Project to deliver antiracist workshops directly to schools and youth groups based around the educational materials that had been developed. The approach built upon the idea of using professional footballers as role models by integrating the educational workshops with coaching sessions delivered by ex-professionals. This model has been successful and well received by schools and young people. NE COMMUNITY EDUCATION PROJECT EVALUATION In addition the Community Education staff undertake the organisation of community festival events and other activities in partnership with local agencies. Conferences and training events aimed at teachers and educationalists have also become part of the team’s portfolio of activities and they also support the football club based events in the region. The Community Education Project has been funded by the Football Foundation and by local authorities in the region. Currently the organisation has partnership arrangements with Sunderland City Council, North Tyneside MBC, South Tyneside MBC, Newcastle City Council, Gateshead MBC, Durham County Council and Middlesbrough MBC. These arrangements allow for the targeted delivery of workshops and activities within these areas. Redcar and Cleveland became the eighth North East local authority to fund the programme during 2009/10. Funding has also been obtained from the Northern Rock Foundation for the development of an additional resource pack aimed at tackling Islamophpbia and for running workshops with schools across the region. 1 2 SCOPE OF THE REPORT The Community Education Project has a detailed monitoring and evaluation system. Reports are prepared on a regular basis for the respective funders and within the Project there is a culture of continual appraisal and assessment. These evaluative processes are focused on specific aspects of delivery and are carried out internally. The organisation decided that after nearly five years of operation it would be an appropriate time to examine the overall performance of the project from an external perspective. The intention in commissioning the study was to highlight and acknowledge areas of strength and to identify areas for development and potential improvement. 1) Information collected by the project would be used to demonstrate the levels of activity and types of activity undertaken by the project across the region over the period from April 2008 to March 2009. (Appendix 1) In undertaking the evaluation it was important that the monitoring data was utilised alongside information collected directly from partner agencies, funders and participating schools. 4) A telephone survey of 32 schools and colleges accessed by the project during 2008/9. (Summary Findings included as Appendix 4) Consequently the following areas of activity contributed to the development of an evidence base:- 2) The Project provided collated monitoring information from feedback provided by pupils, students, teachers and participants in community festivals for 2008/9. (Appendix 2) 3) A survey consisting of semi-structured interviews with key local authority contacts and an email questionnaire survey of partner agencies and funders. (Summary Findings included as Appendix 3) 5) A facilitated workshop with the project staff team was carried using participatory appraisal techniques to draw out issues and potential improvements. 6) Observation of a Football Club based event and a teacher training event. 2 SHOW RACISM THE RED CARD 3 THE PROGRAMME 3.1 Schools Based Activities Schools based activities are aimed at year 5 and above and take place in secondary as well as primary schools. The aims of the workshop sessions are: • To educate young people about the causes and consequences of racism and to explore what forms racism can take. • To empower young people to challenge racism in the communities in which they live and to provide them with relevant knowledge and information to enable them to do this. • To help young people prepare to play an active role as citizens in a growing multi-cultural society. • To enable young people to develop good relationships and respect the differences between people, regardless of ethnicity. The way the workshops are delivered varies between secondary and primary school settings. In primary schools the sessions tend to be class based and last for two hours and are NE COMMUNITY EDUCATION PROJECT EVALUATION accompanied by a ‘Football and Fitness’ session of an hour’s duration. Initial workshops provide an introduction to the subject of racism and are structured around the Show Racism the Red Card DVD. The young people engage in a range of exercises individually and in groups designed to encourage discussion and critical thinking. The ‘Football and Fitness’ sessions are delivered in mixed groups and the emphasis is on team work and mutual support. The young people wear Show Racism the Red Card bibs and prizes are awarded for enthusiasm and participation. The approach of combining the classroom sessions with physical activity is believed to work well. The presence of ex-professional footballers as coaches adds to the enjoyment of the young people and their engagement with the anti-racism message of the session. Secondary schools generally find it more difficult to free up time within the curriculum for class based activities so a different model of engagement has been developed. The Community Education Team work with the schools to provide them with customised 3 workshops using both the anti-racism material and the Islamophobia educational resource pack. Often the package will take the form of a Citizenship Day where an entire year group will participate in a ‘carousel’ of different workshops and events throughout the day. 3.2 DVDs and Teaching materials Show Racism the Red Card has produced four sets of resources for use in and by schools. The Show Racism the Red Card DVD and pack has been developed principally for use with primary school age children. The DVD lasts about 22 minutes and looks at racism, its origins and causes and what people can do to combat it. It uses role models from professional football to provide examples and insight. It also explores particular high profile examples of racism and racist attacks. The DVD is accompanied by a resource pack with activities and discussion points for teachers. The team has recently produced a DVD aimed at highlighting the issue of Islamophobia. The DVD follows the same model as the Show Racism the Red Card DVD with high profile footballers sharing their understanding and experiences of the issue alongside young people. The DVD is accompanied by an education pack. A third pack looking at issues to do with asylum has also been produced entitled ‘A Safe Place.’ Once again the DVD utilises the understanding of professional footballers alongside young people to challenge negative stereotypes and provide information on the realities behind the asylum process. During 2008/9 the team has worked with the Durham Ethnic Minority and Traveller Achievement Service and others to produce an educational pack entitled ‘Out of Site.’ This resource is aimed at tacking prejudice towards Gypsies, Roma and Travellers. This pack is in the process of being piloted. 4 3.3 Teacher training Through feedback from teachers and research the Community Education Team identified that many teachers felt they lacked the specialist skills required to deliver anti-racism and Islamophobia awareness workshops and activities. The Team has since initiated a more extensive programme of training for teachers and trainee teachers in an attempt to support the classroom delivery of the educational resources. The team launched the Islamophobia resource in October 2008 at an event at St James’ Park attended by over 120 teachers, council representatives and youth workers. The team has also worked with universities and colleges and supported conferences and other events including delivering in house training in order to help adults working with young people to operate more effectively. Show Racism the Red Card has received funding to organise four regional teacher training conferences to be held between November 2009 and March 2010 and nine regional teacher training conferences to be held between April 2010 and March 2011. The aims of the conferences are: • To raise awareness amongst teachers of issues of racism and how these impact on young people. • To empower teachers with knowledge to recognise racism and challenge racist myths. • To encourage teachers to be reflective in their work and consider race equality in their lesson planning and delivery. • To equip teachers with knowledge and resources to educate young people against racism in the classroom. SHOW RACISM THE RED CARD 3.4 Community Festivals and Events with Sporting Clubs Important aspects of the programme are the delivering of anti-racism activities at high profile community festivals and using the organisation’s sporting links to deliver anti-racism workshops to young people through educational sessions held in conjunction with professional football clubs and other sports clubs. session with sports stars. The events are designed to be fun as well as educational with competitions and prizes featuring as part of the day. In addition to the full day events, half-day events were held at Hartlepool FC, Darlington FC, Middlesbrough FC, and in the Gateshead Council Chamber. During 2008/9 the Community Education Team held four community anti-racism festivals in Ferryhill (County Durham), Sunderland, Newcastle and Tyneside. Collectively these events attracted approximately 6,000 people. The Sunderland Together Against Racism, Ferryhill is United and the Walker is United events were repeated in the summer of 2009. During 2008/9 the team organised full day events for young people with Sunderland AFC, Newcastle United FC, Durham County Cricket Club and the Newcastle Falcons. The programme for the day tends to vary slightly depending on the facilities available but the morning session will generally consist of a carousel of educational workshops and the afternoon session will take the form of a viewing of the Show Racism the Red Card DVD followed by a question and answer NE COMMUNITY EDUCATION PROJECT EVALUATION 5 4 EVALUATION FINDINGS 4.1 Profile and Branding Feedback from the partners and schools indicates that Show Racism the Red Card has a very strong brand identity. There is a clearly understood message in the brand and the linking of football with anti-racism education is seen by partners to be simple, direct and easily understood. Eight of the partner agencies identified the brand as an organisational strength of the agency. ‘I think Show Racism the Red Card has a strong brand and obviously its links with football are a huge advantage to attracting interest.’ (Partner Agency) ‘Simple but ‘dynamite’ formula, powerful branding, use of sports personalities. Incredibly valuable providing a challenge to young men.’ (Partner Agency) The relatively simple message means that it can be easily understood by a wide range of people and this has helped build a brand identity and forge partnerships with a wide range of organisations and agencies. 6 ‘Its main strength has been to link football with racism. This has undoubtedly enabled Show Racism the Red Card to reach young people who might not otherwise engage in the subject.’ (Partner Agency) The underpinning aim of the Community Education Project is to utilise the football theme to promote and develop anti-racism education. However the strength of the connection with professional football can lead to uncertainty on the part of some partners as to the difference between this approach and that of agencies more concerned with tackling racism within football. ‘They (Show Racism the Red Card) are seen as a resource costing organisation they need to work closer with Kick Racism Out of Football.’ (Partner Agency) In addition there is the perception that sometimes the overwhelming interest of people in football can result in the anti-racism message being lost to some extent. SHOW RACISM THE RED CARD ‘The only weakness I can see is that Show Racism the Red Card is always going to find it difficult to draw attention to the main message when football is the focus of many of the events.’ (Partner Agency) The Community Education Team’s range of partnerships is extensive and includes strong links with local authorities across a range of functions including education, social cohesion and sports. This width of engagement and relevance has meant that the Community Education Project has been able to access funding from different local authority strands. Whilst the focus on educational activities is fairly constant the emphasis of each local authority can differ. Of the six key local authority funding contacts who responded three of them specifically highlighted the work in schools as being a primary consideration with the other three having a wider interest across the community as a whole. This presents some issues to do with consistency of approach and the challenge of developing a monitoring and evaluation approach appropriate, not only across the different authorities but suitable for other funders. It has strong links with the Trade Union movement and this has enabled them to engage with broader campaigning work. In developing the brand the agency has been able to utilise the contact it has with professional football clubs to access high profile players as part of the programme and ex-players to provide the coaching element of the programme. The ability of the agency to access such support has been a major asset in the development of the profile and brand and adds considerably to the effectiveness of the work. The ability to access current and ex-professional football players as role models was identified as a strength by 13 respondents from partner agencies (35%). ‘Show Racism the Red Card are able to use footballers both domestic and international to get the message over to people who otherwise would switch off and not listen.’ (Partner Agency) ‘The organisation has also engaged high profile footballers which has given strength to their message.’ (Partner Agency) ‘Its value in engaging with leading professional footballers in the debate about racism is excellent.’ (Partner Agency) Partnerships have also been developed with community groups, professional sports clubs and their associated foundations, grass roots sports clubs, community arts organisations, the police and local universities. Recently the project has been engaged in developing work in conjunction with the Police Service on the PREVENT agenda aimed at tackling violent extremism which will be another element of the programme’s portfolio of work. Six (16%) of respondents from partner agencies identified the high level of community engagement and connection to grassroots organisations as an organisational strength and a further four (10%) identified the commitment to partnership working. ‘Its open working culture – building relationships with a variety of organisations.’ (Partner Agency) ‘The commitment shown by all the workers and their effectiveness in working with other organisations/partners.’ (Partner Agency) NE COMMUNITY EDUCATION PROJECT EVALUATION 7 4.2 Approach The aim of Show Racism the Red Card’s Community Education Programme is to educate young people about racism through a range of activities with a central football related theme. This approach provides a clear and focused method underpinning the different teaching packs, work in schools and community based events. The combination of a sports / fitness coaching session with classroom activities is a corner stone of the work in schools. As part of the survey of partners and funders, respondents were asked to comment on how effective this approach was in their experience. Similarly in the survey of participating schools the contacts were also asked to draw upon their experience in relation to this issue. Overwhelmingly the respondents felt that the approach was generally very effective in raising the issue of racism and in promoting learning and attitudinal change. 27 (90%) of the schools and 37 respondents from partner agencies (73%) indicated that they felt the use of football and the use of examples from professional football was an effective and successful approach. ‘I think Show Racism’s approach is uniquely effective in terms of football’s value as an engagement tool and in terms of its strength at breaking down barriers. This is important because one of the main challenges in anti-racist work (as I’ve found personally because I’ve done similar work myself in a previous role) is engaging people in a way that makes positive changes in attitudes and behaviour possible – it can be a subject where people are very defensive, fear being singled out or labelled as racist, are reluctant to say what they think because they’re not sure what the ‘politically correct’ terminology is, etc. The particular value of football is both that it provides role models who young people generally admire, and that the modern game itself can be used as a model of multi-ethnic, multi-national cooperation, where team allegiance and solidarity overcomes other differences.’ (Partner Agency) ‘I think this is very effective. Having seen Show Racism the Red Card's activities over a number of years, they have worked constantly to raise & tackle the issue of racism in society. The football link gives them an 'in' & engages young people in 8 a topic to which many of them are ignorant, prior to that engagement.’ (Partner Agency) ‘The football theme is a very effective vehicle for engaging the children. It also helps that Show Racism the Red Card is something they might see on TV - it’s good to make those links between the classroom and the outside world. Good connections made, much more aware of issues.’ (School) Feedback from the teachers suggested that the approach was particularly effective with boys and those interested in football but that it did result in girls and those not interested in sport being successfully engaged. A small number of respondents from both surveys believed that girls were not as fully engaged by the approach and some respondents felt that it would not work effectively with groups where there was not a strong interest in football. ‘There is however a danger of not engaging with those young people who are not interested in football, of course. While these young people may be in the minority, they must surely exist and, I would have thought be over-represented with girls.’ (Partner Agency) ‘However, for some older girls, apart from the cult personality bit, my personal view is that I'm not so convinced of the power of the football theme’. (Partner Agency) ‘The football theme was very effective but maybe only for the boys; I’m not convinced that the girls were equally as engaged by it.’ (School) Show Racism the Red Card has produced an analysis of pupil feedback which suggests that there is in fact little difference between boys and girls in their reported level of satisfaction and enjoyment of the DVD material, the fitness and football training and the classroom work. The approach is obviously seen as successful and highly effective and the feedback from schools in particular indicates that the way the sessions are run does successfully engage girls and those young people not interested in sports. However some partners and schools are reporting that they have concerns regarding this issue and Show Racism the Red Card should take these perceptions and reservations into account and consider some of the suggestions made by schools and partners to improve the materials and delivery. SHOW RACISM THE RED CARD 4.3 Staffing and Staffing Structure A recurring theme in the surveys of partner agencies and schools is the quality of the staff who work within the Community Education Team including the administrative and management functions, facilitators and coaches. Within the survey of partner agencies the quality of staffing, including the classroom facilitators and football coaches, was seen as an organisational strength by 18 respondents (49%) with a variety of attributes being identified including primarily enthusiasm and commitment. A majority of respondents, 19 or 59%, in the survey of schools also identified staffing as an organisational strength. ‘The presentation skills of staff were excellent. Their rapport with the children, the timing and content of sessions – it was all really good.’ (School) ‘Its people- well trained- good communicatorswell organised, committed.’ (Partner Agency) ‘The whole team is good and have masses of enthusiasm.’ (Partner Agency) NE COMMUNITY EDUCATION PROJECT EVALUATION ‘The commitment shown by all the workers and their effectiveness is working with other organisations/partners.’ (Partner Agency) ‘The trainers who run the workshops appear highly knowledgeable and there is no doubting their commitment and enthusiasm, something that clearly rubs off on their audience.’ (Partner Agency) 4.4 Activity Levels and Organisational Capacity An analysis of the Community Education Team’s programme in the region for the period 2008-9 shows that they were instrumental in delivering 229 individual events and activities. As would be expected the bulk of the activities (92%) took part in the Local Authorities which provide funding for the initiatives. There were 172 school based sessions of which 138 were delivered in primary schools and 34 in secondary schools. This equates to between four and five sessions for each week during the school year. The authority with the highest level of school based sessions was County Durham with 34 closely followed by Sunderland (29) and Gateshead (28). 9 During the year there were a total of 54 community events and promotional activities. These activities have been divided up into three broad categories: 1) Workshop style events held at football and other sporting clubs aimed at young people and recruited for through schools and local authority contacts. There were eight such events held during 2008-9 and these were fairly evenly distributed across the region. 2) Community festivals and promotional activities at public events such as the Durham Miners’ Gala and the Newcastle Mela. These tend to be targeted at the general public and have the aim of awareness raising within community based settings. A total of 32 such events and activities were programmed during the year and again they were fairly evenly distributed across the area. 3) Conferences and events which were closed in nature and intended for a professional or targeted audience. This category included trade union based events, teacher conferences and so on. Newcastle featured heavily as a location for these activities with eight taking place during the course of the year. Gateshead (2) Middlesbrough (2), North Tyneside (1) and Northumberland (1) were the locations for the remaining activities. The programme of work undertaken by the team represents a high level of activity for a relatively small staff group. It reflects upon the efficiency of the team’s structure, its organisational abilities and the commitment of the individual members of staff. ‘There is a great deal of enthusiasm and commitment, but also a great deal of knowledge and expertise that has been accumulated over the years.’ (Partner Agency) The survey of schools picked up on the quality of the organisation and ability to deliver with 12 (38%) of the schools identifying good organisation as a strength of the Community Education Team. ‘Organisation was great – very impressed with it.’ (School) ‘Well organised, planned and prepared.’ (School) ‘Efficient, well organised. Reliable.’ (School) The survey of partner agencies did pick up on some concerns about the organisation’s ability to 10 deliver consistently across the full range of activities undertaken. There was the sense that the organisation sometime comes across as overstretched. Similar concerns were echoed by a small number of schools who had experienced difficulties over the delivery of sessions. ‘Perhaps under-staffed/over-committed at times.’ (Partner Agency) Some respondents see this as the organisation being the victim of its own success but there are some concerns that the Community Education Team needs to actively manage the demand upon its resources and be aware of some of the concerns even though they may reflect perception rather than a lack of delivery. ‘The only aspect that would be of concern would be their capacity to deliver - and that is tied to resources.’ (Partner Agency) ‘The capacity of the organisation to engage with larger numbers of young people in school activities - the weakness is not about the content of the programme but capacity.’ (Partner Agency) ‘The only problem I have seen is the demands on the organisation being larger than it can cope with. Essentially the problem of success.’ (Partner Agency) 4.5 Delivery of work in Schools The work in schools is seen as being effective and of high quality by partner agencies and schools. The schools in particular saw the programmes as being highly effective and report a high level of impact and engagement on the part of the young people. This is supported by the analysis of the feedback sheets completed by teachers after the sessions as well as the survey undertaken with schools as part of the evaluation process. In feedback sheets collected over 2008/9 (70 in total) 97% rated the day as excellent or very good and 99% felt that the day had resulted in an improved understanding of racism on the part of the young people. Show Racism the Red Card has produced collated feedback for 2008/9 from 1,194 pupils at 39 primary schools across the region. The feedback indicates a high level of satisfaction and enjoyment on the part of the young people. SHOW RACISM THE RED CARD In total 98% indicated that they had learnt something about racism and 99% rated the event as enjoyable. In relation to the classroom sessions 76% of pupils rated them as very good or excellent and 92% rated the football training as being very good or excellent. ‘We asked every student to do research on campaign and took opinions on racism. This had a positive affect, I feel you need some form of preparation so the students have a good understanding and a wide scope before starting.’ (School) In delivering the workshops the quality of the staff, including the coaches, was seen as a major strength. 13 (41%) schools reported that they did not do specific preparation either because of time constraints or because it was felt not to be needed within the context of the school. ‘Focused day, raises profile in children’s minds. Children are impressed by the link with football and it being such a major campaign. For many footballers are role models who they look up to. The session reinforces the work done in school, but is especially effective as it grabs attention in a way a normal lesson might not.’ (School) ‘The use of celebrity footballers gave the event kudos and engaged the children; they were keen to tell their families what they had done partly because of who they had met.’ (School) ‘Ability to relate to young people, hit right ‘spot’, staff were young which made them role models to the children. The role play activities were enjoyed a lot.’ (School) ‘The children had some really good discussions with the staff. The staff were also very good at answering all of the questions, no matter how trivial they seemed.’ (School) As part of the schools’ survey respondents were asked about any preparation and follow up work they have undertaken using the materials provided by the team. 14 (43%) of the schools had undertaken work in preparation for the visit. Some schools watched the DVD in advance and others looked at the poster campaign and encouraged students to do some research. Several of the schools did preliminary discussions on the subject area and generally laid the groundwork for the visit. The schools that did undertake the work in advance felt that it meant that the event went smoother than would otherwise have been the case. ‘Prep was done with every class. This made them prepared for discussing the issues, as their knowledge base was wider.’ (School) NE COMMUNITY EDUCATION PROJECT EVALUATION ‘No, and I don’t think it made a difference.’ (School) ‘No. What they delivered was excellent, no prep was needed.’ (School) ‘We didn’t feel that any was needed and don’t feel it would have made a difference to the session.’ (School) Four schools stated that they would consider undertaking preparation work in the future. Eleven (34%) schools reported that they have used the Show Racism the Red Card materials after the visit. They have been utilised in a variety of ways but principally as part of PHSE work. ‘Yes, we have used them with the y6s as part of field work, in PSHE and to prompt discussions before doing a piece of writing.’ (School) Six schools reported that they had not used the materials as yet but intended to do so in the future. ‘Not yet, I do want to. They’ll be used in PSHE work and maybe as inspiration for artwork.’ (School) Nine (30%) schools had not used the materials after the visit either because they are using alternative materials or do not want to overuse the resource. Time and the ability to prioritise time for this issue is the main barrier schools identified to further use of the materials. One possible suggestion for the future is the development of specific materials for preparation and follow-up work. This would make it clear to the schools that preparation work is seen as being beneficial and would facilitate the schools to undertake follow up work. 11 Finally respondents were asked to comment on the impact that the loss of the Show Racism the Red Card workshops would have upon the school. The majority of the schools (24 or 75%) felt that the loss of input from Show Racism the Red Card would be detrimental to the work of the school and have a negative impact on their ability to address the issue in an interesting, engaging or even adequate manner. ‘It would be more difficult for the school. Show Racism the Red Card has a strong identifiable theme of anti racism. Discussing the issue without Show Racism the Red Card can be tricky. I think if left to themselves schools would go for a softer multi cultural approach rather than an overt anti-racist stance.’ (School) ‘We’d struggle to do the same kind of thing. The people who delivered it had real presence and the event stuck in the children's minds.’ (School) ‘It would make the school a less rich environment. Visitors and specialists make it more interesting. It sends the message that other people think this outside of school. It creates good links with sport and other life rather than just a school lesson.’ (School) Eight schools felt that they could adequately cover the loss of input either because it is seen as an optional extra to the work they already do or because it could be covered internally or by accessing another organisation. ‘Not a huge negative impact. We see it as an additional benefit to vast amount of work school already does, but not having the session would obviously reduce this and in that respect we would lose some of the ‘richness.’ (School) ‘We would deliver things anyway but I don’t think it would be as fun. I think the message would lose some impact.’ (School) 12 4.6 Effectiveness of the DVDs and Teaching Materials As part of their feedback questionnaire teachers are asked to comment on the effectiveness of the materials. Of the 70 respondents 64 (92%) rated the resource as excellent or very good. Similarly in the collated feedback from 1194 young people 967 (81%) rated the resource as excellent or very good. In the survey of partner agencies five of the 37 respondents specifically mentioned the teaching materials as being a strength of the organisation, as did four of the 30 respondents in the survey of schools. ‘The educational resources are excellent and the staff more than competent.’ (Partner Agency) In the survey of schools 26 of the 32 (81%) respondents described the resources as effective. ‘Very effective, the black footballers on the DVD who spoke about their experiences of racism and how it made them feel was particularly effective.’ (School) ‘The DVD and material seemed very engaging.’ (School) ‘Really good for KS2. It was pitched really well. The material was fast paced and the children were always interested.’ (School) Where reservations were expressed they were in relation to the predominance of male role models within the DVDs. ‘An area for improvement would be more use of women as role models.’ (School) ‘Male dominated with few positive female role models. DVD very football dominated.’ (Partner Agency) The DVDs do feature inputs from young women and Gabby Logan co-presents the SRtRC DVD with Gary Lineker. With respect to role models whist the materials contain interviews with female international players and the current manager of the England Women’s Team these inputs occur towards the end of the DVD and seem to lack impact. SHOW RACISM THE RED CARD A small number of respondents expressed the feeling that the DVDs do not relate to everyday life in a strong enough manner and rely too heavily on examples drawn from football. ‘There was lots of information about race relations in sport but more about race relations in the wider world would have been helpful. I don’t think it was dealt with enough.’ (School) ‘Use of high profile sports personalities - but examples don’t necessarily relate to everyday life.’ (Partner Agency) Ultimately these observations are a question of balance and opinion but as part of its ongoing review of materials and content Show Racism the Red Card should take these views into account. 4.7 Community Festivals and Sporting Club Events The survey of partner agencies generated a range of views regarding the effectiveness and benefits of the community based events and the educational workshops held in conjunction with the professional sporting clubs. The general feedback regarding the community events was very positive. Several respondents commented on the strength of organisation and the commitment the Community Education Team brought to the events. Respondents were clear that the involvement of Show Racism the Red Card was beneficial to the anti-racist work in being carried out in the areas concerned. The overall view was that without the input of Show Racism the Red Card the events would be more difficult to organise, weaker or not take place at all. Specific aspects of the contribution made by the team and highlighted by the partner agencies included skill, expertise, commitment and the ability to draw in other partners. ‘The Show Racism the Red Card events have proved to be hugely successful in bringing people together across the entire community in a fun and educational setting. The 3 events that I have been involved in have been excellent in getting people from varying communities to talk and enjoy leisure activities together.’ (Partner Agency) NE COMMUNITY EDUCATION PROJECT EVALUATION ‘The organisation of special events is excellent. There are numerous examples of these and they have had the added benefit of encouraging more partnership working in terms of anti-racism activities.’ (Partner Agency) ‘A major event in promoting positive cohesion messages and anti racism would not take place without them - they are valuable partners and we can turn to them when we want some help support with doing anti racist work in communities.’ (Partner Agency) ‘Without Show Racism the Red Card we would not have as wide appeal in our anti-BNP work and would not be able to draw young people as easily to our events.’ (Partner Agency) ‘I think that the current partnership working and planning of events with representatives from a wide variety of sporting and community backgrounds is working extremely well.’ (Partner Agency) One response raised specific issues to do with the organisation of the events. ‘The organisation needs to get better organised when organising large events. They do not give me confidence before the event although there has never been a bad one yet.’ (Partner Agency) Whilst recognising the ability of the organisation to deliver successful events engaging large numbers of people a small number of respondents questioned the effectiveness of the events in relation to impact. ‘The festival was a good fun day but I am not sure there was a very strong message.’ (Partner Agency) ‘There is always a danger that people come along to an event, take part in the tournament and go home without being touched by the antiracist message. In the region, they have also been valiantly working with residents’ associations who have been quite resistant to the anti-racist message. These are all issues that we have to address and I have much admiration for Show Racism the Red Card in going into communities with difficult problems and not simply taking the soft option of working with what we know.’ (Partner Agency) These comments are also an acknowledgement of the Community Education Team’s commitment to work in challenging situations and that the task is a difficult one. 13 In relation to the sporting club events feedback was positive and highlighted the benefits of running events at high profile and engaging locations and accessing current professional sportsmen as speakers and participants in question and answer sessions. The attractiveness of the events to young people enhanced the impact of the educational message. The expansion of the programme to include sports other than football was welcomed. ‘I think any sport which provides positive role models has a part to play in promoting the messages, hence why we are more than happy to support the programme.’ (Partner Agency) ‘The use of ex professional players and current players from the major clubs really grabs the attention of the young people who either look up to them as individuals or the clubs they represent.’ (Partner Agency) ‘Attracting positive role models seems to be key to the programme, which is very powerful with the youngsters involved in the activities. The education days which Show Racism the Red Card put on in conjunction with the professional sports clubs are fantastic, attracting a wide variety of contributors to get the same message across, whom children will not have come into contact with before.’ (Partner Agency) 4.8 Monitoring and Evaluation The organisation collects feedback from young people, teachers and participants at the point of delivery. The information collected and the method of collection depends on the event and varies slightly from case to case. to actively reflect on performance and they are committed to improvement and learning. Comments from some of the funding agencies suggest that they would like to see improvements to the current reporting processes with a greater emphasis on long term outcomes. ‘An improvement would be better reporting back on impact and more passing on of skills, but this would probably require additional resources.’ (Partner Agency) ‘I would also like to see more regular updates. Currently we receive an annual report, but I would prefer more regular updates to see if any immediate issues are arising as it could be that either ourselves or our partners could supplement their work with more targeted interventions.’ (Partner Agency) ‘I would appreciate more feedback on outcomes and outputs and would like to see attitudinal shifts over time.’ (Partner Agency) The diverse nature of the range of activities undertaken by the team and the large number of funding arrangements present a challenge in developing a comprehensive monitoring and evaluation system. Show Racism the Red Card is increasing its funders and expanding into new geographical areas and the demand for reporting will also increase. It is important that the monitoring and evaluation system is able to meet the needs of funders and be capable of generating the information required as cost effectively as possible. The organisation seeks to use qualitative as well as quantitative methods. Where appropriate and possible participants are surveyed before and after sessions to measure changes in opinion and in some situations participants are asked to comment on what they have learned. Some group and schools are revisited after six months to a year to gauge the longer term impact of the work. Information is extracted from the overall material and is then collated and presented in line with the reporting processes for individual funders. The staff team and management use the material 14 SHOW RACISM THE RED CARD 5 CONCLUSIONS 5.1 Summary of Organisational Strengths • Show Racism the Red Card as an organisation has a very strong brand and identity. The linkage of football to racism and its use as an educational tool is a simple but powerful concept seen as effective and easy to understand. The accessibility of the approach has facilitated the development of widespread and active support for the organisation’s education work. • The Community Education Team’s work across the different strands of activity including school based work, educational workshops linked to sports clubs, training and community based events is highly regarded and highly respected across a wide range of agencies and partnerships. The team is highly motivated and undertakes a high level of operational activity. NE COMMUNITY EDUCATION PROJECT EVALUATION • Show Racism the Red Card has a range of strong operational partnerships and relationships which facilitate the work of the Community Education Team. The range of partnerships is diverse including local authorities, schools, trade unions, community organisations, arts projects, universities, professional football clubs and other sporting clubs. • The organisation’s relationship with professional football is a major asset to the Community Education Team. The use of exprofessionals as coaches within the workshops is an effective method of engaging with young people and adds significantly to the ability of the team to provide enjoyable and positive inputs. The use of sporting facilities such as St James’ Park and the Stadium of Light as workshop venues adds value to the community based workshops as does the presence and participation of current players. The ability of the organisation to gain the participation of high profile players in the making of the DVDs is a significant factor in the production of the educational materials. 15 • The Community Education Team has been able to produce strong educational materials. The feedback from schools and partner agencies confirms that they are effective in engaging young people and for wider use. • The staff team is highly motivated and skilled and is highly rated by schools and partner agencies. They are perceived as enthusiastic and committed and the team is generally regarded as reliable in delivery. In terms of interactions with young people both the coaches and facilitators are recognised as being skilled and inclusive in the way they operate. • The Community Education Team receives financial support from a wide range of agencies including the Football Foundation, trade unions, eight different local authorities, grant making trusts and central government. This diversity in funding sources is a strength, providing a strong financial base for the development of the team. 5.2 Summary of Organisational Challenges • The use of football as the main tool in the organisation’s approach to anti-racism education presents two basic challenges. The first relates to the need to ensure that the strength of interest and engagement with football and footballers does not over-ride the anti-racism content and message. The second challenge is ensuring that the focus of engagement is upon racism in society generally and not on racism in football. Both these issues are about balance and the Community Education Team will need to constantly review materials and activity to keep the balance right. • Feedback from partners and schools indicates that the approach used by the team is hugely successful at engaging boys interested in football in the educational process. Feedback also strongly indicates that the coaches and facilitators work hard to be inclusive and that girls and non-sports fans have a positive experience and participate fully in the sessions. It may be that the programme would be enhanced if female role models were 16 included more prominently in the DVDs and educational material. In addition the recruitment of a female coach may be something for the organisation to consider. • Several respondents and partner agencies have the perception that the organisation is working to such a high level of capacity that it is sometimes over stretched. This has led to anxiety about delivery and in a few cases concerns about the quality of the work. Generally this is perceived as the organisation being ‘a victim of its own success.’ The balance between demand and expectation and organisational capacity is something which should be reviewed on a regular basis before it becomes a wider issue. • The diverse range of activities and high number of funders alongside the commitment to review and improve practice presents a challenge in developing a suitable monitoring and evaluation system. The demand for reporting will increase as the organisation increases its funding base and operational reach. The current approach works well for individual events and activities and provides material for reporting. It is less well adapted when examining the programme as a whole as information is not collected in a standardised way and extracting and collating information is time consuming. There is also an expressed desire on the part of some funders to have more outcome focused appraisal. Show Racism the Red Card could usefully review the monitoring and evaluation system to address some of these issues. • As a successful organisation, Show Racism the Red Card and its Community Education Team is constantly seeking and appraising new opportunities and developments. Recent examples include the expansion of the teacher training programme and the engagement with the Prevent agenda through the Police. In order to provide a mechanism for planning and evaluating new opportunities the organisation should consider developing an appropriate planning tool. This process would then feed into the development of existing business plan and long term planning processes. SHOW RACISM THE RED CARD SUBSEQUENT APPENDICES NE COMMUNITY EDUCATION PROJECT EVALUATION 17 APPENDIX 1 Collated information drawn from the Show Racism the Red Card Monitoring and Evaluation System. 1) Primary School Pupil Feedback The following information was collated from post session evaluation forms completed by 1,194 pupils from 39 primary schools from across the region during the period 2008/9. Primary Pupils Collated Feedback Forms 2008/9 Question Excellent Very Good Good Not Good 596 371 211 9 What was your opinion of the SRtRC DVD? (50%) (31%) (18%) (1%) What was your opinion 880 216 84 11 of the football training (74%) (18%) (7%) (1%) What was your opinion 463 447 272 7 of the classroom work? (39%) (37%) (23%) (1%) Yes No Do you know more about racism 1174 20 than at the start of the day? Overall Did You enjoy the 1189 5 workshop? Poor 7 (1%) 3 (0%) 5 (0%) Total 1194 1194 1194 Total 1194 1194 Collated Primary Pupils Feedback Forms 2008/9 Chart 1 Collated Primary Pupils Feedback Forms 2008/9 18 SHOW RACISM THE RED CARD 2) Teacher Feedback Collated feedback from 70 teachers (primary and secondary) collected during 2008/9 Collated Feedback – Teachers – 2008/9 Excellent Very Good Good Fair Poor Total Overall opinion of the day 50 (71%) 18 (26%) 1 (1%) 1 (1%) 0 70 How did you find the SRtRC DVDs? 44 (63%) 20 (29%) 5 (7%) 1 (1%) 0 70 Strongly Agree Agree Not Sure Disagree Strongly Disagree Do you think the day has improved the young people’s understanding of racism? 52 (74%) 17 (25%) 1 (1%) 0 0 70 Do you feel the day has encouraged discussion about racism between pupils? 50 (71%) 20 (29%) 0 0 0 70 Do you think the day will have lasting impact on the young people who took part? 32 (46%) 31 (44%) 7 (10%) 0 0 70 Feedback from Teachers 2008-9 Chart 1 Feedback from Teachers 2008-9 Chart 2 NE COMMUNITY EDUCATION PROJECT EVALUATION 19 3) Feedback from community events Attendees at the Ferryhill Is United Community Festival held on the 6th July 2008 and the STAR Festival (Sunderland Together Against Racism) held on the 10th August 2008 were asked to complete feedback forms when they were leaving. 54 forms were collected from Ferryhill and 64 forms were filled in at the STAR festival. Ferryhill is United Sunderland Together Against Racism Totals Collated Feedback from Community Events 2008/9 Excellent Very Good Good Fair 30 (55%) 9 (17%) 15 (28%) 0 35 (55%) 24 (37%) 5 (8%) 65 (55%) 33 (28%) 20 (17%) 0 Poor 0 Total 54 0 64 118 APPENDIX 2 Survey of key contacts, funders and partner agencies Show Racism the Red Card provided the details of 51 contacts within local authorities and other partner agencies. Telephone contact was made with the seven key contacts responsible for the oversight of grants from local authorities. Other respondents were surveyed via email. There were 37 (73%) completed responses, three contacts felt that it was inappropriate given their role to respond and there were 11 who did not return completed questionnaires. The survey was conducted during August 2009. Questionnaire and summary of responses 1. What is your organisation’s main objective when funding Show Racism the Red Card? (Key funding contact only) Six of the seven local authorities responded to this part of the survey. The main purpose in funding the programmes varies from authority to authority with three of them specifically highlighting the work in schools as being a primary consideration with the other three having a wider interest across the community as a whole. 3. The aim of Show Racism the Red Card’s Community Education Programme is to educate young people about racism through a range activities with a central football related theme. How effective do you think this approach is? Of the 37 respondents 27 (73%) stated that they felt the approach was successful. A further 6 felt it was generally effective but expressed some reservations about its ability to always engage with girls and young people not interested in sport. Some expressed the view that the DVDs were male dominated. Three respondents were neutral about its effectiveness and one respondent expressed the view that the use of sports personalities didn’t relate to everyday life. 2. To what extent would you say that the objective is achieved? (Key funding contact only) All six respondents were positive about the work of the organisation in general terms. One identified issues to do with the commissioning process and communication and a second identified the desire for more outcome focused evaluation and feedback. 20 SHOW RACISM THE RED CARD 4. From your understanding of the organisation what would you say were its principal strengths? a) Quality of staffing including the classroom facilitators and football coaches was seen as an organisational strength by 18 respondents (49%) with a variety of attributes being identified including primarily enthusiasm and commitment. b) The overall quality of the programme was cited by 15 respondents (41%) with 5 highlighting the quality of the teaching materials in particular. c) The ability to access current and exprofessional football players as role models was identified as a strength by 13 respondents (35%). d) The strength of the brand with its simplicity and direct message was highlighted as a strength by 8 respondents (22%). e) Six (16%) of respondents identified the agency’s high level of community engagement and connection to grassroots organisations as a strength. Other issues / qualities highlighted were, partnership working (4), strong leadership (2), volunteers (2), political support (1) and access to funding (1). 5. Again from your understanding and experience of the project can you identify any areas of weakness? Organisational capacity Don’t engage girls or young people not interested in sport None Uncertainty on long term impact on attitude / behaviour Strategic positioning / funding Duplication with Kick It and other football related foundations Could make more / better use of volunteering Lack of school engagement beyond sessions Weakness of promotion of events NE COMMUNITY EDUCATION PROJECT EVALUATION 7 19% 6 6 16% 16% 4 3 11% 8% 6. In terms of its operations and activities could you suggest any areas for improvement? None Capacity building in schools / teacher training Better feedback to funders Utilise a wider range of sports 10 4 2 2 16 other suggestions were made:• Adopt rewind anti-racism approach. • Work more with youth sector. • More involvement with grassroots football. • Volunteer development. • Work in partnership with other training providers. • Develop multiple events programme. • More funding. • Link in with CONTEST agenda. • Improve technical quality of some materials. • More press coverage of events. • More work with trade unions. • Link with live music events. • Use current footballers more. • Target workshops to specific areas where there are issues. • Use women as role models. • More schools work generally. 7. In your opinion what would be the impact for your organisation if Show Racism the Red card was unable to fulfil its current role? 27 respondents (73%) indicated that the loss of inputs from Show Racism the Red Card would have a major detrimental impact on their organisation or area of work and / or leave a significant gap in provision. Four respondents felt that it would have no impact or could be covered in some other way and six respondents did not complete the question. 8. Any other comments 2 5% 2 5% 2 5% 1 2% 23 respondents added comments supporting the work of the organisation and generally being positive about future work. Other comments included specific suggestions such as SRtRC involvement in policy development, specific help from football clubs and more work with trade unions. One respondent suggested a possible amalgamation with Kick It. 21 Partner and Funding Agencies - Respondents Ed Cook Football First Coordinator Sunderland AFC Foundation Rob Atkin Football Development Manager Newcastle United Foundation Melanie Magee Community Manager Newcastle Falcons RFC Ginette Tweddle Deputy Programme Manager Help and Hope Project Children’s Society Ciara Van Vogt Volunteer Development Manager VODA Robin Fry Youth Volunteer Dev. Manager VODA Jane Davies Lecturer Sunderland University Leesa Bennison Hate Crime Officer Durham Police Kenny MacPherson PREVENT Officer Durham Police Marsha Garrett BME Officer Redcar and Cleveland Youth Service Tommy Caulker Owner / Manager World HQ Amer Sheikh The Word Cllr David Faulkner Newcastle City Council Mark Ellis Policy and Strategy Officer Newcastle City Council Simon Wood Community Development Officer Newcastle City Council Ann Wood Facilities Manager Newcastle City Council Viki Angel ARCH Project Newcastle City Council Simon Underwood Head of Social Inclusion Newcastle City Council Felicity Shoesmith Community and Voluntary Sector Lead Officer North Tyneside MBC Tony Rice Assistant Manager Burnside Centre North Tyneside MBC Joyce McCarty School Improvement Officer North Tyneside MBC Nitin Shulka Diversity Officer Gateshead MBC Michael Glickman EMTAS Project Coordinator Gateshead MBC Andrew Goulding Area Youth and Community Worker Gateshead MBC John Braithwaite EMTRAS Manager South Tyneside MBC Dave Owen Team Leader Community Safety South Tyneside Les Platt Senior Policy Officer Sunderland City Council Jenny Scott Policy Officer Sunderland City Council Lynne Gregory Head of Learning Support Service Durham CC Sue Green Team Leader – Race Equality and Gypsy, Roma and Traveller Achievement Durham CC Sharon Barker Voluntary Sector Liaison Officer Middlesbrough MBC Cllr David Budd Middlesbrough MBC Jessica Cleghorn Unison Norma Stephenson Unison Ben Sellers Northern TUC Simon Kennedy NASUWT Pat McCourt Unite 22 SHOW RACISM THE RED CARD APPENDIX 3 Summary Results - Survey of Schools Working from a list of contacts provided by Show Racism the Red Card schools were contacted by telephone and asked if they would be willing to undertake a semi-structured interview either at the time or by arranging a convenient call back. Thirty-two schools from a total of 289 who had hosted a session from the Community Education Team during 2008/9 were surveyed in this way representing a sample size of 11%. The survey was carried out in September 2009. Questionnaire and summary of responses 1. The aim of Show Racism the Red Card’s Community Education Programme is to educate young people about racism through a range activities with a central football related theme. How effective do you think this approach is? The overwhelming response (29 out of the 32 schools (90%)) was that the approach was highly effective. Feedback from the teachers suggested that the approach was particularly effective with boys and those interested in football but that it also resulted in girls and those not interested in sport being successfully engaged. A small number of respondents believed that girls were not as fully engaged and one respondent felt that it would not work effectively with groups where there was not a strong interest in football. 2. From the point of view of the school – what are the main benefits of being able to access the sessions from Show Racism the Red Card? Teachers were able to identify a number of benefits, the most common one being the advantage of drawing in an outside agency with specific resources and expertise to undertake the work. This was seen as helping to raise the profile of the issue within school and providing a general stimulus to the school and community. Many highlighted the link with professional footballers and the presence of black role models as being positive in raising enthusiasm and giving the issues importance. NE COMMUNITY EDUCATION PROJECT EVALUATION Main benefits for School in accessing SRtRC Provides outside input and expertise 18 56% Ex-footballers create interest and enthusiasm 8 25% Excellent resources 4 13% Gives issue high importance 3 9% 3. From your understanding of the organisation what would you say were its principal strengths? The schools were generally extremely positive about the programme and highlighted the quality and strength of the staff, both coaches and facilitators. Mention was made of their ability to engage with the young people, their commitment and belief in the work, their presentational skills and overall professionalism. Generally the schools were positive about the level of organisation which was seen as very high, the link with professional football was highlighted as was the quality of the resources. Show Racism the Red Card Strengths Quality of Staff 19 Good organisation 12 Link with pro-football 8 Quality of resources 4 59% 38% 25% 13% 4. Again from your understanding and experience of the project can you identify any areas of weakness? Twenty (66%) of the schools surveyed could not identify any areas of weakness. Three schools picked up on specific issues where arrangements had broken down and organisation / communication had not worked effectively resulting in the session not being what had been expected. Two schools made suggestions about the quality of some of the materials and two others raised the issue of engagement with girls. 23 5. In your opinion how effective would you say the teaching pack and DVD produced by Show Racism the Red Card are in engaging young people? Feedback from the schools on the quality of the DVDs and their ability to engage the pupils on the theme of racism was extremely good. 26 responses (81%) of the total said that the materials were highly effective and none of the schools had any negative feedback. 6. In terms of its materials and operation could you suggest any areas for improvement? Twenty of the schools (62%) did not have any suggestions for improvement. Some of the others made specific suggestions listed below: • Specific follow-up activities / materials for schools x 2 • Longer sessions • More role play • Specific introductory materials • An assembly session to finish off the day • Female role models / input x 2 • More content about racism in society and not just in football • More interactive sessions • Expansion into other sports – cricket • More work on conflict between ethnic groups needed within the context of the school. Four schools stated that they would consider undertaking preparation work in the future. 8. Show Racism the Red Card leave copies of the teaching materials and DVD with the school. Has the school been able to utilise them in any way? If yes- how have they been used. If no - what would you say were the barriers to their use? Eleven schools (34%) reported that they have used the Show Racism the Red Card materials after the visit. They have been utilised in a variety of ways but principally as part of PHSE work. Six schools reported that they had not used the materials as yet but intended to do so in the future. Nine schools had not used the materials after the visit either because they are using alternative materials or do not want to overuse the resource. Time and the ability to prioritise time for this issue is the main barrier schools have identified as a barrier to further use of the materials. 9. In reflecting on the session carried out by Show Racism the Red Card are you able to identify any lasting impact? Can you identify any lasting impact? Yes 17 Hard to judge 12 No / Not Really 3 53% 38% 9% 7. Did you have the opportunity to do any preparation work with the young people before the session? If yes - what preparation did you do and what was the effect? If no why was that and do you think it made a difference to the session? Overall the respondents found it difficult to identify specific evidence of lasting impact of the sessions. However 17 respondents felt that they could see a change in attitude, behaviour and / or awareness as a result of the workshop sessions. 14 of the schools (43%) had undertaken work in preparation for the visit. Some schools watched the DVD in advance and others looked at the poster campaign and encouraged students to do some research. Several of the schools did preliminary discussion on the subject area and generally laid the groundwork for the visit. The schools that did undertake the work in advance felt that it meant that the event went smoother than would otherwise have been the case. 10. In your opinion what would be the impact on the School if Show Racism the Red Card was unable to fulfil its current role? 13 schools (41%) reported that they did not do specific preparation either because of time constraints or because it was felt not to be 24 The majority of the schools (24 or 75%) felt that the loss of input from SRtRC would be detrimental to the work of the school and have a negative impact on their ability to address the issue in an interesting, engaging or even adequate manner. Eight schools felt that they could adequately cover the loss of input either because it is seen as an optional extra to the work they already do or because it could be covered internally or by accessing another organisation. SHOW RACISM THE RED CARD SHOW RACISM THE RED CARD MAJOR SPONSORS: PO Box 141, Whitley Bay, Tyne & Wear NE26 3YH. Tel: 0191 257 8519. Fax: 0191 257 8541. email: info@theredcard.org www.theredcard.org