The evaluation of NOISE allows us to empirically understand the processes and outcomes associated with NOISE, using both quantitative and qualitative research methods.
Quantitative Methods:
All fellows are invited to participate in a pre-participation survey at the beginning of their fellowship in September and a post-participation survey towards the end of their fellowship in March-April. The survey measures academic engagement, civic engagement, and psycho-social well-being.
Survey questionnaires:
Youth Fellows Pre-Survey
Youth Fellows Post-Survey
Qualitative Methods:
Fellows are invited to participate in focus groups and individual in-depth interviews towards the end of their fellowship in March. Written reflections from pods are analyzed using narrative techniques and the social action projects provide documentary material that are also a source of research data. Throughout the academic year, we use developmental evaluation – micro-evaluation – to elicit ongoing feedback and to reflect on and improve our practices and processes.
Focus Group Interview Guides
Youth Focus Group Guide
BSWMSW Focus Group Guide
Emerging Evaluation Findings from Focus Group Interviews
What is working?
Youth are engaged: Importance of youth voice, agency, leadership, asset recognition and development, interactive learning, & incentives for participation
“The ways that you feel welcomed and the ways that you can make your voice heard and put your ideas together with other people, making a project that you believe will become successful and doing it together and getting through it.”
“They weren’t trying to run the show or anything…they made us feel like what we said was actually important”
Psychosocial well-being: NOISE fellows develop positive relationships with each other & express feelings of belonging. Multi-directional inter-generational mentoring is working.
“It helped in making more friends besides making friends in Emery”; “I like being with my pod members”; “I feel more comfortable to speak my ideas”; “they don’t treat you like you’re just a pod member, you’re their friend“ (Gr. 9)
“Through NOISE, you just learn how to belong and help other people have that sense of belonging too” (Gr. 12)
“I’ve become more involved with all the faculty and everything, and I have been talking with the professors here in social work, it’s like I’m joining more clubs and kind of more involved and taking advantage of everything”; “it’s been about confidence”(SSW)
“It wasn’t a hierarchy kind of group… it wasn’t us mentoring the youth telling tell….we all really contributed ideas and collaborated so that part of it was really fun” (SSW)
Expanded learning opportunities: academic engagement (group work, collaboration, project development)
“I know I needed that like extra push in order to get through something. And like I wouldn’t like be like interacting with university students or like other Grade 9’s or like alumni like on my own, right?”
“In my social work classes, it’s all theory and when I’m here in NOISE, um… I can actually see the theory that’s practiced” (SSW)
“I am getting more serious in class”; “I learned different methods to communicate a message” (Gr. 9)
“It’s like really beneficial all of the grades. I think we all need something like this to keep ‘em on track and focused. And actually aware ’cause like if I had this program in grade 9, I’d actually kinda like it ’cause I would need something like this to get me more involved”
Civic engagement – organizing and action, learning that change is possible
“There were some like problems around the school and I used to just accept them as they were…but after you join NOISE, you kinda think like oh wait, that’s not how a school is supposed to be like.” (Gr. 12)
“It like reinforced that it is something that I can do and push me outside of my comfort zone” (SSW)
“Something that stands out is just the importance of coming together and still working it out because, there were times we didn’t really know what was going on but…we still stuck together, we still came up with something great, despite all that” (SSW)
“I think, yes, the NOISE project, there were obviously good things…it just gives you hope that yes gathering community is not that hard” (SSW)
“It wasn’t a hierarchy kind of group… it wasn’t us mentoring the youth telling tell….we all really contributed ideas and collaborated so that part of it was really fun” (SSW)
What can be improved?
Communication – connecting with community outside the university
It would be good to have “outside people who had experience in community development or working with youth come and give us some suggestions” (SSW)
“Different schools. And like having a lot of folks from university like instead of that, like having like two people from university and then like two members of the community somehow, like somebody from the community center or like people who are actually involved in the community so you can get like different perspectives. That would be cool.” (youth)
Communication – connection with each other
“More of the discussion groups [throughout] the year” (youth)
“And if we were updated about what everyone was doing, we could have been more involved” (SSW)
Program structure – clarify program purpose and; operationalization
“It’s been all over the place and there is not a lot of fluidity in our meetings, and I feel like…that’s reflected in like our students’ participation as well, because they do not really know what to expect, because we also do not know what we’re doing” (SSW)
“I thought I would be doing something [different] but to sit around a table, I mean, I already do that in school so I was really disappointed.” (SSW)
Program structure (experiential component)
“But just talking around the table it’s just not working for thirteen and fourteen, fifteen year old… And that’s why we couldn’t build rapport with them” (SSW)
Q: “If you could make noise better what would you do?”
A: “Hands-on activities” (Gr. 9)
A: “Pod assignment based on schedule availability”
NOISE needs to do a better job of communicating the core concepts
“What’s a vital sign?” (Gr. 9)
“I’d ask the GA, “what is NOISE?” and she was hesitant about answering …” (SSW)
“For us not to really understand the project makes it difficult to help [youth] become interested in the project and understand what’s going on…” (SSW)
It would be good to have “outside people who had experience in community development or working with youth come and give us some suggestions” (SSW)
How to find a balance between experiential education and reflective /critical dialogue? (e.g. approach, incentives, documentation?)
Coming Soon: Our 2012-2013 Annual Report with a summary of of the evaluation findings of NOISE 1.0