There is a documented need to improve the level of scientific literacy among the K-12 population in the United States (e.g., Fleischman et al. 2010; National Center for Education Statistics 2011). To achieve this improvement will require not only effective classroom-based instruction, but opportunities for youth to learn science in nonformal settings that occur during out-of-school time (Bell et al. 2009).

The 4-H Youth Development Program is one nonformal youth education organization that has the potential to help address this area of concern (Kress et al. 2008). 4-H has developed into one of the largest youth organizations in the world. Administered through the national Cooperative Extension System, 4-H offers curriculum projects and activities through county-based programs in all 50 states as well as in the District of Columbia. Internationally, 4-H is available to youth through programs in American Samoa, Guam, the Northern Mariana Islands, Micronesia, Puerto Rico, and the United States Virgin Islands, as well as at United States military installations around the world. Approximately half of all 4-H curriculum offerings are science related (USDA 2010) and program delivery emphasizes pedagogical strategies that include hands-on inquiry and experiential learning (Enfield et al. 2007), approaches that have been shown to be effective in advancing scientific literacy (Minner et al. 2010). Furthermore, recent research has demonstrated positive impacts related to youth participation in 4-H programming in terms of their interest and engagement in science (Heck 2009; Heck et al. 2012; Mielke et al. 2010).

Adult volunteers serve as the nonformal educators in 4-H; they lead curriculum projects and activities with youth (Stedman and Rudd 2006). Over 400,000 individuals function in this capacity nationally, and approximately 17,000 in California (USDA 2010). To be successful in this role, 4-H volunteers must engage in effective professional development opportunities (Hoover and Connor 2001). To this end, it has been suggested that episodic workshops that are used most frequently to train 4-H volunteers are inadequate to build their teaching and knowledge capacity in science. Instead, alternative models of professional development that are of longer duration should be explored in order to provide volunteers with sufficient opportunities to develop the knowledge and skills necessary to effectively facilitate science projects and activities (Barker et al. 2009).

The lesson study approach

One alternate approach to professional development for 4-H volunteers who lead science projects and activities is lesson study (Smith 2008), a model that has long been the principal professional development model in Japan (Lewis et al. 2004). Grounded in constructivist thinking, whereby new experiences draw upon earlier experiences and modify them in some way (Dewey 1933), lesson study involves groups of educators who work collaboratively to formulate learner outcome goals associated with each specific lesson; plan and implement the lesson with their target audiences; collect and analyze data on the desired results; and use outcomes to improve the lesson as well as subsequent instruction (Lewis 2002). Taken collectively, these steps set lesson study apart from traditional approaches to professional development for educators (e.g., one-shot workshops or seminars that lack sustained support or followup) in that the work is done “with” or “by” educators rather than being done “on” or “to” them (Loucks-Horsley et al. 2003).

As a professional development strategy in school-based settings, lesson study has gained increasing recognition and use in the United States in recent years (Lewis and Baker 2010). In-service and pre-service teachers from different disciplines, including history, language arts, math, science, and social studies, have made effective use of the process (Blum et al. 2005; Marble 2006; Ogden et al. 2008; Rock and Wilson 2005; Sibbald 2009). Specific to science, educators engaged in lesson study have been shown to be effective in developing and implementing science lessons (Marble 2006; Mutch-Jones et al. 2012). Furthermore, lesson study exhibits certain characteristics of effective professional development in science as outlined in the literature (e.g., Guskey and Yoon 2009; Loucks-Horsley et al. 2003; Penuel et al. 2007): It occurs within authentic contexts over an extended duration; it fosters communication among participants; it involves active learning where participants challenge their existing ideas about teaching and learning; and it develops new knowledge and skills among educators, informed by data collected through implementation of lessons (see page 47 ).

Pilot study in 4-H

This small-scale pilot study used a sequential explanatory mixed-methods design (Creswell and Plano Clark 2007) to investigate the use of lesson study on 4-H volunteers' understanding and use of inquiry strategies and subject matter knowledge related to the Youth Development through Veterinary Science curriculum (Smith et al. 2009). The investigation occurred within the context of 4-H club-based settings in rural California. In mixed-methods research, both quantitative and qualitative data are collected, analyzed, and mixed within an investigation (Creswell and Plano Clark 2007). The first phase of this inquiry was quantitative, using survey data to measure the influence of lesson study on the pedagogical and subject matter knowledge of the participating 4-H volunteers. The second phase of the investigation was qualitative, drawing upon focus group interview data from study participants to expand upon the survey outcomes.

Participants.

Study participants included 16 4-H adult volunteers from three county 4-H programs in California: Lake, Marin, and Siskiyou. Fifteen of the participants (94%) were female and one (6%) was male. Fourteen of the participants (88%) resided in rural areas and two (12%) lived in small towns. All participants had a minimum of 1 year of experience leading 4-H animal or veterinary science projects with 4-H youth members. Specifically, nine participants (56%) had between 1 and 5 years of experience as 4-H volunteers, three (19%) had 6 to 10 years of experience, three (19%) had 11 to 15 years of experience, and one (6%) had more than 16 years of experience serving in this capacity.

The 4-H volunteers were subdivided into three lesson study groups based on county of residence. The lesson study groups in Lake and Marin counties had five members each; the lesson study group in Siskiyou County had six members.

Process and project materials.

Simply organizing lesson study groups is not enough to make them effective: Participating educators also require “models, tools, and structures from which to work” (Wiburg and Brown 2007). Thus, 4-H volunteers who participated in this inquiry were provided an overview of the lesson study process and received necessary support and materials. Instruction on the lesson study process included one face-to-face workshop as well as follow-up conference calls with each lesson study group. Materials supplied to participating 4-H volunteers included resources that outlined the procedures of the lesson study process (Lewis 2002) and additional information on inquiry-based teaching. Each participant also received a copy of the Youth Development through Veterinary Science curriculum (Smith et al. 2009), #which consists of 11 inquiry-based activity modules.

Project implementation.

The lesson study groups in this investigation met on a regular basis over a period of several months. The lesson study group in Lake County met seven times at 3-week intervals (four meetings were in person, three others were held via teleconference), the lesson study group in Marin County held five face-to-face meetings at 3-week intervals, and the lesson study group in Siskiyou County met seven times in person at 3-week intervals. During this period the 4-H volunteers also held regular meetings with their 4-H youth groups to implement the veterinary science activities. Meetings with 4-H youth in all counties were held, on average, every 3 to 4 weeks; one or two curriculum activities were implemented during each meeting.

Agendas for lesson study group meetings were set by the participating 4-H volunteers in each county. The goals for these meetings were to review and discuss curriculum activities, plan lessons, and reflect on data collected during previous curriculum implementations (e.g., observations and authentic work from 4-H youth). Curriculum revisions and adjustments to instructional strategies were based on volunteers' interpretation of their data and applied, as appropriate, to the planning and implementation of subsequent activities.

Data sources

Surveys.

Two retrospective surveys were administered at the completion of the lesson study intervention: the Use of Inquiry Practices Survey and the Veterinary Science Content Knowledge Survey. These instruments were adapted from self-report questionnaires used in previous studies (Gejda and LaRocco 2006; Smith and Meehan 2007). The Use of Inquiry Practices Survey included 20 Likert-type questions that aligned with indicators of respondents' understanding and use of inquiry-based instruction. Three categorical questions that requested demographic information were also included. The Veterinary Science Content Knowledge Survey included 11 Likert-type questions that measured veterinary science content knowledge as it pertained to the Youth Development through Veterinary Science curriculum (Smith et al. 2009) and three categorical questions that were demographic in nature.

A retrospective design was chosen for both surveys because it can mitigate the risk of response-shift bias, a threat to internal validity that can occur when using a pre-/post-survey design. Specifically, response-shift bias can occur when participants have limited knowledge or skills to respond accurately to questions asked on a pre-survey and overestimate their abilities in advance of participation in a program (Pratt et al. 2000). In such instances, pre-test/post-test comparisons can be misleading because participants' frame of reference has changed subsequent to program intervention (Pratt et al. 2000).

The correlation of related survey items on both instruments was tested using item analysis. The resulting Cronbach alpha scores were 0.7495 and 0.7358 for the Use of Inquiry Practices Survey and the Veterinary Science Content Knowledge Survey, respectively, verifying the internal consistency of the survey items in relation to their constructs.

Focus group interviews.

The second phase of this investigation was qualitative, drawing upon focus group interview data from study participants to assist in the interpretation and explanation of survey outcomes. Focus group interviews are appropriate for explanatory studies and are frequently used to supplement quantitative data (Hatch 2002).

Two focus group interviews were conducted separately with each lesson study group at the conclusion of the investigation. The first interview addressed the lesson study process as it pertained to participants' understanding and use of inquiry-based methods; the second concentrated on perceived changes in participants' veterinary science content knowledge. All interviews were recorded and transcribed.

Data analysis

Phase 1: Quantitative data analysis.

Survey data were analyzed using a repeated measures general linear model (GLM) (Hill and Lewicki 2007) and MiniTab software. The factors in the model for this analysis were time and county of residence. Data analysis examined the interactions of these factors on the dependent variable, which was the overall score for each survey. A significance level of P ≤ 0.05 was established for all analyses.

Use of a repeated measures GLM requires that the sample distribution be normal and that it exhibit homogeneity of variance (i.e., the variance within populations must be the same, which is important for meaningful interpretation of data). A Goodness of Fit test (Gravetter and Wallnau 2008) was used to determine if data were normally distributed. In situations where the data were not distributed normally, a Box-Cox Transformation (Hill and Lewicki 2007) was applied. Homogeneity of variance for the data sets was determined using the Bartlett's Test, a measure used to determine whether multiple samples are from populations with equal variances (Snedecor and Cochran 1983).

Phase 2: Qualitative data analysis.

The long-table approach (Krueger and Casey 2000) was used to categorize results germane to the investigation through the development and execution of an initial coding scheme (table 1). Data were then analyzed inductively using the constant comparison method (Hatch 2002), whereby themes emerged from the coded data, as opposed to assigning a priori categories (Patton 1990). Where relevant, some codes were combined based on generalizations during data analysis (Hatch 2002). The interpretation of focus group interview data placed an emphasis on how frequently codes occurred (Sandelowski et al. 2009).

Results

Quantitative findings.

Scores on the Use of Inquiry Practices Survey improved significantly. Data analysis revealed a significant effect of time. The resulting Least Squares Means (LSM) and Standard Error (SEM) for post-survey scores were n = 16, LSM = 3.32, SEM = 0.74. The Least Squares Means and Standard Error for pre-survey scores were n = 16, LSM = 2.92, SEM = 0.74. The difference between post-survey and pre-survey scores was significant: GLM F(1,31) = 13.71; P < 0.01. There was no interaction of county of residence on the dependent variable: GLM F(2,31) = 2.50; P = 0.10.

Scores on the Veterinary Science Content Knowledge Survey also improved significantly. Analysis of the data revealed a significant effect of time. The resulting Least Squares Means and Standard Error for post-survey scores were n = 16, LSM = 3.65, SEM = 0.13. The Least Squares Means and Standard Error for pre-survey scores were n = 16, LSM = 3.09, SEM = 0.13. The difference between post-survey and pre-survey scores was significant: GLM F(1,31) = 9.87; P < 0.01. There was no interaction of county of residence on the dependent variable: GLM F(2,31) = 2.27; P = 0.12.

Qualitative findings.

Emergent themes from the analysis of focus group interviews were based on highest frequency of occurrence of qualitative codes. These themes included understanding and use of inquiry, duration of professional development experience, reflective practice, data-driven decision making, and collective benefits to 4-H volunteers.

Theme: Understanding and use of inquiry.

Several lesson study group members for whom inquiry-based teaching was a new strategy revealed that the lesson study experience helped them understand the process and apply it to their practice. For study participants who indicated they had some prior knowledge of inquiry, lesson study served to reinforce their understanding and increased their confidence to use it. Focus group interview responses included

I would say that it really helped me because I'm kind of a “tell-you-how-to-do-it person,” … so I think it helped me tremendously to find out what inquiry-based learning was and to help me be way open to it now.

I felt that participating in the lesson study group kind of crystallized and fine-tuned a teaching technique I knew was available, but I myself didn't feel like I had the confidence to commit to it because I didn't understand it as well as I could have.

A number of focus group responses were congruent with Use of Inquiry Practices Survey questions and exhibited study participants' knowledge and application of specific strategies associated with inquiry, such as open-ended questioning. For example, interview responses from 4-H volunteers provided numerous references to the benefits they saw in using open-ended questions during curriculum implementations. In particular, study participants noted that they observed how open-ended questions engaged their youth audiences in the learning process and promoted independent thinking. Representative focus group interview responses included

The open-ended questioning … allowed the [youth] to discover answers and discover questions.

The open-ended questions were good because they made [our 4-H members] think.

Another key element of inquiry included on the Use of Inquiry Practices Survey was learner-centered instruction. Participating 4-H volunteers commented several times during focus groups about how they used facilitation strategies that focused more on youth exploration than on providing direct instruction. Among the participants' responses were

I try to incorporate hands-on [activities] for the kids to explore and ask questions and have them formulate the answers and have me guide them.

[Using inquiry has] completely flipped the way we'll run our meetings. [We are] seeing how letting [youth] explore on their own can be a better [experience] than having someone stand up there and throw facts and figures at you.

The application of knowledge was also addressed through the Use of Inquiry Practices Survey and emerged as a theme from the focus group interviews. Interview data revealed how, through the lesson study process, several of the 4-H volunteers had incorporated knowledge application into curriculum activities they were leading with 4-H youth, while others had made concrete plans for their youth participants to apply their learning in real-life settings. Interview responses included

Our goal for the [4-H] group was that when we finished this the kids would be able to call the vet and answer the first five or six questions that the vet would ask them. That was our goal.

[Our lesson study group] talked about visiting a vet clinic where you can learn more by observing the actual live animals. And [our youth] could apply what they've learned to actual live animals.

Theme: Duration.

When discussing the organization and functioning of their lesson study groups, 4-H volunteers commented numerous times on the benefits of having a professional development opportunity that included multiple meetings over an extended period. Participants indicated that meeting regularly was a challenge, but worth the effort. Specifically, volunteers expressed that these meetings helped them improve their time management and meet their goals, and also provided them time to review and discuss curriculum materials, reflect on their prior work with youth and make instructional modifications they thought necessary. Specific comments related to the duration included

Regular group meetings helped a lot. Absolutely! Sometimes the logistics of [meeting] are difficult. It is a big time commitment, but I think in the long run it's definitely worth the time put into it.

We [volunteers] talk about the lack of time for planning when using a new curriculum. Lesson study helped. Meeting every 3 weeks or so helped with preparation and lesson planning.

Theme: Reflective practice.

Having opportunities to reflect on their practice through interactions with other 4-H volunteers during lesson study group meetings was another theme that emerged from focus group data. Systematic reflection helped participants target strategies to improve the learning of their youth audiences by discussing data they had collected and making modifications to their practice and curriculum activities prior to subsequent implementations. Additionally, some volunteers commented that through reflection they had begun to contemplate changes to their teaching practices and organizational strategies with respect to other 4-H projects they lead. Relevant quotes included

As far as how the lessons were presented, if we thought that the kids had not picked up on the curriculum the way we presented it we would bounce ideas off on how we could make it better for those of us who maybe had not gotten to the level that other group members had.

I think that we all had things to bring back and improve upon every time that we met because we were talking about our process, working with different ages and stages vs. just the curriculum.

Theme: Data-driven decisions.

Data-driven decision making also emerged as a theme from focus group interviews. Participating 4-H volunteers collected a variety of types of data (e.g., observations, authentic youth work) and used those data to inform their practice. The lesson study groups in Lake and Siskiyou counties used the data to inform modifications with respect to adapting or modifying individual curriculum activities or how they were taught; the lesson study group members in Marin County shared information on curriculum implementations that helped lead them to combine youth members from three smaller 4-H clubs into one larger group for curriculum implementations, a change in practice that they found to be effective. Representative focus group interview responses included

We talked [about our observations] afterwards. We wrote everything down and summarized it to bring to each meeting, which I think helped all of us to hear what everyone had done.

It was nice to get input from [other lesson study group members] on what worked and what didn't work because I could kind of readjust or think about what things to put out for [the youth] to touch and feel or that type of stuff…. I think it helped a lot.

Theme: Collective benefits.

Beyond their roles as individual educators, participating 4-H volunteers commented frequently on collective benefits they derived from the lesson study process. Benefits included working cooperatively to develop skills, feeling empowered (as opposed to isolated) through collective work, and working toward common goals that benefit not just the youth members but also their county programs. Examples of focus group responses included

I thought it was great that we could all come together and actually work toward a common goal instead of being so segregated all of the time.

I think our goal to function as a team worked for the betterment not only of the lessons that we were teaching and for each other, but also for the county.

Analysis of the qualitative data from this study afforded a more comprehensive understanding of participating 4-H volunteers' lesson study experience. In particular, focus group interview data corroborated the outcomes from the Use of Inquiry Practices Survey with respect to participants' understanding and application of key inquiry processes. In contrast, data from focus groups did not provide strong support for quantitative outcomes from the Veterinary Science Content Knowledge Survey that revealed statistically significant changes in study participants' content knowledge. Although a few individuals commented on specific science content they had learned, the majority of study participants believed their educational and practical backgrounds in animal and veterinary science were already sound, and that the lesson study group process was most beneficial in influencing their pedagogical skills.

Interpreting the results

Constructivist-based professional development models like lesson study help advance educators' knowledge and skills by engaging them in the process of inquiring into their own practice (Loucks-Horsley et al. 2003). The 4-H volunteers who participated in this investigation formed lesson study groups and took an inquiry stance with regard to their practices, investigating strategies that targeted improved learning among their 4-H youth audiences, and did so in a collaborative manner. Through active reflection, a strategy whereby educators challenge their thinking and gain new insights into teaching and learning (York-Barr et al. 2005), participating volunteers systematically developed learner outcome goals, discussed data they collected during a sequence of curriculum implementations, and made modifications to the curriculum activities and their teaching that were informed by data they had collected during curriculum implementations.

Although no prior research had been done on lesson study in nonformal education settings, outcomes from this study were consistent with research in school-based settings where in-service and pre-service teachers who participated in lesson study groups improved their teaching skills and subject matter knowledge (Lewis et al. 2004; Marble 2006; Perry and Lewis 2003; Rock and Wilson 2005). Specifically, both quantitative and qualitative data from this investigation revealed improvements in participating 4-H volunteers' knowledge and use of inquiry-based teaching strategies, methods that are important to the advancement of scientific literacy (Minner et al. 2010).

Gains in science content knowledge among 4-H volunteers were shown through the analysis of survey data; however, these findings were not well supported by focus group interview data. A possible explanation for this discrepancy is that the Veterinary Science Content Knowledge Survey questions targeted subject matter that was specific to the Youth Development through Veterinary Science curriculum (Smith et al. 2009) as opposed to broader animal and veterinary science content knowledge conveyed by study participants during focus group interviews. Participating volunteers had considerable prior knowledge and experience related to animal and veterinary science and may have been considering this when responding to more general focus group questions. Thus, specific changes in their content knowledge related to the curriculum may have been underreported during the qualitative phase of this study.

Implications and recommendations

There is a need to explore alternative models of professional development for 4-H volunteers who facilitate science projects and activities with youth (Barker et al. 2009). Results from this study provided some evidence that lesson study has potential as an alternate strategy. However, Guskey and Yoon (2009) cautioned against embracing new approaches to professional development without thoroughly vetting them. New professional development strategies “should always begin with small-scale, carefully controlled pilot studies” to evaluate their effectiveness within specific contexts. For this reason, it is recommended that efforts be made to investigate lesson study in 4-H through long-term, coordinated efforts that are systematic in nature. The research design used in this pilot study would be recommended as a strategy to consider.

Results from this investigation provided early evidence for the promising nature of lesson study as a professional development model for 4-H volunteers who lead science projects and programs with youth audiences. Specifically, data showed improvements with respect to volunteers' understanding and use of effective pedagogy and some gains relative to science content knowledge. However, it must be noted that the broad application of these results is limited by the size and scope of the investigation. Only 16 volunteers from rural, club-based 4-H programs participated in this study. It is recommended that additional research be carried out on the use of lesson study with other 4-H volunteers involved in club-based programs. Additionally, because lesson study experiences are unique to the particular context within which they are situated (Wiburg and Brown 2007), results from this investigation may not translate directly to lesson study use in other settings. For this reason, the use of lesson study with volunteers involved in other 4-H program-delivery modes (e.g., after-school programming, camp settings) also warrants investigation. Science subject matter beyond the scope of veterinary science should also be explored.