Science communication and engagement in adaptive farm-systems research: a case study of flexible milking research in New Zealand

Year 1 (2019/20)

The first year of the project focused on understanding the information needs by learning from farmers already using FM strategies to help guide development of resources and information. Major activities included the following:

Year 2 (2020/21)

In the second year, activities centred around focus farms where FM strategies were being implemented. Major activities included the following:

Year 3 (2021/22)

Main activities in the third season included the following:

Engagement with website content, podcasts, and webinars

A key channel for communicating results from the project was through the DairyNZ website. Engagement with the content created for this website was followed through analytics of unique page views (termed ‘views’ through the remainder of this paper) and clicks using Google Analytics™. Three webinars were also delivered at different stages of the project, conducted live and recordings were posted on the DairyNZ YouTube™ channel and website. ‘Update 1’ in March 2020 focused on the season-to-date results from the farmlet experiment and learnings from farmer interviews, ‘Update 2’ in November 2020 reviewed project results, including the completed farmlet experiment and partial results from the component experiment, and ‘Update 3’ in December 2020 involved four of the focus farms outlining their experiences mid-way through the season. Engagement was tracked through web views, and time spent watching the content. Two ‘Talking Dairy’ podcasts were created in the final year of the project; ‘Episode 11: better workplaces through flexible milking’ was released 24 August 2021, and ‘Episode 19: flexible milking case study and research’ was released 2 February 2022. Engagement with these podcasts was tracked by recording downloads and average consumption length (% of episode people listened to).

Engagement with email communications

An important communications channel for the project was an email list, to which farmers and other sector stakeholders could subscribe. The email list was advertised on the project website and at early project events such as webinars, and the subscriber list rapidly increased to peak at 552 subscribers. Of the subscribers, 75% were farmers, and 25% rural professional and other stakeholders. Among the farmers, there was a range of farm owners (40%), sharemilkers/contract milkers (42%), farm managers (11%), and other (7%). There were subscribers from all NZ dairying regions, with Canterbury/North Otago being the largest group (26%), and 3% of subscribers lived outside of NZ. The cadence of emails varied across the project timeline; in the first year, fortnightly updates were sent during experimental periods to provide ‘real-time’ information about trial results. In the second year, fortnightly emails were sent to provide updates from the focus farms as well as ad hoc communication of experimental results. In the third and final years, emails were sent less frequently, only when there were major project updates, with the final email being sent in May 2022. Interaction with the email list was assessed by using ClickDimensions™ to analyse ‘open rates’ (how many emails were opened) and ‘click-to-open rates’ (how many weblinks within emails were opened).

Surveys of the email subscribers

Surveys were undertaken at the start and end of the project to assess the knowledge farmers and rural professionals had of FM, and the main barriers they saw to its use on farm. A baseline survey was conducted with email list subscribers from 12 November to 2 December 2019. Respondents were able to indicate whether they were a farmer, rural professional (e.g. farm consultant or vet), or ‘other’. Relevant demographics were collected including region, herd size, preferred information sources, and previous experience with FM. Respondents were asked about their perspective on whether FM had a positive or negative impact on people on farm, milk production, farm profit, animal health, and reproduction. Additionally, respondents were asked which of these five areas they wanted more information on. They were asked questions related to their knowledge and confidence to implement or advise on aspects related to FM. The final survey was conducted between 5 May and 2 June 2022. Respondents were asked to select whether they were a farmer, rural professional, or other stakeholder. Questions mostly matched those asked in the initial survey, with the addition of questions about the project delivery methods.

Co-development of project activities and messaging with farmers

A co-development approach underpinned most of the project. A ‘reference group’ was created to connect the project team with engaged farmers and stakeholders. This group helped co-design the specific treatments to be investigated in the farmlet and component experiments. This flexibility in design was built into the original project proposal. Interaction with, and feedback from, farmers on important topics for the project to investigate was also undertaken through discussion groups, farmer conferences, and question and answer sessions in webinars. These insights were incorporated in the scenarios in the modelling phase, and in the development of website resources. Questions raised by farmers also resulted in an additional experiment being funded in 2022/23 to investigate postpartum changes in milk composition to improve guidance of colostrum management in FM systems. Farmer feedback also led to the development of web pages to fit the following themes:

In addition, results were presented to farmers at 25 farmer/industry events (including annual conferences such as the South Island Dairy Event and Farmers Forum) and 17 articles published in the traditional farming media. Also delivered were six webinar or Facebook live events (via DairyNZ-run Facebook groups), three podcasts, one radio interview, and three international presentations.

Surveys of farmer and rural professional needs

Engagement with email communications

During the project, 552 people subscribed to the email list, and a total of 16 643 emails was sent, an average of 30 emails per subscriber (Fig. 3). The numbers of email sent through the project were 2343 (2019), 9330 (2020), 4567 (2021). Overall, 9057 emails were opened at least once, corresponding to a 54% open rate. Additionally, 1437 emails had click-to-opens, a 16% click-to-open rate, which is considered high for DairyNZ-sent emails (overall average 2.8%).

Fig. 3.

Email send statistics for overall deliveries (emails sent), open rate (% of sent emails opened), click rate (click-throughs as a % of total deliveries), and click-to-open rate (click throughs as a % of unique opens) from 18 September 2019 to 18 May 2022.

Engagement with website content, podcasts, social media, and webinars

Of the pages on the DairyNZ website, the most popular was the page ‘Implementing flexible milking’, where farmer-sourced questions were presented. From August 2020 to August 2022, this page was viewed 7625 times, with people spending an average of 6 min 17 s on the page (Fig. 4). Statistics for the other pages were ‘Flexible milking’ (4531 views, 2 min 53 s average time), ‘Milking intervals’ (3904 views, 1 min 29 s average time), ‘Flexible milking research’ (3774 views, 4 min 21 s average time). The webpages also had different engagement profiles over time, as illustrated in Fig. 4. Practical information, such as the information on the ‘Implementing flexible milking’ page, appeared to have had greater ongoing engagement.

Fig. 4.

Engagement profiles of project web pages over time. (a) Presents views of the ‘Implementing flexible milking’ web page; (b) presents views of the ‘Flexible milking research’ web page.

Social media (such as Facebook™, Twitter™, Instagram™, YouTube™, and LinkedIn™) were used throughout the FM project to promote webpage, webinar, and other content. These platforms provided much greater reach for the project content; for example, a post on the DairyNZ Facebook™ page on 29 December 2020 resulted in 756 clicks through to video content and an overall reach of 19 589 by August 2022. Reach represents the total users with potential contact, on the basis of the network of those who have commented on, or ‘liked’, the post.

Two major articles were also published in the DairyNZ ‘Inside Dairy’ magazine (June 2020, October–November 2021), which is being delivered bi-monthly in print form to every dairy levy payer. While readership of print media is difficult to track, each Inside Dairy issue is also available online and views were tracked. The October–November 2021 issue had 1049 views by August 2022; however, most (750) of these views were in October 2021, 90 views in November 2021, and an average of 23 views per month through to August 2022. This highlights how publishing the magazine online results in a high number of views soon after it is released, but that this view rate rapidly degrades.

Two podcasts were published on the DairyNZ ‘Talking Dairy’ channel. The downloads for Episode 11 and 19 (described earlier) were 307 and 685 in their first 7 days, and 824 and 1049 in their first 30 days respectively. This compares with the last 20 ‘Talking Dairy’ podcasts which had an average of 483 and 790 for their first 7 and 30 days respectively. Episode 11 had 2097 downloads, making it the most downloaded podcast of all episodes on that channel by August 2022. On average, people listened to 94% of the 25 min episode, the highest of all podcasts on the channel. Episode 19 had 1797 total downloads by August 2022 (fourth-most downloaded episode) and an average consumption length of 85% of the 39 min episode (fifth-highest for all episodes).

For webinars, by August 2022, ‘Update 1’ had 1168 total views on YouTube™, and an average view duration of 7 min 58 s (12.4%). It was originally streamed live at 7 pm in March 2020, with 89 participants, then posted online. A post-session survey of the live audience, with 30 responses, rated the webinar 8.5/10 (zero poor, 10 excellent). Comparatively, ‘Update 2’ has had 416 total views on YouTube™, and an average view duration of 7 min 5 s (12.4%). ‘Update 3’ has had 562 total views on YouTube™, with an average view duration of 8 min 53 s (16.2%).

Predominant themes of the qualitative feedback from the ‘Update 1’ webinar were as follows: ‘Ability to interact/ask questions’, ‘Scientific approach’, ‘Clarity and ease of understanding’, ‘Relevant information’, and ‘Good use of interactive webinar technology’. The first webinar was broadcast early in the project, and participant suggestions for project activities included comparing profit among farms, more data on specific milking frequencies such as ‘10-in-7’, more in-depth information for those experienced in the approach, and making the start time earlier in the evening. The overall research design and use of webinars were considered effective by participants, with one suggesting to ‘present more of the DairyNZ research projects in a similar way’.

Assessing impact of the communication and engagement approach

The data showed that there was a high level of engagement with the project through the first 2 years when there were regular updates. The first year, focused on a farmlet experiment, connected with farmers primarily via a webinar, print articles, and the email group. The cadence of updates was regular, and farmers could see how differences among treatments evolved during the season. In the second year, much of the communication related to the focus farmers, and this provided a peer-to-peer voice for farmers through the email distribution list, webinars on specific topics, and presentations at in-person farmer events. The website, print media and webinars were the most popular forms of communication for those on the email distribution list. The third year involved greater extension of the key project messages by using a wide range of communication approaches such as podcasts, conference presentations and media. By the end of the project (June 2022), farmers noted that they had more information to make informed choices than they did 3 years ago (Table 3). They were also positive about how the project delivered interim and ‘on-the-go’ results and insights. However, it should be noted that the response rate for the final survey (8%) of email subscribers was lower than that for the initial survey (34%). This low response rate may have influenced this feedback if respondents were more engaged with the project, therefore potentially more positive about its impact. Overall, data collected showed reducing interaction with email communications over time, and this could be related to much of the data-collection activity happening in the first 2 years. This suggests that providing novel information over the period of the project is required to continue to add value to the audience.

Lessons related to the form of communication used in farm-systems research

The data captured in this project provide insight into effective communication channels for long-term research projects. The range of communication approaches used allowed messaging from the FM project to reach farmers who prefer specific information sources such as print, social media, internet content. The webpage content was vital, not only to provide meaningful content for farmers, but also as a place to direct listeners to during-radio interviews, presentations, and podcasts. Web traffic showed that ‘how to implement’ topics had most ongoing engagement (Fig. 4). Also notable is the difference in ongoing engagement for print versus online communication. Webinars provided an example of this ongoing engagement, with some immediate real-time interaction with farmers during the live webinar, and subsequent question and answer sessions. The webinar recording then serves as an ongoing resource when uploaded to a website, and, in the FM project, this provided much greater reach for the key messages. However, webinar viewing data highlighted that most of those viewing such recordings do not watch the whole webinar (e.g. 12.4–16.2% viewing time for the three FM webinars) and therefore these viewers will be missing part of the messaging. The podcasts had much longer engagement (94% and 85% consumption for Episode 11 and 19 respectively), signalling that more complex messaging may be best delivered through the podcast format.

Communication through the Inside Dairy magazine saw a large peak in views when released (via a PDF or Flippingbook™ available online) but quickly lost traction. Most views of such magazine content online are driven through social media at the time of release, and once this ends, the content appears difficult to discover. Through the early part of the project, such as, for example, in the first webinar, farmers asked to hear from other farmers who were implementing the FM practice. Farmers also wanted to see a range of examples with regional and farm-system diversity. This was subsequently delivered in Year 2 of the project. Peer-based learning, and accessing information from a wide range of opinions, has been shown to be highly valued by farmers when making management decisions (Morgans et al. 2021). The FM project also created a form of community of practice (O’Kane et al. 2008; Eastwood et al. 2012) where farmers, scientists and rural professionals could interact around the FM topic as a boundary object (Klerkx et al. 2012).

Decision-making with imperfect information

In the first 2 years of the FM project, a goal was to get information to farmers as soon as possible. The email distribution list had a high level of engagement (measured through email opens and web-link click-to-opens) and farmer feedback indicated that this was due to on-the-go information such as updates from the past fortnight in experiments and focus farms. The feedback also showed that farmers found the emerging information engaging and useful for their decision making for FM adoption. Adoption of major farming-system changes often does not follow a linear profile (Kuehne et al. 2017). There are a range of pathways that farmers may follow that can involve awareness building, trialling, adoption, non-adoption, or dis-adoption (Montes de Oca Munguia et al. 2021). Within this, farmers are constantly dealing with imperfect information and uncertainty in decision-making (Chavas and Nauges 2020; Eastwood and Renwick 2020). Therefore, the provision of interim (and possibly imperfect) results from FSR may conflict with values of scientists more than it does for farmers. The different timelines for data analysis are also illustrated by the fact that research publications from the project were becoming available only at the end of the 3-year project (e.g. this current paper; and Edwards et al. 2022b).

Implications for farming-system research design

This study highlighted several implications for FSR projects. Adopting a variety of processes to enable integration of farmers helped ensure project findings, and the method of engagement was meaningful for the wider farmer population. One example was the original project design, which mapped out the major steps in the project but left space for farmers to influence the specific design. Adding in a study on colostrum management was one example where farmer questions during the project could be answered by altering the project design. There were opportunities to improve this further with targeted co-design and iterative feedback loops with farmers as end-users (Toffolini et al. 2020; Eastwood et al. 2022). To help facilitate this, a process of reflexive monitoring can be established, where project implementation is regularly reflected on, possibly with a specific role in the team for this skill set (Fielke et al. 2017; Jones et al. 2019). Participation can range from a stakeholder reference group, through to full participatory extension (Murphy et al. 2013; Knook and Turner 2020).

Science-based data are needed to underpin FSR projects; however, the concept of sufficiently credible data may have different meanings for farmers from those of scientists. In the FM project, the engagement data showed that information from focus farms was sufficiently credible for farmers to build confidence in the application of research in a farming context. Of particular interest to farmers was that the milk production loss was minimal for a 3-in-2 milking intervals, which reduce the length of the working day. Experimental data, such as potential reductions in milk production associated with FM compared with TAD milking (Edwards et al. 2022b), have value in providing independent evidence on key adoption triggers for farmers. FSR projects therefore need sufficient farmer input in the design phase, or early stages of the project, to ensure that expensive experimental work is focused on collecting data on these adoption triggers (Rose et al. 2018). An additional benefit of including farmers in the design phase is that it can improve the alignment of project communication with the information needs and preferred delivery methods of farmers. A gap in knowledge identified through this study was the potential unintended consequence of sharing preliminary results. There is a risk that final results may prove interim data and messaging to be incorrect, yet the interim messaging could still be imbedded in online and print communications and therefore cause confusion. This could affect the reputation of research organisations and/or the FSR process. Being clear when interim results are being presented may help alleviate this risk.

A framework for FSR communications

The insights outlined above are useful for the design of future FSR projects. Limited information has been published on communication and engagement in farm-systems research, rather the focus has been on design of the research itself (Packham 2011; Scott et al. 2013). Previous studies on this topic have proposed frameworks for engagement related to FSR (Le Gal et al. 2011; Davis et al. 2021). However, these frameworks do not account for the various factors related to communication and iterative design with farmers. In Table 4, we suggest the important factors for communication and engagement with farmers on the basis of experiences during the FM project. The seven factors identified involve credible evidence, rapid interim results, inclusion of farmer voices, meaningful end-user outcomes, flexible and modern communication channels, feedback loops for iterative design, and adaptable research design.

Limitations and further research

Some of the insights presented in this paper will be context specific, as the questions and interest farmers had in FM would result in different engagement than would some more complex farm-system challenges such as nutrient management or methane emissions. However, the principles in the framework for science communication and engagement (Table 4) should have broad relevance in FSR projects. Another potential influence in this study was the role of the Covid19 pandemic occurring during most of the project. The switch to online communications through this period may have led to greater engagement with webinars and online information sources, as has been highlighted in other studies (Chivers et al. 2021). During the FM project, the quality of online communication tools, and the engagement with these approaches, increased and caused a shift in communication methods. Further research is warranted on whether there has been a defined and permanent change in the preferred information-access channels of farmers. Research is also needed to understand farmer definitions of sufficiently ‘credible’ science, whether these definitions are changing over time, and to determine how FSR needs to be designed and communicated in the future.