Evaluating learning in museums: theory informs practice in Glasgow Science Centre’s ‘Powering the Future’
Rachel Gonzalez, MSc
The Museum Review, Volume 5, Number 1 (2020)
About the Author Rachel Gonzalez is an Exhibit Developer for the Science Museum of Minnesota. She previously worked as an Exhibit Developer for the Field Museum (Chicago), and is a graduate of the University of Glasgow (Scotland) where she obtained an MSc in Museum Studies. In Glasgow, she completed an internship in Visitor Studies at the Glasgow Science Centre, and an internship in Exhibits at the Kelvingrove Art Gallery and Museum.
Powering the Future exhibition background
Powering the Future opened on December 10, 2015, and has since been one of the most popular attractions at the Glasgow Science Centre in Scotland. The exhibition, on Floor 2 of the Centre’s Science Mall, “celebrates the contribution that electricity has made to our modern world” and focuses primarily on the Energy Trilemma. (Glasgow Science Centre 2016, 7) The Energy Trilemma is what the World Energy Council defines as the three dimensions of energy sustainability: energy security, energy equity, and environmental sustainability. (World Energy Council 2015)
The exhibition strives to educate visitors about energy by focusing on diverse themes. One theme addresses energy in general, explaining the primary fuels, including oil, gas, coal, nuclear, hydro, and wind. Energy consumers are included in the exhibition’s discussion, linking energy uses and consumption to how museum visitors use energy in their own lives. A second theme considers the “3 A’s” – energy availability, accessibility, and acceptability – encouraging visitors to create a narrative, and make connections between the personal and the global. (Glasgow Science Centre 2015)
Powering the Future creates a dialogue with visitors, openly asking visitors to consider the future of energy: existing challenges, what must happen to solve diverse issues, and what new technologies and innovations will be developed to create change. The exhibition encourages visitors to explore, discover, and reflect on their own energy impact. Powering the Future is scheduled to run for over six years, and has the goal to reach more than 1.8 million people (Glasgow Science Centre 2015).
Evaluation planning
This analysis was conducted when Powering the Future was a new exhibition. It required a series of evaluations to examine how the exhibition was operating overall, and to answer specific questions about the exhibition’s intended outcomes. The analysis attempted to evaluate the learning experiences of visitors within the exhibition space, and to determine what behaviors could support corresponding learning experiences. Other goals included determining the level of cooperation between visitor groups in the exhibition, finding whether intergenerational activity occurred, and discovering which exhibits had the highest and lowest levels of this activity. The study also sought to determine the exhibition’s overall impact on visitors, to learn which exhibits were most successful in attracting and retaining visitors, and which met the other listed aims, as well as which exhibits needed additional work.
The ability to attract and hold visitor attention is often of primary interest when evaluating exhibitions. It is necessary to assess visitor attention during the evaluation process because it can inform researchers about what elements of an exhibition are “working” or “not working.” This research is especially important for exhibition content that remains fixed for an extended period of time, whether in the same location or when traveling. Powering the Future is an exhibition that will remain in place for six years, so evaluating the positive and negative aspects of this exhibition was and continues to be necessary to the exhibition’s success and longevity.
Powering the Future is an exhibition that focused on science and attempts to contribute to visitors’ scientific literacy. The content is not age specific, nor is it too complex for the average visitor. However, because the exhibition does explain numerous concepts, it is important for the exhibition team to understand if and how visitors are learning.
Defining learning in a museum setting is complicated and often considered subjective. This article does not attempt to claim as a fact what constitutes learning among visitors. Instead, it aims to define behaviors that might suggest learning occurs in this particular exhibition and to see whether these behaviors correlate to other variables, such as high interactivity within groups and intergenerational activity. Although these variables may appear unrelated, it is hypothesized that higher levels of interactive behavior exhibited by a visitor correlate to higher levels of interactivity and even intergenerational activity. For example, if a person is showing positive, inquisitive behavior, i.e. smiling, laughing, communicating or asking questions, in a group of people working together, a greater likelihood exists that the visitor is learning.
In order to obtain the necessary results, a plan for summative evaluation was created and implemented over a period of twelve weeks. Approximately seven weeks were spent collecting data through tracking and observing visitors from 5/30/2016 to 7/18/2016. The following section describes the theories and frameworks that informed the goals and methodology utilized in evaluation.
Learning in museums
Before evaluating an exhibition — or evaluating visitors in an exhibition — it is important to first understand what types of behaviors to assess. Learning in museums is difficult to define—learning in any setting is complex and highly subjective. In its most basic meaning, learning is what people do to make sense of the world. According to Ben Gammon, an expert on visitor studies, learning is “a process of active engagement with experience.” (Gammon 2003, 3) One must be engaged in whatever one is experiencing to understand and ultimately to learn.
Learning is not one-size-fits-all. There are different ways in which learning occurs, and numerous ways to demonstrate learning. Gammon (2003) defines three different types of learning that exist inside and outside the museum. The first type of learning is formal, which usually happens outside of the museum—for example, in a classroom—but can also happen inside the museum with school groups led by teachers or museum educators/facilitators. This type of learning is characterized by an instructor conducting a series of learning activities, usually with an assessment afterwards. The next type of learning is self-directed, which is characterized by learner-led activities that include the use of resources, such as books, websites, exhibitions, and others. Self-directed learning is not typically assessed and is done for a person’s own goals and interests. This type of learning exists in the museum among hobbyists, people who simply enjoy a certain subject matter, and museum-goers in general. The third type of learning Gammon identified is informal or non-formal learning. This can be seen frequently in museums or cultural institutions, and is the spontaneous encounter with an activity, resource, person, etc., that results in new ideas, insights, and experiences. An exhibition could be considered an activity or resource that has an unplanned influence on a person, leading that individual to gain newfound knowledge on a subject.
Understanding the different types of learning that occur within the museum is an important step in understanding how visitors learn, but it does not provide the full picture. Knowing how to spot learning when it happens, or to determine whether learning resources are lacking are crucial to understanding learning in a museum. It is difficult for museum staff to track learning for a number of reasons, primarily because most museum projects run for a short period of time and “require data on visitors learning far more quickly than can be provided by academic research.” (Gammon 2003, 2) Museums do not typically have the resources to conduct longitudinal studies on the learning effects their exhibitions have had on visitors. Museum staff need to consistently test how effective their exhibitions are at creating learning experiences for visitors in order to receive support or funding and ultimately to improve the museum experience for future visitors. If lengthy studies are not possible, then museum staff must turn to other methods of studying visitors, and must find shorter, temporary cues that can generate faster results.
To take this step, one must look to education theory again, and examine different potential learning outcomes. Gammon (2003) suggests using a model of learning that he devised, which draws upon constructivist models developed by George Hein, a museum education expert, for evaluating learning in museums. In this model, the learner is assumed to be an active participant in the process, rather than a passive recipient. (Gammon 2003) By reflecting on their experiences, including what they have seen, heard, or felt, the learner builds an understanding of the world around them. To reiterate, “learning” means to constantly build upon our understanding, and to “accommodate new experiences and new information.” (Gammon 2003, 2) Learning in the museum, however, is not just about gathering information from reading exhibition texts and suddenly having a “eureka” moment. Learning in museums is more encompassing than simply acquiring new knowledge – it is an educational experience that can place learning in museums into several categories. An educational experience in a museum can have different learning outcomes for visitors, as illustrated by Gammon (2003) in the table below.
When learning in museums occurs, the potential for general visitor enrichment happens simultaneously. Not only can a fact be learned, but opinions, beliefs, and values can be changed. In addition to cultural capital, a visitor can gain social capital by being presented with opportunities to work together on something in the exhibition, or to just talk within their group. The potential breadth of new mental and physical skills that could be gained through this learning is boundless, and the opportunity for personal improvement is immeasurable. These possibilities qualify as learning and can result from a single museum visit. The learning prospects in museums are important, explaining why museums, without any other motives, should frequently study their visitors.
Gammon suggests adopting a process to define indicators of learning that has been used by the New Economics Foundation, known as AIM. (Gammon 2003) The system recommends that indicators are action-focused, important, measurable, and simple. To be action-focused means having data that can lead directly to practical recommendations for the institution to improving whatever is being evaluated, like an exhibition or program. The research process must also be important, meaning that the study and its corresponding results provide value to stakeholders, like exhibition developers, museum staff, a museum’s board, or even visitors. The collected data must be feasibly measured, considering time, money, and available staff. Lastly, the data collection process should be simple and the findings should be clear and easy to understand. (Gammon 2003)
The Generic Learning Outcomes model provides researchers a helpful way to look at museum learning in broad terms. Learning within an informal setting can incorporate several different indicators of the learning process. Three of the outcomes that have been most influential on the methodology for this study are the following: Knowledge and Understanding; Activity, Behavior, and Progression; and Enjoyment, Inspiration, and Creativity. The first outcome, Knowledge and Understanding, is difficult to assess but is one of the most important for evaluating visitors in this case. Powering the Future aims to teach visitors about energy and what can be done to improve future energy usage. Activity, Behavior and Progression can assist in understanding how visitors react to the exhibition. Positive types of activity and behavior can suggest that visitors are enjoying their experience, which could reasonably lead to a greater likelihood of learning and impact. Signs of positive activity and behavior can include smiling, laughing, or showing excitement, as well as engaging in conversation, asking questions, or instruction—“positive” in this context suggests a lack of boredom or disconnect. The same can be said with Enjoyment, Inspiration, and Creativity. Powering the Futureseeks to inspire visitors, providing a fun experience while teaching important concepts. It is reasonable to conclude that if visitors exhibit signs of enjoyment, creativity, inspiration, and understanding, whether through actions or verbalization, that a) visitors are learning and b) the exhibition has a positive impact.
Fundamentals of tracking and observation studies
Understanding how learning occurs is a critical step in understanding museum visitors and their engagement with exhibitions, but theory only goes so far. Theories and models inform methodologies, and determining the right methodology to use for obtaining data on visitors is the most important step in the process. Most visitor studies involve several steps of evaluation. This study focused on summative evaluation and is described below. According to the Australian Museum, summative evaluation is defined as using “a variety of methods at the conclusion of an exhibition or program to check whether it delivered the messages that were intended and what learning occurred.” (Kelly 2009, 3) Two methods of summative evaluation that can easily be administered in the museum are tracking and observation.
Tracking is defined by Lynda Kelly from the Australian Museum as “unobtrusive recording of visitor behavior in a museum/specific exhibition.” (Kelly 2009, 1) Information recorded in a tracking study usually consists of:
- which displays visitors attended, either in the museum collection, a special exhibition, or both
- starts and stops, especially noting when visitors enter and exit an exhibition
- types of behavior demonstrated by the visitor at each place they stop, including whether a person reads a text, engages in an interactive or element, or if the person wanders without paying attention to anything specific
- overall amount of time spent in the exhibition and at each stop
- pathways and/or flow, meaning the direction and trail the visitor takes throughout the exhibition
Each variable is important to understand how the exhibition or museum space works, and how strong or weak the exhibition or museum is at attracting and holding visitor attention. These variables can provide answers to questions about the design, layout, and content of the exhibition/museum. Where does the visitor go? What made the visitor choose spending time in this part of the exhibit over another? Did this segment hold visitor attention, or was it briefly visited? Did the visitor take the intended path in accordance to the exhibition plan?
Observations can also be used to examine how visitors interact with the space. This type of data collection involves watching visitors at specific parts of an exhibition, not following them around the space from start to finish. Advantages to this type of evaluation exist, including the ease of more thoroughly evaluating certain audiences, like families, and the studies are “generally small-scale and targeted.” (Kelly 2009, 1) Observations can be less dependent on quantitative data and more focused on qualitative data. Since the researcher does not follow the visitor through the exhibition space recording the exact time of every stop, the researcher can focus on scrutinizing the visitor actions and behaviors to decode the visitor experience and level of engagement.
Powering the Future’s summative evaluation attempted to provide information about how the exhibition worked overall, looking closely at how visitors interact with the exhibition, what visitors learned, and what elements of the exhibition could be changed in the future. To execute the summative evaluation, it was necessary to use structured observations and visitor tracking to determine what attracted visitors within the space, where visitors went and by what path, how long their attention was held, and whether they exhibited signs of learning.
To create a clear plan for data collection, the exhibition map was evaluated by theme or clustered segments. Six general clusters were located on the map, and were confirmed by exhibition staff. These clusters later aided in collecting, organizing, and analyzing data.
A general framework for assessing visitor engagement was then established to maintain consistency when collecting data. A framework described by Barriault and Pearson (2010) as the Visitor Engagement Framework was used by Glasgow Science Centre staff in the past, and the same model was used in this study to assess visitors through observation. The criteria of this framework consist of six different learning behaviors broken down into three categories:
- initiation – incomplete interaction or incorrect use of the exhibit
- transition – repeated use of exhibit and positive responses
- breakthrough – sustained engagement and seeking/sharing information (Barriault and Pearson 2010)
Initiation behaviors comprise two levels: doing the activity and spending time watching others engaging in an activity or observing the exhibit. The first level, “doing the activity,” can be expressed by several actions. Either the individual does the activity in passing, not stopping completely or not finishing the activity, or the person partially completes the activity or does it without exploring further. An example of this would be touching buttons on a screen in an activity, engaging with the activity on the surface but not reading, examining images, or testing any variables on the screen. The other level, “spending time watching others or observing the exhibit itself,” is fairly self-explanatory. This person could be watching another visitor do an activity, and then decide to participate once the person is finished, or could simply be interested in observing other visitors or the exhibit itself, not necessarily participating at a single point.
Transition behaviors also have two levels: repeating the activity and expressing positive emotional response in reaction to engaging in activity. “Repeating the activity” means that a person has done the activity two to three times for a certain outcome, usually to master the function of the particular exhibit. The person may also repeat the activity simply because they enjoy it, or because they are looking for a different outcome. This is a sign of increased engagement. “Expressing positive emotional response” is seen through smiling, laughter, verbally indicating enjoyment, overt eagerness of their participation, and for others to join, etc.
Breakthrough behaviors consist of three levels:
- referring to past experiences while engaging in the activity
- seeking and sharing information
- being engaged and involved which includes testing variables, making comparisons, and using information gained from the activity
“Referring to past experiences while engaging in the activity” is exhibited by a visitor making a simple reference to some comparable experience the visitor had before, whether at another exhibit, museum, or just in the visitor’s life. In addition, the person might make “comparisons and deductions based on observations of similarities and differences.” (Barriault and Pearson 2010, 96) “Seeking and sharing information” describes when a visitor calls someone over to either look at the exhibit with them, or to explain something. The person might also seek information from staff or a member of their group without engaging in deep conversation. A visitor could demonstrate this behavior by reading signage, having conversations about the exhibit with group members, family, or staff, and by sharing information gained from the exhibition with someone. The last level, “engaged and involved,” refers to visitors who remain at an exhibit for two to three minutes, act inquisitive, repeat the activity, read signage, and ask questions. The visitor will look obviously concentrated and will stay on the task for three to five minutes or more. The visitor might also test different variables of an activity and look for different outcomes, resulting in increased concentration and staying for several minutes on a particular task.
Data collection and analysis summary
After a clear evaluation plan was formulated, tracking and observations were carried out for approximately seven weeks. The goal was to track where visitors went and how long they spent at each stop. Observations sought answers about whether visitors exhibited higher levels of learning behaviors, if they worked in groups, and if intergenerational activity occurred within group interactions.
Fifty visitors were “tracked,” meaning that their movements throughout the exhibition were noted. A stopwatch was used to track the time at each stop, and basic demographics were observed. During tracking, researchers observed a fairly even ratio of men to women, and adults to children. Output included color maps showing hot and cold locations for visitor counts and visitor times, as well as maps showing the pathway taken by ten randomly-selected individuals. Similar to visitor studies done for the Glasgow Science Centre’s permanent exhibition BodyWorks, three different levels of cooperation were utilized to assess how people interacted with exhibits. Visitors were assigned a letter grade per interaction: “A” signified no cooperation, or the group split up; “B” signified some cooperation; “C” signified a high level of cooperation. Intergenerational activity was noted with a “Y” for yes, or an “N” for no. Researchers conducted ten observations were conducted per exhibit, and observed thirty-nine total exhibits. The remaining exhibits had too few visits to include in the study.
Sectioning off portions of the exhibition map into clusters was most important for conducting observations, allowing a systematic approach to collecting data. The data yielded slightly different results than those found by tracking visitors, mainly because the variables were different and the sampling pool was wider. Some exhibits were not included in the research results simply because the exhibits did not receive enough visits, or lacked adequate data. For the exhibits that were observed, their evaluation criteria were calculated in order to see what levels of interaction each exhibit element offered. Data results were analyzed and presented to the Glasgow Science Centre with recommendations for improving the overall visitor experience.
The average dwell time in the exhibition was just over fourteen minutes, and the majority of visitors spent approximately five minutes in the exhibition. Exhibits were highly transitional: 53% of observations showed “transition” behaviors; 24.8% showed “initiation” behaviors; and 21.8% showed “breakthrough” behaviors. Of forty-eight exhibits, over half were visited for less than one minute, or were not visited at all.
The results yielded from observations were varied and, in some ways, disappointing. Since Powering the Future is a highly interactive exhibition targeted at a wide audience, the hope was that visitors would demonstrate high levels of transition and breakthrough behaviors, high levels of cooperation/interactivity between groups and that the groups would have some intergenerational activity. As the Glasgow Science Centre attracts families, this was not an unreasonable goal, particularly given that the data was collected over the Scottish summer holidays. Data suggests, however, that visitors largely interact with exhibits alone. Yet rarely was a visitor actually alone in the exhibition; often the visitor observed was in a group of 2-5 people. The exhibition has so many different elements, possibly resulting in groups splitting up or solo activities. If a group is comprised of people of different ages, the visitors are likely to split up to engage with an exhibit that attracts them the most.
That the majority of visitors in this study worked alone does not necessarily have to be interpreted as a negative result. Although it is believed that increased interactivity or cooperation within a group will result in a higher likelihood of learning, this does not mean that those who work alone are not learning or are not having positive experiences. Over half of the observed visitors showed transitional behaviors, suggesting that the exhibits in Powering the Future encourage positive responses. Visitors enjoyed their experiences and often would repeat activities. Visitors did demonstrate breakthrough behaviors, though it was the least observed type of behavior. A correlation exists between visitors who demonstrated breakthrough behaviors and visitors who had high levels of interactivity within groups, meaning that visitors who cooperated with others were likely to have breakthrough experiences. These visitors are more likely to have learned while they interacted with the exhibit.
24.8% of visitors only demonstrated initiation behaviors. There are several explanations offered for this finding. The exhibition is highly interactive, which could be a hinderance in itself. Visitors may have been overwhelmed by the many choices and by the exhibition space. The space is replete with stimuli, so visitors, especially those with children, might find it difficult to concentrate on one exhibit or on a single element for long. Many of the exhibits in Powering the Future also have concepts that are easy to misinterpret. This is not to suggest that visitors are incapable of understanding, but rather that visitors might be missing the instructions or signage, or that the instructions for exhibits need to be simplified, reworded, or reoriented. A visitor might approach an exhibit because it looks interactive or fun, but after five to ten seconds of attempting it without gratifying results, the visitor might give up and move on to another exhibit. In this case, the visitor may not have seen the instructions, or because the activity was too challenging. In such cases, Glasgow Science Centre floor staff could be available to aid visitors. It is also possible to evaluate the text associated with exhibits that have lower levels of visitor engagement.
Overall, Powering the Future provided visitors with engaging experiences and ample opportunities to have fun. The exhibits have a touch component and are stimulating to other senses. To understand the concepts within each exhibit, visitors would have to read much of the surrounding text. This does not include the instructional texts that accompany the exhibits, but the text panels that offer information about scientific concepts, types of energy, careers in the energy sector, etc. Simply put, this exhibition, while engaging and informative, is text heavy. The average human attention span is believed to be shrinking, and even if the visitor is interested in learning about energy, it is unknown how much time the average visitor will spend reading exhibition text in a space that is also highly interactive and distracting. Reading the exhibition text panels is often important to understanding the exhibition’s concepts. There might not be an immediate or simple solution to this problem, which emerges consistently in exhibition development. Some individuals will visit an exhibition with the expectation of spending time reading and subsequently learning something new through that method. Others might have the expectation of glancing over texts, but going straight to the interactive exhibits. Some might do a combination of both, or might read or interact/play with exhibits exclusively. Regardless of where visitors went or what they did during their time in the exhibition, it is worth emphasizing that the majority of visitors who were tracked and observed in this short study did exhibit transitional behaviors that included positive responses.
Final thoughts
As previously mentioned, the practices used to evaluate Powering the Future are just a few examples of how to conduct visitor studies. The best method is often determined by the capability of the museum and the person or team studying their visitors. While there is no perfect method for evaluating learning in museums, the methods used in this study can provide a framework for beginner evaluators. Additionally, for science museum evaluators, it is worth noting that prior knowledge or research in science communication can only benefit the evaluation process. While it is not a prerequisite, knowledge of or interest in best practices and trends in science communication will aid the evaluator in recognizing visitor behaviors and in making recommendations to exhibit content developers or programmers. However, science exhibits do not necessarily require different or special evaluation methodologies than exhibits in other subjects.
Listed below are general suggestions to consider while planning an evaluation:
- Remember the goals of the project – what is to be gained from this study?
- Consider your audience – some methodologies work better than others. If the museum is highly interactive with visitors through interpreters or visitor experience staff, surveys or interviews may be appropriate for your audience.
- Consistency is key – upon decided on a research approach, utilize that approach for the duration of the study to keep your results consistent and clear, unless there are obvious errors or ethical considerations.
- Communicate with all relevant staff and volunteers to make them aware of this study and its intended goals. These staff members and volunteers can be instrumental in ensuring a smooth process of studying visitors and can answer relevant visitor questions.
- Finally, keep up to date on the museum’s code of ethics and privacy rights. Alert visitors upon entering a space that they might be included in a study. This can be achieved simply be posting a notice at the entrances. Be prepared and willing to answer questions visitors raise, and respect their privacy should they decline participating.
List of Figures
Figure 1. Learning in museums (Gammon 2003).
Figure 2. Generic Learning Outcomes from Museums, Libraries and Archives Council (www.mla.gov.uk).
Figure 3. Powering the Future design (map) with clusters identified.
References
Barriault, C. and Pearson, D. (2010). ‘Assessing Exhibits for Learning in Science Centers: A Practical Tool’, Visitor Studies, vol. 13, no. 1, April, 90-106.
Burrow, S., Diment, P., and Youngs, M. (2008). Evaluating Origins: how an exhibition is experienced by visitors. Accessed June 8, 2016. http://www.museum-id.com/idea-detail.asp?id=18
Gammon, B. (2003). Assessing learning in museum environment: A practical guide for museum evaluators. 1st ed. London: Science Museum, London, pp. 1-28. Accessed May 31, 2016. http://sciencecentres.org.uk/events/reports/indicators_learning_1103_gammon.pdf
Glasgow Science Centre (2015). Intergeneration interactions and learning on BodyWorks. Internal document. Glasgow Science Centre, 1-10.
Glasgow Science Centre (2016). Response to the Science and Technology Committee Inquiry into Science Communication. Internal document. Glasgow Science Centre, 1-4.
Glasgow Science Centre (2016). Qualitative Report for the Period 1 April 2015 to 31 March 2016. Internal document. Glasgow Science Centre, 1-21.
Kelly, L. (2009). Exhibition evaluation explained. Accessed May 31, 2016. http://australianmuseum.net.au/exhibition-evaluation
Kelly, L. (2009). Tracking and Observation Studies. Accessed May 31, 2016. http://australianmuseum.net.au/tracking-and-observation-studies
Moussorui, T. and Roussos, G. (2014). “Conducting Visitor Studies using Smartphone-based Location Sensing,” ACM Journal on Computing and Cultural Heritage, vol. 6, no. 3, May, 1-17.
Suggested texts
Bronnenkant, K. and Yalowitz, S. (2009). “Timing and Tracking: Unlocking Visitor Behaviour.” Visitor Studies, vol. 12, no. 1, April, 47-64.
Dierking, L. and Falk, J. (1994). “Family Behaviour and Learning in Informal Science Settings: A Review of the Research,” Science Education, vol. 78, no. 1, 57-72.
Falk, J. (2006). “Free–choice environmental learning: framing the discussion,” Environmental Education Research, vol. 11, no. 3, August, 265-280.
Johnston, D. and Rennie, L. (2007). “Visitors’ Perceptions of Changes in Their Thinking about Science and Technology Following a Visit to Science Center,” Visitor Studies, vol. 10, no. 2, October, 168-177.
Lang, C. (2015). Australian Museum: Aztecs evaluation report. Sydney: Australian Museum, 2-18. http://australianmuseum.net.au/uploads/documents/33859/aztecs%20evaluation%20report%20-%20final.pdf
The Museum Review 