David Seungjoo Baek

UMMA Social

Designing an Artist-audience Co-creation Mixed Reality Experience

David Baek, Raymond Su, Niharika Ved
Keywords Museum Experience, VR, MR, and Artist-audience collaboration.
Approach Research through Design & Design through Research
Duration 4 month
Adviser Dr. Silvia Lindtner

1. Introduction
2. Methodology
3. Design-Research & Research-Design Process & Methods

  • Raw Design Ideation

  • Data Collection (Observation, Interview, and Focus Group)

  • Data Collection Material Example

  • Data Analysis (Synthesis, Reduction, and Validation)

  • Discovering Problems and Design Requirements

  • Persona Creation

  • Design Ideation (Divergent Thinking)

  • Design Decision (Convergent Thinking)

  • Final Solution

4. Wireframe & Prototype
5. Reflection

1. Introduction

UMMA Social project targets to redesign the audiences’ museum experience by using current technologies. 

Conclusively, we created UMMA Social which helps people engage in the process of professional art creation throughout a system-wide Mixed Reality platform, using UMMA Social Website for civic participation and HoloLens Headset.

By doing so, we tackle the conventional concept of artist-centric viewpoint regarding art and cultural creation practice. 

2. Methodology

The question “Redesigning museum experience” is a challenge due to the broad and abstract meaning. So, we firstly framed how we design the UMMA Social project.

  1. Research through Design & Design through Research, which requires iteration between field and user research and design decision.

  2. Exploration & Discovery Research, meaning that we made a naturalistic inquiry approach to understand what was happening in the real field (Museum, in this case) first, and then, to discover problems to address with our design decisions.

  3. Seeking Socio-cultural and -political Implication. We intended UMMA Social to give an impact to the conventional way people produce and consume art and cultural values, which crucially contributes to formulating lopsided power relations between social agents in our times. UMMA Social challenges that convention.

3. Research + Design Process & Methods

3.1. Raw Design Ideation

We firstly brainstormed what kind of future user experience we can create museum audiences, which was a good practice for later works. By doing so, we tried to avoid bias in our user research and design process.


3.2. Data Collection (Observation, Interview, Focus Group)

UMMA Social project initially targeted re-designing audience’s experience with the current museum system. So, we firstly explored what is happening in museum with a naturalistic approach. And we visited 3 museums near Detroit, MI, with the questions as below. We mainly used observation and interview techniques.

We did the field exploration to 3 different museums nearby Detroit, MI, and each trip last 4-5 hours.

  • “What is happening there?”

  • “Who are there?”

  • “What are the audiences doing?”

  • “How is their behavior?”

  • “What do they really want in their museum experience?

3.2.1. Visiting Ann Arbor Hands-on Museum

The first trip was made with an unstructured method, meaning that our UX team made the first observation with an open-ended attitude to the field. 

“Just be there, and see, look, and observe”, as if we have no knowledge about museum experience”.

At this stage, we observed audience behavior and conducted short interviews with 37 audiences regarding their current experience. 

By reflecting on this unstructured method, we more articulated interview questions and observation points for the future field exploration to Detroit Art Museum and University of Michigan Museum of Arts (UMMA).

3.2.2. Detroit Art Museum

We conducted the second field trip to Detroit Art Museum. We used more specified interview questions and more articulated observation protocol, in reflection to the initial visit to Ann Arbor Hands-on Museum.

Using this, we conducted in-depth interviews with fine artists, art educator, museum curator, as well as the audience, to understand their thought, insights, and meaning about museum experience. 

3.2.3. Visiting University of Michigan Museum of Arts (UMMA)

We also reach out to UMMA at last. We interviewed museum curators, audience, and art educators to understand their thought, insights, and meaning about museum experience. 

3.3. Data Collection Material Example

Here I present some examples of our data collection materials.

Screen Shot 2018-12-26 at 2.32.55 PM.png
Screen Shot 2018-12-26 at 2.28.16 PM.png
Screen Shot 2018-12-26 at 2.28.02 PM.png
Screen Shot 2018-12-26 at 2.28.23 PM.png

3.4. Data Analysis (Interpretation, Synthesis, and Validation)

We analyzed all the qualitative data collected from the above field exploration : Notes, Memo, Photos, Interview and Focus group recordings.

Most importantly, we firstly wrote down our preconception on notes, and critically analyzed our collected materials in order to best get rid of bias towards the data. 

For 3 weeks, we conducted Affinity Diagram and Thematic Open Coding by iteratively reflecting on our qualitative data that we generated from Field Notes, Memos, Photos, Interview and Focus Group.


3.5. Discovering Problems and Design Requirements

By conducting iterative data reduction and validation process, we generated key insights of the current audience’s needs and desire in their museum experience. In this section, I reveal the results very shortly with 7 theme cluster concepts as below.

  1. Cultural Capital

  2. Audience Socialization

  3. Willingness for Peer-participation

  4. Resistance to Authority and to Ordinary Life

  5. Herd Instinct

  6. Authenticity

  7. Feasibility

Then, we generated 4 critical questions that our final design solution will answer in order to satisfy the 7 theme cluster concepts (audience needs), as below.

  1. How can we increase participation and co-creation of artworks?

  2. How can we design a platform that empowers the artists, the audience, and the museum?

  3. How can we increase representation of the socio-economically marginalized in popular arts?

  4. How can we make the art-making experience more personal?

3.6. Persona Creation

Based on the Data Analysis and the Problem Extraction, we created 4 personas who are the potential user of “UMMA Social”, the artist-audience co-creation platform.

3.7. Design Ideation (Divergent Thinking)

Based on the personas, we divergently explored how we can re-design the current museum experience by using technologies. And we brainstormed and generated 10 different design concepts. Here I present the initial sketches only.

3.8. Final Design Decision (Solution - Convergent Thinking)

We constantly reviewed our data analysis sets and personas, during the process of the above design ideations. Through the iteration, we made the final design decision to create UMMA Social.


While we went through the final design decision process, we also simulated our design concept with a paper prototype.

3.9. Final Solution

Finally, we created UMMA Social (University of Michigan Museum of Art): a system-wide Mixed Reality platform, using UMMA Social Website for civic participation and HoloLens Headset for artist.

UMMA Social is an innovative approach to engage the audience into the process of art creation by giving them the ability to co-create artworks and to share partial ownership of the final pieces.

Screen Shot 2019-01-08 at 3.20.49 PM.png

UMMA Social enables two distinct type of end user; the audience and the artist. Simply put, it is an inter-communicative co-creation platform to facilitate many audiences engagement in the work of an artist.

4. Wireframe & Prototype

4.1. Wireframe

Screen Shot 2019-01-08 at 3.38.24 PM.png
Screen Shot 2019-01-08 at 3.38.15 PM.png

4.2. Hi-Fi Mockup

4.2.1. Audience Experience

1) Website Landing Page

  • Audience can choose the type of artwork that they want to partake in.

  • Currently active artists are shown on the landing page, to help audience select a specific artist to co-create artwork with.


2) Artist Information Page

  • When audience selects a certain artist on the landing page, the artist’s introduction page shows up.

  • This page shows the artist’s information, so that to help the user to understand the current works.

  • Audience can select which artwork to partake in.

  • Audience can also subscribe or donate to the artist.

Screen Shot 2019-01-08 at 4.43.37 PM.png
Screen Shot 2019-01-08 at 4.43.52 PM.png

3) Co-creation Channel Page

On this channel, audiences can contribute to co-creating an art piece with an artist. This channel enables the audience to chat, create poll, recommend, and pull up reference images, to influence the artist to reflect on their artistic expression.

While the audience have a rather rich interaction experience, the artist only sees aggregated data, if they choose to, to help keep them in the 'flow' during co-creation.

Screen Shot 2019-01-08 at 4.37.06 PM.png
Screen Shot 2019-01-08 at 4.37.43 PM.png
Screen Shot 2019-01-08 at 4.38.14 PM.png
Screen Shot 2019-01-08 at 4.37.29 PM.png
Screen Shot 2019-01-08 at 4.37.58 PM.png
Screen Shot 2019-01-08 at 4.38.23 PM.png

4.2.2. Artist Experience

On the other hand, the artist experience will be Mixed Reality experience - where there are graphics overlay during co-creation. Artists will still be able to draw, paint or create art with the tools they are comfortable or familiar with.


They will communicate with audiences via voice and get feedback through the graphic overlays such as word cloud or 3D charts. If they choose to, they can look at the messaging thread to follow the conversation in greater detail or customize their virtual interface to their preference. Here are a series of sequence to log in into the mixed reality experience.

Screen Shot 2019-01-08 at 9.46.50 PM.png
Screen Shot 2019-01-08 at 9.57.28 PM.png
Screen Shot 2019-01-08 at 9.57.51 PM.png
Screen Shot 2019-01-08 at 9.57.39 PM.png
Screen Shot 2019-01-08 at 9.58.02 PM.png
Screen Shot 2019-01-08 at 9.58.17 PM.png
Screen Shot 2019-01-08 at 10.02.03 PM.png
Screen Shot 2019-01-08 at 10.02.23 PM.png

5. Result

The final result of an art piece :
is jointly and organically produced by the artists and the audiences, like a jazz improvisation where creativity and spontaneity coexist.

For Audience :
The artwork produced will be displayed in the museum, posted on the museum's website. Those who contributed can download and digitally own the piece they have created.

For Artist :
Artists can build a following and have a consistent income stream through the Patron model as they get involved more. The museum would hosts these artists and function as a decentralized guild, by providing these artists with the tools and the space they need, and in return, artists would co-create new artworks through this service.

We hope that this symbiotic relationship will also incentivize the number of people visiting museums, because they have a greater sense of ownership in the artworks that their local museum display.