The Virtual Courtroom

Overview		Challenge		Role
This internship was focused on creating an MVP to address the need of providing a space for students to learn aspects of a UK courtroom. This was in collaboration with a fully remote university called Open University. As the project proceeded there was an opportunity to address the need of creating a social space and a platform for students to practice mooting (mock trials).		Pushing the boundaries of remote learning through using different technologies like Virtual Reality or even immersive environments, create another layer of interaction in education. How might we create a place for Law students at Open University to learn about the UK courtrooms?		My role for this project was leading UI, visual design, and UI implementation. I also worked on UX, 3D modeling (environments and objects), user research, and development (Unity). In addition, I was working with two other Interaction Design students at Sheridan along with a professor who was a mentor/supervisor to the project. This internship lasted three months from June 2021 to August 2021.

Start of the project

Online learning has always been available, even pre-pandemic as it comes with many advantages like learning at your own pace and the flexibility of learning anywhere. This project is a collaboration between two schools, Open University in the United Kingdom and Sheridan College in Canada.

The team consisted of myself, two other Interaction Design students in the program, and a supervisor who overlooked our progress and acted as an advisor/mentor. We all had different skill sets and specializations, which made the project very unique and possible – I wouldn’t be able to do it without them!

Our process and team

For our process, we adopted IDEO’s human-centered design process:

The design process from Design Thinking for Educators

There were several factors why we decided on using this process:

Using a structured design process would help us establish realistic deliverables
It gives us the opportunity to think about each process and how we can tackle them with our constraints
It sets up for success!

As we started with the project, I saw each of us in the team having unique skill sets:

Our specializations and roles!

There was a big opportunity to collaborate on various features and ultimately, we decided on working in UX and UR as a team at the beginning (since we were all Interaction Design students) and then utilizing each of our strengths as individuals for the development phase of the project. We did it this way because it was the most efficient to go about in this project without too much overlap, and it assigned a team member as a “lead” on their specialization! For example, we had a team member teach us how to create character customizations via blend shapes in Blender.

Our research plan

Starting off the Discovery phase in the design process, we had to decide on what we really needed to focus on as OU wanted the tool to be a “virtual courtroom” simulator that is very similar to a video game.

The spec sheet that OU provided to us

Even though OU had specifications on what they wanted, we had three major constraints:

Time: we only have three months
Our technical knowledge for developing the tool: what OU wanted in regard to certain features not feasible due to our technical knowledge
Technology: some aspects of technology is simply not possible and needed something custom

These constraints dictated our decisions moving forward in our research plan, which provided us a path to focusing our efforts in doing primary and secondary research. With this in mind, we decided to focus on:

Create assumptions on what students would want and the prototype itself, based on some of the specifications OU had in mind
Understanding the landscape of Education Technology
Deciding on what to focus on the constraints we had

With a research plan solidified in hand, we did a lot of design research activities ranging from trends/competitor analysis to user spectrums to a defined features page (what we are focusing on).

Convergence map

Competitor analysis

From and To Exploration

Assumptions on essential features for the tool

Research methodology and results

Going into the Interpretation Phase, where we did most of our primary research - We did surveys and interviews with OU Law students, alumni, and also industry to gain an understanding of UK Law and what students wanted in a “virtual courtroom” environment.

Our research data became very clear when we saw:

Students wanting a tool that goes beyond Zoom or Teams for doing mock trials, as they wanted a real experience rather than looking at screens
The importance of video camera feeds to understand the opposition or judge’s faces when they are doing their submissions
Majority of students use laptops, so we had to keep in mind that there are technical limitations such as low-to-medium end hardware

With all this research conducted, we saw opportunities in the 3D space – Although we considered a wide variety of options such as:

2D High Dynamic Range Video
A “deluxe” zoom
2D Virtual Environment that is top-down

We gravitated towards using Unity as a development platform because it had the technology we needed for video and multiplayer functionality. This allowed us to use third-party plugins for the features to be implemented, so we can be efficient in putting together the prototype in less than three months.

We created a proof of concept prototype early on in the project

As we moved into the Ideation Phase, each of us separated in focusing on different aspects of the tool. This included prototyping to setting up the back-end for multiplayer to building the environments in Blender.

Visual design

I handled the visual design for the tool including things like iconography, branding, and also typography. Keeping in mind my goal of keeping things simple for the student, I decided to use the theme of simplicity for the visual design in the tool.

One of the challenges that I encountered was the tool’s purpose was a learning tool and not a game, this changed a lot of the visual design because I had to think of it in a different direction. The importance that this was a learning tool rather than a video game kicked in when we were doing interviews.

Very early mock up of role selection. Please note the character image is a placeholder and would be replaced by 3D models.

Very early mock up of room selection. Please note the character image is a placeholder and would be replaced by 3D models.

Typography was very important to me and I spent a lot of time picking various fonts that would suit the tool. Originally, I selected a font called Industry because of the name “virtual courtroom”. I wanted something futuristic but I was concerned that using display fonts would be difficult to read for some people.

Expermenting with fonts on the menu screen

I gravitated towards using New Hero and Roboto because they were fonts designed for the digital screen and were very easy to read and identify. New Hero had some meaning behind the text where it was designed for “heroes”, where I thought that today’s lawyers are heroes defending their defendant (in a way).

Using the fonts New Hero on avatar customization. Please note the character image is a placeholder and would be replaced by 3D models.

Keeping the simplicity theme was fairly straightforward to design because there weren’t many graphics since the goal was function over form. Having that minimalistic look also helped with the visual design and the majority of students who tested the tool liked the overall look of the UI and visual aspects of the tool. I kept the iconography relatively simple because I believed that icons should be easily identifiable at first glance - The tool didn’t require too many icons, other than the key binds.

In addition to the iconography, I worked on the branding of the tool which was the logo to be placed on the menu screen and also to be used in different things like the launch icon and documentation. I experimented with a team member on various concepts, using courtroom iconic things like the scales, gavel, and other things for the logo.

As we found out from the OU team, they mentioned that the gavel isn’t represented much in actual law but rather in movies or TV shows. We found that interesting as we didn’t realize this and decided to use scales as a better representation of the law.

More iteration of logos

As I went through iterations of different logos, my goal was to incorporate two things into the logo:

Diversity and inclusion
The formalities of the court

I believed in diversity and inclusion as it was equally important to Open University, they believed that everyone should have access to education - Regardless of their standing. I wanted to incorporate this with different colours, symbolizing the various races we have in the world.

Final logo

With the final logo, I used a half and half concept where one side symbolizes the idea of a formal courtroom, while the other symbolizes the idea of diversity and inclusion in the courtroom. This was important to all of us in the team because we were always focused on inclusion when creating the tool, especially designing the character creation.

For the font, I used New Hero and also Freight. I wanted to keep a formal tone with the word courtroom, which was why I used a serif font (Freight). While keeping it modern with the word “virtual”, using the font I selected for the UI, New Hero.

UI Design

I was focused on the UI Design of the tool, which led to several challenges that I encountered when I was developing the UI design for the tool:

From the surveys and interviews we performed with students, a trend of “I’m not tech-savvy” or “I don’t play video games” was commonly said among students. This made me focus on making it as simple as possible, so it can be straightforward and intuitive.

This resulted in collaborating with a team member to create a user flow to envision what a student would go through in the menus of the tool. Our goal was to keep it as simple as possible by using simple UI text that can be easily distinguishable like “Explore the Courtroom” or “Moot practice”. I also implemented additional descriptions, so students can be very clear on what they were hovering over or clicking on.

Menu user flow

Wireframes

Refining the copywriting and UI text was very important, which led me to work with the OU team to further refine and make it consistent with UK English.

The Excel sheet we worked off from, there was a lot of text!

The amount of information being shown was another area of challenge, as we didn’t want to overload the student with so much information. Although, my goal was to keep it simplistic – While another team member had a different vision on how it would be structured.

I and a team member decided to let the students decide and see what level of information density they preferred. We did an A/B test with several students on two versions of the UI in an interactive mock-up in Adobe XD using scenarios to test:

Ver A. Please note the character image is a placeholder and would be replaced by 3D models.

Ver B. Please note the character image is a placeholder and would be replaced by 3D models.

I suggested doing an A/B test because I wanted to find out what students preferred in regard to the visual design of the UI, in addition to the level of information being shown in either of the versions. As these students haven’t encountered many video games before, it can be frustrating when you are confused about where to click or navigate – So my goal was to reduce that frustration.

The findings from the test were split as some students preferred a simplistic approach, while some liked a denser approach in the other version of the UI. Using the feedback provided, I had to balance both opinions of what students wanted and my vision of the UI for the tool.

Another challenging aspect was adding things that students wanted like a timer and a clock to keep track of time

From our interviews, students wanted a timer and a clock as it helped them keep track of their submissions when they were presenting. Typically, if this was done via Zoom or Teams – Someone would indicate to the student by a sign or gesture that they have 5 minutes left. This wasn’t particularly effective as it could be missed by the student. This got me thinking of adding these features to the UI so that it would help students:

HUD

Locations of the timer and clocks were placed in the corners of the screen as not to distract the student when doing their submission. The idea was that since it was in the corner, a student can easily glance at it during their submission. Although, some students requested that they don’t want the timer or other aspects of the UI as it could be distracting – This was resolved by implementing the ability to turn off different parts of the UI on and off in the options.

I wanted the UI to fully customizable as possible because it allows students to tailor the tool to their needs. Not every student would like a timer, as students brought up to me during user testing and it was something I fully understood through various video games where UI customizability was key.

Colour was another large challenge due to how the environment’s lighting was set up in the tool.

I kept things basic by using whites, blacks, and greys in the UI to focus on the environment rather than the UI itself. The 3D environment was a little dark due to the lighting nodes placed around the environment and we had to be very careful not to overdo it with the lighting.

Using basic colours allowed for the font to have great contrast against the background for heavy text descriptions

Using basic colours like white on the UI, made it clear to read and contrasted very well with the environment - Since the environment was very muted in colours (except for the hallway since there were pictures).

The UI is still very visible against the environment

Debugging, 3D modeling, and a lot more

As the team kept working on the technical aspects of the tool and the 3D environment, I wanted to help in some way as the UI and visual design were finalized. Some of the areas that I assisted, and I felt created an impact were:

Addition of detail in the environment with small 3D items

I wanted the environment to be realistic as possible and the idea that I brought to the team was to add in small objects that would simulate an office or courtroom setting. I and the team looked at various photos of UK courtrooms and determined from those references what kind of items we would add-in.

To save time, I modeled several things that I wouldn’t be able to find on the Unity store or places that offered free 3D models. I wanted to save a lot of time, which finding models that were free for some objects like plants or books – Made things a lot more efficient rather than modeling them from scratch.

Debugging and error handling

Debugging and error handling was something done towards the end of completing the prototype and during user testing. I would admit that it was something we haven’t considered due to the more important aspects of the tool, we had to implement but I still realized that this was very important that feedback should be given if something goes wrong.

Being inspired by vague error messages in video games, my goal was to be as clear as possible to let the student know what the error is.

The result and user testing

As the MVP is finalized in the last few weeks of the project, such as final UI implementation, 3D environment work, and character customization, we tested the MVP with 10 students and 3 alumni. We tested over a period of two weeks and continuously iterated and debugged after each test was run.

Some of the goals of testing were:

Exploration: we wanted to see if students are able to get around without any issues and the environment was free of issues (i.e. falling through the environment, any open spaces due to modeling)
Feel of controls: I was very concerned that the controls will be hard for students as they told us they don’t play video games
UI: Can the student understand the menu and navigate around easily?

The result of testing was very positive:

Students were generally positive and excited about the tool as they haven’t encountered something like this in their program
Students felt very intimate just like in real life due to the moving avatars, webcam, and audio. They mentioned that it felt like an actual physical interaction but digitally
The overall UI was very positive as students were able to connect to a room without issues and were very appreciative of the tutorials provided as some students were not familiar with controls from video games (WASD)

Final MVP

Playing around with the MVP!

Students can interact with picture frames in the hall way to learn a bit more about the history of the UK courtroom

Exploration mode: In the court

Exploration mode: Another view of the courts

Inside the menu/setup screen

Avatar edit/creation

The Takeaways

Talking to the user base is important

I learned that talking to users was very important to identify their needs. Talking to the OU Law students was very important and proved our assumptions at the start of the project. What OU assumed was way different compared to what students told us, which was that they wanted an interactive experience with video – Rather than a virtual world with just simply avatars. We saw the tool providing a way for students to practice their presentation skills in a safe environment without trouble.

Working collaboratively with team members who have different skillsets

This was the first time that I worked with two other peers from the Interaction Design program in this internship, who had different skill sets than myself. I learned to rely on my team members when I have problems like in Unity, coding, or 3D modeling. Relying on my teammates when I need it is great as they have some skills that I don’t necessarily have. In addition, if my teammates are having trouble, they can lean on me for help – Whether it is design-related or anything in-between.

The value of user testing and proving assumptions

Throughout this project, we user-tested with students to ensure that we are on the right track. I learned that doing this keeps us on our toes as sometimes students find things that we don’t realize that would be an issue. For example, a student pointed out that the signage should be improved as it was a bit confusing. Originally, I added the signage to make it realistic but didn’t realize that it would help students to find their way around the 3D environment. By constantly getting users involved with the design process, I learned that feedback will shape the final product that the students could happily use in the end.