Scan - Know - Eat

SafeBite: AR App

Institution

The University of Sydney

Master of Interaction Design and Electronic Arts

My Deliverables

UX & AR Product Design, Background & User Research, User Testing, Low-Fidelity & High-Fidelity Prototyping, Visual Branding

Tools

Unity AR, Vuforia, Figma, Procreate, Photoshop, Illustrator, InDesign & After Effects

Timeline

10 weeks

Individual Work

AR UX/UI Design

Florence Zhu: Architecture Student

(Architecture background)

Challenge

Food allergies affect an estimated seven hundred million people worldwide, making them a substantial health concern with a continuously rising prevalence. Peanut allergies, in particular, can trigger severe, life-threatening reactions. Those with allergies, especially when traveling to unfamiliar destinations, encounter difficulties in identifying allergens in food products. The introduction of a digital assistant equipped with augmented reality (AR) technology to provide ingredient information can significantly enhance their travel experience by addressing these challenges.

"How can AR technology enhance the user experience for individuals with specific dietary needs?"

Solution

SafeBite AR represents a user-friendly solution tailored to individuals with food allergies. It incorporates essential features, including ingredient detection across languages, an emergency call button, and information on available allergy medications at the user's travel destination. This feature streamlines the process of allergen avoidance and accessing necessary medical assistance. It saves time and provides peace of mind, particularly in foreign countries.

Personalised Allergen Profiling

Empowering Users with Customised Safety

SafeBite revolutionizes dietary management through its intuitive allergen registration system. Users can effortlessly curate a comprehensive list of ingredients to avoid, tailoring the app to their unique health needs. This user-centric approach ensures a personalized and adaptive experience.

AR Ingredient Detection

Transforming Food Labels into Interactive Safety Guides

Harnessing cutting-edge AR technology, SafeBite turns smartphone cameras into powerful allergen detection tools. Its innovative system swiftly identifies and highlights potentially harmful ingredients, transforming routine label checks into an engaging, visually intuitive experience.

[data-wf-bgvideo-fallback-img] { display: none; } @media (prefers-reduced-motion: reduce) { [data-wf-bgvideo-fallback-img] { position: absolute; z-index: -100; display: inline-block; height: 100%; width: 100%; object-fit: cover; } }

Unity AR Prototype: Vuforia Image Detection and C# Programming have been used

Impact

97 %

Conducted three rounds of user testing with participants who have food allergies, achieving a 97% positive feedback.

A+

people

Received excellent feedback from my tutor, who praised the project for its practical application, awarding it the highest grade.

DESIGN PROCESS

Understanding The Problem

A recent study has revealed that food allergies affect ten per cent of the global population, with their prevalence steadily increasing over the past decade. Food allergies have become a significant health concern, leading to frequent emergency room visits worldwide. Particularly vulnerable are children who struggle to decipher the ingredient labels on food packaging (Food Insight, 2022).

"Over the past 25 years food allergies have skyrocketed to become a major health issue affecting 220 million people globally"

Among food allergies, peanut allergy is one of the most common and severe, potentially triggering life-threatening reactions like anaphylaxis, throat swelling, difficulty breathing, sudden blood pressure drop, and more (FARE, 2021). Consequently, individuals with peanut allergies face the risk of severe consequences upon consuming such food items, necessitating strict avoidance. Unfortunately, the cause of food allergies remains unknown, leaving ingredient avoidance as the sole means of survival.

The challenges escalate further when individuals with food allergies travel to foreign countries where they are unfamiliar with the spoken language. While it is relatively easier to avoid peanuts in restaurants, identifying the allergen in supermarkets becomes arduous without the aid of a translator or assistant, particularly when accompanied by children (Allergy & Anaphylaxis Australia, n.d.).

In light of these difficulties, the presence of a digital assistant capable of providing information about allergenic ingredients would greatly benefit individuals, enhancing their travel experiences by facilitating easier avoidance of allergens. Augmented reality (AR) technology holds promise as an ideal and practical solution for this purpose.

Why AR?

AR technology is the ideal choice for providing a seamless and intuitive user experience for people with food allergies, especially when travelling to countries where the language they are not familiar with is spoken. With the accessibility and interactive nature of AR, displaying allergen information on a smartphone screen through visual elements can save time and reduce stress for users, blending the digital and physical worlds effortlessly.

Benefits of AR

Easy to use

AR technology can provide intuitive and user-friendly interactions, allowing users to easily navigate and interact with the augmented content.

Immersive Experience

AR enhances the user experience by overlaying virtual elements onto the real world, creating an immersive and interactive environment that can be engaging and enjoyable for users.

Easy to Access

AR can be easily accessed through smartphones, which are widely available and commonly used. This accessibility makes it convenient for users to experience AR content without the need for additional hardware.

User Engagement

With AR, users can actively participate and engage with the content, resulting in a more interactive and memorable experience compared to traditional interfaces.

User Persona & Storyboard

Using the findings from previous research and insights, I generated a user persona and storyboard to gain a deeper understanding and foster empathy towards the target users. These tools proved instrumental in shaping a more compelling design concept.

Essential Elements of The Design

By holistically synthesising all the data collected from previous research, I identified and extracted the essential elements necessary for the final prototype.

Allergen Detection

Identify allergens like "peanut" on food labels, displaying a 3D object and warning sign for visual impact.

Language Translation

Translate detected allergens into the user's preferred language for easier comprehension.

Interactive UI

Implement virtual buttons for user engagement with 3D objects and additional information.

Safety Features

Include an emergency call function with country-specific contact numbers and provide access to local allergy medication information.

Animation & C# Code

Several animations have been implemented to make the app more enjoyable and playful. Firstly, I created the animations to make the 3D objects move when the virtual button is activated. Secondly, "Transform.Rotate" has been used to spin the exclamation mark constantly. Thirdly, the shader that makes an object bounce has been added to every 3D object. Lastly, "Application.OpenURL" has been utilised for the emergency call and medicine information buttons.

AR Design: Unity Prototype

Utilising Unity AR and Vuforia, renowned AR software, I successfully developed a prototype for user testing. Despite encountering challenges with Vuforia's text tracking feature, which was not working as intended, I sought guidance from AR experts at the University of Sydney and online AR communities to resolve the issue. Ultimately, I pivoted to image tracking for the prototype, resulting in successful development for testing.

UI Design: Low Fidelity Wireframes

I also created wireframes incorporating essential interfaces and pages to enhance the user experience of the AR detecting feature. These make the product more usable and accessible to a broader range of users, catering not only to individuals with food allergies but also to those with specific dietary preferences, such as those who cannot consume coriander and other polarising ingredients.

User Testing & Evaluation

Using the AR Prototype and Lo-Fi Wireframes, I conducted user testing employing various Design Thinking methods, including the Think-Aloud protocol and SUS Questionnaire. Carefully selecting participants with food allergies and specific dietary preferences, I immersed them in a real-life experience at a Korean market using the AR prototype.

The results of the user testing were overwhelmingly positive. Participants found the product easy to use and highly beneficial, particularly when navigating countries where unfamiliar languages are spoken. Additionally, feedback indicated that children would enjoy using the product due to its engaging interactions.

Key Insight & feedback from user testing

FINAL DESIGN

Design System & Branding

After successful user testing, I began developing the final design, incorporating feedback from the participants. Given the relatively complex technology utilised in the product, such as AR, which may cause user reluctance, I prioritised designing the brand identity to foster a friendly and easy-to-use approach.

The colour palette and fonts were carefully selected from WCAG (Web Content Accessibility Guidelines), balancing aesthetics with usability to ensure accessibility for all users. Also, I created the logo to establish a memorable brand identity for the app.

SCAN • KNOW • EAT

I crafted a friendly and inviting brand identity for SafeBite, encouraging users to give it a try. Additionally, I designed a poster featuring the slogan 'Scan · Know · Eat', simplifying the product's usage steps and piquing the audience's curiosity. Furthermore, I created 3D icons using the AI tool DALL·E 3, refining their details in Photoshop to ensure they aligned with the brand's identity and maintained consistency.

Final UI Design & Prototype

After receiving feedback from user testing, I redesigned the final prototype. Firstly, many participants suggested that 'View Inventory' and 'Scan History' did not need to appear as separate pages. Therefore, I removed the 'Scan History' page and integrated it into the 'View Inventory', simplifying and enhancing usability.

Secondly, some participants expressed confusion with the layout and scrolling system while navigating the Lo-fi prototype. To address this, I created a separate page for recommending allergy-friendly restaurants. As a result, participants were pleased with the final design's layout and structure, finding it extremely easy to use and providing only essential information and buttons.

INITIAL ITERATION

Lo-Fi Prototype: Home Screen

FINAL ITERATION

Hi-Fi Prototype: Home Screen

Add Food to Avoid

To use the AR detection feature, users can add foods they want to avoid, such as ingredients they are allergic to or dislike. Simply click the 'Add Button' at the bottom.

Detect Food Ingredients with AR

After adding foods to avoid to their inventory, users can start using the AR detection feature. The app displays AR animations when detecting any of the added ingredients, whether on food labels or restaurant menus.

In addition, users can access emergency contact numbers and medication information specific to their location by clicking the 'SOS' and 'Medicine Icon' buttons at the bottom. For quick instructions, users can click the 'exclamation mark' at the top, which provides a brief sentence: 'Scan food labels or menus to detect ingredients you cannot eat'.

Analyse Usage History

The app automatically maintains a scan history and provides usage analysis. Users can review their past activity patterns and trends based on the analysis data.

Discover Allergy-Friendly Menus

SafeBite also provides information about restaurants or cafes offering allergy-friendly menus. This feature is particularly useful when traveling in countries with limited options for people with food allergies, such as South Korea, Japan, Bangkok, etc.

What I learned

01 Adaptive Problem-Solving

Turning obstacles into opportunities through innovative solutions

Quickly adapted to the unavailability of text tracking in Vuforia
Sought guidance from multiple sources to find alternative solutions
Successfully implemented image tracking to overcome the initial challenge

02 Collaborative Innovation

Leveraging collective expertise to drive breakthrough results

Engaged with Vuforia's support team and Usyd tutors for expert advice
Synthesised insights from various sources to create a novel approach
Implemented suggestions effectively to enhance project outcomes

03 User-Centric Design

Addressing real-world needs with empathy-driven technology

Recognised the global significance of food allergy challenges
Developed a solution tailored to a pressing user need
Received positive feedback, especially from the target user group

04 Agile Development

Maximising impact within constraints through strategic prioritisation

Worked efficiently under time constraints
Delivered a functional prototype that addressed core user needs
Balanced project scope with available resources effectively

Sophie Adams

· Travel Blogger

SafeBite AR would be a game-changer for my travels. It gives me confidence in navigating foreign cuisines without risking allergic reactions

David Maas

· Business Traveler

As someone with severe food allergies, SafeBite AR would be my essential travel companion. It’s incredibly reliable and easy to use

Emily Jackson

· Parent of a Child with Allergies

SafeBite AR would make family outings stress-free. It'd help us quickly identify safe food options for our child, no matter where we are

Rachel Patel

· Health Coach

SafeBite AR aligns perfectly with my clients’ needs. It’s intuitive and empowers them to make informed choices about their diet

Chris Bouras

· International Student

Navigating food options in a new country was daunting until I found SafeBite AR. It’s like having a personal food safety guide everywhere I go

Linda Smith

· Retiree with Multiple Allergies

SafeBite AR would simplify my grocery shopping and dining out. It’s a must-have for anyone managing food allergies