What I've Done

There are a few instances where I did projects out of my official academia journey, some while working as a research assistant, a freelance UX Developer (working for the university), and many when I was actively participating in hackathons. Here is a brief outline of the projects I will showcase in this page. I do want to acknowledge that many of the projects are done in a team and I am truly grateful of everyone's contribution to the success of the projects.

In a pioneering research project exploring the impact of virtual reality (VR) cues on vaping behavior, I played a crucial role in managing data analysis and authoring the resulting academic paper. The study leveraged VR environments to investigate how virtual cues can elicit subjective cravings among different user groups, particularly focusing on vapers. My responsibilities includes the intricate analysis of both physiological and subjective data, employing robust statistical methods to unravel the complex interplay between VR-induced cues and vaping cravings. The insights derived from this analysis were instrumental in drafting a comprehensive paper that not only presents our groundbreaking findings but also sets the stage for future research in VR applications for addiction and rehabilitation. This project greatly improved my skills in data analytics and allowed me to practice more scientific writing. It also underscore my contributions to cutting-edge research at the intersection of technology and behavioral science.

I have done quite a few projects while I was in university. All of them are affiliated to the university in a way but most of them have external collaborators which I interact with as my clients (people who I am developing for). To have a coherent understanding of what I've done, I classified the projects I've done via the clients.

University of Otago

During and after my PhD, I served as a research assistant for the University of Otago, participating in several intriguing research projects. One such project aimed to overlay historical photos onto the present environment using Augmented Reality (AR). This innovative approach allowed users to view a location as it appeared in the past, superimposing old photos of buildings onto their current counterparts, creating a compelling mixed reality experience.

My role in this project involved developing a proof-of-concept prototype using Unity3D. The prototype utilized a 360-degree video recording of an environment, with the historical image superimposed onto it. Users could adjust a slider to control the opacity of the photo, blending the current reality with the historical image. I also developed an AR prototype for indoor use, although outdoor functionality was not achieved during my tenure. The project was subsequently continued by an honours student, resulting in a working prototype that adjusted the photo to match the viewer's perspective.

Another project I contributed to focused on transforming standard 2D videos into 3D videos for VR headset viewing. While existing solutions required extensive time, costly hardware, and significant resources, our approach was designed to work with a standard consumer PC. Although I didn't develop the VR video pipeline, I designed and conducted a user study to evaluate its quality, providing essential feedback to the principal researcher for product improvement.

After completing my PhD, I transitioned to a part-time research assistant role, focusing on modularizing and revamping my PhD project—an AR sports spectating app. The app, built on the work of numerous contributors, needed a comprehensive overhaul and documentation. My task was to simplify and streamline the development process, making it easier for future researchers to continue the project. This work will significantly benefit the application's development and the progress of future research.

Otago Boys High School (with University of Otago)

In response to pandemic-related travel restrictions, the University of Otago's Science Communication department collaborated with Otago Boy's High School (OBHS) to create a virtual reality (VR) solution for their field trip to Matukituki Valley. The project involved filming a series of 360-degree videos, which were then edited together to form a cohesive narrative. I assisted with the video editing and providing feedback on the footage and filming methods.

Once the video editing was complete, I utilized Unity3D to incorporate interactive elements into the VR experience. This allowed students to engage with points of interest (POIs) within the virtual environment, triggering the display of additional information for an educational and personalized experience. The final prototype was tested with the cohort of students who were unable to participate in the physical field trip. To ensure broader accessibility, we also developed an iPad version of the experience, enabling more students to partake in the virtual field trip.

Throughout the project, I was responsible for the development and visualization of the VR experience, while also providing expert feedback related to VR technology. This project was a testament to the power of VR in providing immersive and educational experiences, even when physical travel is not possible.

Baba Nyonya Heritage Museum

I had a fantastic experience developing interactive exhibits for the Baba Nyonya Heritage Museum, located in Malacca, Malaysia. The museum is dedicated to preserving the local history of the Peranakans, an ethnic group of early Chinese migrants who intermarried with locals between the 14th and 17th centuries. I began working with the museum during my undergraduate final year project, where I developed a conductive paint poster. This initial collaboration led to further opportunities to assist a PhD student in creating two additional multi-sensory interactive exhibits: a goldsmith simulator and a "batu boh" (stone grinder) simulator.

The goldsmith simulator provides an authentic and engaging experience by incorporating actual hardware components, such as a blowtorch foot pump. We embedded sensors in the foot pump to detect pressure and used a Leap Motion sensor to track users' hand movements. Participants follow on-screen instructions to virtually create a pendant, selecting their desired frame and coin, weighing the materials, and using the blowtorch to solder the pendant together. My role in this project primarily involved evaluating the system, while also contributing to brainstorming sessions and project discussions.

The "batu boh" stone grinder simulator aimed to recreate the traditional grinding process while teaching users about the role of the stone grinder in Peranakan cuisine. We collaborated with a stage set craftsman to create a custom-made, realistic replica of the grinder. I designed blueprints to accommodate the necessary hardware components, which included a Raspberry Pi with an RFID reader and infrared sensors to detect grinder rotation. After several iterations, we achieved a visually authentic grinder that provided a semi-realistic grinding interaction suitable for all ages.

Users engage with the exhibit by inserting RFID cards representing different ingredients in the correct sequence and grinding for a specified duration. Upon completion, they receive a toy capsule containing a sticker depicting a local cake based on the chosen ingredient combination. This sticker can be exchanged for a real sample of the cake at the museum cafe. Throughout the experience, voice narration guides users, and a puff of vanilla scent enhances the immersive atmosphere upon completion of the grinding process. For this exhibit, I was involved in all aspects, from designing the grinder and contributing to code development, to conducting user testing at the museum.

River of Life Outreach Program

The River of Life Outreach Program is an educational initiative affiliated with the Malaysian government, designed to raise awareness about the importance of rivers and their protection. This program is a collaborative effort with Asia Pacific Environmental Consultants Sdn Bhd (ASPEC) and focuses on creating interactive exhibits tailored for the general public, with a particular emphasis on engaging the younger generation.

In this project, my supervisor and I collaborated to develop two interactive exhibits. The first exhibit was inspired by the conductive paint poster I previously mentioned in my work for the Baba Nyonya Heritage Museum. The second exhibit introduced a new experience, incorporating RFID tags, LED light strips, miniature landscape models, and animation videos. By integrating these innovative and immersive elements, we sought to foster a deeper understanding and appreciation for rivers and their conservation among visitors. Our goal was to encourage responsible environmental stewardship through engaging and educational experiences.

The first interactive exhibit we designed was an intricate landscape model that depicted a section of the riverbank, complete with commercial and industrial areas, as illustrated in the image above. To create the base of the model, we carefully carved a block of styrofoam using specialized styrofoam cutters. We also embedded a Raspberry Pi and LED strips under the styrofoam that acts as indicators and feedback to guide user attention to a specific area. We then adorned the landscape with a combination of arts and craft materials, as well as miniature toy models, to bring the little town to life.

In addition to the commercial and industrial zones, the model also showcased an area where deforestation was taking place, highlighting the negative impact of human activities on the environment. Furthermore, we incorporated an interactive area that utilized RFID cards and a reader, allowing visitors to engage with the exhibit in a dynamic and immersive manner. This combination of visual and interactive elements aimed to educate visitors about the importance of preserving our rivers and the surrounding ecosystems.

The exhibit features an attached monitor that displays various issues affecting the river, which are linked to numbered RFID cards. Users can select a card, scan it on the reader, and observe the corresponding area in the landscape model light up. The monitor then plays an animation we created that illustrates the specific problem. Following the animation, the screen prompts users to choose the correct solution for that particular issue using the RFID cards. If the user selects the right solution, the area in the landscape model lights up, and another video demonstrates how that solution effectively addresses the problem. However, if an incorrect card is scanned, the unrelated section of the landscape lights up, and the screen encourages users to try another card. This interactive experience not only educates visitors about the challenges faced by the river ecosystem but also engages them in problem-solving, fostering a deeper understanding of the potential solutions and their impact on the environment.

The other interactive exhibit I've done is a reinforced version of the conductive paint poster I've developed before. From past experience, we realised that users, especially children tend to touch the poster with force. This time, we created the posters with wood and printed large stickers to stick on them. The conductive paint are painted on the wood which still works being underneath the sticker. The poster contains an illustrated map of Kuala Lumpur, Malaysia with various touchable point of interest (POI) as shown in the image above. When users touch the different POI and audio track regarding that POI will be played. We removed the projection component and added LED light strips and sound as feedback, making the interactive exhibit much smaller and easier to operate by the ASPEC team without our assistance.

University of Malaya Medical Center

In a freelance project I did shortly after my undergrad graduation for the University of Malaya Medical Center, I designed and developed a comprehensive Android application called "Pain Diary". This tool was designed to assist caretakers of children with cerebral palsy in tracking daily symptoms and observations. It was meant to simplify the doctors' appointment session by having the doctors understand what was going on in the past period of time rather than relying on what the caretaker remembers. The intuitive interface and straightforward design of the app facilitate the easy logging of daily entries, creating a detailed, digital record of the child's condition.

Through multiple interviews and meetings with key stakeholders (hospital staff and doctors), I meticulously gathered requirements and translated them into actionable design elements. The initial prototypes were created in Adobe XD, ensuring the design was user-centric and intuitive for both caretakers and doctors. Once the prototype was refined and approved, I developed the final application using Android Studio, delivering a functional tool that would enhance the communication between caretakers and doctors during consultations. Although the Pain Diary app is just a prototype for research trial, it has streamlined the communication process between the caretakers and doctors.

During my time in Malaysia, I had the opportunity to participate in numerous hackathons. These events, typically spanning 24 to 48 hours, are intense and exhilarating experiences where teams of talented individuals come together to "hack" out a minimal viable product or prototype. The goal is to transform an idea into a functional prototype within the given time frame, often working through the night to achieve this.

Over the course of the 10 hackathons I participated in, my team and I developed a wide array of innovative prototypes. Each event was a unique challenge, pushing us to think creatively, work efficiently, and collaborate effectively. The fast-paced, high-pressure environment of a hackathon is truly a crucible for innovation, and it's amazing to see what can be accomplished in such a short period of time. Among the numerous prototypes we developed, there are a few winning solutions that stand out and I'd like to share.

Materialize 3D Printing Hackathon 2016 - Winner

During the weekend of February 20th and 21st, 2016, I had the opportunity to participate in a 3D printing-themed hackathon organized by Materialise Malaysia Sdn Bhd. My team, called BrailleHub, developed a knowledge base website designed for educators of the visually impaired. The website functioned as a search engine that could translate definitions into Braille, the tactile language used by the visually impaired. Additionally, it could look up a 3D model of the object being searched. Both the Braille text and the 3D model could be printed out using 3D printing technology, enabling the visually impaired to "visualize" objects. Our innovative concept won us the Most Impactful Award. We were awarded a 3D printed trophy and an Apollo 3D printer sponsored by Designex3D. (Link to article)

Smart City Banking Hackathon 2018 - 2nd Runner Up

In this Smart City Banking Hackathon, my team and I introduced an innovative concept: a communal power buffer system for commercial buildings. In Malaysia, commercial power costs are often calculated based on peak usage for the month. Our solution proposed a shared, large-scale battery system that could serve multiple commercial entities. This system would supply electricity not only during power outages but also during predicted peak usage times, effectively reducing peak charges. The battery system would charge using solar power and off-peak power supply, promoting energy efficiency and sustainability. We envisioned this as a way for companies to maintain their normal power usage without incurring excessive costs.

To complement this system, we designed a user-friendly app mockup that allowed vendors to effortlessly pay their bills through integrated partner banks. This streamlined approach reduced the steps required for bill payment and provided a transparent view of power usage. As a key contributor to the UX/UI design of the app mockup, I utilized Adobe XD to bring our vision to life. Our innovative solution and comprehensive presentation earned us the position of second runner-up in the hackathon.

JomHack Smart Cities with LoRaWAN 2016 - Finalist

This Smart City Hackathon centered around the use of LoRaWAN technology. LoRaWAN, short for Long Range Wide Area Network, is a protocol designed for wireless battery-operated devices in a regional, national, or global network. It provides long-range, low-power wireless connectivity that is ideal for Internet of Things (IoT) systems. Leveraging the benefits of LoRaWAN, we conceptualized a system of smart traffic lights. Our system was designed to prioritize emergency vehicles, ensuring that traffic lights would always favor their route.

We brought our concept to life with a prototype consisting of multiple LEDs connected to a Raspberry Pi 3B, equipped with a LoRaWAN module to receive signals. In a scenario where an emergency vehicle is dispatched and expected to cross a junction, our simulated traffic lights would turn green upon receiving the corresponding signal. This concept could be further expanded to street lights, providing visual cues to drivers that an emergency vehicle is approaching. This would allow ample time for road users to clear the way, facilitating quicker and safer passage for emergency services. Our innovative approach and practical demonstration earned us a place among the finalists in the hackathon, securing us a consolation prize.