WayangWayang Preserves Heritage with Innovative AI-powered Features

Students from APU’s Software Engineering, Computer Science, and Cyber Security programmes — Johnson Chin Hong Wei, Sean Ng, and James Wong Chai Fuu — developed WayangWayang, an award-winning AI project preserving heritage through innovation, won the gold medal in the ICT category at the Malaysia Technology Expo’s (MTE) 2024 Asian Youth Innovation Awards (AYIA).

Learning and cultural appreciation should be smoothly integrated. This is the vision behind ‘WayangWayang’, a ground-breaking project created by a team of passionate students from Asia Pacific University of Technology & Innovation (APU) that recently won the Gold Medal in the Information and Communication Technologies (ICT) category at the prestigious Malaysia Technology Expo’s (MTE) 2024 Asian Youth Innovation Awards (AYIA).

Held as a hybrid event focusing on innovation and technology, the MTE 2024 attended by companies, entrepreneurs, investors, innovators, and students at the World Trade Centre (WTC) Kuala Lumpur from 22 to 23 February 2024, to showcase and explore the latest trends and solutions in the field.

The purpose of APU School of Computing (SoC) and School of Technology (SoT) sending one team under the Youth Category for this year’s AYIA was to provide them with valuable practical experience, improve their skills, get them ready for future job opportunities, and give them a platform to showcase and discuss their work with the public.

Developed by APU’s Software Engineering, Computer Science (IS/AI), and Cyber Security students — Johnson Chin Hong Wei, Sean Ng, and James Wong Chai Fuu, — ‘WayangWayang: Malaysia’s Cultural Tapestry in Shadow Puppetry 1 Stop Learning Experience’ transcends the limitations of traditional e-learning platforms.

This award-winning AI project offers an engaging exploration of Malaysia’s rich cultural heritage, particularly focusing on the traditional shadow puppetry art form of Wayang Kulit, through its meticulously designed elements that deliver an immersive and unforgettable educational experience.

More than just entertainment

One of the highlights of WayangWayang is the ‘Shadower Intelligent Chatbot’. This AI-powered tool engages learners in dynamic and interactive conversations, making knowledge about Wayang Kulit accessible and enjoyable in multiple languages.

Additionally, the ‘Pose Animator’ feature allows users to animate Wayang Kulit characters in real-time using their body movements, adding a layer of gamification, and fostering deeper engagement.

Wayang Kulit is deeply rooted in Malaysian culture, and WayangWayang recognises its potential to contribute to several Sustainable Development Goals (SDGs) outlined by the United Nations.

By promoting cultural learning and heritage preservation, WayangWayang contributes to several Sustainable Development Goals (SDGs), including Quality Education (SDG 4), Industry, Innovation, and Infrastructure (SDG 9), Sustainable Cities and Communities (SDG 11), and Peace, Justice, and Strong Institutions (SDG 16).

This ensures the survival of traditional art forms in the digital age, fosters social cohesion, and promotes social justice and harmony by conveying moral values and encouraging dialogue and understanding among diverse communities.

A journey of discovery, celebrating innovation and heritage

The team behind WayangWayang embarked on a journey of discovery themselves, delving into the fascinating world of Wayang Kulit across different countries.

They were particularly struck by the unique characteristics of the Malaysian (Kelantanese) style, known for its sturdiness, single movable arm, and vibrant colours visible during performances.

Ignited by the desire to preserve and share a cultural gem using innovative technology, the three students translated their passion into action by participating in MTE 2024, celebrating both innovation and heritage.

Earning the gold medal not only validated their hard work and creativity, but also opened doors to valuable connections and insightful feedback, fuelling their determination to refine the system and leverage its potential for future impactful change.

Johnson, representing WayangWayang, expressed his group’s fascination with the diversity of Wayang Kulit across countries, stating, “Each nation brings its unique interpretation and style to this traditional art form.”

“The Malaysian (specifically Kelantanese) version was an eye-opening experience for us and by thinking out of the box, we decided to use the advancements of AI to preserve the culture of Wayang Kulit.”

​

Elaborating on their project, he mentioned collaborating with their mentors, Assoc. Prof. Ts. Dr. Vinothini Kasinathan, Dr. Julia Juremi, and Ms. Nur Amira Abdul Majid, focusing on leveraging Artificial Intelligence (AI) and culture for preservation and dissemination.

This, he explained, aims to showcase how Wayang Kulit can transcend its boundaries and be utilised for diverse purposes and further shared his excitement at sharing their innovative approach with an international audience, including individuals from Vietnam, Taiwan, Indonesia, and fellow Malaysians, calling it “Truly exhilarating” and “Emphasised the importance of safeguarding cultural heritage while embracing innovation for continued relevance and appreciation”, highlighting the significance of such moments.

James, reflecting on their MTE participation, stated that it was a significant achievement, offering valuable insights and connections.

Winning such a prestigious competition, he said, “Has inspired us to further refine our system based on the constructive feedback received. We are grateful for the opportunity and look forward to leveraging this experience to drive impactful innovation in the future.”

Similarly, Sean highlighted the value of participating in MTE 2024. He added that the experience provided an invaluable opportunity to assess the reception of their idea, with the gold medal win serving as a strong validation of its worth.

“This success motivates us to strive for continuous improvement and explore further avenues for enhancing our system’s capabilities in preserving traditional shadow puppets in a modern context,” he said.

WayangWayang exemplifies the power of innovation and cultural appreciation, reminding us that technology can safeguard our heritage, ensuring its continued relevance and appreciation for future generations.