November｜The Integration of New Media and Art
December｜The Application of Artificial Intelligence in Music
Written by Stella TSAI
The Art Generation Accompanied by Technology
As 2023 draws to a close, industries are gradually revitalizing. The “technology” that was massively discussed, applied, and grew at the speed of light during the pandemic, is now blossoming post-pandemic. Arts Talk has invited FENG Chien-chang, the creative director at FREE’S who has extensive experience in pop music concert production and straddles both traditional fine arts and new media arts, as well as LIN Yi-chun, the director of the Tsinghua University AI orchestra. The former discussed “The Integration of New Media and Art,” speaking about his 20 years of experience transitioning from manual to digital creation and creativity, while the latter shared her practical experience since 2019 in AI research, applying artificial intelligence to music composition and event planning. The speakers’ personal experiences are a testament to the extraordinary levels of creativity and artistic milestones that have been catalyzed by technological advancements.
The Millennium as a Boundary: The Rapid Progress of New Media Art in the Entertainment Industry
In the 1990s, FENG Chien-chang broke into the television industry as an assistant production designer, a time when the concept "new media" had not yet entered the lexicon. Production designers at the time relied solely on their boundless creativity and a diverse array of materials. As Taiwanese singers began touring China, production design swiftly evolved into a lucrative business. The advent of 3D drawing technology marked a significant departure from traditional manual sketching, enabling designers to bring the visions in their minds to life with greater speed and less effort. It was a moment of realization for FENG, who recognized the shift towards “thinking in a virtual world.” The new millennium heralded a surge in televised concerts, and with it, new media soared in tandem with the pop music industry.
By the 2004 Golden Melody Awards, LED displays have flooded the physical sets, signaling a broader adoption of LED technology across TV shows, ceremonies, and large-scale outdoor concerts. Notably, in the 2019 TV talent show "Jungle Voice," digital technology was employed to craft a vibrant and dynamic forest scene, achieving visual splendor without the environmental cost of felling trees. This era saw new media art flourish, gaining remarkable momentum during the pandemic and advancing at an extraordinary rate.
During the pandemic, physical concerts came to a standstill, giving way to the golden age of online performances. The shift from the conventional 16:9 horizontal video to the 9:18 vertical format tailored for mobile viewing was just the beginning. Audiences were treated to a rich tapestry of experiences with the integration of augmented reality (AR) animations, live interactions, and the simultaneous filming of live performances to craft a visual feast. The design approach that layered virtual imagery over actual footage began to see widespread use. Today, the scope of new media art encompasses a broad spectrum, including digital art, interactive installations, virtual reality, internet art, and AI algorithms, all of which are extensively utilized across various forms of entertainment. Take, for example, the music video by Flesh Juicer, where the lead singer, filmed in a 4D View, transitions into an animated character within a virtual realm, marking the convergence of reality and virtuality. Similarly, at the Kaohsiung Lantern Festival's "Light Kaohsiung Century 2021,” a fusion of land-based light sculptures, maritime stages, drone shows in the sky, and fireworks displays leveraged the full potential of new media. This not only maximized the use of diverse media but also ensured a captivating viewing experience for audiences attending live, tuning in online, or watching through broadcasts.
Harnessing Technology to Enhance Musical Creativity
From her early days learning the flute, LIN Yi-chun has always had a penchant for integrating multimedia into her performances, pioneering unique forms of artistic expression. In 2019, she ventured into marrying the growing field of AI with the art of music performance, introducing this innovative approach to the freshmen Music majors in Tsinghua University Interdisciplinary Program.
Composers are well aware that music creation follows certain standards and structures. With the advent of computers, the principles of musical forms, lengths, and intervals have been encoded into software, allowing for precise analysis within set parameters. When composers utilize computers in their musical creation process, they can efficiently alter sections, craft individual parts, and even engage in “automatic composition.” This function not only streamlines the compositional process but also acts as a muse, providing fresh inspiration. Since musical creation is often a reflection of a composer’s emotional state, the melodies generated by automatic composition may function as a creative spark. LIN contended that while composers may not need to depend exclusively on AI-generated melodies, they can certainly extract inspiration from these algorithmically created tunes, combining them with their original concepts to produce inventive and authentic works. It is in the intersection of human ingenuity and technological assistance that the true value of technology shines brightest.
Among the prevalent music software today are "Magenta Studio" and the online tuning software "Drumbot." The former, developed by Google AI, utilizes data analysis to teach machines how to compose music; the latter is an interactive robot capable of providing real-time accompaniment to live performers. An intriguing example of technology's role in music is the collaboration between Grammy music producer Alex Da Kid and IBM Watson. Watson analyzed Alex's social media behavior, including his likes, shares, and posts, to discern patterns. It then translated these insights into music, reflecting his life, cultural background, and style. The first album born from this collaboration, "Not Easy," even soared to the top of the U.S. Billboard charts.
For musicians and composers, AI's assistance extends beyond automatic composition and inspiration. It includes features such as automatic scoring, accompaniment, genre classification, and automatic post-production. The efficiency and output driven by AI have long been a part of the music creation process, offering significant benefits to artists and composers. This symbiotic relationship is set to continue its evolution, shaping the future of music production.
Arts is a summary of life experiences.
Arts Talk is a project which gathers together renowned artists from different fields such as music, theater or movie domains for sharing their thought about art with the public, telling us how art can communicate and resonate with its audience.
Creative work is like an extension of every incident in one's life. The accumulation of your everyday experience, the moments when your imagination runs wild, and your persistent cultivation of hobbies can all be the sources of your inspiration, enabling you to write stories that recount your memories of life.
So why not do some brainstorming to see which part of your life, no matter how trivial it is, can be turned into a work of art?
Arts Talk gives you an opportunity to interact directly with art and brings wonderful surprises into your life!
