Statement of Purpose

On November 30, 2022, ChatGPT was released.
The world was drowned in a combination of excitement and fear
as the mastery of natural language, a trait once considered exclusively human,
was acquired by a seemingly intelligent large language model.


Stable Diffusion and Midjourney were also introduced in 2022,
further challenging humanity’s monopoly in the domain of art.
Developers certainly won’t stop here,
and as a content creator posting music on Bilibili as a part-time interest,
I could feel the tension alongside the overwhelming opportunity.

I have since dreamed of researching human-oriented content creation aided by large models,
focusing on instructed music and video generation.
It would be exciting if an app could design sounds, add effects, perform arrangements,
and bring the song in my head into reality just from a hum!

But designing such a system introduces many hidden challenges.
Not only should the model accept multimodal inputs and take copyright into account,
but the composer should also retain control over every fine-grained aspect of the final product.

Such a challenging project requires expertise in multiple disciplines,
including music theory, high-performance programming, and computer security.
Nevertheless, this research topic unifies my academic pursuit and personal interest,
and I am ready to devote myself to this complex and prospective field in my graduate years.

I have adequate experience in AI research.

My work with Prof. Zuozhu Liu in my sophomore year was my first exposure to advanced AI models. Our team worked on self-supervised semantic segmentation and defect detection on 3D oral mesh.

• Names like “point-cloud masked autoencoder” and “bootstrap your own latent” sounded terrifying, but they also motivated me to complete the Dive into Deep Learning online course throughout my first summer break, thoroughly study the papers, and gain hands-on experience on the entire pipeline from data preprocessing to model evaluation.

I further developed my skills when I joined Prof. Yang Yang’s lab in Summer 2022. The team was working on identifying power-stealing behavior from electricity usage data in Hangzhou.

• I was amazed by how their “Time2Graph” architecture observed a typical time series classification task from the perspective of transitioning shapelets (representative subsequences) in a graph.
• After teaching myself the Stanford CS224w course (Machine Learning with Graphs) in the first two weeks, I proposed incorporating Graph Attention Network with KMeans clustering in the Time2Vec-embedded space. With dynamic time warping turned off, I managed to accelerate shapelet extraction by nearly 100% with comparable accuracy to 10 benchmark SOTAs.
• I enjoyed this research experience, pushing forward my intellectual boundary and applying acquired skills to real-life projects while gaining invaluable insights into the lifestyle and expected capabilities in academia.

I adopt an interdisciplinary way of thinking.

As an Electrical and Computer Engineering student at ZJU-UIUC Institute, I am honored to receive undergraduate education from the top universities in the East and West.

• The International Campus of Zhejiang University is known for a highly integrated research environment where engineering students from different majors frequently cooperate – my “oral mesh” project, for example, benefited from medical expertise.

Moreover, the ECE major is known for its curriculum focused on both hardware and software, the best example being ECE385 (Computer Systems Laboratory), where we switched back and forth between SystemVerilog and C programming to build a system-on-chip with FPGA.

• I started imagining a 38-key rhythm game during the winter break, developed a scripting language with my knowledge of music theory three months before the deadline, and designed a hardware-driven GUI hand-crafted at pixel level during the semester.
• It was a complex protocol featuring “flowing notes” animation, skill and life mechanism, and SGTL5000 sound codec, but I nonetheless enjoyed the process.

My love of music continued into my research with Prof. Sever Tipei in Summer 2023, where we added functionality to the 30-year-old digital sound composition and synthesis software DISSCO.

• Using C++ to facilitate spatialization and reverberation by partials in a sound requires backend infrastructure to render multiple channels, a frontend interface to pass extra information through an XML document, and a middleware workflow to concert the process.
• Co-designing system components and familiarizing myself with multiple disciplines both require efficient learning, quick adaptation, and team cooperation, skills in which I have been thoroughly trained in my undergraduate years.

I am confident in my teamwork and problem-solving skills under pressure.

Numerous ECE courses have considerable workloads, including ECE391 (Operating System), where my team built a working Unix-like OS in six weeks.

• We developed kernel components such as terminal and system calls from scratch, incorporated extra functionalities like signal handling and dynamic malloc, and ranked Top 5 among 50+ teams in the final showcase.

Such skills also apply to extracurricular competitions – we have merely 100 hours to compose a 25-page paper in the 2022 Mathematical Contest in Modeling.

• Competition was fierce: the gold-bitcoin portfolio problem was selected by 40% of the teams, and the data provided is relatively scarce compared to previous years.
• After applying multiple dimension-rising techniques, I utilized the grey model for short-term and multi-layer perceptron for long-term analysis. Together with my teammates, we creatively modeled investors with “vision” and “risk tolerance” parameters in the Capital Asset Pricing Model and achieved a 3200% investment return over five years.
• Our numerical solution was suboptimal, but we won the Finalist Award (top 2%) probably due to our comprehensive comparison between algorithms and rigorous analysis despite the time constraint, a trait vital to graduate research.

During my internship at Baidu in August 2023,
everyone in the company was talking about “AI native”.
It is a research frontier amidst the buzz of large models that, in the long run,
instead of integrating AI into existing cloud services through crafted prompts and APIs,
we should reimagine future software and design the applications based on the capabilities of AI.

As a researcher and content creator, it excites me to imagine such a native composer AI.
I hope to bravely push forward this frontier of large model research
and embrace the endless possibilities AI will deliver to the ecology of creators.

Ziyuan Chen

November 2023

---

Let it be code or music, do it like you never did before.