Miao Ying's Pilgrimage into Walden Twelve is a live simulation software that can be run on computer and smartphone. The simulation has three distinct chapters: The Honor of Shepherds, The People’s Quest, and Battle for Glorious Spells. The project is inspired by AI neural network machine learning, medieval christian indulgences mechanisms, conflicting political ideologies, popular online Chinese S&M romance novels, and the recently emerged “social credit system” in China. The simulation takes place in a medieval magical fantasy land--Walden Twelve. Walden Twelve refers to “Walden Two”, which is a utopian novel written by behavioral psychologist B. F. Skinner, a society where perfect behavior is achieved as a result of reinforced positive conditioning of the desired behaviors while meeting basic needs of citizens. During the Medieval Era, the institution interpreted the religious ideology for the people who were illiterate, which led believers to exchange money for their sins--the indulgence system. The social credit system in China is a digital indulgence system, which records big data from its citizens and rewards or punishes them according to their social credit score. In a structure similar to that of the medieval church, the Chinese government explains the Marxist doctrine to its people and has implemented the technology to enforce citizen behavior. pilgrimageintowaldentwelve.com

Artist Statement

In our first meeting, I asked Michael and Ben if they can hack the Great Fire Wall, they looked at my excited face then they looked at each other and said “No.” Then, they patiently explained to me the difference between computer science and hacking. I often think of that moment and laugh at myself. I have learned so much from them and it was such a privilege to have an experience like that.

I am very proud of our final project Pilgrimage into Walden Twelve, all three of us see how many layers of process and labour there are in the piece. It was a great challenge to all of us, I was frustrated about learning 3D modeling and game engine but I am so glad I pushed myself.

The first chapter, The Honor of Shepherds, shows 6 Artificial Intelligence “social shepherds” who represent different classes: urban migrant worker, outsourced American farmer, kung fu master, Chinese hero, Bourgeois ruling class golfer, and sophisticated intelligentsia. All 6 characters have different AI brains. Each shepherd is trained on millions of examples of human-like movement patterns as a metaphor of Big Data and are scored according to a deviation from a set of ideal expressions. The Shepherds try to minimize the difference between what they believe is a correct movement to express and what is known to be the true movements shown in the data using neural network supervised learning.

In addition, the characters must learn how to act correctly, while also accounting for the natural physics in the game world. Namely, not only must they behave a certain way, they must do so while using their own natural physics. During the simulation, the system samples, and tries to determine which ones are underperforming. Those that are deemed unacceptable by the algorithm, have their neural network replaced, and the associated character model is discarded.

The second chapter of the adventure, The People’s Quest, represents the inner struggle of the kingdom. It is a self-played survival game/simulation utilizing machine learning takes place in a medieval village. In Christianity, the role of the shepherd is to guide and protect the weak. Cockroaches are despicable and disposable, but are also resilient. In People’s Quest, the shepherd AI hunts the roach AI who are trying to escape. Each round they learn from each others behaviors and will evolve their strategies.

The ultimate goal for the shepherd is to protect the roach from the ultimate danger—capitalism demons, with the powers of Communist magic. The last chapter, Battle for Glorious Spells, is the outer struggle of the kingdom, a combat game engine between the different ideologies. Each attack has a unique name that is inspired by different political ideology systems and medieval fantasy games. Such as ”Class Struggle Cracking Smash”, “State Capitalism, Enforcing Flush”, “Rugged Individualism Slash”, “Nimble Chain of Social mobility”, “Bourgeois Exploitation Force”, etc. Chaotic magical effects manifest arbitrarily.

All chapters are live simulations, each time you run the software, it will appear differently. The reality of present China could be a simulated plausible future to the rest of the world.

Collaborator Statement

It’s hard to know where to start. Backslash has been such an incredible experience, and one that has taken so many turns that I feel like we could write volumes about our journey. Despite this, and the number of times we made those turns and pivoted to new ideas, we eventually made it! And I am so proud of what we were able to accomplish. I learned so much about the artistic process–and subsequently how it interacts and sometimes clashes with the common engineering mindset. I also internalized what it takes to shape a singular concept and a plethora of ideas into a polished piece of art.

In this case that singular concept was the implications of the Chinese Social Credit System and how modern technologies enabled its conception. This in itself is a huge concept to explore and it led us to so many different ideas of how to express it. We explored abstract data visualization techniques to try and represent data that might be used to judge people’s credit score. We also implemented different generative art techniques blending Chinese propaganda with Medieval Catholic Imagery using various Neural Network Architectures. And we even made instagram bots trying to craft internet personas as reflected by the ads served to them. It took many attempts and tries, but eventually we landed on building Machine Learning powered 3D humanoids in the game engine Unity. Even at this point we weren’t all convinced or assured that this would finally bear the clear vision that we sought. But with much determination and trial we made the initial models work and provided a technical foundation for Ying’s design chops to really shine. I’ll always remember the day when Ying finished the first 3D model for us to use in Unity. After months of wondering when we would figure this out, we finally had some semblance of what this piece could be, and how awesome it would become.

Lastly, I like to think that if this pandemic didn’t happen, we may have actually continued on working and finding ways to continue making art + tech projects happen. But we cannot change what happened, and we as a team made the best of what we had. Working with Ying and Michael throughout the year and especially during the start of the pandemic made me appreciate how awesome they are and how much fun working with them has been! They were both excellent mentors in their respective fields and our endless conversations and exchanges were second to none. They really are what made this experience great. And although the project is over–for now–I hope to carry on the spirit of this fellowship in my own creative pursuits in music and wherever else life leads.

Collaborator Statement

Creating an artistic piece that was both technically compelling and integral to the actual thematic goals was a constant source of brainstorming, recalibrating, and continuously collaborating. While we knew that the focus was largely around the social credit system (SCS), what that would mean from a technology standpoint, and as an art piece was not. To that end, it was us prototyping, seeing how it fit the themes, realizing the issues, and starting over. Collectively we explored generative images, generative text, reinforcement learning, and finally 3D procedural animation. Beyond the many, many technical challenges, the solution we ultimately arrived at was creating something that used AI in a way that actually mirrored SCS and still could be compelling from an artistic standpoint.

To give some context, each character is a representation of an archetype persona that is ultimately following the master/roach (system). This is expressed via both their character design, and from an ML perspective, how they literally learned to move and react in a physics based environment. Each character is trained on a subset of 3D movement data that is tuned to certain behaviors (running, fighting, dancing, etc). They then try to emulate a primary character that is perfect modeling a specific sequence of actions. Because of the nature of their training, and the physics of the environment, each behavior is continuously different and unique, and can never truly conform to the master/roach, but there’s always an explicit instruction to do so.

In terms of actualization, I think what was probably the ‘whoa’ moment (and I’m sure Ben would agree) was seeing Ying’s artistic characters and models implemented in the 3D environments we had all designed. Having these ML models live in a 3D simulation with physics wasn’t entirely meaningful until it had a ‘face’. It also viscerally reinforced how important it is to have a team, creative or otherwise where everyone has a diverse background and perspective.

Creating art isn’t grounded with an explicit template. It really needs an openness and level of exploration that just wouldn’t have happened if we were working in isolation. Or maybe it would have. But, it certainly wouldn’t have been satisfying. For me, working with Ben and Ying was as important as the project’s result. I’m proud of what we accomplished, but more so that everyone was so willing to engage, collaborate, and genuinely embrace each other’s ideas and perspectives.