Let’s take it from where we left off, when I was sitting on an airplane heading for Expressive and Siggraph 2017. Hello Los Angeles, it’s been a while since I enjoyed your sunshine! Coming to Los Angeles made me feel a bit nostalgic. For those of you who might not know, I spent almost 3 months in Los Angeles with Marcus LeVere and Hanna Radek trying to get our startup, studio.coop, started in 2013. For more information on this, make sure to check out this blog post. Being back in LA was great, even though my Airbnb was in a sketchy part of town far away from everyone. Oh well, this happens when you are given a fixed budget and need to spend over half of it only on the flight… Nonetheless, it was great to be back and have some tasty Mexican food with free flow of guacamole around the corner!
Expressive and Siggraph 2017
— July 30th-August 3rd
Expressive was co-located with Siggraph at the Los Angeles Convention Center and started on the 28th of July. It was wonderful to be back at Expressive! Right from the beginning I finally got to meet Oliver Deussen from the University Konstanz, who has done research in watercolor stylization back in 2006 and Holger Winnemöller from Adobe, who has also done some amazing work in expressive rendering. It was also great to see the familiar faces of Amir Semmo and Ergun Akleman and get to know other people like Paul Rosin, Mandy Klingbeil and Felice House. I even got to know a new professor from my alma matter, the Stuttgart Media University!
I presented during the first morning session, which was great! All attendants (there were around 45) were awake and eager to know what new developments had been made. My presentation started right away with a live demo, showcasing the new effects in real-time (no crashes, yeah!) and then explaining our novel approaches, enabled through our closer investigation of edges and substrates in watercolors. The presentation went quite well, and I made extra effort to not use the podium, but use the entire stage instead and move while explaining. This is something I especially like, as it brings me closer to the audience. The feedback was phenomenal, good comments were given for the technical contribution and praise was given for the aesthetic results (time well spent) and the presentation. I also got some good suggestions regarding temporal coherence and PPI (pixels-per-inch) relative effects. You can get the demo video, presentation slides and the actual publication at Edge- and substrate-based effects for watercolor stylization.
Unfortunately, due to overlapping schedules, I had to do the tech setup for Real-Time Live! (RTL) at Siggraph that afternoon. This meant that I missed almost the entire Expressive afternoon session and some presentations, which I would have liked to see. On the other hand, I got to know the chair of the event Cristobal Cheng, the other presenters of RTL, The Blackbird car rig used to insert any type of car in a VFX shot; and I got to see my work on the biggest hall at Siggraph! (I then realized why Real-Time Life! has an exclamation [!] mark). The conference hall is overwhelming in size and had close to 3000 seats. Thankfully, the lights pointing at you are so strong that you barely see a third of it while presenting. My 5-day old workstation worked and performed like a champ during the tech setup but, for some reason, after a specific amount of time being idle, Maya crashed… like, seriously… now? I fired up Visual Studio and tackled that nasty bug on the actual stage, so I could return to Expressive with peace of mind. I attended the final presentations of the day and participated in the Adobe Dinner. This dinner was held at a Mexican restaurant nearby with free flow of food and drinks… One feels appreciated as a researcher under these conditions, thanks Adobe! More importantly, it was a great opportunity to catch up with everyone and have a lovely time.
The next day (30th of July) was the last day of Expressive and the first day of Siggraph. There were conflicting schedules again, as I had the Real-Time Live! rehearsal the entire morning while Expressive continued. The two rehearsals went quite well in my case and it was fascinating to see all the other projects. As expected, I was the only student presenting and there were some seriously skilled technical artists presenting amazing projects from all over the world! I felt humbled to be presenting along these people, they have created truly great technology in real-time and I was between them somehow. It was truly rewarding to receive their positive comments and feedback regarding the stylization and the presentation. I returned to Expressive for lunch, as I wanted to take part in the steering committee meeting and help with what I can. In small specialized conferences like Expressive, they are always happy for an extra hand. I stayed there for the remainder of the afternoon and attended the Technical Papers - Fast Forward at Siggraph with all other attendants. Now with Expressive over, I could focus all my attention to Siggraph.
On the 31st of July I could explore Siggraph to my own pace and liking. There is so much going on at the conference at the same time, that it is hard to decide where to go. There are the must see courses related to Advances in Real-Time Rendering in Games, the technical papers catering to all graphics related interests and some amazing production sessions where studios get to share many details behind their latest works. This all comes together with the exhibition halls and many more courses and talks. In the evening, the Siggraph reception took place, which was held at the California Science Center. All conference attendants were invited to food and beverages, while networking and exploring the museum at night. It was cool, especially getting to see the Endeavour - the last space shuttle of NASA that brought passengers back and forth to space.
The 1st of August was the big day… During the day, I got to see a Giraffe in the middle of the Conference (things that randomly happen at Siggraph) and met with some friends that were also attending the conference including Beibei Wang, David Vanderhaegue and Jason Schleifer. It was great seeing them again or for the first time outside of Skype. I also got to attend the “VIP” lunch for all Siggraph contributors (the food was nice). Afterwards, I headed to the Birds of a Feather talk on Remote Studio Productions: Online Collaboration to represent studio.coop (at least in spirit) and got to also see Bobby Beck and Bernhardt Haux again. Things are going great at this front with Artella, syncsketch and Nimble Collective steadily growing and changing the way digital productions are made. After this session, Real-time Live! was about to happen. And it happened! Gosh I was nervous… The fact that they would record and live stream the entire event meant that there was no room for mistakes; and it is especially hard for me to talk from memory without any slides to support me. I’m a casual presenter and usually do quite well, but I had only 6 minutes to bring everything across and wanted to optimize this time the best way possible. Enough words, you can see for yourself..
Even though I forgot to mention something during the presentation, I’m happy the way it turned out (and I kept it to 6-min, yeah!). After the presentation, I couldn’t have wished for a better audience, and the reception was great! Amir Semmo and Mandy Klingbeil were there, so we head out together to the Texas A&M Viz reception to let go. Great people, good drinks, no worries, all good!
On the 2nd and 3rd of August, I experienced and enjoyed the rest of my time at Siggraph and got to know some amazing people that had seen my presentation. Among those people were Cassidy Curtis and Juan Miguel de Joya from Google Spotlight Stories. Cassidy did a groundbreaking short called Fishing back in 1999, which has been an incredible inspiration during my research on watercolor rendering. During lunch, James Jackson from Pixar also joined us and we were geeking out on stylized rendering. They all liked the work that I’m doing and it felt awesome! During the last few hours of Siggraph, I finally had the opportunity to properly talk with Daniel Sýkora from the Czech Technical University in Prague. He has been supervising research in image synthesis for stylized animation and we had a good discussion about what is still missing in terms of the nuances of emulating watercolor, digitally. And then… just like that, Siggraph came to an end. Unfortunately, I wasn’t feeling well that last night, so I decided to stay in my sketchy Airbnb and recover a bit before my long flight home.
Back home going multi-platform
— August
Coming back from Siggraph in LA to Singapore was strange. So much had happened that I was in a bit of a limbo… what now? I kept busy catching up on emails and writing to all the new contacts I had made. Networking takes time and writing emails all day is not what I would call a fun activity - but it was definitely worth it. There were many new interested parties, so one thing became unavoidable at this stage, I needed to make the technology multi-platform. I began getting acquainted with Linux and after a prolonged battle getting the right version of Linux, Maya and NVidia drivers working together with Viewport 2.0, I was compiling code again. Pierre Bénard, one of my French supervisors, thankfully translated the shader code to GLSL, so we could start sharing Linux and MacOS implementations of it. These have remained mostly in the dark for now though, as big updates were coming that would change everything anyways.
I also moved places in August with my flat mates to a beautiful apartment across the pool in the same condominium. This time, however, we live on the 12th floor and have a sunset view! It’s a big upgrade from our previous apartment and I can’t wait to go home on sunny days now. Sun, couch, cocktail, you get the drill!
A new framework
— September - October
Whenever I wanted to extend, tidy up or optimize the code that I had developed these past years, I realized how cumbersome/inflexible it was to work with (and to understand, for an outsider). If I were to open source the code in the future with the hopes that people would adopt, use and extend it, I had to re-write it from the ground up - with all the tricks learnt in the last 3 years. I originally created the watercolor renderer by adopting a plugin available in the Maya devkit called viewRenderOverride and extending it to fit my purpose. This proved to be a good solution at first, because there is little learning material available on overriding Viewport 2.0. But, the more complex the stylization grew, the more cumbersome the plugin architecture became to work with. Something simpler was necessary, where a few lines of code in a single file could create a custom render operation with all parameters, targets and textures. From there, the Framework would take care of most UI related elements and offer useful/lightweight tools for debugging. I wanted to create the development framework that I wished to have had when I first started my PhD - this would hopefully motivate developers to use the code!
It turns out that this was a bit more challenging than I originally expected, and it took me most of September and October to do it. Nonetheless, I made it and am happy with the result. The framework is quite optimized and it is incredibly simple/fast to create a direct stylization pipeline with an arbitrary number of render operations. Things don’t mysteriously break and it is fun to use and debug! One could always invest more time on it and continue improving things, but I can’t do that right now, as I still need to do research. Things like these are considered engineering in academia and they won’t bring much to your research contribution.
During this period, Alexandre Bléron from Inria in Grenoble, Rhône-Alpes, came for a 3 month internship at MAGIC. It was great seeing Alex again and, finally, I had a fellow student in our lab focusing on stylized rendering to geek out about pixels, shaders, art and stuff. Overall, I believe he had a great time here and all of MAGIC benefited from his expertise and wonderful French accent.
Keying of effects
— November
November came abruptly and with the new framework ready, it was time to start developing new ideas. We started brainstorming on out-of-geometry (out-of-the-box) ideas for stylization control in object-space, but we seemed to go in circles. It is difficult to think on novel ways to control things with such an established pipeline being used in the computer graphics industry nowadays. There were many ideas, but these all presented the dilemma of what the actual technical contribution would be. What works in 3D, what works 2D? These simple questions proved difficult to conceptualize and have everyone on the same page.
Not all was abstract thinking though, I also took some time to implement something that I’ve wanted for a while now: keying of effects. Managing multiple vertex color sets and keying the right color channel for the desired effect can be tricky, especially without an intuitive workflow. Additionally, whenever you assign or key a vertex color in Maya, the vertices of the adjacent face is assigned/keyed, not the vertex itself. This means that if you key a single vertex with 4 adjacent faces, you would have 4(faces) * 4(RGBA) = 16 animation curves, for one vertex! This is prohibitive, as you will most likely be keying effects at hundreds of vertices. Unfortunately, we can’t get around Maya’s way of assigning vertex colors, but we could optimize how many channels we key with the click of a button. The result proved to be quite fun to use and play around with. I can’t wait to see what other artists end up doing with it.
Apart from that, together with Dr. Amir Semmo, we have volunteered to be Publicity Chairs of Expressive 2018. Since then, we have collaborated remotely to create its new website using Jekyll (the same framework that I use to create artneering.io). The collaboration has been quite fruitful and we have worked well together. In fact, it has gone so well that MAGIC has invited Amir to come to Singapore for a few months next year to collaborate on more things, more on this in a future post though.
That time of the year again
— December
One of my best childhood friends decided to marry the girl of his life on the 9th of December back where I grew up in Guayaquil, Ecuador. So, being held at such a convenient date close to Christmas, there was not much choice for me but to take most of the month off work. After more than three and a half years - and 26 hours inside an airplane -, I came back to the place where I grew up. I can’t really put to words all the mixed impressions and feelings I had while I was there (otherwise this blog post might get too personal - and long), but it was really nice remembering so many things, people and flavors. I would have never imagined though, that I would find my home town of Guayaquil to be cold. Nonetheless, after living in Singapore, warm weather has a completely new definition for me.
After my one week visit in Guayaquil, I flew to Stuttgart, Germany, with my parents to spend Christmas over there. My brother, Sebastian, lives there and we planned early in the year to spend the holidays together. Stuttgart is great! I did my bachelor degree at the Stuttgart Media University, so the city has a special place in my heart. I still have good friends from my university days and it was wonderful to see most of them and catch-up with our lives. Spending Christmas together with my closest family was just what I needed to close this wonderful year of 2017.
With that being said, dear reader, I want to wish you an amazing 2018! It’s going to be a great year, full of surprises and new adventures. Thank you for reading my longest blog post yet and see you next time with some exciting news!
Happy new year!
Santiago.