Artificial Images: RunwayML Deep Dive
Overview
Artificial Images: RunwayML Deep Dive is a course for image makers (graphic designers, artists, illustrators and photographer) to learn about RunwayML.
Class meets on Tuesdays Apr 7 - May 5, at 8:30pm for 1.5 hour sessions.
Taught by Derrick Schultz and TA Lia Coleman.
Course Syllabus
- Week 1: Intro to ML, ML inspiration, Runway Setup.
- Week 2: Image generation models in Runway, Project Inspiration
- Week 3: Image generation models, Chaining Models, Advanced Usage
- Week 4: Training StyleGAN
- Week 5: Next Steps: Using plugins (p5, photoshop), Google Colab; Open Q&A
Week 1, April 7
To-do before this class:
- Bookmark this page!
- Fill out the pre-class survey.
- Join our Slack and poke around! For our class, we will be communicating through the
#runwayml-deepdivechannel. - Introduce yourself to your classmates in the
#runwayml-deepdivechannel. Say hi to each other! Some starter questions: Where are you located? What do you do? What experience do you have already? What do you want to learn or make? And links to your IG / twitter / website. :) - Read our class Code of Conduct and Zoom Guidelines.
Class Materials
Homework
- Find a model (or a few!) in Runway and play with it! In Slack, post:
- What model did you use?
- What the inputs and outputs were.
- What questions you have about the model.
- Show us what you made!
- Think about a bigger project you want to work on.
- For inspiration: the Runway Slack, the Runway Youtube.
- Consider with inputs you have available to you, and what you want as outputs.
Links and Resources
- Join the Runway Slack
- For a little more about Machine Learning and to see more examples of ML art I recommend my Week 1 video from a previous course (note that a lot of the inspiration I show here isn’t doable in Runway, but we could think of ways to do similar things!)
- A playlist of a bunch of dataset creation demos (best for people who have some coding experience or want to learn to use the command line)
- DeepDream using Google Colab. A better introduction to DeepDream and a much more flexible library. (Google Colab is free but might require a little coding knowledge. Try it!)
Week 2, April 14
Class Materials
Homework
- Start thinking about datasets.
- Watch some of the dataset videos
- If you want to make a personal dataset or something by hand, start collecting!
- If you’ve never done scraping before, we might recommend looking for Instagram accounts (Instagram is the easiest to scrape). For instagram scraping, watch this video for installation steps.
- Keep exploring models.
- We’ve covered a handful of models so far in class, but there are many more! Keep digging around.
Links and Resources
- Style Transfer explainer video
- Style Transfer Livestream using Google Colab
- Getting really smooth Style Transfer videos (note requires you setting up a server)
- Runway/Gene Kogan tutorial on GPT-2
-
Anna Ridler, Mosaic Virus: Video Project
Week 3, April 21
Class Materials
Homework
Get ahold of a dataset.
- Make your own!
- Scrape your own!
- Find a premade one on the internet.
- Let us know if you want a premade one from us. Runway also has some premade ones.
Links and Resources
- Artificial Images intermediate course from January
- Pix2PixHD demo
- Style Transfer for video using Optical Flow (this requires a cloud server or a local computer with an NVIDIA GPU)
- Topaz Labs AI tools
Week 4, April 28
Class Materials
Homework
Finish your dataset and Train a Model!
Week 5, May 5
Class Materials
p5.js links
- (Linear Interpolation)[https://editor.p5js.org/dvs/sketches/ab2iTZQtb]
- (Noise Loop Interpolation)[https://editor.p5js.org/dvs/sketches/Gb0xavYAR]
- (Interpolation using a CSV)[https://editor.p5js.org/dvs/sketches/ab2iTZQtb]
- (Generate images using text)[https://editor.p5js.org/dvs/sketches/fFEJqxxgJ]
Homework
- Please fill out our class survey!
- Prep for the end-of-class showcase on May 19. Slot signups here!
- Optional!
- 5-7 mins per student.
- Open format. Show and Tell, performance, teach, etc. Play a pre-recorded video. Present live with slides. Live demo + Q&A. Any mix of the above!