Multimodal AI Projects with Emily Bruff

A few episodes ago, I talked with Marc Watkins of the University of Mississippi about the many ways that generative AI is beginning to intersect with student learning. Marc noted that the newest versions of ChatGPT and similar tools are no longer just text generators, but multimodal in nature. That is, they can work with text and images and audio and in some cases video, too. 

To help us better understand what roles these AI tools might play in multimodal assignments, this episode features an interview I conducted earlier this year as part of the slow read I ran of my 2019 book, Intentional Tech. One chapter of that book is about using technology for multimodal assignments, and to bring some fresh perspectives to that topic, I talked my wife, Emily Bruff.

Emily is a marketer for Zondervan Academic, and earlier this year she completed an interesting multimodal project to support the release of a new book called Know the Theologians. In the interview, Emily shares her experience using an AI image generator to make theologian trading cards, and we extrapolate from that experience to working with students with AI.

Episode Resources

·       Emily Bruff’s website, https://emilyhbruff.com/ 

·       Know the Theologians, https://www.zondervan.com/p/know-the-theologians/

·       Visio Divina conference, https://emilyhbruff.com/visio-divina-conference  

·       The Parthenon digital restoration project, https://www.youtube.com/watch?v=YBHYVIyEVjg 

Podcast Links:

Intentional Teaching is sponsored by UPCEA, the online and professional education association.

Subscribe to the Intentional Teaching newsletter: https://derekbruff.ck.page/subscribe

Support Intentional Teaching on Patreon: https://www.patreon.com/intentionalteaching

Find me on LinkedIn and Bluesky.

See my website for my "Agile Learning" blog and information about having me speak at your campus or conference.

Chapters

• 1:56 Meet the Educator
• 3:32 AI for Marketing
• 5:19 Theologian Trading Cards
• 14:17 AI and Expertise
• 18:57 Multimodal AI Assignments
• 25:09 Outro

Transcript

Derek Bruff: 0:05

Welcome to Intentional Teaching, a podcast aimed at educators to help them develop foundational teaching skills and explore new ideas and teaching. I'm your host, Derek Bruff. I hope this podcast helps you be more intentional in how you teach and in how you develop as a teacher over time. A few episodes ago, I talked with Marc Watkins of the University of Mississippi about the many ways that generative AI is beginning to intersect with student learning. Mark noted that the newest versions of ChatGPT and similar tools are no longer just text generators, but multi-modal in nature. That is, they can work with text and images and audio and in some cases video too. To help us better understand what roles these AI tools might play in multi-modal assignments. This episode features an interview I conducted earlier this year as part of the slow read I ran of my 2019 both Intentional Tech. One chapter of that book is about using technology for multimodal assignments and to bring some fresh perspectives to that topic. I talked with someone I knew who was using an AI image generator in her professional work that someone was my wife, Emily Bruff. Emily is a marketer for Zondervan Academic, and earlier this year she was completing an interesting, multi-modal project to support the release of a new book called Know the Theologians. In the interview, Emily shares her experience using an AI image generator to make theologian trading cards. And we extrapolate from that experience to working with students with AI. I've been sharing Emily's story at my talks for months now because it has a lot to say about the relationship between expertise and generative A.I., and also because I like talking about my wife. I have shared this interview with Emily with my Patreon supporters back in February, and now I'm excited to share it here on the podcast. Hi, Emily.

Emily Bruff: 1:58

Hi.

Derek Bruff: 1:58

Welcome to my podcast.

Emily Bruff: 2:00

Thanks for having me.

Derek Bruff: 2:01

Glad to have you. Before we talk about this project in front of us, can you tell us about a time when you realized you wanted to be an educator?

Emily Bruff: 2:08

MM Well, wow, that's a question. I think it was probably way back when I was in school, in my undergrad work, and I was teaching some kids at camp how to do art and I was like, Well, that was exhilarating. Maybe I could do that for a living. And so then I did that for a living for a while.

Derek Bruff: 2:39

Yes, you did. Yeah. And what do you do for a living now?

Emily Bruff: 2:42

Now I am a marketer for Zondervan, academic. So I get to market books, which sounds like a long way away from teaching art. But it has more overlaps than you think.

Derek Bruff: 2:55

as we will get into today. Yeah. So what kind of books do you market?

Emily Bruff: 3:02

I market theology books specifically for seminary level, some graduate, some undergraduate classes and some, you know, interested, curious laity.

Derek Bruff: 3:22

Okay , Gotcha. So if I were if I were teaching in the seminary, I might assign one of the books that you market.

Emily Bruff: 3:29

Yes.

Derek Bruff: 3:30

Yeah. Okay. What? Prior to this project, what kind of experience did you have with generative AI in your marketing work?

Emily Bruff: 3:42

Mostly we have we use a tool called Jasper through my company. And, you know, we have a big corporate license, and so we're able to really tailor it for our uses. If we needed to come up with 20 different tweets for a particular book launch, it would help us as a starting point.

Derek Bruff: 4:06

Okay.

Emily Bruff: 4:07

I never leave them as is mostly because they have way too many emojis for my audience. But but it gives a good place to start and then we tweak it down and fine tune it and it saves us a lot of time on that front. And we do other things like summarizing manuscripts or coming up with engaging questions for study guides or things like that that might be based on the manuscript that we need more content for.

Derek Bruff: 4:39

Okay. So you can feed in an entire book.

Emily Bruff: 4:42

Yes.

Derek Bruff: 4:42

And have it summarize things, generate questions, etc..

Emily Bruff: 4:46

We can feed it an entire book and we can assign it a particular brand voice or a particular human voice, if you wish. Not a literal voice. When I say voice like, not like you're talking to me, right? But like, how would you talk on Twitter?

Derek Bruff: 5:01

Right. My writing style.

Emily Bruff: 5:03

Yes.

Derek Bruff: 5:03

Yeah. So you can you can tell it to sound like one of your authors.

Emily Bruff: 5:08

Yes.

Derek Bruff: 5:08

Okay. How does it do? A good job?

Emily Bruff: 5:11

Sometimes.

Derek Bruff: 5:12

Sometimes, Yeah. Okay. All right. Which is why I never just take its ouput and run with it.

Emily Bruff: 5:17

No.

Derek Bruff: 5:17

You've got to. Okay. Well, you recently spent some quality time with Jasper's image generator for a project. Can you. What can. What can you tell us about this project?

Emily Bruff: 5:32

Well, I was wanting to do a pre-order bonus for one of my books. That's coming out called Know the Theologians by Jennifer and David McNutt. They're professors at Wheaton, and they have this incredible book that is very accessible. You know, it's very academic, but it's also very accessible, which is a really fun book for me to market. Not that I don't love marketing commentaries, but this is just a little more fun. And so the audience reach was going to be a little wider, more undergraduate and more, you know, curious person who might just want to know more about theologians. And so I wanted a fun pre-order bonus, and we were brainstorming and we thought it would be fun to make a set of theologian trading cards. And, you know, I have some budget, but I didn't have a budget to pay an artist to create 18 different portraits of theologians.

Derek Bruff: 6:35

Sure.

Emily Bruff: 6:36

So I did have in my budget some money to spend on a designer. But again, I needed to give her a head start. So I used the image generator and one of the templates within the image generator to come up with the first pass of these portraits of the theologians. And I thought, Hey, this will be a really quick, easy. Stick it in and see what it spits out kind of situation. And it was a little more involved than that, but I ended up coming out with a pretty good, pretty good set of trading cards. But I will definitely say with the help of the PhDs who wrote the book and also the designer to tweak and refine all of those images.

Derek Bruff: 7:22

Okay. So I want to get into that. But first, what do they look like? Can you describe. We have a we have a printout of these trading cards in front of us here. What do they look like on the front and on the back?

Emily Bruff: 7:34

So the front is sort of a monochromatic printmaking style portrait of the bust of whatever the theologian, whoever the theologian is. And there's fun facts.

Derek Bruff: 7:48

Fun facts about theologians.

Emily Bruff: 7:49

Yes, there's lots of fun facts about theologians. And then there's also on the back of all the cards is the kind of a remix of the book cover.

Derek Bruff: 8:01

Okay. Yeah. And so we have I see Augustine of Hippo and Martin Luther, John, Calvin Anselm of Canterbury, Thomas Aquinas.

Emily Bruff: 8:14

Julian of Norwich.

Derek Bruff: 8:15

Julian of Norwich, Teresa of Avila and the Cappadocian Four.

Emily Bruff: 8:21

Yes.

Derek Bruff: 8:22

Is that a a rock band or a superhero team?

Emily Bruff: 8:27

I like to think they're superheroes.

Derek Bruff: 8:30

Fourth century theologians. Okay, so. So we have the portraits, as you said, monochromatic, so kind of black looking like a print on a on a solid colored background. So what was a tell me about the process of working with the AI tool here? What what, what was it easy to get the AI to generate and what was what was harder to work with?

Emily Bruff: 8:59

It was easier to get the AI to do an image of someone particular. Well, you would think that it would have been easier for them to get for the AI, to get an image of someone that we knew what they look like. But it's funny because that ended up being one of the more difficult. Mm hmm.

Derek Bruff: 9:22

Like a like a martin Luther or a John Calvin.

Emily Bruff: 9:25

The Wesley Brothers. We we that one was in particular very difficult for some reason, for for AI to understand. For example, it came out with three or four brothers and there was only two.

Derek Bruff: 9:41

Okay.

Emily Bruff: 9:42

It came out with, you know, even though I had. So I would input the name of the theologian and I would say, come up with a portrait of this theologian in this century and create it in the style of a trading card or something like that. So it would come out with first, I made the mistake of saying sports trading cards, and then all of them had, you know, sports equipment and baseball caps.

Derek Bruff: 10:11

Okay.

Emily Bruff: 10:12

So that was a no go and then if until I realized I need to start saying bust of said theologian. It would have all, you know, full body length, crazy hands, multiple, you know, five or six hands sometimes.

Derek Bruff: 10:29

All right.

Emily Bruff: 10:31

And then there were some you know, it would hallucinate glasses for people in the fourth century or, you know, so really strange kind of things that you would think maybe it wouldn't do because even when I would say no glasses or no eyewear, it didn't understand that command at all. So that's where the designer really came in to be useful, to be able to really fine tune fine tune those specific details.

Derek Bruff: 11:03

So you mentioned using a template in the tool. What what do you mean by that?

Emily Bruff: 11:08

Well, I tried to use the free form version of the art generator because I thought that might work better because I wouldn't constrain it as much. And I thought if I could say specifically what I wanted, like monochromatic ink drawing, then it would come out with maybe what I wanted. But it didn't understand any of those words. And so it would have, you know, full fledged paintings full color when I said monochromatic ink drawing. And so there was a template of ink drawings.

Derek Bruff: 11:43

Okay.

Emily Bruff: 11:43

And that's what I ended up putting in, because I suppose they had been able to train that particular template on what an ink drawing was.

Derek Bruff: 11:51

Gotcha. And I assume you had a lot of different options for templates.

Emily Bruff: 11:54

Yes.

Derek Bruff: 11:55

Okay. But this was the one that matched the look that you were going for.

Emily Bruff: 11:58

Mm hmm.

Derek Bruff: 11:58

So then everything that came out was any drawing of some sort.

Emily Bruff: 12:03

Yes.

Derek Bruff: 12:03

Okay. And then you said there were some kind of keywords that you needed to to either use or not use.

Emily Bruff: 12:11

Yes.

Derek Bruff: 12:11

Right. So bust gave you a kind of shoulder up image of the person in question. Mentioning glasses was problematic. Mentioning sports was problematic.

Emily Bruff: 12:23

Yes. Yeah.

Derek Bruff: 12:26

So. So is there anything that wasn't problematic?

Emily Bruff: 12:35

Martin Luther actually came out pretty well. And I think it's because he's one of the more famous theologians. There's lots of images of him. And so that one was fairly straightforward. Yeah.

Derek Bruff: 12:52

Otherwise, it was a lot of.

Emily Bruff: 12:53

A lot of them were problematic.

Derek Bruff: 12:54

Yeah. Tell me about the Cappadocian Four.

Emily Bruff: 12:57

Well, AI doesn't think that women are theologians. Okay. Apparently. So I eventually, instead of saying.

Derek Bruff: 13:07

So, that so the cappadocian four, I'm seeing three men and one woman.

Emily Bruff: 13:10

Yes.

Derek Bruff: 13:11

Right. So that was that was your goal?

Emily Bruff: 13:13

Yes. And and they're from the fourth century. And so if I used create a group portrait of the cappadocian Four theologians from the fourth century, it would not only generate a group portrait of like six or seven people, but they were always men.

Derek Bruff: 13:36

Okay.

Emily Bruff: 13:37

And so I had to be very specific. Three men, one woman, fourth century. And even that didn't always work. It would hallucinate more. It didn't unders. It can't count apparently. It doesn't like to count.

Derek Bruff: 13:52

It doesn't like to count. And it you know, I've talked to many times about the biases in the training data for these AI tools, and presumably it doesn't have a lot of experience working with images of fourth century female theologians.

Emily Bruff: 14:07

That's probably true.

Derek Bruff: 14:09

Right, right, right. And so you had to actually kind of take out the theologian part in order to kind of overcome that.

Emily Bruff: 14:15

Yes.

Derek Bruff: 14:16

Yeah. So one of the things that I've been saying a lot for a while when working with faculty is that experts can use AI to get expert level results, but novices can't. They tend to get novice or maybe kind of novice plus results. Mm hmm. Does that resonate with your experience on this project?

Emily Bruff: 14:34

Absolutely. I mean, you know, I have a degree in fine art. I have a I have a doctorate of ministry, but I am not an expert in any way of because my training is Protestant. I don't have much knowledge at all about all of the symbolism that goes into clergy icons or symbols or, you know, attire and what that might mean or even what a shaved head means versus a head full of hair.

Derek Bruff: 15:12

Mm hmm.

Emily Bruff: 15:12

And so when I was passing my proofs back and forth from the authors to the designer, you know, I could only go so far. I was a very good translator because I was an I'm an artist with some theological training between the Ph.Ds and the designer. But there was a lot of things like the miter for Anselm of Canterbury, like he needed to have that hat, that very specific hat. And even the authors were like, We teach in our classes. You'll know Anselm by his hat, or you'll know these theologians by their attire, their symbols. And so it was really important to get, you know, the the clerical robes right for the Wesley Brothers. And, you know, Teresa of Avila was Spanish. So her particular head covering needed to look different than Julian, Julian of Norwich.

Derek Bruff: 16:14

Mm hmm. Yeah. And it's interesting to hear you say that, because I have gone to art museums with you. And if there's a, you know, Renaissance painting of some scene from the Bible, there's a lot of symbolism that's used. And you're really familiar with that kind of symbolism, because that's something you would have taught in your art history courses. But the head coverings of particular orders of of nuns is not something that you're that hasn't come up in art history very often.

Emily Bruff: 16:45

At least not in the art history that I studied, I'm sure. I mean, there's layers and layers upon layers of art history. And that was just not one that I, as a Protestant, particularly, am well, well-versed in culture.

Derek Bruff: 16:58

But the authors of the book were able to give feedback and say, No, it's more like this. It should be that.

Emily Bruff: 17:03

Yes.

Derek Bruff: 17:03

Yeah. That's fascinating. Yeah. Are there any other particular theologians where you remember some some back and forth on.

Emily Bruff: 17:17

Karl Barth's hair was a was a thorn in the side. I don't know why AI wanted to make his hair very poofy, but.

Derek Bruff: 17:27

Okay.

Emily Bruff: 17:27

We know exactly what he looks like. And so that was kind of humorous. But we have many, many conversations about Karl Barth's hair.

Derek Bruff: 17:37

Not something that's happened in my day job.

Emily Bruff: 17:42

Menno Simons And I'm probably not even saying these correctly, but he's wearing a skull cap and he has a much longer beard. And the hat he was originally in apparently looked more like a fisherman's hat and yeah, there was a Dorothy Sayers had a hat. For some reason AI thought she should be wearing a hat and that's not really something she did.

Derek Bruff: 18:08

So, no. You know, you had said you had told me because. Because the tool keeps a record of all of your prompts. Yes. Like, how many prompts do you think you use for all this?

Emily Bruff: 18:20

I had to go back and look at a couple of previous and at that point it was over a hundred. And so I don't actually know how many I ended up with, but for 18 portraits, there was over 100 prompts.

Derek Bruff: 18:34

Yeah. Yeah.

Emily Bruff: 18:35

And each prompt generated four options.

Derek Bruff: 18:38

Oh, sure. Okay.

Emily Bruff: 18:39

So.

Derek Bruff: 18:40

Yeah. Yeah. So. So you are you are an educator among the many hats that you wear even. Well, I guess Dorothy Sayers, I don't know if she wears hats at all. You wear a lot of virtual hats or metaphorical hats. So if I were teaching a course that used this book, Know Your Theologians as a textbook, and I wanted to give my students the assignment of making more trading cards with different theologians, how might the students use of AI be useful to their learning, or maybe not useful to their learning? What How do you think this would play out as a student assignment?

Emily Bruff: 19:27

That's an interesting question. I think it would be really fun to just assign, you know, create, fill in the rest of the theologians from the book and maybe you pick the one that you want and you assign everybody in the class a different theologian. You know, it would be interesting because the student would have to be very well versed in the theologian and all of the symbolism that went with that theologian's, you know, whatever their job was, whatever their order, whatever their, you know, denomination in the church, whatever century they lived in, you know, it would they would have to have an incredible amount of background knowledge to understand the image that they're producing with AI was accurate or potentially accurate because, of course, we don't know what all of them looked like. But but that would require a great deal of research on their part just to and it would be something that would probably be fairly quickly quick to grade for the professor because they would be able to look at it and immediately know, well, clearly, you didn't read about the Wesley Brothers because there's four guys here, right?

Derek Bruff: 20:45

Right.

Emily Bruff: 20:46

Or this person is wearing a the completely wrong habit for their order of of monks or whatever it is. So.

Derek Bruff: 20:56

Yeah. Yeah. You don't have the right miter.

Emily Bruff: 20:58

That's right. And I don't again, I would probably not do well at this project because I have not well enough versed in this.

Derek Bruff: 21:06

Yeah. So it's interesting because I've been thinking about the use of AI image generators in multimodal assignments where we might ask students to create something with images or video or audio.And my initial thought was that for something like this, I was going to focus on the text, on the training cards. What are the like two or three most important things you would want to say about this person? And the student could use the image generator to quickly generate an image that would go with that. But in fact, it sounds like creating a good image is is the harder task here and kind of the richer task in terms of a learning perspective.

Emily Bruff: 21:51

Mm hmm. Yeah, because it was much more labor intensive to create these images than it would have been for me to upload their manuscript and ask AI to summarize it.

Derek Bruff: 22:03

Yeah, right. It's going to do a pretty decent job doing that.

Emily Bruff: 22:08

Mm hmm.

Derek Bruff: 22:10

Well, and it's also I'm thinking of another student project that I knew about at Vanderbilt, where students were taking parts of the Parthenon, the frieze, the sculpture on the exterior of the Parthenon that had been worn away through time and creating a digital copy of that and then filling in the gaps. Right. Kind of reconstructing what was missing and thinking about how it would have been painted thousands of years ago. And it was not really a creative project because students weren't allowed to kind of go in any direction they wanted to. They needed to make educated guesses about how were horses depicted, Right. How can we how can we fill in this half a horse, Right. What?What paint, what materials were used at the time and what would that look like? And so there wasn't a right answer because we don't exactly know what the Parthenon would have looked like thousands of years ago. But there were lots of wrong answers. Right. And so it was it was a way to kind of help the students dig into the the art history. There. And I'm sensing the same kind of thing here that, like, there's a lot of the lot of bad portraits of Anselm of Canterbury. I can imagine to create a good one. You have to make a lot of educated choices and then also work with the AI to give you the things that you're you're after.

Emily Bruff: 23:36

Yes. That's 100%.

Derek Bruff: 23:40

What did the the graphic designer end up adding to the illustrations?

Emily Bruff: 23:45

Mm hmm. So to make them look more cohesive, the designer added a couple of layers of texture to make it look more like printmaking. She actually took out the white from the image so that it was. It looked more like a print off a print block. So instead of being a stark white, black and color, it was purely monochromatic with black. And then, you know, she was able to just add some mark making with, you know, the outlines and and parts of their clothing that sort of tied everything together in a unified way.

Derek Bruff: 24:35

Gotcha. And that's where I can imagine a student assignment might not include that piece unless they were in a in a design course. Yeah. Okay. Well, thank you for this, Emily. This has been great.

Emily Bruff: 24:48

Yeah.

Derek Bruff: 24:49

I appreciate you doing a little deep dive into this project and Know the Theologians is coming out soon.

Emily Bruff: 24:55

Yes. Hey, if you want your very own set of theologian trading cards, you can preorder the book. Preorder bonus right here.

Derek Bruff: 25:03

Absolutely. Well, thank you, Emily. Thanks for coming on the podcast.

Emily Bruff: 25:06

Thanks for having me.

Derek Bruff: 25:10

That was Emily Bruff, marketer for Zondervan Academic, and also my wife Emily is far more comfortable interviewing others on a podcast than being interviewed herself. So I'm extra thankful that she sat for this interview and shared her experiences with our audience here. If like Emily and me, you find the intersection of art and theology an interesting place, then you should go visit Emily's website. See the show notes for a link where you can learn more about her research, her art, and her consulting work. As I record this, Emily is busy preparing for Visio Divina, an art and faith conference she's hosting in Nashville on September 14th. Again, see the show notes for a link for more information about that very interesting conference. And if you'd like to hear more about the intersection of AI and art, then I recommend you scroll back a few episodes and listen to my interview with Pary Fassihi in episode 35. Pary talks about her multi-modal AI assignments with great nuance and care. Intentional Teaching is sponsored by UPCEA, the Online and Professional Education Association. In the show notes, you'll find a link to the UPCEA website where you can find out about their research networking opportunities and professional development offerings. This episode of Intentional Teaching was produced and edited by me, Derek Bruff. See the show notes for links to my website, The intentional teaching newsletter and my Patreon where you can help support the show for just a few bucks a month. If you found this or any episode of intentional teaching useful, would you consider sharing it with a colleague? That would mean a lot. As always, thanks for listening.