How to Use the Triple E Framework for Edtech Evaluations

00;00;00;03 – 00;00;03;16

John Davis

This is the 10 Minute Teacher Podcast with your host, Vicki Davis.

Show Sponsor – TARA: Your Teacher’s Assistant

00;00;03;25 – 00;00;30;00

Vicki Davis

Thank you, TARA, for sponsoring today’s show. A new tool to help teachers streamline their biggest burdens and create more time. Stay tuned at the end of the show to learn how to get your free account and about the powerful tracking abilities for teachers and special education teachers. 

Introducing Dr. Liz Kolb

So, so excited about today. We’re going to be talking to Liz Kolb, who is clinical associate professor of educational technologies at the University of Michigan in Ann Arbor, Michigan.

00;00;30;02 – 00;00;51;28

Vicki Davis

I know Liz from some years a go when she authored Toys to Tools about connecting student cellphones to education and a lot of things about cell phones. But today we are going to totally focus on the Triple E framework. A lot of folks were talking about this at ISTE. I attended virtually this year. 

What is the Triple E Framework?

And just to give an overview, what is the Triple E framework?

00;00;52;11 – 00;01;14;00

Liz Kolb

Yeah. Thanks for having me. So I developed the Triple E framework about ten years ago, and I originally developed it for my pre-service teachers at Michigan because they were having trouble understanding how to take TPACK into the classroom. Like what that sound felt like. They just really struggled with what does a lesson look like that has all the TPACK.

00;01;14;08 – 00;01;39;07

Liz Kolb

So I realized that they needed something less theoretical and more concrete. So I developed the TripleE framework in order to fill that space. 

And the framework is it takes research around ed technology and pretty much puts it into nine questions that you should be asking when you develop a lesson. There’s three questions around engagement, three around enhancement, and three around extension.

00;01;39;07 – 00;02;08;18

Liz Kolb

And it’s all focused first on the learning and the fact that there are learning goals and thinking about how the technology is engaging students in the learning goals, enhancing the learning goals, and ending those learning goals through this nine question framework. 

And in the end, you get a score and the score tells you if you’re doing a nice job connecting the learning goals and the technology choices and the pedagogy around that, or if maybe you need to make some different decisions to strengthen that connection.

00;02;08;18 – 00;02;20;25

Vicki Davis

I love it because several of the presentations I saw used your framework. You and I’ve been around long enough that people fall in love with the shiny and the new. But doesn’t it come down to how we use it in the classroom?

The Effectiveness of Technology Tools Comes Down Pedagogical Choices and Instructional Strategies

00;02;20;25 – 00;02;48;08

Liz Kolb

And that’s what every single piece of research that I’ve come through when I was developing this framework said. Ultimately it said that there’s no such thing as a bad tech tool or a great tech tool. It really comes down to the pedagogical choices and instructional strategies the teachers is using in conjunction with that tool. 

So you could have two studies on like Seesaw, and one study would show that students improved in their growth, in their learning growth, and another study might have the opposite.

00;02;48;11 – 00;03;01;01

Liz Kolb

And it had nothing to do with the fact that they chose Seesaw. It really had to do with those small instructional moves that the teachers made around it, which once again shows us how important the teacher is when it comes to the lessons with technology.

Do you get pushback on the framework?

00;03;01;19 – 00;03;04;02

Vicki Davis

Liz, do you ever get any pushback on the framework?

00;03;04;03 – 00;03;30;08

Liz Kolb

You know what? I haven’t gotten much pushback on it. And I think it’s because this particular framework is very practitioner based and it really tries to bring the research to practice in a way that’s digestible and easy to do. And I think sometimes in my research world that I’m in now, we tend to stay in the clouds where we make everything very lengthy and wordy and difficult to assess.

00;03;30;19 – 00;03;50;23

Liz Kolb

And I think what I’ve tried to do with this framework is to make it so that you can get to the point where you never even have to look at it, just in your brain, you can quickly go through each of these and see if the lessons there. So I’ve actually gotten more thank you’s than pushback that, oh, “this is what we needed for four tech coaches working with teachers” or for an administrative walk through.

00;03;50;23 – 00;04;01;01

Liz Kolb

I know a lot of schools are using it that way as a way for the administrators to have conversations with teachers about what they’re doing with technology, knowing that it’s researched, informed and evidence based.

Do You Every See the Triple E Framework Misused?

00;04;01;06 – 00;04;15;10

Vicki Davis

I remember years ago talking to Bernie Dodge about the misuse of webquests (Women of Web 2.0 Show), and he kept saying how sometimes it was misused. Have you ever seen that Triple E framework not used in a way that you really intended it to be used?

She does not recommend using it to evaluate a whole curriculum.

00;04;15;12 – 00;04;39;19

Liz Kolb

I have seen people try to use it to evaluate a whole curriculum, and sometimes the curriculum didn’t even have technology involved in it. And this is really specifically when it’s lessons in and tech tools involved. So it’s not something you can just use where technology is not involved in it. So I have seen it used that way and I wouldn’t I would not advise it.

What Liz wishes every teacher and administrator understood about education technology.

00;04;39;25 – 00;04;50;17

Vicki Davis

So fill in the blank. Liz. I wish every teacher and administrator understood _______ about educational technology.

00;04;50;21 – 00;05;21;01

Liz Kolb

I wish that they understood that there’s no such thing as a bad tool when it comes to education technology. In other words, just because it’s an older tool doesn’t mean it can’t be really effective. And just because it’s the shiny newest tool does not mean it’s effective. 

Don’t evaluate teachers based on the newness or popularity of a tool but the pedagogy used with a tool.

So I wish every administrator knew not to evaluate a teacher based on the newness or the popularity of a tool that they’re using, rather the pedagogy that they’re using in conjunction with the tool, whether it’s old or new.

00;05;21;06 – 00;05;47;01

Vicki Davis

Sometimes I see research with tools where basically they’re overpromising the tool’s ability to deliver, and it does come back to the pedagogy. I would think that might bother you some because you see the studies and first thing you do is you create the tool, then you get another round of funding, you fund the research and then you tell everybody that your tool is going to solve blank, how do you feel about that?

Instructional design and how students learn is most important.

00;05;47;06 – 00;06;07;25

Liz Kolb

Yeah, I don’t think we actually I think we have lots of tools. We have plenty of tools now. And it truly is about the instructional design and if you don’t understand how students learn and how students how to create interventions for students when they’re struggling with things, those kind of basic educational teaching and learning practices, which is a skill.

00;06;08;07 – 00;06;28;06

Liz Kolb

And if you don’t understand that no matter how the tool use you have, you’re never going to really get the learning growth that you want, ultimately. We need teachers to really understand how students learn best and then add the technology to it. Rather than trying to have the technology somehow replace a teacher that maybe doesn’t have those skills and ability because that’s not going to work.

What excites Liz in classrooms today.

00;06;28;07 – 00;06;35;04

Vicki Davis

I know you observe a lot of teachers. What does it take to get you excited about a class and what’s going on in the class?

00;06;35;04 – 00;06;52;25

Liz Kolb

One of the things I always look for is I look that they are really putting that learning goal first, that they are telling me this is what we wanted to do. They’re giving students agency around that. They’re having them come in and say, “Oh, we’re interested in learning about.” 

 

00;06;52;25 – 00;07;14;28

Liz Kolb

I had one teacher I wrote about in a book where he had students who wanted to learn about the Flint Water crisis and it blew him away. And so then he started using technology tools that made sense for that learning goal, connecting with another classroom through Skype and co-writing through like Google Docs. And so the tools fit in with enhancing what he wanted to do with the learning goal rather than jumping in and saying, we’re going to make an imovie.

00;07;14;29 – 00;07;19;08

Liz Kolb

He went to the students and said, “What do we want to learn about?” And I think that’s the most important thing.

Does the Triple E framework need to evolve?

00;07;19;16 – 00;07;45;01

Vicki Davis

So when you think about where education technology is heading, we’ve got a lot of convergence happening. We have Artificial Intelligence converging with augmented reality and virtual reality and all. Of course, A.I. is the big buzz word now as you see us moving forward, because a lot of people are promising big things with AI. I actually read in a book recently that the answer to the teacher shortage was VR.

00;07;45;06 – 00;08;04;28

Vicki Davis

Like, I literally read that and I’m sitting here thinking, so as you see all these technologies converging and as people evaluate something like, say, Grammarly, which gives feedback to students as they make those mistakes, do you think the framework needs to evolve, or how do you evaluate when the technology is actually teaching a little piece of the lesson?

00;08;05;07 – 00;08;22;18

Liz Kolb

I actually designed the framework, so it’s not it’s not about a particular tool in time. So it really comes down to we want students to meet learning targets, right? And that’s not going to change whether we’re using AI or VR or some kind of new emerging tech, we still are going to want students to meet those learning targets.

00;08;22;20 – 00;08;44;27

Liz Kolb

So you can use this framework to evaluate a lesson with adaptive software or the latest VR goggles that classrooms are using. And ultimately, it still comes down to the learning goal. And if you’re using VR goggles and it doesn’t make sense for the learning goal, that’s going to show up in the framework and it’s going to cause you hopefully to think, is this the right tool to meet this goal?

00;08;44;27 – 00;08;59;25

Liz Kolb

Or maybe we need to modify the pedagogy, right? 

Yeah. There’s a lot of big promises out there and ultimately it comes down to the teacher understanding how students learn best and knowing the pedagogy and being able to select tools that will meet the learning goal. And good instructional design.

How to Learn More About the Triple E Framework.

00;08;59;25 – 00;09;20;26

Vicki Davis

Teachers and administrators are listening to you talk. A lot of folks already know about the framework, but others may not. Where are the best resources and places to go so that they can jump in and really focus on the pedagogical use of education technology because it’s really all about how you use it and how you teach with it.

00;09;20;26 – 00;09;42;03

Liz Kolb

Yeah, and I think that’s what a lot of schools are looking for now, especially once they got all these tools with the COVID pandemic. So I have an open resource ,https://www.tripleeframework.com/  where it has the TripleE framework on there. You can use it has some information about how it was some case studies in there, has contact information for me as well.

00;09;42;03 – 00;09;58;13

Liz Kolb

And then I also have in beta version, I want to emphasize that the TripleE PLN which we’re starting to develop as a way to build some community around this and that’s still in progress. So if you have tips on that, you can also let me know. But those are two great places to get started with.

00;09;58;13 – 00;10;01;15

Vicki Davis

Excellent. Are you seeing a lot of conversations happen in the PLN?

00;10;01;15 – 00;10;12;06

Liz Kolb

Starting and we’re trying to figure out what’s the best way to organize those conversations. So we actually are doing a bunch of focus groups right now. That’s another thing that people can get involved with if they’re interested in weighing in on that.

Traction with the TripleE Framework

00;10;12;07 – 00;10;38;28

Vicki Davis

So, Liz Kolb, the Triple E framework will include everything in the show notes. 

I was just so impressed, Liz, with the presentations I saw that would say, okay, now we’re evaluating this use of technology using the triple-E framework. And I don’t think it you know, it’s been a couple of years since I’ve been to an ISTE, but I don’t think I’ve seen it in as many sessions as our site this year, which was like, Oh, that’s great, it’s getting traction.

00;10;38;28 – 00;10;41;18

Vicki Davis

Do you feel that way? Do you feel like it’s finally getting traction?

00;10;41;21 – 00;11;06;07

Liz Kolb

I do. And it’s just music to my ears that you say that just makes me smile for the day. I actually got a lot of text and email from friends and colleagues that were saying, Oh, you’re they’re talking about your framework in this session or they’re singing the praises of the Learning First book in this session. So that was the first time I felt like at the ISTE Conference that there was a lot of conversation around the framework.

00;11;06;07 – 00;11;19;02

Liz Kolb

I think in the past I’ve been if I presented they talked about it, but so that was very exciting to me and I’m glad to hear so many K-12 districts using it, not just higher education, because my goal was get it into the hands of practitioners.

00;11;19;12 – 00;11;44;16

Vicki Davis

Yeah. And it’s just very practical. Thank you for being on the show, Liz, and I hope that anyone listening that wants to know more about the framework will check out all your resources. Thanks. 

Thank you TARA: Today’s Show Sponsor

TARA sponsored today’s show. TARA stands for the Teacher Assistant for Resources & Administration which keeps tools, resources, lessons, IEP’s, and meetings tracked in one place. It also includes vetted resources that you can download and use in your classroom immediately. As I reviewed this product, I’ve seen special education teachers and organizations raving about how TARA has helped them organize IEP progress, meetings, and result tracking. TARA is free for up to five people to work together as a team or just for you to use in your classroom to organize your lesson plans. Go to coolcatteacher.com/tara to sign up for your free account. Thank you TARA for sponsoring this show!

00;12;10;25 – 00;12;30;13

John Davis

You’ve been listening to the Ten Minute Teacher podcast. If you like this program you can find more at coolcatteacher.com. If you wish to see more content by Vicki Davis, you can find her on Facebook and Twitter under @coolcatteacher. Thank you for listening.