HILT 2019 Conference: Learning from Each Other, Learning Online

HILT 2019 Conference: Learning from Each Other, Learning Online


Well, hello. My name is Kristin Sullivan. I am the senior director
for educational technology at the Kennedy
School, and I’m also the director of our recently
lodged flagship online program known as the Public Leadership
Credential, one of the projects that you’re going to
be hearing about today. Thank you very much for coming. As you can see, we
have an excellent cast of characters to talk to
you about pure learning and online environments. We have Bharat Anand
from Vise Provost for Advances in Learning,
as well as a faculty member at the business school
and a senior associate dean for HBS Online. So Bharat, thank you very
much for joining us today. We’re very happy to have you. We also have my good friend
Mary Godfrey from Harvard X. But today we’re not
going to say she’s a staff member from
Harvard X. We’re going to say she a GSE student
who is going to be talking about the new online
pre-matriculation experience for incoming students at GSE. So Mary, thank you very much. My friend Dan Spratt, who is
not known as Daniel, but Dan; and we’ve confirmed that
I go by Kristin, not Kris. Dan is an instructor
at DCE as well as a professor of medicine
at Maine Medical Center and the Tuft School of Medicine. So Dan, thank you very
much for joining us. And a partner in crime of
which there are many in the PLC is Teddy Svoronos, who’s is
a lecturer in public policy at the Kennedy School and also
a faculty lead in the Public Leadership Credential. So the way we were
going to do this today is I was going to just sort of
set the table for a few minutes and then invite our
speakers to each have about 10 minutes to talk a
little bit about pure learning and online environments. If you think of a question,
how about if you’ve got it down and we’ll just try and
get through all of it. 10 minutes is always in air
quotes with this sort of thing because we’re going to try
and keep to 10, but who knows. We have some padding there
because what they have to say is very interesting and I’m
sure you want to hear about it. At that point,
we’ll take a minute. We’ll see. Do we want to ask the
panelists questions or would we like to do a little peer
activity in the room? So if you’re OK with winging
it, that’s how we’ll do it, and then the hour
will probably be up. Any questions? All right, let’s get started. What I wanted to do to start– I only have five minutes. Let me just start at the top. I was thinking, I’m
no anthropologist, but I have a suspicion
that since people came on this earth, people
interact with each other. They learn from each other. They help each other learn. Whether that means I’m going
to help you cook something, you’re going to
help me learn how to build shelter in a formal
schooling environment, I’m going to have trouble
with an assignment and I’m going to
ask you a question, you’re going to help
me out, vice versa. And so I think when you’re
in an in-person environment, we naturally want
to help each other and we naturally
seek each other out for help and to help each other. So I think that is
a baseline reality. And so then I started
to think about, well, what about when we take it
out of the physical space? How do we interact with each
other in formal learning experiences as well as
informal learning experiences? And so I just wanted to
think for a second, OK, today we’re going to talk about
online environments, but what other non face to
face environments have there been over time, and how has that
peer interaction taken place? So thanks to the
interwebs, I just did a quick Google search
and I thought, wow, the first time that we were
learning in a distant way was way back in the 1800s
where you received something in the mail, and that
was a way of learning. That doesn’t seem like there
was a lot of peer interaction unless your neighbor also
got that package in the mail. Came over to your house
and you talked about it. Hundreds years later
we moved on to radio. So you could sit
in your living room and listen to
learning come at you. And again, how are
you interacting with somebody else who is
also listening to that? Unless you invited your neighbor
over to listen to the radio with you, which would
be nice, but it’s still an in-person experience. A few years later we moved
to the same kind of concepts, but in a television format. Again, not to beat a dead horse,
but again invite your neighbor over and watch TV with you. There might not have
been any opportunity to interact virtually. It wasn’t until
possibly the ’80s when we first had computer
conferencing able distance education. That doesn’t sound
very friendly, but at least it was an effort. We went into the
late ’80s where we started to move on to the
internet, and in the early ’90s where browsers enabled
probably more deeper learning online, but the
first fully online programs. And this was
something interesting I wouldn’t have thought
of if I had to guess and why U online was
one of the first fully online programs in 1998. And then I want to bring
it back to me for a second. I used to be a teacher actually. In the olden days, I
went to the Ed school. I used to be known
as Ms. Lofblad, and I was running
a writing center out a half hour from here
in Medfield, Massachusetts, at Medfield High School, and I
had computers in my classroom. A lot of the classrooms at that
time didn’t have computers, but I did. And students would
come into my classroom after school and they
said, what are you doing? May I help you? Are you working on
your college essay? Like all that friendly stuff. They’re like, no, I’m
taking an online class through virtual high school. And I just remember thinking,
shut the front door. What are you doing? And they’re like, well, we don’t
offer Russian at Medfield High School, so I’m taking Russian
online or other courses like that. I just thought that
was so fascinating. Sidebar, I quickly
left K through 12 and got into the online
space and ended up here. But anyway, even in that
context in the late ’90s, the interaction I saw in
terms of pure learning was again the physical,
I’m doing Russian online. My friend is doing
Mandarin online. We’re both coming into
Ms. Lofblad’s classroom after school and we’re
talking to each other IRL, in real life what the
kids say, but there wasn’t a lot of interaction
on the online environment. So that was 20 years ago. I hope now we’ve done better. And we’re going to
hear from our panelists today about some of the ways in
which peer engagement and peer learning have taken place. In the last 10 years or so– I’ve been at Harvard
for 15 years. Who’s been here for
more than 20 years? Any long timers? Remember, before any
of this was happening? But I think like
around 10 years ago I started hearing a
lot more about online thank you to Harvard X. There’s
HBS Online obviously, HMX. DCE has done so much with
online, both informal learning experiences and toward a degree. Chan has the low
residency degree now. Who am I forgetting? The online pre-matriculation
experiences at the Ed school, and I just learned about
OL here at the law school, and of course at the Kennedy
School our big bet in online is something that Teddy
is going to tell you about called the Public
Leadership Credential. So it’s starting to happen. And I think this
is a great moment to go from I received a book
in the mail in the 1800s to what’s happening
in 2019 at Harvard and how are peers
interacting with each other virtually to help
each other learn and engage with each other? And with that, I would
like to turn it over to Bharat to go first,
if you don’t mind. Here is the– Thank you. Morning, everyone. Morning. So thank you Kristin for
moderating this panel. Thank you all for coming. I have three slides,
which I’d like to talk through in
the next 10 minutes and make a few observations. The first one actually
picks up on the last thing that Kristin says. And in fact Cecil
also mentioned this. By the way, thank you for such
an pop provoking morning panel. It was delightful. When Cecil was talking about
the future and the hope that online learning
might emulate some of the peer interactions
that we can take advantage of in the physical setting. So I’ll talk about HBS
Online a little bit here, which was what I got involved
in about seven years ago. The thinking behind
HBS Online was sparked by a classroom with
an architecture like this. So we’re in the law school. This is the physical
architecture to have conversations. And the thing that’s most
interesting about of classrooms like this is that sitting
anywhere in the classroom you can see everyone
else without having to turn around and so on. And so the architecture
in some sense enables certain
conversations that wouldn’t be quite as easily
possible if you had rows back to back. That was the DNA of Harvard
Business School as well. And when we started the
online experience, what we were trying to
think about is, how do we recreate that
pure conversation online? So that was really the
first order question. One of the things
that’s interesting about those technologies that
Kristen was talking about is with radio and broadcast. Those were broadcast media. We didn’t have the possibility
for peer to peer interaction. And I’ve been studying
the media entertainment space and how they have reacted
to digital over the last 25 years. I ended up writing a book
about this, which was really talking about the possibility
for this spoke to spoke of peer to peer interaction
as being what’s profoundly different
about online than those earlier technologies. And yet, what was
interesting to us when we started thinking
about online education, was you go back to
the last 20 years and most of online education
was still broadcast. So the question is, can we
take advantage of that somehow? And I’ll just mention a
few things on this slide. First, we had to be
very intentional to put peer learning at the
center of the conversation. So that whiteboard
there was literally a snapshot of the whiteboard
which we were scribbling stuff on for three months. And you might be able
to see social learning at the top of that chart. We had broken it down
to different forms of learning, active
learning, adaptive learning, social learning. So we were thinking intensely
about how do we make this work? And I just want to mention that
because it doesn’t just happen. It’s not that these
conversations just happen because you’re online. I’m always amused by
newspaper companies who tell me, oh, we’re in
the peer to peer space. And I say how? And they say we have discussion
forums, which I often label as crimes against humanity. But if you spent 99% of your
time thinking about the content and then 1% thinking about
how to make those interactions occur, it’s not going to occur. The thing that was
interesting was after three months of this
endeavor we had three MBA students working with us who
were giving us advice on how to do this, and I noticed
they’d been saying something that we weren’t listening to. They kept saying we learn in
the classroom from each other, but we also learn outside
the classroom, in the lunch corridor, in the gym
conversations and study groups. And they kept talking
about social learning. And what I realized was
even though we were talking about it, we hadn’t made
it central to the platform building effort, which was
particularly sobering since I’m writing this book talking about
spoke to spoke interactions. That’s how insidious
this problem is. So we basically pivoted
the entire effort to focus on social learning. And this is the first page
of the HBS Online platform. There’s no content. It’s just a global map, and with
pulsating bubbles and people can see who’s on and what city. And if you click
on a bubble, it’ll show people’s profile pictures
and then you can message them. The first day that
we launched this, there were 300
people who logged in. There were 13,000 profile views. All they wanted to do
was check each other out. But I just want to pause
that because in some sense we can’t have
conversations unless we’re familiar with each other. So familiarity is almost
a precondition, and that was profoundly important. The next thing
that was important is trying to think
about incentives. Why would people want
to talk to each other as opposed to just looking at
the content from the faculty member online? How do we make it easier for
people to talk to each other? Because most of the discussion
threads and online forums or online courses
are just hugely long. You can’t figure out
what topic is where. People get turned off. So we realized that in
the physical classroom, we actually give
incentives to students. So in our courses, we
say half your grade depends on class participation. Meaning, that’s a
pretty strong incentive to talk to each other. So we said, why don’t we
try the same thing online? So to complete
the online course, we said part of
your grade depends on the extent to which
you answer other people’s questions. We didn’t say whether
it’s 10% or 50%. That’s all we said. Just part of your grade. And on the first day,
75% of the students went to the peer forums. So incentives matter. Searchability matters. As opposed to having the
discussion thread on the course homepage, we had it
local to each page where the questions on
that discussion forum are relevant to the
content on that page. Again, it’s such a
simple thing, but it just makes it much easier
to search then. And then we also made it fun
because a lot of students are scared of asking questions
thinking, if I ask a question, someone will say what
a stupid question. So we said when
you ask a question, tag your question so
you can label it saying I’m asking a stupid question. The moment you say
that, you’ve taken away the license that I have to
say it’s a stupid question. So I mean these are
just behavioral things that we were thinking about. There’s one thing I
just wanted to mention, which we borrowed literally from
the physical classroom, which is the famous cold call. So in the physical
classrooms, we often just call in students
at random and it’s a way for keeping
them attentive. By the way, what’s interesting
is they get really scared not because they might say something
that doesn’t make sense in front of the
faculty, because they’re worried about what
their peers might think. So we created something called
the online cold call where you’re going through the course
and suddenly at random a pop up appears and it says Cecil,
you’ve been cold called, you have a minute to
answer this question. There’s a clock
ticking in the corner. 30 words or less
and your answer is visible to the entire cohort
with your profile picture. So suddenly it sort of becomes
not just an act of learning, but a social event. By the way, I will say Sal Khan
has copied this feature now for the Khan Academy, which
I was very proud to hear, although– and this is
just an observation. –the effectiveness
of the cold call depends on cohort familiarity. Meaning, if it’s a
lonely experience up until I get cold
called, I sort of don’t care about
what my peers think. So these things must
go hand-in-hand. So that’s basically
just to give you a glimpse of the
platform that was built. How has it worked? So first of all, in terms of
the effectiveness of questions and answers. So we can’t have content
experts or faculty intervening every time there’s
a question online, so we needed peer
learning to work. The first three
weeks we were just looking at the questions
ready to jump in if the discussion threads
went off in the wrong way. The number of times
we had to jump in was precisely zero, every
question correctly answered by the peer group. OK? And this was stunning. Now, that comes with a caveat,
which is it doesn’t always happen on the first answer. You’ve got to give
them space and time, just like we do in the
physical classroom. So we often say
when you’re teaching in this kind of classroom,
trust the students. Give them space to
talk to each other. It’s the same thing online. It’s really hard not to jump
in, but if you give them space, within half an hour
or so they’ll have basically converged to the answer. The other is the diversity
of the cohort, which is exemplified in this chart. This was someone who
posted this on LinkedIn. And I was gratified to see
that the bomb diffuser actually has time on his hands to
take an online course. But what’s more interesting
about the diversity is they actually throw out
ideas that we would not even think about. So in the economics
course, I was talking about coupons as a
way of price discrimination. Some student in Holland
writes, never heard of coupons. We don’t have that in Holland. And that started a really
stimulating conversation. There was another example
where I was talking about McDonald’s Happy Meals as a
way of price discrimination, and I jokingly mentioned in the
video isn’t it interesting that adults would never go by
themselves to get Happy Meals in McDonald’s? And then someone from Malaysia
posted a picture of a huge line outside of McDonald’s where
there were only adults standing to get the toys from McDonald’s. So you start seeing
that diversity in ways that are pretty profound. The third and probably
most important thing is confidence and deeper
understanding of the material. When you are forced
to teach others, you actually understand it
better yourselves and students take ownership of
the experience. So those are some of the
things that we’ve seen. Are there things that
we can do more of? Yes. The three I would just throw
out are group discussions and spontaneous breakouts
on the platform; peer grading, which we’ve
done to some extent, but there’s still huge
opportunity there; and peer tutoring. So in some sense,
the live interaction comes entirely asynchronously
in this platform, but there might be ways to
have synchronous interactions. The one thing I’ll say that
we were really surprised by was students then started
having physical meet ups outside the course
content or structure just to get to know people
in their different cities, saying who’s in Tokyo? Who’s in Bangalore? Who’s in Singapore? Why don’t we just meet? This actually gave
rise to something called HBSO Connext, which
is an event we have once a year on campus where
we invite people who’ve taken a course to come to campus
and we run sessions for them. We send one email. Typically 600 people
typically show up to campus, and they fly from
as far as Australia and Denmark and Brazil
just for that one day. So this is the kind
of human interaction that is now being built online. There are 29 chapters now. This is community chapters
across the world which have been self
organized, and all we do is literally just
a light touch guiding. The most inspiring
thing that happened was in this last
Connext event, this May, we had a challenge for
them in partnership with the Greater
Boston Food Bank, saying there’s food available. There’s people who need it. How do you manage inventory
supply chain reliability of delivery issues? And so the CEO of
Greater Boston Food Bank basically posed this challenge
to the HBSO Online community. You had I believe nine
teams self organized in different cities
who basically submitted sort of proposals. We then convened
them through HBX Live to talk about their
proposals and one was chosen as the winner they
were literally building apps to solve this problem,
which is sort of inspiring. You can now think about
communities sort of going far beyond the online content. The last thing I’ll
just show you– Dan’s I think going to talk
about a version of this. –is the synchronous
classroom at HBS Online, which is basically a
physical classroom, but instead of 60 seats,
we have 60 TV screens, and you can run
a case discussion with the only difference being
you’ve collapsed geography. So you can be
anywhere in the world. This is now changing
executive education completely for the school and
potentially for the university as we start making
these classrooms available to the rest
of the university. By the way, the
true test of this was when our dean was
testing the facility with a group of people online. He walked up to one
person on the wall and grilled that person
for three minutes in a case discussion. He said at the end of the three
minutes I saw her sweating. At that point I knew
this would work. The other thing
that happened was we ran a program of seven sessions. On the last session
people refused to log off. I mean they were
literally crying. They had developed those
interactions with each other. So I’ll just pause there. By the way, in terms of scaling
this, you have 60 on the wall. You can have 1,500,
3,000, 10,000 people watching the livestream
and chatting through the chat bar. So there’s a way
now to scale this as well, which is
really encouraging, and by using both
platforms now going forward for university wide courses. The Harvard X team, many of
whom are already in the room, have been building
out courses to take advantage of what we have. Thanks, Bharat. That’s so thought provoking
and it’s interesting. It was sort of with academic and
social online and an in person component because we just as
people can’t not be together. So that’s interesting. I wonder if that’s going to
come up with the other speakers. And so with that, I’ll pass
it on to Mary, GSE student. Good morning, everyone. So as Kristin emphasized, I
am here today as a learner. I think many of
you know me as Mary from Harvard X as the
production manager, and I’m going to put that hat
aside and speak to you fully and wholly just as a
learner at the Graduate School of Education. So when Kristin and
I were chatting, we were talking about the
concept of pure learning and the idea of
learning as a peer, but also teaching as a peer. And so I tried to think about
how that experience showed up for me in the how
people learn course. So very quickly show of
hands, how many people here have heard of the How
People Learn course or taken The Help
People Learn course? OK. And then how many are
not so familiar with it? OK. So bear with me for about
20 seconds for those of you in the room that know something
about how people learn. This is a
pre-matriculation course offered through the Graduate
School of Education. It’s eight weeks, six modules,
following two weeks working on a design
proposal, and then it culminated in an
in-person component at the very end
where we actually got to meet and greet and
share with many of our peers that we were
engaging with online. So that’s sort of
an overall format for how that experience went. A couple of other
important things I think that are
key to understanding what made it effective for
me were small group cohorts. Those were cohorts that were
designed for us to engage with each other on
discussion forums, and they were the
same set of people that we engaged with over
the course of those modules. So we got to know
each other over time. Anyway, that’s sort of the
broad strokes around some of the things that I’ll
show you up here on slides and also some of the format
for how people learn. A few other things I think
that’s important to hear about me as a learner because
this was so important to me on my learning journey for
how people learn is this. I’m not somebody
who grew up learning through online
education platforms. My pre-K through 12 experience
was purely residential. At this point in my life, I
am a mid-career professional. So online learning just
wasn’t a reflexive component to how I learned over my career. I also am somewhat
of an introvert. So I’m not inclined to engage. It’s just not my MO. In fact just for fun, we
don’t have to do this now, but for giggles and for a
laugh, you could go and find me on Twitter– I am on Twitter. –and you’ll find zero
tweets and zero followers. And that’s just by way of
example of how uninclined I am to sort of get out there. So going into how people learn,
I knew this about myself. I also knew there was going to
be an expectation that there was going to be some
component of this, and that actually
kind of scared me. So if I can just show you at
the beginning of the course there’s a learner
profile you fill out and I was so
cognizant of the fact that I was going
to be intimidated by this idea of engaging with
my peers in discussion forums. I actually set a challenge
for myself and this learner profile, that I would
challenge myself to post one interesting
idea or observation a week. That was how I was
going to try to overcome this kind of reticence. OK. So hopefully at the end of
this, we’ll find out how I did. So just to move a
little forward here, as I thought about the theme
of this particular talk, I wrestled with what
could I sort of tell you about peer learning in
my experience of it? Because there were so many
rich opportunities for peer learning in this course. I landed on three
approaches that I thought jumped out at me that
were really effective for me, but I just want
to say that there are a lot of opportunities
in this course for this sort of engagement. So again, show of
hands, how many people are familiar with Yellowdig. So Yellowdig was
the digital tool that functioned as
the discussion board for this course. So there were sort of
two elements to the way that we engaged with Yellowdig. One was required
postings that we had to put on to the
discussion forum each week. Each week we had two prompts
that we were to respond to. One new insight that you
learned from the course material in that module, and
then one thing you might carry forward
in your careers or in your future
educational endeavors. What I liked about
this for me is that it provided some grounding
and accountability for me as a learner in the experience. Grounding because we were given
things, like best practices for how to post word
count, that sort of thing; grounding because I got to
know my peers over the course of the modules;
and accountability because I had to do it. So unbeknownst to me when I
set that challenge for myself, I was actually already
going to succeed. That was sort of a fun
little takeaway for me. The second way in which
I engaged with Yellowdig was in an optional capacity. So sprinkled
throughout the modules were little moments
where you were invited to go ahead and connect to
each other in your cohort on Yellowdig around a concept
that jumped out at you. And again, these were optional. But I think that the
optional was something that was made easy
because we were already in the practice of
doing it due to the fact that we had to do these
required postings each week. So to my surprise, I
actually went ahead and posted optionally. I mean I didn’t have to,
but I was so inspired and I felt so comfortable
in that space by module 4 that I was actually back and
forth with some of my peers. What you’re looking
at is a little bit of reflection and
connection, and it’s actually a couple of my peers
in this cohort going back and forth about this
idea of hidden curriculum. Turns out just as a fun fact,
both of these peers of mine are in classes of mine
right now at the Graduate School of Education,
and one of them I am locked in with
for a group project for the rest of the semester. And I think it’s just
very, very meaningful as we talk about the networks
that happen, the trust building that happens, and
how that can carry forward into your face to
face experiences. So the sort of final
kind of peer learning moment for me that
was very significant was the final project. And a little bit about the
final project and then I’ll just show you some of
the work that I was doing in that final project. It was a design proposal
that we were to create. We were paired with
somebody in our cohort, and we were invited to figure
out amongst ourselves how we were going to engage. And what my peer
and I landed on was we would just share our notes
with each other via Google Docs. We went back and
forth over email. We said, OK, I’ll give
my comments to you and you can give
my comments back. I want to pause here
and say that I’m not sure I could have gotten to that
comfort level of feeling like I could give my peer feedback. I don’t know that I could
have played teacher to my peer had I not already been
stepping through these very intentional exercises
through the discussion forums and through those opportunities
to connect and reflect. So this is not exciting
in terms of a slide, but this is sort of a
little bit of the commenting that we’re doing back and forth
on the final design proposal. Another sort of important
point too I think is that we were
really set up to give each other structured
and significant feedback. You don’t exactly see
it in the comments, but it was meaningful. Again, I don’t
think we could have gotten there had we not had six
weeks of material leading up to that. And again, that included some
of the very necessary exercises that were going on in
the discussion forums. So how did I do on
my challenge, rather? I did it. I did it, and I was
really enthused by it. Again, I didn’t know that I
was going to do it anyway. I didn’t find out until I
was in the middle of it, or starting it rather
that I was going to be required to do this
posting and that sort of stuff. But it was very
invigorating to actually be able to meet that challenge,
very invigorating for me to interact with my
peers and share ideas, and very invigorating
to then in turn meet them in person in school
now and get to kind of share with them and laugh
about some of the things that we put up
there, and oh my god. I can’t believe–
what did I say? That was actually
really smart of me. So just to wrap up and before
I hand it over to Daniel, I just want to say that some
of the kind of ideas that have been surfacing
this morning, things around familiarity,
I’m hearing words like team culture, incentives. In my case, I think
those incentives were they’re required posts. In order to participate,
you have a certain grade and we have to make sure
that you did these things, and sort of networks. And I think that those
are really important words as we think about peer learning,
both as our own learning, but also as we are
teachers to our peers. Thank you. Thanks, Mary. [APPLAUSE] A couple thoughts strike me and
then we’ll pass it on to Dan. But again, I’m hearing about the
in-person component as well as an online component,
but my mind is sort of tipping toward
it’s a porousness. There isn’t like online
versus in person, but it’s just a porousness. Also, thank you
for sharing how you self identify as an introvert. I wouldn’t have
thought that that would be a factor in
an online experience. I would have expected that
an online experience might have been a safe space
for an introvert. So thank you for sharing
that facet of it. It’s going to give me food
for thought for later. And with that, I’m going to pass
it on to Dan to talk about DCE. So good morning. And first, I’d like to thank
Kristin and the organizers for putting together
this conference as well as this session. And second, I want to
say I’m not as much of an introvert as Mary. I have five followers
on Twitter– [LAUGHTER] –and a few postings and tweets. So I’m fortunate to have one
of my jobs in a setting that encourages and supports the
innovative use of technology and enhancing teaching. And that’s the Extension School. I teach science courses
at the Extension School, endocrine physiology and
evolutionary reproductive biology. I also teach medical
students and residents at the Tufts Medical
School system. Now, at the Extension
School, one of the chief aims is to nurture the
peer to peer learning among a group of students with
differing in base of knowledge and differing in
comfort with engaging. In addition, we work with a
slightly different setting that we’ve talked about so far. It’s a hybrid setting. Now, how many of you are
familiar with the HELIX classrooms? Great. OK. And so this is also one of
the audience response systems that we still use, in
addition to Poll Everywhere. So the HELIX classroom has
three sets of students. The one is the in class students
like you are here right now, the second is the live students
joining us through Zoom like you saw that huge
panel of students, and the third is the
on demand students who watch the videos
of the lectures. So the idea is to have them
all involved and all involved in peer learning. So a sense of a
community and comfort as you’ve both mentioned is
really important in this. And we start off with
that with two things. One is we assign pre-work
for every lecture to help them get to a
level of playing ground when we start into the lecture. And that can be reading or 15
to 20 minute video or both. The other thing is
that they all create profiles, like you
mentioned, so that they get to know each other. And the profiles have to have
the picture going with it. The first year I
did, it has maybe 10% of the students in the profile. Next year I decide one point
to their final grade, 100% did. And you don’t have to
have a lot of incentive, but incentive works. So after the pre-work
I also ask a couple of questions at the
beginning of the class through Poll Everywhere
just to make sure we’re all on the same page. Now, the class
consists of segments of didactic presentation
material that average maybe 15 to 20 minutes,
sometimes less, rarely more. And these are interspersed
with breakout sessions where students break
down into small groups to discuss various
questions or scenarios. The breakout sessions
last four to six minutes. And we actually have four to
six of these breakout sessions for every two hour class. So there’s a lot of them. And the timing of
these is intended to optimize the attention
span and concentration throughout those two hours. Each breakout session is
followed by a reporting session where spokespersons for
each of the small groups provide a piece to answer the
question or a scenario that was put forward. And I cold call on them
to be spokespersons. And so every student has ample
opportunity in the course to be a spokesperson. Now, I do give them
the opportunity to have a lifeline so
they can assign somebody else as the spokesperson. And I find that’s fairly
frequent in the first week or two, but by the
middle of the term nobody is using the lifeline anymore. OK. So at the same time, we
have our demand students who are watching the video,
and I ask the demand students to pause the video
while they work through the questions
or the scenario, and they have their answers
before they restart the video and watch the reporting session. I should also mention that in
class, the breakdown groups– and this is actually an
idea Kristin gave us. –the breakout
groups are combined with the Zoom
Online live students with the in class students. And I’ll show you
an example of that. So I’m going to then– we have our my
first slide up here. So this is an example of a
typical over at the Extension School HELIX classroom. And you can see this
is a view from where I am with a podium there looking
out over the students way back to that screen in the back,
and that screen in the back is going to be where our
students show up that are joining us through Zoom. And they are visual and we
can talk to them like you mentioned before with Zoom. So I can have conversations
back and forth with them. And you will see also on the
podium there is another screen. Their faces will
be there too where I can read their names so I
can call them then by name. Just above that screen,
you will notice a camera, and that camera is
what brings the Zoom students into the classroom. And this is the
view that they have, looking down up to
the front of the room across their fellow
students who are in class. And you can see on the
left is the screen that is PowerPoint presentation is. On the right is the
screen where another screen where the Zoom
students’ faces are and talk back and forth
with the students in class. And then in the
middle is a Blackboard where we work through
the reporting session. And my experience is
the same as yours, is that I was surprised that
when they do these peer groups and do the reporting
session, they basically always get all the right
answers of the questions. But the other thing is they
come up with right answers that I didn’t even think about. So I’m learning during this
period of pure learning too. So it’s been really nice. Now, this is my Thursday
night class getting ready just before a lecture, and you can
see on that screen in the back the Zoom student’s faces
are beginning to appear and that’s going to be
filled up with Zoom students. And this is one of
our breakout sessions and you can see the faces of
the Zoom students are gone. They have descended into
their breakout groups. And so they’re online in
these breakout groups. And you’ll also notice
just below that screen there is a student looking
at his computer that’s isolated from the other
students with headphones on. He’s joined into the
Zoom breakout sessions with the online
students so that we’re mixing the online students
with the live students in class in each one of these
breakout sessions. And then the last thing
we’re going to do this term to enlarge this is– and all these things
we do are enabled by the Extension School. And we’re going to
add on iPads on stands that will be on the table around
the in class of breakout group so each of the Zoom students
will have their own iPad, be able to join in with
the group in class. So they further integrate this. So these breakout sessions are
progressive through the term. We talk about comfort. When I start off the
first breakout session, it was really easy to make
sure everybody is comfortable getting their feet wet. So for the example
of a first question will be, name a
hormone, or a name a gland that makes
a hormone, or name a tissue that makes a hormone,
and I would get almost 100% involvement on that. Then as the term progresses,
these sessions, these scenarios where you make them
more complex layering new information upon previously
presented information. So for example, in
my endocrine class, something that I might ask
midway through the term is, imagine yourself
running the Boston half marathon or the 10K, and it’s
a hot day and you’re sweating. Think of all the physiologic
responses in your body that are enabling
you to run that. And then when you get
to the drink table and you’re offered water or
electrolyte solution, think of how your choice
is going to change your physiologic responses,
which is way harder than what’s a hormone? OK? And my very last class ends with
a one hour breakout reporting session that presents
two complex scenarios that the students
will have to draw on information that
has been presented them throughout the term. So an example of that
in the endocrine class is first scenario is, imagine
that despite the advice of your parents, you’ve
decided to go to Pamplona for the running of the bulls. And when those
bulls get released and you start running, what are
all your physiologic responses that allow you to
survive and they all think of them pretty quickly. Then we have the
reporting session where we work out through the
board, the complex reactions and the interactions
for all these responses that they’ve learned
through the term. Then I proceed to the
second example, which is even a little
bit more complex, it’d be typically
something like imagine you have a critically ill
patient in the critical care unit, now bringing them
into my world a little bit, and that patient has a severe
infection, low blood pressure, and is not doing well. Imagine all the
additional responses, besides running at Pamplona,
that that patient is having physiologically to survive. And then we fill up the rest of
the board with those responses. So the breakout
sessions not only are really a core
part of this class for teaching, but for the
peer to peer community and teaching each other. But they provide
limited opportunity for the on demand
students and involving the on demand students in
this peer to peer learning and community of
the class I think is one of the most
difficult challenges. So that’s one of
the areas that we’re working with the Extension
School on developing. They’ve provided us with four
tools so far that we’re using. First is online small
groups that we’ll have a project twice per term. So the project is to
generate a detailed report on a specific subject. And there’ll be four
to five students in each of these groups. They’ll be mixed between
on demand, Zoom students, and in class students. So they’ll be mixed together
and working on this. Each student will be
assigned an area of expertise that they’re supposed to
learn with this topic. They will share that expertise
with the other members of the group, and then
they’ll synthesize these different areas into
a concluding statement. So it took us a
while to figure out how to do this for people
thinking it’s fair, but we’ve gotten good
feedback on this. The second is Yellowdig. Again, fortunately I don’t
have to go in Yellowdig. We basically use it the
same way that Mary does. But that’s an integral
part of our course, and we assign substantial
number of points to this so that we get
robust involvement. But even if you only
assign one point, Yellowdig has really
enthusiastic involvement from the students where they
can post ideas, questions and articles, and
then they respond to other people’s questions and
ideas by expanding it on them or answering them. And we virtually
never enter into this to answer the questions. It’s all done by the students. Dan, do you mind if
I jump in for one second with a reflection? I have my eye on the
clock too, so some sort of during my moderator
thing for a second. But I did hear in
what you’re talking about both instructional
moves as well as technology to facilitate this
peer to peer learning. And I just want to
comment for people that haven’t taught
before, it’s hard enough to teach to physically
present students while you’re preparing a lesson
and getting your comments out and allowing the students
to learn with you, in front of you, et cetera, and then
in the simultaneous delivery that DCE is known
for that brings it up a level between the
physically present students and the live online students. And I just was so
struck by the fact that you also are
talking to the people who are watching their recorded
versions of this simultaneously delivered course to work as an
instructional move, pause this, do the work, and then come back. I just think that
that’s fantastic because it’s hard
enough to teach and then it’s hard enough to
teach in simultaneous delivery. But to also honor
the people that are watching the recorded
versions, that’s just really a dance that is not a
simple thing to master. And then the second thing
that I heard you say was about the variety
of technologies that you use both to facilitate
the teaching and learning, but to facilitate the
engagement across the peers. I’m glad that you are
taking me up on this idea to mix up the physically present
and virtual learners in groups. And thanks to Zoom
for facilitating that, but also with Yellowdig. You as well, Mary, mentioned
a technology that helps. And I feel like a
lot in these peer learning and online
environments it’s a combination of the pedagogy,
but also the technology. And obviously, Bharat,
you mentioned that a lot, and HBS Online technologies
really invite that. In the interest of
time, if you don’t mind, we could come back
at the end, but I want to give Teddy a moment– I actually was down
to five seconds. Five seconds? All right. So the last two things are
that we use an online thing for the on demand students
where they can comment, record comments and questions
as they watch the video, and then we incorporate those
into the following lecture or review session. And that was the last
thing I was going to say. Oh. Thank you. I’m sorry that I
cut you off then. I was just worried about
the clock looming in my eye. But thank you very much. And Teddy, I’m going to pass
on the clicker to you, sir. Thank you. Is this till 11:30? What’s that? We’re till 11:30? OK. Good morning. Morning. I had to do it because
everyone else did. My name is Teddy Svoronos. I’m representing here the
Public Leadership Credential at the Kennedy School
just to give you a sense of context and scale. Our current run of
three of the six courses has about 450 learners
across the three of them. So about 150 of them each. And so we think of it as
sort of this middle space between a residential
course and a– and that’s a thing that
I’m going to kind of return to a couple of times. I want to take my first
a very few minutes to acknowledge all the
people that are involved in creating and running PLC. So first, certainly our
illustrious leader, Kristin, and Dan, who is not
here at the moment. I’m one of three faculty
leads across the three domains of the Public
Leadership Credential. And then we have the
course design team, the learning designers
who were involved in creating this thing. So a thing that’s
going to come up a lot is that pure learning is
at the absolute center of this credential and
how we decide to plan it. So the people
designing these courses had a huge role
on that, and then also the team that’s actually
involved in delivering it. And speaking of the folks that
were involved in delivering it, our small and
growing group of TAs who have helped across
these courses who actually I think a lot of them are here. So people, anyone
on the screen right now, can you stand
up for a second? Thank you very much. So first of all,
thank you very much. Second of all, know
who they are so you can ask them
questions afterwards because they know a lot
more about this than I do. There are two reasons
for me to put this up. One is that I’m extremely
grateful for all the work these folks have done. Two is that a thing that
has become clear to me in administering this credential
is that peer learning is work. It’s a lot of work. It takes effort. It takes time. We were tempted at
trying to think, whoa, we can scale this
kind of arbitrarily. The marginal cost some
additional learners zero. I’ve come to believe,
at least in the way that we have administered
this, that it really does take people
and time and energy to make peer learning
work in a good way. It’s not really
a substitute for, we don’t have the
resources to do this. So let’s just have people
learn from each other. To that point, the reason for
us incorporating pure learning so deeply into PLC is twofold. One, as evidenced by the
existence of this conference, there’s lots of
evidence to suggest that peer learning
and peer instruction can be incredibly helpful
for learning outcomes. That I think sort of
goes without saying. But the second is that
when we were trying to decide what we wanted
our offering to be, we thought about this notion
of signature pedagogy, which is to say, what are things
that people who graduate from this program need to
be able to do in their work, in their lives? What do we want a
graduate of this to do? But what became clear to
us is that all those things involve peer learning
and peer instruction. So being able to
learn from cases get deeply into an example
and sort of elucidate all the different nuances of it. Talking to one another and
collaborating on projects is going to happen constantly
to these folks as policymakers and decision making. Excuse me. And then finally being in
the role of a decision maker made clear to us that we
need to incorporate things like simulations into the group
into the credential in terms of trying to be in the
role of the decision maker, walking through
different choices and how they affect
your outcomes. That was also a
really big part of it. So those three notions,
learning from your peers, learning from case studies,
and learning from simulations all necessitated a kind of
peer learning that we thought was really important. So to that end, we
decided that we’re going to kind of keep this
central to the program. So the way that it
works is that we have these five week long courses. Within those courses, you are in
a group of about five learners, and that’s your group
for all five weeks. You have the same
group, the same people, you collaborate on the
same projects over time. The reason for that
was that we were trying to think of trying
to be able to create the kind of collaboration
and knowledge that we were aiming
for to begin with. So Bharat’s point
earlier that you can’t learn from one another
until you know each other. This is a way to get people to
really truly know each other. So the way that PLC
works is in addition to quizzes and individual
assignments and things like that, every week has
a group assignment where you collaborate on a project. You work on a Google Doc
or something like that. You meet on Zoom and you submit
a group assignment every week. So that involves getting really
deeply into the kinds of case studies we were talking about,
talking about the simulations and reflecting, and
in my case, learning about how to use data and make
data usable as a policymaker. So this is just an example
of one of the assignments that we have that I think sort
of takes into account a bunch of different parts of it. This is sort of a
Google doc template. It’s a case study,
so it involves one of our very many
faculty guests speakers who we have in our courses,
one of whom is David Eaves. They’ve been a really
huge part of it. You’re supposed to
name all the people that you worked with
and sort of indicate who was actually involved
in the sort of decision making process. And I’ll get back
to that in a second. So the tough part about this
is that we’re asking groups to self negotiate. So we help them sort of find
each other in terms of timing and location, stuff like that. At that point, they’re sort
of left to self manage. So built into these
assignments are these moments of metacognition. So in the first assignment,
they discuss and establish group norms. They talk about what they think
their group needs to look like and where it needs to do
as part of the assignment. A couple of weeks
later they reflect on how it’s going,
what should change, what should sort of be
happening with the group work. And at the end, we ask
them to identify what we call exceptional contributors. So not just who
was really great? Who was really
good in your group? Who was an exceptional
contributor to your group? So that has a small effect
sort of on your performance in the course and
things like that, but it’s also a
chance to acknowledge that different
people in the group have different abilities to
contribute, have different time commitments to contribute,
and acknowledging the sort of hard
work of the people that had a bigger
sort of role in that. Part of that involves
a lot of TA work. So we have TAs that
are grading each of these group assignments. Very few of them are just
multiple choice questions. Those are quizzes. The group assignments
involve writing. So TAs are providing
feedback to groups. TAs are grading
groups and TAs are kind of also playing
an informal role in helping to deal
with group dynamics, because, as you can imagine,
working with people is hard. Working with people is messy. Maybe people aren’t
listening to you or you don’t like someone
or something like that, and that kind of persists
across the course. So helping to manage that
and helping to kind of deal with that is a big
part of the course. I will say that in our
experience, at least after talking with Neil, that
hasn’t come up nearly as much because we were kind of
worried that it would at first. Groups are sort of able to
self manage and create dynamics wherein they’re able to
collaborate together, but that ends up being kind of
a big part of the role of a TA. So two of the three domains
are about leadership and policy design. So that involves a lot of
simulations and case studies and things like that. Even in my Evidence
and Design course, we have a final test at
the end of the course, and that takes
place in two stages. They do it individually,
then the group assignment is that they do it
again as a group and submit their
answers as a group, and then we can sort of compare. We can essentially see did
they do better as a group than they did individually? And we actually
construct a metric for how well they collaborated. So part of incorporating
evidence and decisions into the course involves
creating metrics for collaboration and having
people reflect on how well they were able to collaborate. Again, I’m running
really short on time, so I want to say a
couple of best practices that we’ve come
across for people who are thinking
of doing something so much what we’ve
been describing. As I said, the role of the
TA is for informal support is a really huge part. We found that
designating head TA– so somebody who sort of helps
coordinate and calibrate across TAs can be
really helpful. Making clear to TAs sort
of what is to be expected and what isn’t in these
sorts of contexts. So learners are coming to
them with lots and lots of questions, both in terms
of the content of the course and also advice on managing
group dynamics and things like that. TAs have had to learn
to kind of adapt to that, which can be quite
different from what they do in a residential course. The last thing I’ll say
because I was talking very fast is that we’ve been soliciting
feedback from learners to sort of see what
they liked and didn’t like about the course. And a recurrent thing,
there’s one learner in particular who
we said, what was the high point of the course? And they said the group
was by far the best part. I have continued
collaborating with them after the course ended. It’s been wonderful to get
to know these people from all around the world with lots
of different experiences and things like that. And then we said, what was
low point of the course? And they said definitely
the group part. [LAUGHTER] The group part was
really difficult. There were times which
we really disagreed and we had to argue and kind
of get into the thick of that. That to us is also
an end in itself. We’re both trying to
teach them the content and teaching them the
ability to make decisions and to work in this group
collaborative environment. So that’s all I’ll say so we
have some time for questions. But very happy to talk
about more of this after, and certainly all
the people that have had a huge role in this. Thank you. Thanks, Teddy. What strikes me
when you just said that the best and the worst,
the tragedy and how meaningful the group work was. So last fall we piloted
the first round of courses and we had this great idea
to have a super learner in each of the courses that
were Kennedy School affiliates. And not only did they
participate in the courses, but they gave us feedback
on being a learner and what the experience was. And I recently ran into one
of those super learners. And so we’re on
the third running of these first set of courses. She’s still in touch
with her group. So that’s a relationship
that’s persevering past the end of that
pilot course that happened almost a year ago. So that’s interesting
that the best are also the hardest in terms of
the people element of it. So I would like to
thank the panel. We have a parlor game moment. We only have a minute more. I was trying to honor the
concept of pure learning by giving an opportunity for
you to turn to a neighbor and talk and digest
what you heard, but in the interest of time,
I will just put these up on the screen for
you to think about because you can find
each other later, or you can engage with
each other online. Shoot an email, have
an in-person meet up. Think about and process
what you heard today.

Danny Hutson

Leave a Reply

Your email address will not be published. Required fields are marked *