Tuesday, April 8, 2014

More Beefs

image

A week or so ago Barry Joseph - a close and valuable colleague and contributor the Open Badges community - posted  “My Beef with Badges”, where he calls for a healthy dose of skepticism and honesty about our successes and our failures with open badges. I do not disagree with Barry’s main point: the goals with the badging work are lofty and tough, and we won’t see any significant impact or change if we aren’t watching closely, sharing (all) findings, recalibrating or evolving as we go. But there are specific points that Barry makes that I, errr, have beefs with, or that I feel deserve more context and discussion.

The problem that concerns me the most is the lack of a broad ecosystem for badges. I want to tell youth in our programs their badges will have value outside our museum, and many even need to hear that as a condition for participation. But without such an ecosystem in place, I’d be lying. 

Again, I do not disagree with the general sentiment: the badging ecosystem is still young and while there is a lot of adoption and interest, there is still much more growth necessary to recognize the potential. It’s true that the ‘issuing’ side of badges has received and continues to receive the most attention from the community. Why? Because without valuable badges out there to earn, the conversation about systemic change stops pretty quickly. That said, in the last year, there has been significantly more interest and work on the ‘consumption’ side of badges - employers using badges in the hiring process, universities using badges for admission, etc. -  and we’ll see even more of that this year, as it’s a top priority of the Badge Alliance. But there are a few things I’ll say:

1) I think we’re selling ourselves and our learners short if we ONLY link ‘value’ with our own top-down predefined measures (i.e. got me a job). There is a lot of value that can come out of learning in and of itself, community participation, as well as reputation and identity building. Before the open badges work, we weren’t doing a very good job recognizing any of that stuff. Now we’re starting to change that, and there’s some value in simply calling it out to youth (or learners of any age for that matter). Recognizing learning can help them know what they know, learn how to learn, discover themselves. And unlike anything else they may have experienced so far in their education-related trials and tribulations, the badges they earn are theirs. They own the data about their learning. They can decide what they value, what is reflective of who they are or want to be. With that as a new starting point, they can begin to build a personalized, customized story in a way that’s valuable to them. So, if we as badge system builders get stuck in a cycle of trying to determine what’s going to be valuable for learners upfront, we’ll find ourselves reinventing the same system we’re dealing with now. Not saying that we shouldn’t be considering how to build badges that are valued and used by employers or admissions folks, but we can’t limit ourselves - or our learners - to that alone.

And its not just about personal gratification or meta-cognition. Look at sites like StackOverflow, who have badges for what seem like completely context-specific behaviors. “Your answer got voted up 100 times!”, “You were the first to answer a question 5 times!”, “You edited someone else’s answer!” In isolation or with our prescriptivist glasses on, some would say that those badges are insignificant and meaningless outside of that context. But turns out those badges, in combination with other badges like “knows Javascript”, start to paint a pretty solid picture of what kind of worker and colleague this person will be. Ultimately those badges could get that person a job, so who are we to condemn them based on our own perceptions or preconceived notions. Let a thousand flowers bloom, something something. Or, as my colleague Carla so eloquently put it, let’s not get stuck behind “The Myth of the Lightweight Badge”. 

2) Don’t wait for the ecosystem, build some of those connections yourself. I’ve endured a lot of finger pointing and curved-eyebrow questioning over the last few years. Which employers are accepting badges? Which badges are being accepted for credit? By whom? Where can I use them? What’s the currency? These are all extremely important questions and as I mentioned before, a top priority of the Badge Alliance. But why wait for it at an ecosystem level? Build in some of the currency directly. Reach out to local businesses, forge that relationship with an institution. You know your learners better than anyone else, so figure out what they want with them and start to layer that into your badge system design thinking. That only makes your badges, your entire offering and the ecosystem more valuable. Win-win-win.

I mean I love them for what I’ve seen them actually achieve: new literacies amongst youth to describe their learning within a Brooklyn  after-school program; new motivation within an Atlanta private school;  pride in portfolios within a Bronx library; a new understanding of how  to use learning technology in a New Orleans day school; the emergence of  formative assessment within a New York museum. I am informed by the  theoretical but guided by practice, by what I have seen with my own eyes  over the past five years…

…But I preferred to focus on that achievement rather than the majority of youth who displayed little interest in badges as their design offered   scant value beyond an additional form of grading. 

I have to say I’m pretty sad if we can’t celebrate the individual learner anymore. Sounds like there were some pretty positive things that came out of the experience for some youth. Let’s not discount that. Indeed, let’s celebrate that! If we are going to hold ourselves to solutions that work for everyone out of the box, we’re on a slippery slope towards standardized testing. 

But OF COURSE the badges didn’t ‘work’ for all youth (although we really need to define what ‘work’ means). Badges are not a silver bullet. They are not a magical solution you can overlay and expect them to enlighten every type of learner out of the box. Does anything work that way? Badges are a tool for recognizing more and connecting more learning than we were able to do before. We still need to approach badging by being thoughtful about how we’re developing them, using them, and consuming them, all the while paying close attention to our learners and their needs, etc. Barry is totally right that we need to be honest about what worked (and who it worked for), and what didn’t work (and who it didn’t work for), so that we can build better systems that have different badges or options for different learners. But we’ve still got to do the work.

But, I do harbor concerns. Not concerns about extrinsic versus intrinsic motivation, or whether badges are the right focus for advancing  alternative assessment. Those don’t concern me. 

Interestingly, the things that Barry is not concerned about are the exact elements that we don’t have enough information about. Those are the things that we need the honest feedback and findings about. They are exactly the elements that play into how we design learning experiences and align badge systems that cater to each of our learners.

image

We all have a beef, or severals beefs with badges. I would be worried if we didn’t, because that would mean that we weren’t taking this seriously enough; that we didn’t think it had enough potential to warrant the tough questions. “Hopeful skepticism" is a common thing I hear, and even feel myself at times. We don’t have all of the answers figured out, but we agree that there are some problems that need solving and there is definitely some promise, some potential resident in the idea of badges that’s worth exploring.

My ‘yes, and’ to Barry’s general call for sharing and honesty about the failures, would be for us to be open and persistent about our beefs. And equally open and persistent about addressing and solving those beefs. Not to just state them or poke holes - that part is easy - but to commit to doing the hard work of finding answers, finding solutions, and suggesting alternative approaches.

I’m also pretty hopeful that the recent announcement of the Badge Alliance @SRL14 will help in this direction (Marc Lesser, from comments on Barry’s post)

Marc Lesser, of MOUSE and Open Badges community fame is spot on. I too am hopeful that the Badge Alliance will be able to move us towards progress, honesty, and impact. Simply creating the Badge Alliance (with close to 300 organizations already signed up as members), is a statement that we’re committing to collaborating and zeroing in on these issues. And now its my team’s (and ultimately the wider network’s) job everyday to ensure we are not only just talking about our beefs, but actually addressing them.

IN FACT, the Open Badges community call TOMORROW is dedicated to talking through some lessons learned so far. Join us.

So thanks, Barry, for your important and timely post. Looking forward to digging in together.

image

-E

(image attribution: www.buedelmeatup.com, www.insidearm.com, www.brandsoftheworld.com)

Tuesday, March 25, 2014

Truth in Tagline

When we decided to soft launch the Badge Alliance at the Summit to Reconnect Learning, my Communications Director had about 5 days notice to pull everything together. She did, of course, pull everything together as she always does, but there was definitely some throwing ‘good enough for now’ words up on a site at the last minute. As we are starting to more formally kick off some branding work, we went back to the tagline we used. While it might not be where we end up, there was and is a lot of truth in those words, and there are layers that really get to the heart of what we’re trying to do (and I suppose, what we’re now on the hook for doing!)

The Badge Alliance is “a network of organizations working together to build and advance an open badging ecosystem”

Let’s break that down a bit…

"The Badge Alliance is a network of organizations…"

The Badge Alliance is made up of organizations that want to work on these issues together, want badges to succeed, believe in a similar vision. These organizations (and in some cases, individuals) are volunteering to contribute to working groups, to roll up their sleeves and do the work necessary to move the badging work forward. They are the lifeblood of the Alliance, and of the ecosystem we are building.

After a relatively low profile soft launch, we already have close to 300 organizations that have not only expressed interest in the Badge Alliance, but have signed up to participate in at least one working group. In many cases, they have signed up for several. The initial response has blown me away and I am more convinced than ever that the Alliance is so important and timely.

"…working together…"

One of the reasons we have created the Badge Alliance is that this work is so much bigger than any individual organization. It’s going to take a village. It’s going to take an ecosystem. That means non-profits, tech providers, agencies, institutions, schools, corporations, foundations have a role to play. It’s only through connection and collaboration across these organizations and sectors, that we will make significant progress for learners and workers across their lifetimes.

As I mentioned before, we’re pretty serious about the collaboration part. So much so that the Badge Alliance is completely built around working groups. All of the work is/will be done through working groups. All of the key issues are tackled through working groups. Most of our job will be to facilitate, recruit for and shepherd working groups. Working groups, working groups, working groups! We’re all probably going to start getting so sick of hearing those words, we’ll have to make up some others. Constellations! Action teams! Etc. But for now, working groups.

It’s a super exciting approach with lots of potential. We’ve already got a really healthy mix of organizations that seem energized and ready to dig in. But its also a little scary. I’ve always been of the ‘if it needs doing, I’ll just do it" mentality, but now our role will to wrangle, recruit and ultimately rely on lots of different players to move the ball forward. Again, together is the only way we succeed. We’ve been spending the last couple of weeks trying to create some guidelines and process to create a layer of accountability and confidence in working group outcomes, which we’ll share in the next week or so, but really these are working theories. This is going to be a constant work in progress, with payoffs so much bigger than any of us could accomplish on our own.

"…to build and advance…" (Or "…to build and grow…")

This is a pretty heavy piece. Building and advancing. What does that mean? Where do we start? What does success or advancement or sufficient growth look like? One deliverable that we are on the hook for, with our Steering Committee, is an initial definition of umbrella goals, strategy and metrics for the badge ecosystem, that we will then vet with the broader network. This will help to provide a larger context for all of your work, while also connecting work across the ecosystem, highlighting gaps or new opportunities to put some attention into and giving us all way to determine if we are winning.

But we don’t need that, and frankly, can’t wait for that to keep the momentum going. Each of you can probably list a few things that you think are needed to advance the broader badge work, or maybe even your own badge systems. Issues that need tackling, hurdles that are in the way, use cases you need to see, questions you need answered…I guarantee if you all did write them down, there would be a lot of overlap. And I can also guarantee that we’ve probably been talking about many of them since very early on. So let’s stop talking and dig in. That’s the purpose of a working group. Let’s pick one of these key issues and work together on it. Let’s set concrete goals and divide and conquer.

With our soft launch of the Badge Alliance, we tried to capture what to us felt like some of the most critical issues/topics through the initial working groups:

  • Open Badges Standard - shaping the evolution of the open standard for badges
  • Endorsement - how to build functionality and practice around third party endorsement of badges
  • Cities & Network-wide Badge Systems - how to support network level badging systems
  • Badges Messaging - how we talk consistently and effectively about badges to different audiences
  • Globalization, Localization & Badge the World - how to encourage and support badging in other countries and cultures
  • Web Literacy & Digital Literacy Badges - a shared badge system(s) for promoting and recognizing important digital skills
  • Badges for Admission to Higher Education - how to get badges into the admissions evaluation process
  • Recruiting Next Generation Workforce & Acceptance by Employers - linking badges to jobs, internships, career advancement and other opportunities
  • Badges for Educators & Professional Development - granularrecognition for teachers and educators

This initial set of working groups isn’t comprehensive, of course, but reflects where we and the broader community sense some the biggest urgency or heat. It’s a healthy mix of driving adoption on the issuing side, while also really starting to dig in on the ‘consumption’ or currency side of the ecosystem as well. We already have begun to identify a fast follow set of working groups that will most likely include things like research, validation (although this one is too big for one working group), K12/schools (also too big), pathways and privacy/data.

(you can still participate in these working groups - visit http://badgealliance.org to sign up. You can also suggest additional topics/working groups you think should be represented)

"…an open badging ecosystem…"

Oh you thought the last part was heavy? Wait for this one! :)

An open badging ecosystem. I could write several blog posts on the meaning/importance of that phrase, probably at least one post on each one of those words (the one on “an” would be a page turner ;)).

But while we *could* (and I am sure at some point *will*) get existential and philosophical, this doesn’t have to be that complex. If we agree that ultimately this is about recognizing and connecting learning of many (and of more) kinds across contexts and across lifetimes, and leveraging that recognition to better connect people to jobs, additional learning, personal growth, advancement, social connections and more, then there are some pretty obvious and important lines in the sand, but only a few. Two actually.

1) Badges must be interoperable. In this case, that means badges must align with the open standard, which is the ‘information model’ for badges.

2) Earners must own their badges and have control over where they are stored, how they are shared, etc.

That’s it really. If we all agree on those two points, then we have the makings for a healthy ecosystem. If badges are interoperable, then they are stackable, we can ensure we have enough information for making sense of them and we can always build tools and processes on top of them to better issue, manage, understand, etc. And if earners are in control, then badges cannot get ‘stuck’ in a silo and we can continually build in more connections and opportunities for that badge earner. The work doesn’t stop with these two ‘principles’, but these are the minimum required to ensure that the ecosystem can grow up around the badges in a way that places the learner at the center.

We haven’t gotten so far as to finalize a manifesto or formalize any requirements for membership, but I can’t imagine these two NOT being in there in some shape or form. We’ll be co-creating these types of things with our Steering Committee and Alliance members, so obviously much more to come here.

So…we didn’t do so bad with our first tagline. There’s a lot in here, maybe some of it is controversial. Certainly some of it I intend to dive into much more deeply in subsequent blog posts. But don’t wait for that - I’m definitely interested in getting feedback and hearing more from you on these somewhat rambling thoughts. Where do you agree? Where do you disagree? What are other working group priorities? What does open ecosystem mean to you? What other names for working groups should we use? :) This is a pretty critical juncture in the Badge Alliance and the overall work so now’s the time to weigh in!

And don’t forget, another way to get involved is to participate in one or more of the working groups. You can sign up on the site http://badgealliance.org

Much more soon.

-E

Friday, February 14, 2014

Badge Alliance

Finally enroute back east after a quick and power-packed trip to San Fran for an awesome Summit to Reconnect Learning

At the Summit, we announced a new venture to build a Badge Alliance. The Alliance is a network of organizations and individuals working together to build and support an open badging ecosystem. Members of the Badge Alliance will collaboratively tackle important issues, questions and opportunities to continue to push the work forward.

We also announced that I will be transitioning out of Mozilla to build and run the Badge Alliance. Over the the course of the year, I’ll move to the Alliance full time to drive this work.

I am incredibly excited about this opportunity - it really feels like the natural and necessary evolution of the badging work and of my role in it.

We really need this. The badges work relies on a robust ecosystem and while there are many many folks playing in the space, we aren’t leveraging each other enough. We aren’t sharing enough. We aren’t building a knowledge base. We aren’t moving the ball forward as quickly as we could. And that’s not because we don’t want to. It’s not your job. You each need the space to do your work, your slice, with your agenda, timeline and perspectives. Someone else needs to help connect the dots, set overarching goals, drive the conversations that need to happen. Well, now that’s my job. :)

So I am just thrilled. This is going to be fun!

As this has been unfolding, I keep getting pulled nostalgically back to a dusty corner in a Barcelona museum…we were at the Mozilla Festival in 2010 in “the Badge Lab” and posed a set of questions: What if we reimagined credentials to support a broader definition of learning? What if we thought about digital badges as a way to capture more about what we know and an do? Folks then wrote down the questions, issues and ideas that came to mind. 

Now that was a relatively small group of people - maybe 15. But let me tell you, the amount of post its produced was astounding. We’d both hit a nerve but also hit on something worth talking about. And talk we did, we spent the next 3 days digging in.

This story is relevant to me right now for a couple of reasons:

First of all, many of the folks in those conversations then are still in conversations today. In fact, many were at the Summit with me. That day, we started a community, we started the ecosystem that is vital to the success of this work. That has grown significantly over the last 3 years. There are so many of you contributing to major ways to the work and that’s absolutely critical. And there is so much potential when we start to leverage each other. I feel so motivated and inspired by this opportunity to get up everyday thinking about how to support you, connect the work and move us all forward.

Second, many of those questions that were written on those post it notes are still open questions today. In fact, some are topics we spent the first day of the Summit talking about. We’ve been thinking about this stuff for a long time and we are at a moment where we can make decisions and really accelerate by more formally working together.

The Badge Alliance will operate through working groups that are facilitated by the Alliance team. This isn’t really new - as evidenced by the 300 people who showed up to the Summit to work through key issues - this is how we’ve been working already. The Badge Alliance will just layer in more intentionality, accountability and support.

We are going to hit the ground running. Today is the soft launch, with a formal launch planned for Q2, but in the meantime, we are going to kick off the initial working groups. These working groups will help us continue the conversations and work needed to move the ball forward on important issues. You can visit http://badgealliance.org to view the initial working groups, sign up to participate and suggest additional ones.

I may be taking a leadership role in the Alliance, but this is way bigger than me. Just as the Alliance is a network of organizations building the ecosystem, it needs those organizations to help shape it. It’s incredibly exciting to me that a number of organizations have already stepped up as founding members:

Huge hugs and high fives to these folks for their initial interest and support. And it’s not too late to contribute as a founding member. If you are interested visit http://badgealliance.org or email me at erin@badgealliance.org.

Also, definitely sign up and secure your spot on the initial working groups that are important to you. You can do so at http://badgealliance.org.

Many many more posts to come from me with more information as this unfolds.

-E

Monday, September 30, 2013

CSOL - How Did We Do?

The mornings are crisp and the leaves are turning rich colors here (you should come!)…it’s officially Fall. We’ve wrapped up the Chicago Summer of Learning and have had some time to reflect on how it went. 

It was helpful to start the debriefing process with a look back at what our original goals were back in what feels like years ago, but was only February of this year. Looking at the numbers through that lens has helped us start to thoroughly evaluate this summer. We, of course, want to supplement the raw numbers with more real feedback from youth and organizations and research, all of which is forthcoming.

THE GOALS

  • Challenge all Chicago youth to learn over the summer
  • Call on all youth serving organizations to collaborate and support learning initiatives over the summer
  • Highlight opportunities in the city related to STEAM

Internally, we also aspired to:

  • Reach as many youth as possible, especially low-income or minority kids, get them to earn at least one badge (goal: 100,000)
  • Provide pathways for youth to encourage additional learning and ‘leveling up’ (goal: 1000 kids leveling up)
  • Get as many participating orgs as possible (goal: 100)
  • Make the badges ‘worth something’ (goal: avoid all ‘participation’ badges, engage CPS and local institutions to recognize badges)

THE NUMBERS

The CSOL numbers:

  • Total orgs issuing badges: ~125
  • Total of badges available for earning: >1000
  • % badge types: 3% participation, 92% skill, 5% achievement
  • Total badges issued: ~150,000 (note: number includes some badges that are still in the process of being issued)
  • Total badges claimed to date: ~50,000
  • Total badges wish listed: >5000
  • Gender breakdown: of reported 53% Female, 47% Male
  • Race breakdown: of reported, 56% african american, 7% white, 6% asian, 29% latino/hispanic, 2% other
  • Age breakdown: of reported 26% under 13, 60% 13-17, 9% 18-24, 5% over 24
  • Average time on site: ~6 minutes
  • Total youth who earned city-level badges: ~600
  • Total youth who completed challenges, or successfully leveled up: ~300

THOUGHTS ON THE NUMBERS

What Went Well

When I look through these numbers, the initial reaction is wow. We can go down the goals and check them off across the board. Over 100 orgs and 1000+ badging opportunities represents not only a lot of coordination and collaboration that I am still shocked that we all could pull off in just a few months, but also an incredible coverage across the city and a heck of a lot of learning opportunities. The badges themselves were really great - with only a few orgs issuing badges simply for participation*, but most badges focused on skill development with robust assessment and criteria. The Chicago organizations really stepped up and came into this summer and this new concept of badging with open minds and a dedication that was astounding. Huge kudos to all of them - they are the ones with the content, experiences and relationships with youth that matter. The badges are just the recognizer on top of that learning. That said, the badges were very strong and DePaul University has already stepped up to say that they will accept some of the badges for credit in their institution, and CPS is working to define rewards and advancement that they can offer for the badges.

(*Note: this is not to say that participation badges are *bad*. There are many reasons why rewarding someone with an initial participation badge could be valuable, including motivating additional work, recognizing time commitment, etc. But we wanted to hold ourselves to a goal of having the majority of the badges tied to a deeper assessment, which they were)

Orgs issued over 100,000 badges this summer - that represents a lot of learning and a lot of youth. And there is diversity among those youth. Time on the site shows that youth (and parents) were spending time searching for things to learn or digging into the self-paced challenges that were available for youth on the site, and the wishlisted badges show that some youth were setting goals and into learning more.

Even though we had set a goal around 1000 youth leveling up, I was pretty worried about this one given the fact that many programs started later in the summer and often required multiple weeks of participation and learning before a badge was issued. In order to level up, kids had to not only earn a badge, but earn several within the same category (Science, Technology, Engineering, Arts or Math). This most likely involved doing a combination of local learning programs and online self-paced challenges. So I was pleasantly surprised to see that almost 600 youth got to the city level badges and had access to the challenges, of which almost half of those completed challenges, which were not easy and required commitment and hard work. 

What Could Have Gone Better:

The part I am less wowed by is the disconnect between the badges issued and the amount that have been claimed. This number is less stark as it looks but only about 50% of badges issued were claimed and added to Backpacks. To explain this a bit, badges were issued one of two ways: 1) through an issuing tool built by Mozilla called OpenBadger that had an admin interface for org representatives where they could issue badges to youth emails, the youth or their parent had to click through the link in the email to accept the badge and have it show up in the youth’s backpack.  2) Organizations with offline learning experiences could also hand out paper badges with unique claim codes printed on each. The earning youth then needed to log on to the website and enter the claim code to see it in the Backpack. 

There is still some research to conduct, including some surveys and focus groups that folks are conducting to learn about what worked and didn’t work for kids this summer, but its likely that the lower number of badge acceptance most reflects breakdowns in the overall user experience. Many youth haven’t checked their emails, or didn’t click through links of the original, text-only emails. Some might not have had access to a computer at all to claim or accept the badges. Others may have lost the paper badges - its likely there are lots stuffed down in corners of physical backpacks - before they could claim them. Another option is that kids just didn’t care about the badges, or didn’t understand the value, so that’s something we’re exploring more. 

While, as I mentioned, I was pleasantly surprised by the number of kids leveling up, we did not hit that initial goal of 10% of kids. Some of this can also be attributed to user experience design, some of this was the relatively short amount of time to learn and earn and level up. I also think there is an opportunity to design and promote challenges in a way that even more connects directly to youth interests and real opportunities. I’d love to see more challenges next year that show youth completing real projects for their neighborhoods or the city of Chicago. Or developing skills that are directly tied to career pathways, jobs and internships.

In general, the numbers were again, impressive, but I think we can do more. Through a combination of improved tools, better user experience design, more and better training and more marketing and communication, I think we can bring more organizations on board, reach more youth and see more accepted badges.

WHATS NEXT

There is a big assumption in that last statement: that we are doing another CSOL in 2014. I think there was a resounding consensus in a recent meeting in Chicago that the results of this year’s summer were compelling enough to demonstrate the value and promise of this work. So there will be a CSOL 2014, and in fact, maybe something more on the lines of CYOL (Chicago Year of Learning or something like that). Lots of details to work out on the goals, roles and specifics for that work, but given that we are building from a foundation and set of findings, as well as starting those conversations in September instead of a few months before launch, to me, means the possibilities are enormous and exciting.

We’ve learned so much this summer about what it takes to build a badge system at this level, how we can better the tools, who to invite to the table from the get-go, and more. Lots of work to do to get to a 2.0, but there are a lot of pretty clear marching orders. Look from more blog posts from me and the team on these lessons learned and thoughts for moving forward on our pieces.

Also, turns out that Chicago is and will continue to be a hub for innovation and advancement with badging, but another exciting development is that more cities are interested in building something similar to CSOL in their neighborhoods and I think we’re going to see not only more geographical reach, but also a significant amount of more learning and youth impacted in less than a year.   

Thanks,

Erin

Wednesday, August 14, 2013
As I’ve written often in the past, I believe the automation of recruiting, job seeking and hiring has exacerbated America’s employment crisis. Online forms and tools like the “apply with LinkedIn button” make it too easy for the wrong applicants to apply for jobs, and harder for employers to find the right ones. But when a job applicant’s position on the stack of resumes can be bought, the search for the best-qualified candidates is even further compromised, and so is our economy.

Source: http://www.pbs.org/newshour/businessdesk/2013/08/ask-the-headhunter-is-linkedin.html

Sounds like we need better, more verified tools for understanding skills and matching people to the right jobs. If only there was something….

(More) Reflections on the Badge Summit

My last post on the Badge Summit was more of a ‘here’s what happened’, but wanted to follow that up with some of my own reflections and inspirations. 

Distributed leadership: We talk a lot transparency and community-driven development at Mozilla, and certainly within Open Badges. Our aspirations are so high, that it would be impossible to begin to even start to meet them without a movement, without markets, without more and more people not just involved, but driving things forward. We want to build a new system for credentialing and accreditation, we want to redefine learning - there’s no way one, or even three, organizations could do that. We need the full ecosystem fully realized and fully empowered. But one thing that became even more clear at the summit was that we are well on our way. I was awed by the leadership and work that folks in the room represented. There is so much that other people are already doing and contributing, we just have to make sure we are coming together from time to time to swap stories and celebrate the success. 

Exemplars: The word of the day(s) at the event was “exemplar”. There were a few that even threatened to start a drinking game. But whether it was in the conversations about marketing and communications, or in a group focused specifically on higher education or workforce, it’s clear that the most critical way forward is to have working proofs of concept for each of the stories that we need to tell or use cases that we want to show. Exemplars allow us to scope and focus in on building something specific quickly, that then can be a model that other similar groups or organizations can follow. They also can also function as a laboratory where were are advancing the tools and experiences, while also learning a significant amount about what works and what doesn’t. CSOL is a great example of an exemplar because it includes a bunch of ‘new’ stuff that before were often only ideas or theories: network-level badging, earning badges across many informal learning experiences, leveling kids up through pathways and translating learning back into schools and local businesses. It’s a mini version of the ecosystem that we can see results of in a 3 month time period instead of 3 years, or whatever turns out to be. We need to do more of these. (And I don’t mean just ‘we’ as in Mozilla, but ‘we’ as the broader badging community. Mozilla will most likely do some but there are many that need to be defined, recruited for, designed, tested, communicated, etc.)

Local: Building off of the last one, there was a lot of interest in ‘cities’ at the summit - thinking about badges in a local context as a key set of exemplars. To me this is really exciting for a number of reasons, including the same thing I said before which is that we are building mini ecosystems, with all of the players - issuers, learners, validators, employers, schools, etc. - in the mix and at the table from the get-go. This is super powerful because it moves us away from some of the philosophical rabbit holes that we often get stuck in - like “How are badges validated? Do they have meaning?” - and make those questions concrete and addressable. City-level anything, let alone badge systems, are also really hard to do and there is tremendous learning involved. If we can make it work at the city level, then we’re in a good place for addressing needs at each org-level and even at the ecosystem-level in many ways. And finally, badges in the local community / city context have real meaning for those earners in their lives right away. So regardless of what happens in the broader ecosystem, those badges meant and will mean something to those learners. Let’s do more cities.

Small data: Another recurring theme, which I actually first heard at the DML conference, but then came up again at the summit was “small data”. The idea of personalized data tracking or even local community-level data, that can give you real-time feedback on who you are, what you can do, what you are doing, what role you play, etc., and help you choose your pathway or alter your behavior in the process. It’s the quantified self idea, but for learning. I’m not sure exactly what it means yet but I love the idea of learners owning their data and that being a powerful mechanism for driving choices, opportunity, evidence, etc. 

Learning GPS: As we sketched out our own perspectives on the current state of the ecosystem a new role appeared several times across the sticky notes. The idea of a person or an organization that helps learners choose pathways. We dubbed it “learning GPS”. This exists right now in coaches or career advisors, but badges actually makes this an easier and more powerful role because they can surface the universe or ‘map’ of learning opportunities, all backed by information and data to show the value and endorsements of each step along the way. Exciting possibilities!

These were just a few of the themes and reflections that I left with. I’m sure there are more and will try to add to this as I continue to process all of the goodness that came out of the two days in Toronto. 

-E

Friday, August 9, 2013

Badge Vision Summit

This week kicked off with a meeting of badge minds in Toronto. Around 30 of us packed into the lovely Centre for Social Innovation to discuss the emerging badge ecosystem and how to drive the next era of the badge work together. We were, you know, socially innovating. :)

A common theme was that the badges work is bigger than one organization, certainly bigger than Mozilla. There is SO much to do, and there are many roles that need filling, and more everyday as the ecosystem and market(s) grow. We’re already seeing leaders emerging beyond the initial Mozilla, MacArthur, HASTAC work and this meeting was about celebrating and honoring that, as well as encouraging more ownership to tackle key pieces that the ecosystem needs. 

Day 1 was about visioning (what’s our shared vision of success) and level setting (what’s out there already). Day 2 was digging into some of the key questions, opportunities and challenges, with the focus on building working groups around them and getting actionable plans in place.

Core topics included:

  • Ecosystem: We talk about this all the time. But what does the ecoystem actually include/look like at this point? What are the roles to be played? What are the pieces that still need special attention? More to come - including a fancy visual - on this piece.
  • Sector specific needs: What are the specific needs and value propositions for badges from each sector? What’s the right messaging for each? What features, answers or examples do they need in order to move forward?
  • CGI: How can we get to 2M better futures in two years?
  • Exemplars: What stories do we need to tell and what are the partnerships, strategies and badges systems that we can help build to tell those stories. 
  • Cities: how can we capture all of our learning from CSOL in a way that makes this easy for other cities to do this ‘right’?
  • Federation / Toolstacks / Localization: How can we design the infrastructure, tools and services to support the most scale, adoption and contribution? Another way to say this, is how do we design and build for a healthy and thriving ecosystem?
  • Endorsement: This one is huge and means many different things to many different people. Let’s put some definition around it and figure out the short term plans for demonstrating the idea (hint: exemplars!)

We also have plans for a shared user research framework and outreach plan, building the overall foundation of research and findings to back the work, creating a shared resource repository, encouraging innovative assessments within the badge ecosystem, and more. It was a busy two days… 

There is a core documentation team that will be generating a report to share all of the conversations, plans, etc. Look for that in the next couple of months. 

All-in-all, I was excited and inspired by the two days and really looking forward to keeping the momentum going. Many thanks to the participants who traveled from far and wide, including across oceans, as well as our fantastic facilitator, Gunner, who kept the motley crew on task.

Additionally, its worth noting that this event was by no means the comprehensive set of leaders in the badge work. We wanted to ensure that key perspectives and specialties were represented, but above and beyond that, last minute plans, summer schedules, budget, etc. also influenced the attendee list. We hope and plan to pull many more into the conversations and work as we go (if there are any up there that you know you want to be part of, let us know!). That said, don’t wait for an invitation - keep doing what you are doing, share your work, jump into conversations, etc. The only way we win is together. 

More to come - thanks!

-E

Monday, July 29, 2013

Badge Camp

My two weeks of vacation conveniently ended with a fun, relaxing week with my team in a remote oceanside town in Northport, Maine. Not sure there’s a better way to get reintegrated after vacation.  

The retreat, affectionately called Badge Camp, was focused on reflecting on the Chicago work, as well as digging into the core features and directions ‘next’. We all stayed in an old inn where turns out we had limited internet, no cell connectivity, one semi-working refrigerator and a barn with a DJ machine - all of which were unanticipated, and only one of which was a happy surprise (hint: karaoke!) Sounds like the makings of an incredibly UNproductive week, right? Not with this crew. We capitalized on the opportunity for uninterrupted facetime to take planning walks, hold paper prototype demo sessions on the front porch, make new prototypes with felt and glue guns, huddle over meals with project teams and host mini focus groups over the camp fire. 

image

image

image

image

Before Badge Camp, we had broken people into core project teams and had them brainstorm for 3 weeks, and everyone arrived with solid thinking and prototypes. In addition to feedback and (of course) a much deeper dive on the overall ideas and details than anticipated (but most welcome), we ended with roadmaps from now through MozFest for each project (some artistically written with glitter glue and construction paper (soon to be transcribed into a digital format). Unsurprisingly, the team was eager, serious and excited about the next phases for us - we’ve all been anxious to get back to the core product and features for the broader badge ecosystem. All-in-all, we made some significant progress in our plans and thinking on key projects and features, most of which will be coming through blog posts from the team, but I’ll summarize and highlight the core areas here:

P(riority)1s:

Production Backpack: cleaning up code, fixing bugs, finalizing 1.0 features that need more love, knocking off the low hanging fruit want-to-have features and ux improvements. Also includes building some acceptance tests, which will significantly improve our ability to monitor and test our stuff, but also hopefully give the community more insight into what features we have with each release and help them help us understand what’s not working the way its supposed to be, etc. 

Prototype Backpack, or ‘Backpack 3.0’: the bigger vision for the Backpack including more tools and features for the earners. More to come on this as the thinking develops. 

Federation: Allowing many Backpacks to exist across the ecosystem and still work together at the ecosystem level. This will require a standard for a Backpack and APIs for ‘registering’ Backpacks, all as seamless as possible to the end user. 

Open Badges Badges: Wait, we’re the Open Badges team and we aren’t issuing badges?! Blasphemy! Don’t worry, Open Badges badges are on their way and they will not disappoint. 

CSOL: Continuing to support and push CSOL through the end of the summer. 

P2s:

Endorsement: Endorsement is allowing third parties to sign badge classes to formally endorse the curriculum and assessment that they represent. This is in the P2s not because it isn’t as important, but it relies on some pieces from the projects above, namely Federation and Prototype Backpack that this extends from. We’ll be kicking off some user testing and research on this in the meantime. 

Productizing CSOL: We now have a pretty solid ‘tool stack’ for issuing, including the Badge Studio, OpenBadger and Aestimia (peer assessment). We also have some really cool new Backpack features like recommendations and COPPA-compliance. But these were all really built and customized specifically for CSOL. So we want to go back in and make all of this stuff accessible, useful and available to the wider community and ecosystem. This is also very important but will most likely be something that gets more dedicated time through additional use cases and projects that we take on. 

Community Site 1.0: Last May we launched a mini community site, meant to be a placeholder for a bigger, more community driven repository for examples, resources and celebrations. The latter clearly needs to be designed with and through the community so we’re looking forward to kicking that off soon. 

Ongoing:

User Research: We are kicking off a new approach to our work, that includes more user research from the beginning! How revolutionary of us! :) We’ve always tried to loop the community in from early on and get feedback along the way, and that works great for the issuing side of things, and we will continue to do that, probably even more targeted. But we also need to talk to other audiences that we don’t usually reach on an everyday basis like earners and employers, so Emily on our team, is stepping up to map out our research process and bake it into all of the projects outlined above.

Partnerships: This could also be called adoption, but its working with issuers to build and issue awesome badge systems, and with employers and institutions to use badges on the other end. We’ve been doing this since day 1 and will continue to make this an ongoing priority. We’ve now got a powerhouse team to help us drive this side of things, including a set of key emerging community leaders who are owning these conversations within particular sectors or locations. +1 to this.

Marketing + Documentation: We’ve got a firehose of inbound demand, with almost no marketing efforts. I can’t imagine that will change significantly given that we are grant-based, but there are definitely things that we know we can do to make things more accessible or approachable, and Meg has joined the team to whip us into shape here. 

What’s Next / How to Get Involved:

  • Let us know if you have feedback on this list, including what’s missing. Best place for this is our mailing list.
  • Watch our blogs for more info on each project. (btw: Planet Open Badges is coming back soon, which will be a one stop shop for all of our individual blogs, in the meantime, we repost most at the Open Badges blog)
  • Come to the community call this Wednesday at noon ET to hear more about the outputs of Badge Camp and weigh in through conference call and etherpad magic

Thanks,

E

Tuesday, July 9, 2013

Badges Have Gone Global

If you’ve ever been to one of the many educational conferences in the US, you will hear a lot about North American-centric issues and proposed solutions. While making change in our country/continent is important, we often fall into the trap of only building solutions for our particular context and miss bigger opportunities for global collaboration and scale. This is why I’ve been happily surprised at the upswing of interest and adoption of badges outside of the US. Badges are flexible enough to be used in many different ways, to solve many different problems, and we are seeing that in action as they are being adapted and implemented across the UK, Italy, Australia and more.

I’ve just returned from a quick, but power-packed trip to Milan, where we met with our innovative colleagues from the Hypermedia Open Center (HOC) Lab at Politecnico di Milano. They invited us to Milan to talk about the badge work in North America and kick off a national conversation about badging in Italy. I was impressed at the reception and interest from the many Italians in the room - not only to explore badges for their own schools and workforce, but also across countries in the EU. There were handfuls of initial pilots that attendees signed up to lead on the spot. There is real opportunity and momentum here. 

image

(that’s me on the right with the translation headphones :))

At the same time, just a few countries and only one hour time difference away, badges are booming in the UK. Together with, Doug Belshaw, our resident UK spreader-of-the-badge-word, Tim Riches and the folks at DigitalMe have built a significant community there already exploring badges. They are doing really interesting things with badges because they are more tapped into the schools and really pushing buy in from the top. They recently launched “Badge the UK" which is a campaign to get organizations and institutions to issue and accept badges across all of the UK. It’s incredibly exciting and we are learning a lot from one another. 

We’re also seeing interest emerge from Australia, China, Estonia and more.

We are excited about the globalization of badges and one thing that is a high priority on our end is a solution for localizing badges. Good news is that all of our tools are open source, so countries or organizations can set up their own, localized Backpacks relatively easily (although we still have a bit more work to do to make them easily localizable and federated). But there will be, and already are, badges that are meant to be used across languages or learners that span several localities and languages, and so thinking through how the badge and the metadata become localizable is important and top of mind. 

Would love to hear other ideas on how to best localize the badging experience, as well as places you’d love to see badges expanded to. 

-E

Friday, July 5, 2013

Why the Chicago Badges Work Matters

We recently formally launched the Chicago Summer of Learning badge work. This has been a pretty big undertaking for us, to say the least, we started early conversations back in January and for the last 3 months, my team has been almost solely focused on it. The all-hands-on-deck was because the scope of work was so big (and ever-growing), but also because of the potential weight and impact of the work. I’ll detail each below to help explain why CSOL matters for us, for Chicago and for the world. 

What We Built: 

The Chicago badge work started as a ‘simple’ statement of work - overlay badges on top of all of the amazing programs and learning content that local organizations and cultural institutions already provide across the summer. The goals were:

  1. Help every kid in Chicago, or even visiting Chicago, learn something and have evidence of that learning (i.e. get a badge)
  2. Provide pathways and encourage kids to learn more (i.e. discover and motivate more learning through the badges)
  3. Communicate the learning back to schools and local businesses in the fall (i.e. show the badges are worth something)

The Mozilla contribution to the project was mainly two-fold. While we played a role in project management, we mainly focused our attention on 1) designing the overall badge system, including all of the organization-level badges and 2) building all of the technology components. 

1. Chicago Badge System:

The Chicago badge system design was no small task. We had over 100 organizations all teaching different things in different ways across various age groups, some online and most face-to-face with limited connectivity. We needed flexibility in the badges to ensure that they reflected each unique organization and their offerings, and yet were still seamlessly connected for learners. We needed to make sure the badges were worth something and met some level of standards. And oh yeah, we need to have all of this defined and designed in 3 months time. No problem. 

Solution:

  • All badges mapped against the core theme of CSOL which was STEAM (STEM + Arts)
  • City-level set of badges that were standardized and controlled. These were the City of Chicago Scientist, Technologist, Engineer, Artist and Mathematician badges. 
  • Organization’s had flexibility and freedom to design badges that reflected their programs and learning. We did the work to help them ensure the badges were robust, and then mapped them against the STEAM categories. 
  • We reviewed all of the badges and came up with a taxonomy of levels - participation (not assessed, earned through attendance), skill (assessed badges, aligned with a particular skill or competency) or achievement (bigger assessed badges, combination of several skills or accomplishments, typically take longer to earn)
  • We then developed the algorithm to unlock the city level badges based on the available badges. It ended up being a fairly simple algorithm - 3 ‘points’ for the city level badge, participation were worth 0, skill 1 and achievement 2. 
  • Once youth get the city-level badges, they unlock access to city-level challenges where they can use their new skills/role(s) to complete projects and learn/earn more. 
  • The summer ends with a celebratory Summer Faire, where youth are displaying the work they achieved over the summer. 
  • All of the badge information is then connected back to schools in the fall so that teachers can have a better understanding of the work over the summer and in some cases, award credit or other advancement. 

The (outdated) internal diagram for how this works:

image

The prettier, external napkin sketch:

image

(it’s worth noting that Carla Casilli built this thing from the ground up, including working through each of the 1000+ badges. She’s awesome. We also had tremendous input and support from Nichole Pinkard, Caitlin Martin and the rest of the DYN crew. Well done, team!)

2. Technology

The CSOL work had some complicated technical requirements, including servicing badge definition and issuing from representatives from more than 100 organizations; supporting kids 13 and over, as well as under 13 in collecting badges across experiences, with required parental controls where necessary; suggesting pathways and more learning opportunities; sharing badges with schools and social networks; and providing the end user ‘site’ to find learning, badges, track progress, etc. It was really multiple systems that we needed to build in parallel, and yet again, still needed a seamless experience for all of the participants over the summer. Oh yeah, and 3 months, something something. 

Solution:

  • Advanced OpenBadger tool to support badge definition, design, awarding and posting to the Backpacks
  • Aestimia, mentor assessment tool that allows for badge pledging, assessment and awarding
  • New and improved Backpack with recommendation system
  • Custom authentication
  • COPPA compliant Backpack for kids under 13
  • Organization / program / badge listing, search, filtering
  • Playlist functionality allowing kids to build a set of projects that they wanted to complete or badges they want to earn over the summer - their learning playlist for the summer
  • CSOL site: chicagosummeroflearning.org as the front face of all of this 

image

image

Why It Matters:

  • This is the first badge system at this kind of network level. Badges that connect across organizations, across the city. Badges that define pathways and push youth into deeper learning. Badges that truly connect learning of all kind. We learned a lot about how hard this is, but also landed on a model that we feel can be replicated fairly easily in other communities or networks, so that’s really exciting.
  • These badges mean something right away. We’ve been working at the ecosystem level for a long time and despite lots of progress, there is still a long way to go before there are enough badges to truly capture your skills and learning. But with something like Chicago, that is local and self-contained and comprehensive, in just 2 or 3 months, we’ll see the full value proposition of badges in play. Youth will earn badges for all of their learning and that will lead to advancements and opportunities in school and with local businesses right away.
  • All of the pieces we’ve built are open and replicable anywhere. We’re set up to roll this out in many cities, networks and communities. Our tools and documentation are open, we’re working on publishing toolkits and experiences to support other groups pursuing things at this level, and our services as now veteran advisors/consultants on these types of projects are available. We could see a huge set of real badges effecting real lives in a very short amount of time.
  • Chicago is innovating and leading the way with this. Despite some of the tough times Chicago is experiencing, they are fighting back by embracing their local organizations and their youth. The Chicago Summer of Learning is a celebration of what Chicago has to offer, through its organizations, cultural institutions and people. It’s a celebration of their youth and the opportunities available to them. More than a celebration, its about starting to unlock possibilities for youth that were not available or discoverable before. 

Many thanks to my team and the extended CSOL team, including the City, DYN, MacArthur, Hive Chicago and more.

And thanks to our Open Badges community for your patience and support as we’ve focused in on this important endeavor. We are excited to get back to the core infrastructure in a big way. But I hope you’ll join me in congratulating the team, and congratulating Chicago for building something special. 

-E