World of E's

OBI Public Beta

We are announcing today that we launched the Public Beta of the Open Badge Infrastructure. Huge milestone and huge kudos to the team for making it happen. 

What’s the OBI?

The OBI is the ‘plumbing’ of the badge ecosystem. It is a specification for badges, set of repositories (“Backpacks”) for storing badges and APIs for pushing badges in and pulling badges out. It’s an important piece of this badge experiment because it moves us beyond more silo’d systems, allows the learner to collect badges from lots of different learning experiences and provides the structural components to enable badges to be transferred and leveraged across the ecosystem for real results like jobs or credits.

What’s Public Beta?

With this Public Beta launch, the OBI is now publicly available for use. Badges can be pushed in and pulled out and earners can store badges in the middle in their Backpacks. And more! Specifically, Public Beta includes:

• New and improved issuer API
• Backpack feature upgrades:
• Store badges
• Manage badges
• Import badges
• Group badges
• Publish groups to a unique URL and add narrations/notes around each badge to share
• New displayer API
• New documentation
• Legal docs! Privacy policy, terms of use and FAQs specifically for the Backpacks.

Wait, weren’t you already in beta?

Yes and no. We were calling it ‘beta1’ which was a made up word to mean that it was a step up from alpha but not quite all the way to beta. It was essentially the initial issuer API and Backpacks, but was available basically by invite only. We should have called it a ‘developer preview’ but hindsight, something something. This Public Beta (capital B!) is a proper Mozilla beta (security review, user data committee review, on Mozilla servers, etc.) and its publicly available! Woo!

What does it look like?

Technically like this…

But really like this…

(Sample Badge Backpack)

And this….

(Published group of badges)

How can I get involved?

• Be a badge issuer or a badge displayer
• Earn badges and push to your Backpack - in fact, you can earn your first badges at openbadges.org
• Check out the code. Fork it. Get Technical Documentation. https://github.com/mozilla/openbadges
• Join our community calls: https://openbadges.etherpad.mozilla.org/openbadges-community

What’s Next?

We are moving to a much shorter release cycle - releasing things at least every two weeks, but possibly more quickly as we go. But we are aiming to move from Beta to 1.0 by the end of the year. In addition that work, plus bug fixes along the way, we are also working on some lightweight tools that make creating and issuing badges easier, and eventually will most likely do the same for displaying badges. 

Who should we congratulate?

The team for being some of the smartest, hardest working game changers I’ve known, as well as our community who have been advising us every step of the way. Thanks to you all - congratulations!

-E

Reflections on Reflections on Badges

There have been a bunch of posts from really smart people reflecting on badges over the past month, leading up to and following the DML Competition culmination and DML Conference. There is certainly a dose of skepticism across some of the posts (like here and here), mostly coming back to the question around motivation and rewards. In fact, Mitch Resnick held a session about his motivation-related issues with badges at the DML Conference, but unfortunately the room was so small, that most of us weren’t able to squeeze in, so we formed an Occupy Badges makeshift session to talk about badges ourselves. 

After getting an update on Mitch’s session and catching up on some of the posts, the common concern is around introducing badges as extrinsic rewards into learning experiences where intrinsic motivations may be at play, and potentially disrupting a delicate balance of motivations or existing interest-driven learning. (It should be note that this is a generalization and there is more nuance to their claims - definitely worth a read).

I’ve been wanting to add some of my reflections on these reflections (get all meta) for awhile now and finally scheduled some time - a meeting for myself - to dive in so here it is: 

On intrinsic vs extrinsic motivations:

There is a classic scenario referenced a lot: kid gets good grades in school because he wants to do well and then his grandparents start giving him money for every A. When the grandparents stop paying the kid later on, the kid suddenly isn’t motivated to get good grades anymore. It’s called ‘crowding out’ - the intrinsic motivations get crowded out by the extrinsic motivators. That’s the core of the argument against badges - that badges will be yet another extrinsic motivator that will squelch any existing intrinsic motivations.

This binary view of intrinsic and extrinsic motivation is too simplistic. Dan Hickey, an assessment and motivation guru out of Indiana University, has a nice summary for those of us with less expertise on different theories of motivation and learning, and points out:

One of the things that has been overlooked in the debate is that situative theories reveal the value of rewards without resorting to simplistic behaviorist theories of reinforcing and punishing desired behaviors.

The ‘crowding out’ concern is real (and should be considered with grades as well!) but too simplistic for learning and these complex social environments. We all agree on the issues, and we run the risk of doing nothing about them if we cling to overly simplistic interpretations of theory or research studies. It’s also worth noting that badges do not have to just be a carrot, but can be built as tools for formative assessment, empowerment, roles/identities, etc. This means we need to put some thought into the badge system design, but that’s exactly what the competition and other parallel work right now is focused on. 

Don’t muck up interest-driven learning

There is another set of related concerns that go something like this: there is a lot of youth interest-driven learning already happening and its awesome because it is separate and pure and we aren’t mucking it up with adult-imposed rules or rankings, etc. Badges are just another top-down adult-driven system of rules that will just interfere with the learning. 

There are some HUGE assumptions in here. The first is that all youth have opportunities for interest-driven learning and the second is that those that do understand that this is valuable and legitimate learning. I don’t think these are true. I don’t think most kids have opportunities to explore their own interests - instead are forced down the pathways we prescribe for them in school. And if they aren’t inspired by the topics or projects at school, then they are labeled as bad students and that’s not something kids can rise above very easily, or in most cases at all. Most don’t understand that there are other avenues. For those kids that are lucky enough to have some opportunities to explore interest-based stuff, usually in afterschool programs, I doubt that many understand that this learning at all, and that its legitimate and important and could lead to a lot of opportunities for them. They aren’t as empowered by these experiences as they could and should be. These are the gaps that this badge work is looking to fill - to recognize learning and help learners use it for real results like jobs or credits, as well as to help learners find other learning opportunities.

There are some smaller assumptions like badges are only for youth, which they aren’t and that badges are only created and issued top-down and they don’t have to be. But the big assumptions are the dangerous ones.

Badges as a silver bullet:

There were some concerns around badges being positioned or thought of as THE solution. It might have seemed that way at the DML Conference because there was so much attention paid to them. But badges are not THE solution. In fact, badges themselves are not even A solution, but part of a toolkit and common approach of redefining learning to be something that occurs beyond classroom, beyond age 22, etc., recognizing and legitimizing more types of learning and helping the learner have more choice and control about pathways and interests. Badges are the representation, the gateway, the conversation starter, but its really about this new way of thinking and approaching learning that is the powerful part.

I’ve also heard things like “why are you focusing on only one approach” or “one form of assessment”. It’s worth reiterating that badge itself only represents the learning, assessment, experiences and evidence behind it. There aren’t any constraints on the learning or the assessment behind the badge - and that’s by design at this point. If you stop and look at the badge systems people are developing, you will see that there is a lot of thought going into how to utilize badges for specific learning experiences and how to be innovative about assessment, etc. Badges don’t limit this at all.

Another flavor of the silver bullet concern is that we are moving too fast and have one standard too soon. But again, the standardization is only at the level of what information is included with the badge - there are no constraints on the learning and assessment part, at least not from Mozilla or the badges themselves. If there is still concern about the standardization at the level of the badge -  I’m not sure how we would really truly give this a solid try if we weren’t working together. A bunch of siloed systems are not going to help empower the learner or help them create their own pathways. We need some way for the badges to work together - for the learner - and be tapped into a larger ecosystem of opportunity and access. That’s what the Mozilla Open Badge Infrastructure is supporting.

A few last small(ish) reflections:

Education vs Learning: I think its worth making a distinction between ‘learning’ and ‘education’. Education is a set of policies, content, structures and expectations that we define and force youth through. That sounds negative and its not meant to be, education systems are important for many reasons. But learning is so much more than that - it’s any experience where people learn something and that can happen inside a classroom but can also happen in a seemingly limitless amount of ways outside of classroom, and across lifetimes. It’s all that other learning that isn’t currently consistently recognized or valued. That’s where badges can fit in, or at least that’s the current hypothesis we are working under. That’s not to say that badges don’t or won’t have some value in formal education, but there are some bigger questions to think through there - it won’t work if we just overlay badges on the existing system or trying to force the existing system on top of badges.

Badges are not a Mozilla solution - this experiment, and its success, is not dependent solely on Mozilla. We are building the infrastructure to support the badges, but its on everyone else - the learning providers - involved. It’s on them to continue to offer awesome learning experiences, be innovative and authentic about assessment, design badges that amplify that learning and empower learners, etc. But again, if you look at the types of badge systems proposed for the competition, this is exactly what people are doing. 

More to come I’m sure.

-E

More reading:

Mimi Ito: Reflections on DML2012 and Visions of Educational Change

Alex Halavais Badges: The Skeptical Evangelist

David Theo Goldberg: Badges for Learning: Threading the Needle Between Skepticism and Evangelism 

Dan Hickey Open Badges and the Future of Assessment

Audrey Watters (who I finally met in person at DML!) Thinking (Strategically) About Badges

Cathy Davidson Can Badging Be the Zipcar of Testing and Assessment?

Philipp Schmidt Let’s Make Badges Not Stink

Badges 101 Webinar Follow-up

Last Thursday I presented on a Badges 101 webinar run by HASTAC. The goal was to give some foundational information and answer questions about badges in general. It was well attended - over 200 attended and more weighed in with questions. 

You can see the recording here: http://www.dmlcompetition.net/Blog/2011/10/badges-101-webinar-follow-up-and-recap/

We were only able to address a handful of questions during the webinar since time was running out, but we are all currently weighing in on the great questions that came in and will be continually posting our responses via the HASTAC Badges forums. There were a few that caught my attention and I wanted to answer here as well:

Isn’t grade just another form of a badge I can post A,B,C,D, F on fridge, The internet just makes fridge bigger?

I love this question. The internet as a big fridge just makes me smile. But it is also a really good question. Are badges the same as grades? Aren’t we just reinventing the same system? I think this is exactly what we should avoid. Grades are a very limited and in many cases, ineffective system. A grade is abstract and often tells very little about what was behind the grade. Even for those situations where the criteria differentiating an A from a B is clearly defined, once removed from that experience, the grade loses all of that information. Even just looking at the system - 5 grades - imposed on everything regardless of what is being taught/learned, feels irrelevant and artificial. And then there is normalizing and grade inflation… 

A driving principle behind the badge work is that we will use badges to capture a wide, granular range of learning so that the skill, competency, achievement, etc. is explicitly expressed through the badge, and of course that badge carries with it all of the information needed to understand the badge, including the criteria/assessment behind the badge and potentially even a link to the learner’s work as evidence (an optional piece of metadata). Thus badges move us away from a standardized, artificial system and start to lay the foundation for an authentic, personalized system that captures and continues to communicate the learning and skill development that occurred. 

And returning to the fridge metaphor - in my childhood, only the papers, quizzes or homework that got the A made the fridge. Regardless of the work, when my parents saw an A, they implicitly knew what that meant and knew how my work compared to others’. Badges sets us up for a different, more personalized system where learners can collect badges for a wide range of skills and achievements. Badges can represent unique and individualized pathways of learning. It becomes less about comparisons to other learners, and more about personal interests and accomplishments. It becomes less about the rating and more about the work itself. This makes some people nervous - how will we make sense of it all if we don’t have the standardization? It will be a different system, that’s for sure. But what gets me excited is that it opens up the opportunity for so much more authenticity, flexibility and recognition. A learner’s collection of badges could be on that fridge but instead of just a few papers with A’s, and it would represent a much more complete picture or narrative around that learners achievements, strengths, interest and skills. 

Seems like the success of badges at least partly depends on educating employers and the public about their value?

Yes, there will certainly be a learning curve. We are starting something new here so it can’t be expected that HR departments are going to suddenly know what to make of badges on digital resumes or applications. But that said, we have talked to a bunch of employers and hiring managers and many are open to the idea, and beyond that, almost all (if not all) recognize a need for a new system. They are looking for a way to get more contextual information on applicants, including the evasive social or softer skills that are so important and relevant to employers and success as an employee. Badges can offer a way to present more granular and comprehensive information about a person, and that information is more than just something flat listed on a resume but instead is linked to information about criteria and evidence to validate the badge. So while, yes, as with any new system, there will need to be some initial education around badges, the potential is so huge that I don’t anticipate it will take much to tip folks in favor of a the new approach. But we’ll see!

There is a follow-up Badges 101 webinar on October 17th at 2pm ET so if you missed that one, or still have more questions - check that one out. Also, for questions specific to the competition details and instructions, check out this webinar tomorrow (10/11) at 3pm ET. 

-E

Join the Open Badges Team!

The Mozilla Open Badges Team is expanding! 

As you might have guessed, there is a lot going on in the Open Badges world right now. We recently launched beta1 of the Open Badge Infrastructure (OBI) and are hard at work on beta2 and 1.0 development and preparations. We are developing documentation and materials to onramp folks as seamlessly as possible to the OBI, but also want to provide channels for direct assistance to those partners that need it - whether that be through advising on badge system components, developing interfacing widget technology to assist with connections to the OBI or custom building technological components. We are working with the MacArthur Foundation and HASTAC on their DML Competition to facilitate a set of high-quality badge systems and badges that will all be plugged into the OBI and thus, the wider badge ecosystem. We are also working on badges within various Mozilla programs, such as School of Webcraft and Hackasaurus, and are starting conversations about an even bigger Mozilla badge system that would extend across various projects and programs and create a consistent experience and pathway for Mozillians to participate and grow with us and through us. And more. As I said, TONS a’happening in Open Badges land. 

But all of this will not be possible without some really good people to help get us there. So we are currently hiring for 4 positions, with another one soon to follow. Please share these with your networks and if you fit the bill for one and are interested, please let us know! From my own experience, its a fun, chaotic, innovative and inspiring project to work on, so join us!

Open Badges Developer

• Work with the tech lead on OBI development - create and iterate on the core infrastructure for the badge ecosystem!
• http://hire.jobvite.com/j/?cj=oq5XVfw7&s=MoFoPost

Open Badges Partner Manager: Business and Design

• Support partners on badge system development and design. Help to build badges into various programs and environments. Become/be an expert in an emerging field. 
• http://hire.jobvite.com/j/?cj=ot5XVfwa&s=MoFoPost

Open Badges Partner Engineer: Tech and Support

• Support partners in connecting badges systems to the OBI or pulling badges out. Develop tools and widgets to streamline the process.
• http://hire.jobvite.com/j/?cj=ox5XVfwe&s=MoFoPost

Mozilla Badge and Assessment System Desginer/Specialist

• Design and build a badge system for Mozilla that motivates and rewards participation, provides learning pathways through various programs and experiences and fosters the next millions of Mozillians to be involved in the open Web.
• http://hire.jobvite.com/j/?cj=oz5XVfwg&s=MoFoPost

We are a distributed team - all positions are remote and flexible. All are one year positions with potential for extending based on our success together. So come on board!

-E

Badges for Lifelong Learning: Join the Conversation

As I (and numerous others) have already mentioned, last week in addition to our announcement of the Open Badge Infrastructure beta1 release, the MacArthur Foundation and HASTAC announced the 4th annual Digital Media & Learning Competition, which is focused around badges. These two efforts work together well in that we at Mozilla are building the infrastructure to support an ecosystem of badges-as-credentials for learners, and MacArthur is bootstrapping that ecosystem with high-quality, high-value, learning-focused badge systems through the competition. Together with MacArthur and HASTAC, as well as all of the people that join the competition (and conversation), I think we can take this experiment and exploration pretty far, pretty quickly, and learn a lot.

The announcement was made in DC at the Hirshhorn (which somehow was my first time there despite living in DC for 7 years - cool museum). It was a jam-packed few hours that went from the conceptual introduction of badges, through goals and aspirations for this work, to very real examples of badge systems that are already out there now. It took an initiative that was certainly publicly talked about (we are Mozilla, afterall), but within a limited range, to a much bigger stage with much wider reach. Needless to say, there has been a lot of chatter about it since then - this post from my colleague, Matt Thompson, highlights some of that. While much of the feedback was positive, there were definitely some concerns and issues raised among the voices, some of which I want to address here.

But first let me just say that for me, it was an incredibly surreal day, given that I could remember back when some of these initial conversations were happening in a not-so-well-lit corner in Barcelona at the Mozilla Drumbeat Festival. And fast forward a relatively short amount of time (and a TON of work) later and I was sitting in front of Secretary Duncan from the Department of Ed and Administrator Bolden from NASA, among others, saying some pretty inspiring and generally awesome things about these efforts. 

Now. I realize that I am coming from a different vantage point in this than most. I have been thinking about badges, talking about badges, designing badges, critiquing badge systems, building the badge infrastructure, etc. So I’m a little close to all of this. Some may argue that makes me blind to some of the concerns raised in the blogosphere and twit-o-sphere (?) after the event, but I would actually say it makes me more tuned into them. And appreciate them even more.

I still very much see this as a conversation, or as I said before, an experiment or exploration. Not so much in a starting-from-scratch or winging-it way, since again, there has been A LOT of work and thought put into this, but instead in a we-don’t-have-blinders-on, let’s-tackle-these-issues-together way. I think as many perspectives as possible are important for this conversation and I would be a little nervous if there wasn’t any push back from anyone. We don’t have everything figured out yet, and its only through this conversation and these explorations that we will make progress. (Note: I do not, however believe that not having all the answers means we shouldn’t try)

Again, most of the feedback was overwhelmingly positive and Matt captures some of that, but there were a few common themes in some of the more concerned feedback that I wanted to address here. This is a conversation after all, and so as conversations go, let’s talk it out. 

A few themes emerged in some of the feedback:

Theme #1: DANGER! This is being done by technologists, not education people

There seems to be some confusion about Mozilla’s role and the role of the technologists in all of this. Mozilla is building the technical infrastructure - the plumbing, or the part that makes sense to have techies involved in - but the key goal of that infrastructure is to support (and not constrain) innovation at the fringes. This means that the education folk have complete control over their badge systems - they can decide what the badges are, what the assessments look like, how the learning experience plays out, etc. So the educators are VERY much involved and in fact are driving the early badges, criteria behind those badges and ultimately the value of the badges.

Theme #2: This is going to be a top-down system forced on everyone that will ruin our motivations for the things we love to do just because we love to do them

Exactly the opposite, actually. This is at heart, a bottom-up, grass roots, OPEN ecosystem that is working to provide recognition where there currently isn’t any option. Issuers can issue badges for any number of skills, qualities, achievements, interests, affiliations, etc. Each issuer can decide what badges to issue and what things to recognize, just as each learner/user can decide what badges they care about. Learners/users control their collection of badges and can decide what to accept, what to reject, what to share, etc. For some, tinkering with robots may just be something they love to do in their free time (doesn’t everyone?) and that’s enough…for them, great. But for some, all of that experience and skill development could lead to job or further education opportunities and right now, there is no real way to get recognition for it. So that’s the gap badges are trying to fill. 

On the intrinsic/extrinsic motivation issue - I think this is definitely something that we want to track very closely. I agree with some of the more current thinking that this delineation is not as binary as we used to talk about it and that there are different motivations mixed into everything we do so I don’t think that the introduction of badges to interest-driven learning experiences is necessarily going to immediately bring us to the classic kid-getting-good-grades-because-they-want-to versus kid-getting-good-grades-because-they-get-paid situation. But again, this is definitely an area to research more and discuss further as we proceed. I am clearly not an expert on this and another great aspect of the competition is that is it supporting the people that are to do solid research on open questions, effectiveness, etc.

Theme #3: Oh great, more gamification, this time attacking education and learning.

Badges are linked in many people’s minds with gamification and so a natural reaction to these efforts is that this is another example of layering gaming on top of yet another space or discipline. These assumptions are not totally off-base, I think some of the concepts of game design and motivation will come into play in some badge systems, but this is definitely about more than that. Certainly, many of the badges developed through the competition will be backed by rigorous assessments and evidence. And we will see a lot of innovation that I think will move the learning/education space forward. This is not about just slapping some badges onto existing frameworks and calling it a day. This is about turning assumptions on their heads and taking a fresh, hard look at learning and how to support it. One of my Open Badges teammates, Carla Casilli, wrote a great post about this as well.

Theme #4: What happens when this gets out of hand and there are badges for everything?

This has come up since day 1 which is kind of funny given that we’ve all jumped from just talking about the potential for using badges to some future world where this has taken off so much, that there are a ton of badges in the ecosystem. I think that that future day is a real possibility, and in some ways would validate some of our thinking about the potential for badges. But while I understand the concern, I myself am not that concerned about this. One, we aren’t there yet. Two, we are building things into the Open Badge Infrastructure that can help with some of this. And three, if badges do become as big of a ‘thing’, then there will be markets and tools that emerge around them. Look at the Web in the early days - thanks to the simplicity of HTML among other things, there was an explosion of websites. It could have been complete chaos but then third party tools (and arguably quite lucrative markets) emerged to help us filter, rank and make sense of this world. I think the same thing will happen for badges. There will be tools that emerge to filter and visualize badges. Endorsements and other information built into the technical infrastructure (and the badge itself) will help people make sense of badges. And again, users will have complete control over their badge collections so can decide which badges to let in, how to value those badges, who to share them with, etc.

Anyway, I am super excited about these efforts and really looking forward to continuing the conversation. I encourage all of you to get involved in that conversation - in a constructive way of course (nothing is more frustrating than a negative (or positive!) remark without anything context or thoughts that we can all react to or build from so let’s just avoid that if possible). So join the conversation, join the competition. Explore this with us. 

-E

OBI beta1

Hey all - 

As many of you probably heard, we formally launched the Mozilla Open Badge Infrastructure (OBI) last week in conjunction with the MacArthur/HASTAC Digital Media for Learning Competition focused on badges (more on my thoughts/reflections about this in the next post).

The OBI is an open infrastructure to support an alternative credentialing (badging) ecosystem where there can be lots of different badge issuers and any given learner/user can collect badges across those issuers, pull them into one collection that they control and manage, and then share them out as needed with various audiences, websites, etc. We are building the specifications/standards and core reference repository software to sit in the middle and make this ecosystem a reality. More on the OBI here and here and here. 

As of the middle of last week, we released the beta1 version of the OBI which is very exciting progress and means that there is now a there there. 

beta1 includes:

• Reference Badge Backpack technology
  • Repository of all badges for a particular user
  • Unpacks the badges to provide user with a view into the metadata behind the badge
  • UI for management and privacy control
  • Grouping - ability to create groups of badges (and then share them out)
  • Coming soon - access to display sites that have registered with the OBI, so can share out groups of badges directly from the Backpack

• Specifications/Standards
  • Metadata spec for badges - Badges pushed into the OBI from the issuer are simply JSON blobs, in other words, simply a set of metadata that defines the badge. This means a badge at any given time, carries with it all the information needed to understand the badge. Includes badge title, badge description, issuer, issue date, criteria to earn the badge (URL), badge image (URL) and optionally unique URL back to user work or evidence.
• Communication channels/API
  • Issuer API (to push badges in)
  • Coming soon: Displayer API (to pull badges out)
• Mozilla Baking Service
  • Service to embed JSON into PNG files*
• Line-up of Initial Issuers
  • We are working with a number of initial issuers (around 10 so far), including Remix Learning, Open Michigan, P2PU and Parsons New School, to push their badges into the OBI. They will be rolling out badges systems starting next week, through the next month or so. beta1 is a ‘private’ beta meaning that we are working closely with the initial partners versus releasing it widely for public use. However, we are definitely still open to working with more beta partners so if you have a badge system and are ready to plug in, just let us know!

*More on PNG files

We have taken a somewhat unique approach to the badges themselves, based on some awesome ideas from the ever-impressive Dan Mills at Mozilla. Instead of badges existing as the raw data - JSON blobs - throughout the system, we are ‘baking’ (embedding) them into PNG files so that each badge becomes a portable, ‘tangible’ thing that can be exchanged more easily. This means that issuers could email them directly to users, and users could email/send them directly to consumers, etc. This approach gives users more control over their badges and creates a more decentralized system. Users can decide to forward badges onto the OBI, or they could store them in their own Backpack that they host or even store them locally. The one caveat is that the embedded PNG is fairly unreadable without some unpacking software - this means that the badge is a viewable/exchangeable image but the metadata behind it is fully embedded into the PNG and unreadable unless unpacked. There are several ways to access the metadata: the OBI Badge Backpack will unpack badges for users (so if they are stored there, users can see the data), some technically-able users may write their own tool or third party tools may emerge within the community. But the idea that the badge can be easily exchanged/stored and all the information needed to understand that badge lives with in means that ultimately, the PNG approach gives users and the ecosystem, more flexibility and control.

Why ‘beta1’?

What’s this ‘beta1’ thing all about, why not just call it ‘beta’? Basically because ‘beta’ has come to mean something different than originally intended. Today we see a beta stamped on just about everything, sometimes for long periods of time. It seems to have come to mean, this is almost 100% but there may be a little flakiness or we may change a few things around from time to time. Our beta is much more in line with the traditional beta which means, its the step up from alpha that is critical feature complete, but there’s still a lot of tweaking, updating and building out to be done. Hence ‘beta1’.

What’s Next?

beta2: We are working to release the beta2 version later in October that will include the displayer side API and communication channels, enhanced Backpack UI, scalability considerations, etc. More issuers and the initial displayers will start to plug in at this point. We are also building a Facebook widget as the first displayer widget example.

Public 1.0 release: We will be releasing the fully functioning OBI as GA/1.0 in January of 2012. This will include documentation and materials to support issuers, users and displayers who want to use the system. At this point, the OBI is fully public and anyone can plug in on either end.

Where Can I Learn More About the OBI/beta1:

• Website: http://openbadges.org
• Documentation: http://openbadges.org/infrastructure-tech-docs/
• Source Code: https://github.com/brianlovesdata/openbadges

Kudos:

Many thanks and praise go first and foremost to our rockin’ technical lead, Brian Brennan, who architected and built the current version in record time. Additionally, again, thanks to Dan Mills and the Mozilla Labs folks who assisted and guided the work. And finally, we are incredibly fortunate to have a totally awesome advisory group made up of folks spanning academia, development, federal agencies, industry and the open source world. They have been with us every step of the way, helping us vet ideas (and many times kill ideas), think through assumptions, etc. Thanks everyone!

-E

Badge Working Group

Last week, I trekked up to NYC for a two day meeting with the badge working group. What is the BWG you might be asking - its a pretty frickin’ cool group of folks exploring badges and innovative assessments for learning. 

This was actually the second meeting of the BWG which is funded by a MacArthur working grant. We first assembled the group from people we had met through various conferences, festivals, meetings and conversations we’d been part of for the previous year and met in Brooklyn, NY at the end of January 2011. These meetings are run “Gunner-style” (ala Allen Gunn, of Aspiration the best meeting facilitator you will ever encounter) meaning that they are unconference-y, participatory and interactive and the agendas are driven mostly by interests and issues of the participants themselves. We do a fun ‘post-it note party’ as I like to call it where all the participants write down topics, questions or issues on post it notes and we combine them all and somehow always manage to let order emerge from the chaos. The post-its are arranged into common themes and those themes then become topics of breakout sessions moving forward.

During the January meeting, these topics ranged from abstract to foundational. Questions like “what do we mean by badges”, “what might a badge system look like for my program”, “how will people distinguish badges”, etc. It was a great meeting for getting people on similar pages, airing concerns and planting the seed for potential projects.

This time around, 6 months later, we were hoping to be able to get more concrete. Our expectations were exceeded by far. We had allotted 1 hour to do short presentations about existing badge projects and ended up spending about 6 hours on them. There were not only more projects than we were aware of, but those projects were far enough along to warrant fairly detailed presentations. On top of that, the participants were so engaged in the presentations that we often had to cut off discussion to make time for others. It was incredible how far people have come in such a short time. We have gone from conceptual conjecturing to solution developing in just a few months. There are some amazing, game-changing things in the works and I am so honored to be working with these people on these important problems. 

If you are interested in joining the BWG moving forward, let me know.

-E

Oh yeah, here is the short report back I drafted for the powers that be. 

Meeting Details:

• July 18th and 19th at the Social Science Research Council in Brooklyn, NY (super kudos to NYCLN for organizing everything for us and being awesome hosts)
• Planning etherpad: http://etherpad.mozilla.com:9000/badge-working-group
• This is the 2nd meeting of the BWG, the first was in late January 2011 (initial planning etherpad: http://ierick.primarypad.com/7?)
• These meetings were funded by a MacArthur working group grant

Goals:

For this second meeting, our goals were to dive into much more concrete discussions and call-to-actions, including:

• Catch up on progress people have made since last meeting.
  • Report back from pilots that have run - SoW, Quinnipiac
  • Update on current projects in the works/currently running - Q2L, MOUSE, Global Kids, etc.
• Discussion of key research questions and plans
• Facilitate partnerships, feedback or assistance on badge pilots or ideas that are in the works or planning phases
• Update on the open badge infrastructure (OBI)
  • OBI Requirements gathering sprint

Participants:

• Participants of the BWG have been identified across the various conversations, meetings and conferences we have had with people, starting with a meeting on Open Assessment in September 2010 in Palo Alto, through the Drumbeat Festival and other meetings/conference calls that we have had in the first half of 2011.
• There were 16 participants in NYC, which worked out as about half returnees from the previous meeting and half, new faces. There were also more folks that could not make it to NYC but are participating through the etherpads and mailing list. In total, the working group now consists of approx. 30 people. 
• Participants included game designers, educators, academics, researchers, open ed folks, youth developers and programmers/web developers and spanned formal-informal, product/implementation-research, K12-adult, even open-closed, etc. This added a depth of perspectives and insights that is typically difficult to achieve and really added value to the discussions and breakout sessions.

Agenda:

• These meetings were held in the “Gunner-style” meaning that most of the agenda was determined by the attendees, based on key questions or topics that were important or relevant to them. We did a post-it note exercise early on the first day to identify these topics and plan out breakout sessions. 
• We also planned for time for people to present their projects, but actually needed much more time for this than anticipated (both b/c there were more projects than we knew about and also because people were eager to discuss each project at length).
• The agenda can be found here: http://etherpad.mozilla.com:9000/badge-working-group

Accomplishments/Evaluation:

• The goal of having this meeting be more concrete and focused on actual implementation specifics was met and exceeded by far. It was incredible to see the progress people have made, both in their own thinking and understanding of the badges work, but also in their own implementations and planning. As previously mentioned, we initially slotted an hour for mini-presentations of projects, but ended up spending about 5-6 hours total on this. This was because more people had things to present than anticipated, but also because the group was very interested in discussing each at length, which was also exciting. There were many points of collaboration that came out of these presentations as well. 
• We had fewer breakout sessions than originally anticipated, mostly because of the interest in exploring the specific examples and collaborations. Those breakout sessions we did have were incredibly robust - extending well past the allotted hour with deep dive, energized discussions from all participants. Participants are still adding notes, but many can be found here: http://etherpad.mozilla.com:9000/badge-working-group-notes
• Many of the topics and questions came back to the specifics of the open badge infrastructure since that is the core underlying technology to support everyone’s efforts. We were able get some very very helpful requirements gathering done, which also included defining what the infrastructure should not do (i.e. push the innovation out to the edges to the issuers/displayers), which was also very important to work through

Next Steps:

• There was a great deal of interest in continuing this working group in any way we can. Our grant is up at the end of August so we are exploring ways to leverage social media and other channels to continue discussions, share resources, etc. If we do find additional funding, we would hold another meeting in early 2012 to review all the badge systems that have most likely existed (and have been plugged into the OBI) for a few months at that point and build research agendas for moving forward.

New Badges and Assessments (help needed)

We are currently in the midst of planning for the second phase of the School of Webcraft assessment and badge pilot and one of the key elements of this phase is the addition of more skill badges (and associated assessments). 

As I have previously detailed, last February we launched a pilot with 14 assessments and badges including skill badges (Javascript Basic/Expert, PHP Basic/Expert, Open Source Contributor), value badges (Accessibility Foundations/Expert), peer/community badges (Team Player, Peer Mentor, Good Communicator, Community Builder, etc) and some P2PU-specific badges (P2PU Veteran, Course Organizer). 

In this round, we are building out the skill badges significantly, adding at least 12 new skill badges to the mix:

1. HTML Basic
2. HTML Expert
3. CSS Basic
4. CSS Expert
5. Python Basic
6. Python Expert
7. JQuery Basic
8. JQuery Expert
9. HTML5
10. CSS3
11. Popcorn.js Demo
12. Popcorn.js Plug-In

The important part is to make sure that the assessments behind the badges are appropriate and effective at demonstrating the right skill. We don’t want something to easy or too hard, but that’s tough to tell since we aren’t experts in most (or all of this). So we need help. 

Here is an outline of the current thinking on the assessments. Please give us your feedback. 

3 key areas to focus your attention and feedback:

1. Filling in the blanks
2. Reviewing existing assessments
3. Reviewing rubrics 

1) FILLING IN THE BLANKS

You will see that there are two that are still pretty blank:

• Python Basic 
• JQuery Basic

What are some challenges, exercises or projects that demonstrate basic understanding of python or jquery? All ideas and resources are welcome and appreciated. 

2) REVIEWING EXISTING ASSESSMENTS

For the other badges, do these challenges/exercises make sense? Are they the right level? Do they sufficiently demonstrate the skill?

3) REVIEWING RUBRICS

For the rubrics, are these the right things to be looking for? What else should be in there to ensure quality work and skill? I am SURE that we have missed things since again, we are not experts in these technologies/approaches. 

Please have a look and give us your feedback either as a comment here or within the etherpad. As with everything we do, we are moving at warp speed but we want to make sure that we get this right so thanks in advance for all of your help! We are happy to give you acknowledgement for your contributions on the assessments and of course, there will be a badge for those who suggest ideas/feedback that gets incorporated. :)

-E

Badge Pilot - Phase 1 - Evaluation

I have several posts that I have been meaning to do over the last few weeks but there has been so much going on that I have been remiss. So expect a flurry of posts (or a few at least) from me in the next few days.

But to kick things off - we have completed the first phase of the P2PU and Mozilla School of Webcraft Assessment and Badge Pilot. It’s a mouthful and rightly so, since it was full of a lot of very cool stuff. These previous posts here and here give some background on the pilot but to quickly summarize, the pilot consisted of new assessments and badges for skills, values, community interaction and participation in the School of Webcraft. These badges are meant to be an alternative pathway to accreditation and credentialing that SoW community members can earn to demonstrate skills and then share with stakeholders like peers, formal institutions or potential employers to network, progress careers and/or find jobs. 

This initial phase of the pilot included 14 pilot badges (ones designed by us and aligned with specific skills, values and community behaviors relevant to web development) and a bunch of participation badges that came with the core system we were using for the dedicated badge environment, OSQA. The latter were meant to encourage and guide participation in the site as a question and answer forum. Since we were not using it as a true Q&A system, but instead simply leveraging the functionality to support the assessments and badge issuing, many of the OSQA badges were not relevant or achievable by users but some were, such as First Responder, Popular Answer, Editor, etc.

The full evaluation report is available here, but for those that don’t want to read a (titallating) 17 page report, here are some highlights below:

Goals of the Pilot:

• Build proof of concept for a badge system for web development training 
• Create and roll out initial taxonomy for types of badges 
• Develop and roll out assessments that fit the peer and interest-driven learning environment
• Get initial feedback and reactions from the community
• Learn as much as possible that can be applied to later versions of the pilot or integrated solution
• Prototype and pilot the open badge infrastructure

Key Findings

Participation 

• Overall: Participation was lower than expected, with only 52 registered users (in the dedicated badge environment) and of those, 21 active users (earned a badge, assessed work, etc.) We feel there are a couple reasons for this low participation: 1) communication and 2) lack of integration. 
  • Communication: From a communication perspective, this pilot was intentionally tightly controlled, mostly because we wanted to make sure that course organizers were prepared and that we had assessments closely aligned with relevant courses to encourage more active participation and assessing. But this meant we only touched a small portion of the wider Webcraft audience and did so through course organizers who rightly passed the message along (if at all) on their own schedule, so traffic and attention was intermittent at best. We intend to communicate to participants more directly moving forward so that we can ensure that they are fully aware and have all of the information (including why these badges are worth their time). 
  • Integration: On the integration side, as mentioned before, we used an OSQA system that is separate from the P2PU platform and thus required learners to log into a separate site (we built it so that they could use their P2PU account to reduce this issue but it was still a separate action they had to actively take). We plan to integrate the assessments more directly into the learning environment and experience moving forward to make it more seamless.

Assessments

• Overall: Feedback on the assessments was very positive and it seems like we are on the right track with authentic, relevant challenge-based assessments.
• Types: Of the different types of assessments, we really only saw examples of peer assessment, which again were encouraging, with examples of constructive feedback and reworking of submitted work, as well as learners discussing how much they learned from the process of assessing peer work, but there was some struggle with ensuring that there were peers to assess submitted work. That incentive structure is still a gray area for us - we need to figure out how to attract quality people with the right skills to assess submitted work across the system. We will be exploring this more moving forward. We did not have any submissions for expert-level badges (see below) so we did not see any guru assessment, but hope to in subsequent rounds. There was some stealth assessment in the OSQA participation badges, but none of these were directly tied into the learning. 

Badges

• Overall: The main feedback was that people wanted more badges to cover more skills which we totally expected and plan to build out further as we move along. 
• Types: There was a good overall response to the types of badges we had and people felt it was important to have a mix of hard skills and soft skill badges, which we also know are important to badge consumers like potential employers, so we will continue down this path. 
• Levels: There were no submissions for the expert badges which makes some sense given that all of the courses were entry level with some pushing into intermediate for some skills. We do feel the expert level badges are important to have as a goal or benchmark for people to work towards, but we will need some more advanced courses and active advanced community members before we will get more traction on the expert badges. 

Infrastructure

• Prototype: We were planning to run the first phase of the pilot with a prototype of the open badge infrastructure (OBI) that would allow us to port the badges from OSQA into the infrastructure, and then display them on other sites including the P2PU profile. But due to development cycles on both the OBI and P2PU platform, we decided to push this to the end of the second cycle, which will be in late June. 

Summary

Overall, the initial phase of the badge pilot was a positive step in the right direction for our assessment and badge work. We had initially planned on starting with 2 badges and ended up with 14 badges which allowed us to explore more types of assessments and badges in this phase. While participation was low, we learned a lot that we will apply to the next rounds in terms of communication and outreach, and have identified areas that need dedicated focus like driving more peer assessors to be actively involved.

Revisiting our goals, we met most of them by building and launching a quality proof of concept badge system, which included a basic taxonomy for badge types and various assessments approaches built around peer learning. We got some great feedback and interest from the community, as well as other stakeholders, and have some solid direction around future versions of our efforts. The only goal that we were not able to meet was the prototype of the badge infrastructure, which again, was pushed because of delayed contingencies on the development sides, but is targeted to roll into the second phase of the pilot. This will allow us to port the badges out of the OSQA environment and into the P2PU profile to give learners more control over sharing and using the badges in other contexts.

Overall, we feel that we produced a good proof-of-concept to build off of moving forward, and initial responses and observations indicate that it is important and valuable to continue to move in this direction.

Phase 2

We are rolling all of the stuff that we learned from this pilot into the second phase of the pilot which will launch in early to midJune and run through July 2011. Look for another blog post shortly detailing the plans for that phase of the pilot.

Over and out,
E