How MongoDB University Online Courses are Produced

MongoDB online courses are built on the edX platform. We entered into a collaboration with edX to use the edX software. As part of that agreement, we contribute back any improvements we make to the software. We thank the good folks at edX, including Anant Agarwal and Rob Rubin for working with us.

Although we are using the edX platform, our class flow is most similar to that of Udacity. We strive for short lecture segments (3-7 minutes long), each one with a specific learning objective in mind.

The lectures segments were all recorded using a Wacom tablet and an overhead camera to also record hand movements. See below for more detail on the technique we used to record the video segments.

One of our video recording stations

Initially, we used a fairly simple assessment engine for in-course quizzes, homework problems, and final exams. We could handle only multiple choice questions, check-all-that-apply, and fill-in-the-blank questions. We have extended the types of questions we can handle to include JSON and JavaScript answers. We also use an in-browser Mongo shell and a client-side assessment tool (MongoProc) that allows us to set up MongoDB deployments and test solutions to a variety of complex homework problems covering just about anything one can do with a MongoDB deployment.

Our staff consisted initially of one engineer, one video editor, and our instructors. The team has now grown to three engineers, one curriculum developer, and our video editor. In addition, the Director of Education and all instructors contribute to our curriculum.

All videos are hosted on YouTube and have captions that are transcribed in English by 3playmedia. We embedded the video player in the edX stack. We also have captions in Spanish contributed by a small team organized from our online ed community.

Students primarily interact with each other in the discussion forums, built into edX. Initially, the forum software was probably the least useable part of the system. We have made a number of improvements and continue to refine this essential component of all our courses.

We have found that designing and recording lessons has a ratio of approximately 10:1 with the finished product. That is, 10 hours of prep and recording goes into each hour of lesson video on average. For more fundamental topics an instructor has already presented many times, the ratio is considerably smaller (e.g., 5:1).

The experience of creating classes is intense and fun. For the first courses, Dwight and Andrew both logged long hours and late nights recording material, designing quizzes and writing programming assignments. The engineers worked crazy hours to perform customizations to the platform, and our video editor saw the sun rise at the office many times.

With several courses produced and a more well-developed course platform, creating new courses is a little less frenzied, but no less exciting and we still work through the night on many occasions in the days leading up to a launch.

Recording Video Lessons

Recording the video lessons is at the heart of delivering a good online education experience. Our videos look like this:

We show hand movements composited on top of writing or terminal windows. We like this style because it is more engaging for students. Even when the instructor stops writing, gestures keep the student engaged.

Stanford’s SITN has been teaching via TV since 1969. When Andrew was a student there in the 90s, professors would sit at a table with an overhead camera looking down at a paper tablet. Professors would write out ideas and equations using a Sharpie.

With an electronic tablet it is, of course, possible to capture what someone writes without a camera. The problem is that, as a viewer, you don't see the hand if you just capture the tablet video stream. The solution is to composite two video streams: one from a tablet and the other from an overhead ("overhand", actually) video camera.

We use a Wacom tablet positioned in a copy stand with a video camera positioned to record the hand writing on the tablet. The copy stand is a Bencher 910-60. Note that our host computer for the Wacom is a 27inch iMac. In the setup shown, we are using a predecessor to the currently shipping Wacom Cintiq 22 inch pen tablet. The benefit of this large Wacom tablet is that it tilts forward and makes writing comfortable. It’s also a bit brighter. But the 22 inch tablet is expensive (we had one prior to online Ed) and you can also use the most more reasonably priced Wacom Cintiq 12WX. We have four setups currently and all are the 12 inch version.

We use Screenflow to capture the video appearing on the Wacom tablet. You can buy it directly or get it via the Apple App store on the mac. I prefer the App store because it allows me to install it on the computers that I use.

Because you want the final video to be legible when displayed at web resolutions of around 700x400, we mark off, using painters tape, a space of approximately 1024 by 585 on the tablet. Make sure the aspect ratio of your marked off space is 16x9, like HD video.

We have tried several video cameras to capture the hand of instructors. The video camera we recommend is the Canon XA 10HD. It has good manual controls and takes XLR audio. Don’t forget to buy a handful of large capacity SD cards. There is nothing more frustrating than sitting down to record and finding that there is no card in the camera!

You will need a ball head to align the camera properly. We bought some inexpensive Oben BA-2 ball heads that we would not recommend. They flex too much, vibrate too much, and are difficult to adjust. We recommend the Arca Swiss Z1. You might need a mounting plate and we recommend the Arca Swiss universal plateNote that the Arca-Swiss is threaded to attach to a 3/8 inch screw and the Bencher copy stand has a 1/4 inch screw. You can get a reducing bushing to fix this.

Ok, now you are almost ready to shoot. To balance the light of the copy stand with the light of the Wacom display, we found it best to reduce the intensity of the fluorescent lights on the stand by covering them with a Rosco neutral density filter. We bought one sheet and cut it in half, attaching them with painters tape to the lights.

Alignment is critical for later compositing. It’s a huge hassle for the video editor to have to remove any parallax error from the overhead shot. It’s a bit easier to align when use the 12 inch Cintiq because your tablet is flat on the tablet and you can use the copy stand alignment grid to align the camera in dead center. We aligned the 12 inch Wacoms in the center of the copystand because that was easiest, but it does make it hard to reach as you write. Instructors find that they often want to stand while recording lessons. It allows one a better perspective when writing on the 12 inch Wacom.

You should use manual controls on the camera to force an exposure that exposes the hand correctly. With the neutral density filter, we found that exposure tends to overexpose the tablet itself slightly, which makes for easier compositing and whiter backgrounds. You should also adjust the white balance so the wacom appears white, while not causing the hands to be too strangely colored. The light bulbs in the copy stand are fluorescent like the backlighting of the Wacom, so this turns out to be pretty easy to do. On our Canon Mark III, we wound up using ISO 4000, f11, and 1/40th of a second. You want to be fairly stopped down (bigger f-stop, smaller lens opening) to get adequate depth of field. Otherwise, the text will be blurry or the hand will be blurry. We used custom white balance on our Canon Mark III.

We used Autocad’s Sketchbook Pro for whiteboard work on the Wacom. As with Screenflow, we recommend you buy it from Apple’s Mac App store. Don’t use Sketchbook Express, which is bundled with the Wacom tablets. It lacks certain customization features that are useful.

Good audio is critically important to students. It will make you easier to understand and it will make your caption transcriptions more accurate. There is no reason not to have amazing audio. We use an over the ear Countryman E6 Flex Microphone with cardioid pickup pattern and headroom for general speaking, wired for XLR. There are many variants of the E6. This one is tuned for general speech, is directional, adjust to multiple people (the flex) and has an XLR balanced audio output (pro audio standard).

The E6 isolates your voice very well. As you can see from the photo above, we do have some acoustic insulation on the walls, but even if there are people talking nearby, the E6 will not pick them up much. From an acoustics standpoint, you just want a room that is acoustically dead without many reflections. Any room full of stuff will work pretty well. Carpet on the floor helps too. To attach the E6 to your computer, you will need something that converts XLR to USB. We use a M-Audio Mobile Pre.

We record with the lights in the room off. Otherwise, we get specular highlights on the Wacom from the overhead lights. There is a checklist of things that the instructor must do taped to the copystand. It’s easy to forget to turn off the lights. You won’t be sitting in darkness. The lights from the copystand are pretty bright, even behind the neutral density filters.

Now you are ready to record. Start your camera and then start Screenflow. Make sure that Screenflow is set to pickup the Wacom, the entire screen. We start the camera before Screenflow so we can capture the countdown produced by Screenflow. This makes it easier for our video editor to sync up the shots.

Teaching Using a Wacom Tablet

Generally, we subscribe to Salman Khan's approach to online teaching, which is to view it as a one on one tutoring session.

Writing material out has two benefits. The first is that it paces you. You can’t throw up a tremendous amount of material at once. The second is that it limits you in the total amount of information you can present per square inch.

These instructions are for the Wacom Cintiq 12 WX pen-based tablets.

A Wacom tablet is a secondary display for your computer with an integrated writing surface that is sensitive to the included pen (stylus). The computer sees the tablet as a display and driver software on the computer integrates the pen as a type of pointing device.

image

Setting up Your Wacom on a PC

The Wacom tablet comes with a small interface box with a DVI video output and a USB output. Plug the USB cable into a free USB port on the computer.

There are two included video cables with the Wacom, a DVI-to-VGA cable and a DVI-to-DVI cable. If you have a DVI port on your PC, that’s the preferred connection. The DVI output will provide better calibration, less flicker, and tighter alignment of the pixels and screen output.

There is a switch on the interface box that must be manually set to DVI or VGA. Make sure it is set correctly for the cable you are using.

image

At this point, you are ready to turn on the display and install the driver software. The driver software comes on a CD in the Cintiq box or you can download it from Wacom’s website.

Once you touch the pen to the tablet, it should offer to let you calibrate the tablet. If not, go into Start->Programs->Wacom->Wacom Tablet Properties and calibrate your tablet.

Setting up the Wacom on a Mac

Plug the USB cable from the interface box into the USB port of your mac.

There are two included video cables with the Wacom, a DVI-to-VGA cable and a DVI-to-DVI cable. On a mac with a mini-display port, you will need a converter cable. The mac supports converting its minidisplay port to either VGA or DVI, but DVI works better. The DVI output will  provide better calibration, less flicker and tighter alignment of the pixels and screen output. 

Here is the connector you need for a Mac:

image

You can buy it from Apple or from Amazon

There is a switch on the interface box that must be manually set to DVI or VGA. Make sure it is set correctly for the cable you are using. See the PC section above for a photo of the switch.

At this point, you are ready to turn on the display and install the driver software. The driver software comes on a CD in the Cintiq box. But if you don’t have that, you can also download the driver from Wacom’s website

Once you touch the pen to the tablet, it should offer to let you calibrate the tablet. If not, go into Apple Menu->System Preferences->Wacom->Calibrate and calibrate your tablet.

Using the Wacom as a Whiteboard

To use the Wacom as a whiteboard, you need a good pen-aware drawing program. Sketchbook Pro from Autocad is excellent and I recommend it. It’s works the same on PC and Mac.

On a Mac, you can buy Sketchbook Pro from the Mac App store. On a PC, you can buy it directly from Autocad. You need the Pro version as the non-Pro version is missing certain critical features.

Once you install Sketchbook Pro, you are going to want to customize it to make it easier to use. We will make the following customizations:

  • Customize the features available when you touch the bottom bottom of the stylus while hovering above the surface. This is called the default menu.
  • Customize the size of new canvases.
  • Customize line thickness

Customizing the Default Pen Menu

The default pen menu is the menu that comes up when you hover above the tablet with the pen and push the bottom pen button.

With Sketchbook Pro running, open the preference pane. On a Mac, this is under the Sketchbook Pro Menu->Preferences. On a PC, this is under Edit->Preferences. 

image

Choose Lagoon at the top. Now customize the features for the default menu as shown below. This involves choosing each of the items in the wheel and then choosing an appropriate feature. Starting at 12 o’clock, set the items to:

  • View->Zoom/Rotate/Move Canvas
  • Colors->Black
  • Colors->Blue
  • Colors->Red
  • Colors->Green
  • Colors->Orange
  • View->Actual Size
  • File->New

Customize the Size of New Canvases

We are going to customize the size of the new canvas to be the full width of the Wacom but three screens long. This will let you use the scroll tool to scroll down the page and get more paper without losing your existing work.

Open up the preferences pane and makes the following changes to the default canvas. Note that the default width of the Wacom is 1024 on the 12 inch version and 1280 on the 22inch version. 

image

Customize the Pen Line Thickness

The pen tool works best. You will need to choose the pen and change the default line thickness and opacity. Click on the icon for the Pencil in the lower left hand corner of the Lagoon. 

That should bring up this menu:

image

Now choose the ball point pen and click on brush properties. That should bring up this view:

image

Change the line width size to 2.5 and the ink opacity to 90%.

Using Sketchbook Pro for drawing

Here are some pointers:
  • The pen is magnetic so you can rest your hand on the surface without drawing.
  • The flip side of the pen is an eraser.
  • You can choose a new color by hovering above the canvas and pushing the bottom button on the pen. That brings up the default menu, which you customized above. Now gesture outward and choose a color.
  • To get a new page, push the bottom pen button and choose the lightbulb. That’s New File. You can choose to save or discard your old work.
  • To scroll down the page, push the bottom pen button and choose scroll/pan.  If you accidentally zoom, you can follow the next point.
  • To return to 100% view, push the bottom pen button and choose “Actual Size” which is the icon that looks like a blank page.

You should now be ready to use the Wacom as a whiteboard for teaching. Refer to the previous post on setting up the rest of your recording station.

Compositing and Editing Video

It all begins with the raw content produced by the instructors. We are given two assets for each video segment. One is the video from the overhead camcorder, the other is a Screenflow bundle. Screenflow is the application we decided on for screen capturing but we could have just as easily used Camtasia or any of the other screen capture software out there.

The first thing to realize is that camcorders record video at 29.97 frames per second (or 59.94). Even if it is set on the camera to record 30p, you’re still going to get 29.97 fps. Screenflow by default records at exactly 30 fps, so the compositing gets out of sync to a noticeable extent about every minute and a half or so. So to get around this we have to tweak the export setting in Screenflow.

Go to file->export (you could also use E)->customize->video settings and change the frame rate to 29.97.

After exporting with these settings, we import the Screenflow feed and the camera feed into Final Cut Pro. We use Final Cut Pro X on a Mac Pro.

Having both feeds with the same frame rates in Final Cut, we composite the feed from Screenflow on top of the feed from the camera. The camera feed is going to be the bottom feed, so we start with that.

Because of the copy stand configuration, the video footage comes out looking something like this. Notice, if you haven't already, that the video camera is down.

We rotate the overhead shot by 180° and zoom appropriately. We delete the audio because the audio we get from Screenflow using an M-Audio box and Countryman over-the-ear ear mic is much crisper.

We have to get the footage from the camera to be as perfect as possible because this is the bottom feed, meaning everything you see in the finished video except the writing comes from this feed. This requires color correction, resizing, and if the camera added any fisheye, or barrel distortion, or wasn’t 100% aligned, we transform it to be perfectly aligned.

We also try to get the writing or text on the screen to be as light as possible so it won’t be visible underneath the Screenflow feed.

After we tweak the camera feed to our liking, we add the Screenflow feed on top of it in the timeline and detach the audio from the video so we can edit them separately:

Final Cut works like any other layer-based application, so layers on top take precedence, meaning with this configuration, all that you see is the Screenflow footage. To solve this problem, we use a keying affect, which basically tells Final Cut to look for all the white in the Screenflow feed and make it transparent, so you can see the footage beneath it. We use Final Cut X’s luma keyer which performs best.

By using keying and inverting it and playing with the rolloff and matte settings we can do a pretty good job of keying out all the background white leaving us with just the writing or text. Here are our settings:


Now that we have our bottom feed (the footage from the camera) and the top feed (the footage from Screenflow minus the background), it is just a matter of resizing and aligning the top feed to completely cover the writing and text on the bottom feed (not a small feat but with some patience and a bit of luck we get it to work).

We also have to sync the two feeds visually, so we go frame by frame on both feeds and find an easy place to sync the feeds, like a window opening or a new canvas popping up. We put a marker on each feed and line up the markers.

After we have the feeds composited, we use the New Compound Clip feature in Final Cut so we have 1 video file to edit in the timeline:

This makes is so we do not have to make 2 cuts every time we need to edit the video.

At this point we can begin the editing process. Since these are online classes and not in-person lectures, there is an opportunity to remove mistakes, and fix pacing. Editing makes everyone better.

Any “ums” and other accidental sounds are cut out (unless we are very pressed for time). Also, since you talk faster than you write or type, a lot of video needs to be cut out to keep the pace steady. We have to strike a balance though. We don't want to over-produce the videos and make them feel artificial.

When videos are complete, we upload the finished product to YouTube and add the URL to the appropriate lesson in our course builder application.

Because of everything that needs to be cut, the ratio of length of raw footage to length of edited footage is about 4:1. The ratio of editing time to finished video is about 8:1. We leave it as an exercise to the reader to figure out the effort involved to edit about 16 hours of online video per class for two classes over a period of about about eight weeks!

Curriculum Development

Now that you know how to setup the recording station, teach using a Wacom tablet and how to edit video streams, the big question is, how do you organize yourself to actually deliver the course?

We use Google Drive to organize curriculum -- spreadsheets to organize the lessons into chapters and Google docs to outline lessons. We do not script course lessons, but we do outline them as a series of talking points with bullets. We share all course materials between stakeholders including instructors, curriculum developers, content editors, reviewers, and video editors.

Most courses are seven chapters long. Each chapters delivers about two hours of lesson video with quizzes throughout and homework exercises at the end. Typically, our lessons are 3-5 minutes long, each one with the following structure:

  • A short video lesson that strives to achieve a goal posed as at the end of this segment, students should know X. X can be a piece of knowledge or a skill. For example, ..students should know how to use the $inc operator in a $set command.
  • A short quiz.
  • A short solution video or writeup that exposes the right answer and explains the reasoning.
  • From a curriculum development standpoint, it’s a very linear process to build a lesson. We specify the learning goal and write a quiz that tests the goal. We add these to the Google spreadsheet, one row for each lesson. The columns for each lesson are:

    1. Chapter (eg: 1)
    2. Unit (eg: Aggregation - true for the whole week)
    3. Key concept (eg: pipeline)
    4. Learning Objective (eg: students should understand that mongodb aggregation is like a unix pipeline)
    5. Quiz: (eg: which of the following are stages of the aggregation pipeline..).
    6. screenflow filename (more on this later)
    7. Quiz Preview URL (more on this later)
    8. comments to video editor (more on this later)
    9. YouTube URL (more on this later)

    We usually write an entire chapter and then record it. As part of the recording process we build each lesson in our coursebuilder. This involves creating a segment that will (once it's finished and uploaded to YouTube) include the lesson video URL, building the quiz, and either writing up the solution as HTML or adding the URL for the solution video. Depending on the lesson, we may also upload handouts as attachments to the lessons. The coursebuilder allows us to preview a quiz using the same skin as the edX courseware so we can record solution videos directly from coursebuilder rather than having to deploy the course first to our edX instance.

    We frequently set up a recording station for instructors at home to make it more comfortable to do long runs of recording. So we need a means of file sharing. We found a Dropbox (paid account) to be a pretty effective way of moving files around. The Screenflow files are small, but the overhead video files are enormous.

    Homeworks are a little different. Those we develop mostly using a text editor and the command line. Many require the use of handouts and some programming. We store the source for these homeworks in a GitHub repository and create handout bundles that are uploaded to coursebuilder for distribution to students.

    As you can see from the process above, designing good online classes is a significant amount of work and planning.

Recording Headshots

We use what we call headshot videos to advertise classes and to introduce each chapter. These segments look like this and are mid-shot videos of a single person, typically, talking directly to the camera:

The one above is to advertise the M102, MongoDB for DBAs class. Students appreciate these because they put a face to a voice. Since most of our teaching segments are composited overhead shots of the Wacom tablet, these shots represent the only time the students see the teacher.

For recording the video, we use our Canon XA 10 Video Camera. It has XLR audio inputs. We have two Sony UWP-V1 wireless microphones systems. We use the balanced audio cable to go from the receiver to camera. We own two microphone systems in case we want to have two people talking on camera. That’s useful if there are two instructors or if you are conducting an interview.

With only one microphone running, we get sound only on one channel (left or right). We clone the audio onto the second channel during article processing.

The wireless body mics are pretty good. They don’t isolate the speaker as well as the over-the-ear countryman mics, but they are isolating enough that we can shoot in a crowded office.

We wind up tweaking the gain levels on the mics a bit to get the right volume. You don’t want the mic to clip. It’s easy enough to make a soft signal louder and you don’t lose much fidelity doing so. But if your microphone signal is clipped, information is entirely lost and the clipped parts can’t be recovered. Audio that is clipped sounds bad.

We tripod mount the video camera on a Pearstone tripod that includes a cheap video head and try to put the subject in a place that respects the rule of thirds.

Lighting is always tricky with humans. Although we have a small LED lighting system, the problem with it is that the subject has to be really close to the lights to get even illumination. We probably need bigger lights. Instead, we have moved to shooting these type of shots with a window behind us and using natural light.

We are also never quite sure what the backdrop should be for these shots. We frequently use an open shot of the NY office as the background. We have also purchased some solid colored cloth backdrops but were ultimately not happy with the look.

We do sometimes write scripts for these shots. We don’t read from the scripts (we don't own a teleprompter) but writing them out often helps us decide what we want to say. Note that we don’t write out scripts for lesson videos at all.

These also tend to be some of the more stressful videos to record. Maybe not surprisingly, it’s harder to look right into a camera and speak versus work at a tablet while being recorded. And we record these out in the middle of the room so people are acutely aware of how many takes we do (sometimes more than five - they usually decline in quality from the first onward).

You also get more comfortable being on camera the more you do it, and ultimately the first video you record is the promo video for the course. If timing permits, we would recommend re-recording it after you have finished recording the course.

Costs of Teaching Online Classes

There are two sets of costs to running online classes: the capital cost of buying the equipment and the variable cost of the labor. In this analysis, we will look at first year cash costs. The capital equipment can be amortized over multiple years in a true accounting analysis.

To be able to teach a single class online, you need a recording studio, the equipment for standing news reporter shots and the equipment to edit video. The total cost as configured for us was about $13k for this equipment. Here is how the cost broken down:

Note that the video editing station specified is good enough to edit more than one class. You could probably teach 5-6 new classes per year using this setup, sequentially.

We own four recording studio setups, but only one is strictly necessary.

In addition, we employ a full time video editor. Our video editor estimates that it takes him eight hours to edit each hour of finished video. Our courses are about 14 hours of total edited lesson material, so that’s 112 hours per class. That’s about three weeks of work at 40 hours per week. It's difficult to edit more than 40 hours per week.

To get our online program started, also required a full-time engineer. That cost dwarfs the capital equipment.

We also pay approximately $1,700 per month to host the classes on Amazon Web Services ($20,400/year).

We also had all video captioned at 3play media. Putting all that together, and assuming that we it takes two full time people on the online ed effort to produce the first three classes, we get the following annual costs.

Note that we dropped in $100k for the fully burdened cost of a head. That is not our number at MongoDB University. We just put that in as a placeholder to be concrete. On our team, we have more than two heads, but some folks are working on more than just online education.

Although the startup costs are not insignificant at $250k, the incremental cost of adding one student or running the classes again is very low. Even considering the high startup costs, we registered more than 50,000 people in the first year, so the cost was under $5/student.

The economics of online education are amazingly good. There is at least an order of magnitude improvement over the costs of teaching in person.