First month at Nava

23rd September 2015

It’s been ~1 month since I’ve joined Nava! In addition to having a website that reminds me of David’s Tea, we have this audacious goal:

Create software that radically improves how our government serves people.

In some ways, I feel like I’ve turned back time to four years ago, and I’m back at Khan Academy in 2011. There’s this < 15 person team, everyone is passionate about huge positive social impact, and we’re all trying to figure out what that means together. We’re definitely shooting for the moon (or is it Mars?), but we’re taking these smaller pragmatic steps to get there. Though we aren’t a non-profit like KA, we are a public-benefit corporation, and I again find myself building useful services for people who are possibly in need and most likely not directly paying for it.

In other ways, Nava is distinctly different. Instead of education, it’s government. Instead of discussing growth mindset or throwing a cheeky “you can learn anything” at coworkers, casual conversations take on a more political / historical default. Supreme Court rulings, debates, lessons about WWII. “Did you know I was on the same flight as Ted Cruz?” comes up way more often than I would have guessed. I’ve learned so much about the surrounding ecosystem, and I’m starting to pick up all the three letter acronyms that everyone naturally speaks in (SEP: special enrollment period, CiC: change in circumstance, ESC: employer sponsored coverage, etc).

Engineering wise, it’s been fun. Since KA was such an early adopter of React (thanks Alpert and Joel), I’m pretty familiar with it, and one of my first Nava projects is to help us switch over to it. It’s a neat way to learn a new codebase + process, while still having React as a familiar face. Quite a few common English words have taken a more nuanced meaning to me: grunt, jasmine, node, babel (is this English?).

I’m still stumped when it comes to pronouns. When folks ask me about previous work at KA, I default to “we did that”, “we do this”, and “we’re planning to do that” — but at some point, I wonder if I’ll get used to thinking of KA as “they”, and then Nava as the only “we”? We shall see! We are hiring.

On celebrating small steps in the right direction

20th April 2015

It’s easy to remember to celebrate big milestones — company wide announcements, launch parties, cakes, etc. However it’s done, the team is awash in recognition, and everyone is a little bit cheerier than normal (or is it just sleep deprivation?).

Even medium sized milestones are easy to remember. This past week, the SAT team reached such a milestone — private beta. We reflected and celebrated in our weekly sync, and we each walked away with a small wobbling solar flower to brighten our desks.

I guess this is obvious, but we should remember to acknowledge and celebrate the small milestones.

Small steps in the right direction, they happen frequently on any well functioning team. Someone wrote up a clear project doc. Someone shared a neat design. Someone fixed a nasty bug. Someone crafted an artful diff. Someone expertly fielded questions with external partners. Someone automated a process that previously took 20 minutes of someone else’s time every week. Someone gave sage advice in code review. Someone helped the team make tough decisions together. Someone swooped in and caught a ball before it dropped all the way. Pair programming, brainstorming, rubber ducking, venting sessions, etc.

Whatever they might be, these small steps don’t require the same public fanfare as big milestones, but they should still be acknowledged. If only from personal experience, being acknowledged for these small steps brightens my day. It can be as simple as saying, “Hey, thanks!” :)

Without these small steps, how else would your team reach that big milestone?

PS Hey, thanks! Thanks to the SAT team for doing all those “small” things above all the time. You’re a dream to work with, and it’s a privilege. Thanks to Jordan for writing that blog post above, which inspired me to write this “obvious” one. Thanks to Kamens for reminding me to say thank you and to eat my vegetables.

Too much of a good thing

10th December 2014

Is it possible to have too much of a good thing? Too much call to action? You decide!


See the impact of your work, in person!

24th October 2014

During our weekly company updates, we read out testimonials from students, teachers, and parents, and I’m reminded of our work’s impact. Even so, there’s a distance between them and me (literally!).

That distance shrunk to 0 at the Grace Hopper Celebration career fair a couple weeks ago, where Pamela, Kayla, May-Li, and I represented KA. We met over 800 of the 8000 GHC attendees over the course of three days, and so many of them came up simply to say:

I love what you’re doing. Keep it up!

I didn’t understand my professor, and then I found Khan Academy.

You helped me pass Calculus

I’m not looking for a job yet, I just wanted to come by and say thank you.

Coming up on four years at KA, I’m somehow still surprised when I meet someone in real life who learned something with KA. It’s nice!

Show and tell: testing assumptions

6th August 2014

I love working at KA because we can test our assumptions at scale.

For one small example, we recently experimented with the layout of math subjects in the menu.

Here’s one version we tried, where the grade levels appear in a row at the bottom:


Here’s another alternative, where the grade levels appear in a column alongside the other subjects:



Which one is better?

They’re so similar, perhaps, that you have no opinion! Maybe you like the first because of the row’s aesthetic. Or, you might like the second because it emphasizes grade levels. (“I’m in 8th grade, so I want to look at 8th grade stuff!” vs “I’m interested in all of Geometry, give me all of it!”)

If we were a static textbook, we’d make a decision without knowing whether it was the right one. Maybe some feedback would come from an opinionated reader, but more likely not. Thankfully, we are not in the business of making textbooks, and we inform our decisions with data.

With 1 – 3 % gains in return visits and completing different types of content, the second alternative won. Yay!

Invest in onboarding (continuously)

6th May 2014

Every time new devs join the team, they get a mentor, they get some onboarding docs, they ramp up, and they become wonderfully productive people. 

Considering that we’ve had ~50+ (?) people flow through some sort of onboarding (counting our ~25 person dev team and 3 classes of summer interns), we might think we’re done with “working on onboarding”. Our onboarding is reasonable, with docs about the dev environment, code review, unit testing, style guides, what to expect on your first day / week / month…

…but we don’t update them as much as we should. There are pages that mention workflows that have completely changed, eg post-commit code reviews with mercurial/kiln when we’ve long switched to pre-commit code reviews with git/phabricator. I looked around today, and some pages literally make no sense given our “new way of doing things” – but how would a new dev know that? (Thankfully they have mentors to guide the way, but we shouldn’t have confused them in the first place!)

Our docs have a bunch of potholes that our brave new devs encounter, and though each blip might take just a little bit of time, they can certainly add up to a whole afternoon (or gasp! a day!) of slogging through boring mind-numbing things that literally tens of other people have already done. [very dramatic]

Instead of each person tripping over the same obstacle, why not clear it? Give them the onboarding that will help them thrive, not onboarding that they will thrive in spite of.

We’re about to absorb ~15 interns for the summer, so there’s no time like now to update our docs! Both for all the interns who will undoubtedly ask certain questions, and for their mentors who will be there to answer them. Summer 2014, here we come!

non deterministic bug or ndb?

17th March 2014

We recently made a bunch of perf fixes to reduce homepage load times (by 30% in some cases!).

Sounds too simple to be true, but we reduced time on both server and client by doing less stuff:

1) Remove unnecessary RPCs

2) Load less css / js

After these fixes were deployed, we noticed a small trickle of errors on homepage load:

TypeError: '_BaseValue' object is not subscriptable

The errors seemed non deterministic, and they seemed to happen more often when hitting non default versions of our app. After some sleuthing, we figured out what was going on, and this inspired the following quiz.



Dashboard: six months later

20th February 2014

Happy half-birthday to you, little dashboard! Since then, the core idea has remained the same (a good sign!), which lets us iterate and refine (also good!).

Six months ago: original-dashboard


Can you spot the differences?

Originally, we launched the dashboard with one mission: “The World of Math.” In December, we introduced more missions, and you can see that I’m focusing on 6th grade right now.

Here are the missions we have so far:


Since launch, we’ve also refined the dashboard’s appearance. We grouped logical components together – like the mastery challenge with the user’s list of upcoming practice tasks. We organized the mission progress by topic, so users can better see and understand the context of their progress. We’ve iterated on color and layout, and all of this combines to make the dashboard feel cleaner and better.

As you explore the dashboard, you might notice a few new, possibly experimental, features. You might see a video in the dashboard (which typically only had exercises). You can decline tasks that you’re not interested in. When you master a mission, you’re greeted with a celebration of your achievements.


And that’s not all! There are experiments for how you’re first introduced to the dashboard, and under the hood, we’re tweaking how we suggest exercises to users (which exercises to suggest and when to suggest them). The list of iterations we’ve made and will make go on and on.

With most (if not all) of these changes, we listen to our users’ reactions. Quantitatively, we measure the impact on their retention and engagement through a/b testing. Do they come back more often? Do they practice more? Are they getting more problems correct/incorrect? Qualitatively, we listen to all the different ways they can communicate their delight or distaste. Is there a huge uptick in bug reports because they hate the new layout?

Can’t wait to see how the dashboard grows in the next six months – by then it’ll be a toddler.

Implementing color palettes with LESS mixins

16th October 2013

In March, we launched a redesign of Khan Academy where we gave the world the gift of color! [1]

Screen Shot 2013-10-16 at 7.28.53 PM Screen Shot 2013-10-16 at 7.29.03 PM Screen Shot 2013-10-16 at 7.29.08 PM Screen Shot 2013-10-16 at 7.29.15 PM

As you meander about Khan Academy, you’ll see that each domain’s color scheme permeates all the way through. For example, “Discoveries and projects“ falls under science, so it’s pink.


Instead of copying and pasting domains + colors everywhere, we used LESS variables and mixins.

More specifically, we did something like this gist [2]

As a fun demo, open some science page and then do this in the console:


and you’ll see the theme switch just like that!

blue science

[1] – Where Jason and Tabitha both blogged about it

[2] – I should figure out how to embed gists some time. Or move off of wordpress.

tales of app engine: how to reference an ndb entity from a db entity

24th September 2013

approach one: make a custom property that converts between ndb and db keys.

this is OK, but db validation will complain when you try to build queries that filter by  ndb key. this approach does work in the bare-bones case where you have a db entity handy, and then get the referenced ndb entities via the custom property. (thank goodness for ben ko for asking me to test that!)

approach two: store the urlsafe version of the ndb entity’s key as a db.StringProperty.

this is OK, but annoying in that you have to remember to convert from string to ndb.Key and then get the entity. and do the reverse when you assign.

approach three: rewrite your entire codebase to use ndb.

this is OK, but only if your codebase is two lines. oh ndb.KeyProperty!

is there an approach four that is not a compromise at all?