Young Reacts #169

I am going to the office tomorrow to see my teammate visiting the area. He will be my first Square coworker that I meet in person after five months of working here. I look forward to meeting a real person behind the tiny rectangle on my screen!


Software Engineering ⚙️

Resilience engineering: Where do I start?

I like learning about how systems fail in surprising ways to further my understanding of their behaviors. This short article contains many links to the foundational academic papers. If you find all these papers daunting, How complex systems fail is a great starting point.

How to publish Deno modules to NPM

Deno is a new serverside Javascript runtime, an alternative to Node.js. It has a different set of native APIs, making it challenging to write a code that works on Deno and Node.js.

The author uses dnt to publish their Deno module to NPM. I am amazed that they could cross-publish an HTTP framework with a few configurations and code branches.

Imbalance detection for healthier experimentation

My team recently adopted A/B testing, so this level of sophistication is not yet desirable. However, I learned two things in this article: 1. since achieving true random is difficult, imbalances can happen. 2. online traffic is precious, so every time we waste a user session, we miss out on a learning opportunity.

People ❤️

What We Lost

We’ve been working remotely for two whole years now. Many of us have changed our lives, assuming that we will work remotely permanently. I moved away from where my office and many colleagues did too. That flexibility is a win, but I miss being in the office; jokes, walking meetings, serendipities, all of them.

The skills product managers need to become a Director, VP, or CPO

This table in the article summarizes well. In short, what got you here won’t get you there.

Business 💰

Apple CEO Tim Cook tells employees the return to offices will begin on April 11th

As the COVID case rates continue to go down, companies plan to bring their workers back to the office. As Apple always planned to return to the office, I am not surprised by this news. But I wonder if companies that declared to go permanently remote will reverse.

Kroger expanding into Oklahoma City as it builds national delivery network

As more spending moves from offline to online, even traditional retail chains like Kroger are building fulfillment centers only for online orders.

Young Reacts #168

I shared this message with my team on Thursday morning:

Hi all, you may have heard the news about Russia invading Ukraine. I recognize that the last couple of years have been difficult and this invasion makes it worse. If you need time to clear your heads or to learn more about the crisis, please take the needed time off. If you’d rather focus on deep work, do go ahead and cancel meetings.

In crises like this, it is paramount to care for ourselves. At the same time, we have to consider our civic duty to one another. I am currently thinking about the “banality of evil” and how silent complicity enables opportunists who seek to take advantage of the people.

This kind of intro probably isn’t why you subscribe to my newsletter, but I didn’t have mental space for a different one.

Software Engineering ⚙️

Typescript Type Assignability Matrix

Captured from the above link. Each cell indicates whether a variable of type [row] is assignable to a value of type [column].

I wish I had something like this when I first started learning Typescript. The matrix may be confusing. But when you read each cell carefully and consider why they behave like that, you will understand it.

People ❤️

Support open-source that you use by paying the maintainers to talk to your team

Speaking engagements sound more beneficial for engineers than direct sponsorships (though direct sponsorships are essential, too!). I wonder what the reasonable rate for the speakers is.

Times are great for programmers now. How does it end?

I can’t imagine a world where technology gets less critical in businesses. We have barely digitized our messy real world. I predict that the story will differ based on the engineers’ skillsets—while machine learning engineers are on-demand, Fortran or Cobol programmers are not. Individual engineers need to continuously learn and adapt to stay relevant.

Business 💰

Shopify’s Evolution

Shopify is moving from an integration hub to a full-stack service provider for eCommerce. It now has its payment system, a fulfillment network, and more. However, its evolution is creating a platform risk to its partners:

Announcing our $100M Series C funding

Hasura, a major contributor to the GraphQL ecosystem in both content and OSS, got a new round of funding to invest in its cloud services. I am happy for the team, and I hope that they keep the current effort in pushing the ecosystem further.

10 Lessons Learned after $5B of M&A

The most surprising lesson was the second one:

Startups are sold to individuals, not to companies. The champion – often a product leader, the CEO, or a general manager – risks their career by buying a startup.

Young Reacts #167

I have found myself quite a bit more stressed since I became a manager. That’s because I have far more time-sensitive tasks, such as interviews, performance reviews, quarterly planning, and so on, and I don’t own the deadlines as company processes set them. Worse yet, only I can do some of these tasks (most people management tasks).

On the contrary, if you are a trusted engineer on a well-run team, you can generally set your timeline and rarely come under time pressure. Also, your team should have enough context to cover for you if you cannot get to them in time.

I started thinking about how much I am willing to sacrifice my well-being to do the job I like.


Software Engineering ⚙️

How not to learn GraphQL

As I only worked with Apollo’s flavor of GraphQL (Apollo Client and Apollo Federation), I am not well-versed in other GraphQL tools. This article taught me that other tools, like Urql and Stitching, are worth looking at.

People ❤️

Engineering Org Structures— The QRF Team Model

I’ve seen many teams where they took weeks to respond to customer requests, which take a few min because of how they prioritize. The author astutely observes that “the team is operating at a tempo that is slower than the environment demands.” So the team should create a separate workstream with a faster tempo by dedicating some of the team’s time, nominating someone on the team, or creating a whole new team.

The perils of a writing culture — the Amazon hammer to decision making.

I started reading Working Backwards, a book on Amazon’s work culture. I admit it’s easy to get hooked by its rituals as Amazon’s business success is undeniable. But as the author of this article says, we need to use the right tool for the right job, and a 6-page memo may not be it.

That Wild Ask A Manager Story

The article has an extreme case of a well-meaning process gone wrong. When you run a retrospective, it’s tempting to create a new process to capture “what we should have done better.” Unfortunately, even well-meaning processes have overhead costs and should be considered the last resort, not the first reaction.

Business 💰

The App Store has a ‘Too Big To Fail’ problem

The critical insight in the article is that “the App Store is the Games Store” as mobile games generate the bulk of the App Store’s revenue. While Apple is willing to bend over backward to keep the larger apps, it won’t budge over the games.

Can a $310M startup avoid due diligence?

I am not sure if this article is accurate about Merits.com. Only the top executives at the company will know. But this is another reminder that startups are full of gotchas.

Young Reacts #166

I’ve been trying the templated response to recruiters (I shared this in the last newsletter). And I was surprised that a recruiter sent me salary information for an engineering manager role. It might not be much, but it is worth more than the last fifty recruiter emails I received before. So why don’t you try it yourself?


Software Engineering ⚙️

READS: Service Health Metrics

READS stands for request rates, errors, availability, duration/latency, and saturation. I have a reasonably good understanding of our services’ error rates and availabilities but not the other three. I will need to look into them.

Observing FinTech Services For Seamless Payments

This article goes nicely with the READS article above as this provides concrete examples of the theory above, such as latencies of order checkout and CPU usages.

Immutability isn’t free

Javascript’s spread operator surprised the author with its O(n) performance characteristic. Javascript developers need to understand how the Javascript engines work to avoid similar mistakes. But that’s easier said than done. This surprising behavior is a reason why performance-sensitive apps are written in other languages like Go or Rust.

GraphQL Network Inspector

I would have loved to use this Chrome extension when I was an engineer. Finding the request in question by clicking through a list of GraphQL requests was not fun. Netflix eventually created a Chrome extension in-house to tackle this.

People ❤️

My 3 Favorite Types of Manager Meetings

I am not used to being a full-time manager. During my previous manager roles, I did quite a bit of coding, which helped me feel a sense of achievement. But I no longer code and have lost that immediate positive feedback from coding. While most of my managerial work is not visible, I’d love to have a meeting like the Managers Demos Meeting to share my achievements.

How to Write Something Compelling

The author provides valuable axes to evaluate our writings: an idea’s impact, simplicity, universality, and a word count.

Business 💰

Tech questions for 2022

The author’s chosen keywords are Crypto; AR, VR and the metaverse; Games; Regulation; Privacy; Cars; China; Macro; Everything else. I would also add cybersecurity.

Young Reacts #165

I happened to chat with an old-timer at Square who was leaving soon. They shared a backstory on how our engineer leveling changed over time and what pain points it has. All companies have their internal struggles, so hearing this wasn’t upsetting or anything. But I started thinking about how this trend will impact how I manage my team.


Software Engineering ⚙️

Updating React at Etsy

I was surprised to read that Etsy decided to migrate from React 15 to Preact 10 instead of React 16 or beyond. I don’t know how Etsy got stuck with a five-year-old version of React. But a decision to migrate with a React-compatible library, not React, will have long-term ramifications, the risk I would avoid.

The Time Value of Shipping

Since I started working at larger companies, I ran into many projects that dragged on a couple of years since their inception. Given that context, the article taught me two things: we need to anticipate how our customers will change in the coming years and stop the project if we miss the window.

How I Discovered Thousands of Open Databases on AWS

The author found 337+K IP and port combinations open to the general public with a simple set of tools, and many were actual vulnerabilities (some were already victims of ransomware). Security through obscurity isn’t enough. Configuration is messy.

Tools for Fast (and Less Furious) Frontend Development

I liked that this team introduced debugging facilities to their internal framework to speed up their development (see “4. ‘Observability’”).

People ❤️

Career Advice Nobody Gave Me: Never ignore a recruiter

I am guilty of ignoring recruiter emails or just outright rejecting them. If we know what we want to learn about the market and prepare to get the information, recruiters can provide great signals about the market.

Titles

I was curious why Square does not have titles like a “Director” or a “VP” and was pointed to this article. While I don’t hold the idealized view such as this article (I can still see how large of an org someone manages, which affects my perception of their work), I think Square found the right balance to provide a structured growth path while somewhat protecting against HiPPO.

Red Light Green Light

Where we sit changes what we see. When we disagree with someone, the disagreement usually comes from different vantage points.

Business 💰

Sony buying Bungie for $3.6 billion

Game platform companies are gobbling up studios to take dibs on their IPs, betting on the future expansion into other media or their metaverses. Microsoft bought Activision Blizzard, and now Sony bought Bungie. Some companies, such as Riot Games, have leveraged their IP incredibly well, but many failed. As a gamer, I’d love to see these IPs pan out, but I am not holding my breath.

Young Reacts #164

Instead of an introduction, I am linking to my short blog post based on a coffee chat: A Manager’s Incentives.


Software Engineering ⚙️

Fixing Performance Regressions Before they Happen

Detecting regressions, especially non-deterministic regressions, is about finding the tricky balance between false positives, which will reduce the trust in the detection, and false negatives, which will release regressions to customers. Netflix’s TVUI team adopted statistical analysis (anomaly and changepoint detection) to reduce both falses.

Deno in 2021

Deno is worth following if you are a Javascript developer. Their focus on Node compatibility, which started with the 1.15 release, is a smart move to get wider Deno adoption.

Announcing Parcel CSS: A new CSS parser, compiler, and minifier written in Rust!

Not just JS tools but CSS tools are now written in Rust.

People ❤️

How can a Scrum daily not be a status pull?

Throughout my career, I was disappointed that the standups eventually turned into mere status updates and, worse yet, daily guilt trips. The engineers felt compelled to show that they were productive the day before. This StackOverflow thread showed me an alternative; teams focus on what they need from each other, not what they did or will do.

How “Engineering-Driven” Leads to “Engineering-Supremacy”

This article reminds me of my past experiences where the more mature, successful companies (Netflix, especially) had far more engineers interested in the companies’ businesses than the failed startups had. Curiosity and respect for other functions and customers will make us better engineers and more effective teams.

On Leaving Facebook

This first-hand account of looking for a new role as a senior engineer explains why he decided to leave and how he looked for the next opportunity. I especially found the idea of treating compensation as a trailing indicator enlightening.

Business 💰

Algorithmic Safety: Mitigating Bias in Workforce Decisions

A group called the Data & Trust Alliance created a guideline to safeguard against bias when algorithms are used to make HR decisions, such as hiring and promotions. The fact that this guideline exists shows that employers are already using algorithms to make those decisions. Companies like Meta, Starbucks, and Pfizer are members of the Alliance.

A Manager’s Incentives

Last Friday, I had a random coffee chat at work with an engineering lead (meaning they are a manager of managers). I regularly do these chats because I enjoy meeting new colleagues and learning about their work. But I got an unexpectedly profound insight into how a manager differs from an IC.

“ICs grow upwards, but managers grow downwards,” he said. He explained that an engineer grows by generalizing their work. They’d start by building a custom UI component, then create a shared component library for their team, and, eventually, tackle how UIs built in the industry (not everyone gets here).

On the contrary, a manager grows by going deeper into their domains. As their team proves its impact, the team will get more investments. Once their team grows enough to warrant a split, the manager can then add more layers and headcounts under them. This is a typical pattern at fast-growing companies. He said it is unusual for a manager to grow by inheriting new domains unless their own manager somehow leaves.

There is another important distinction, according to him. A feedback cycle for a manager is longer and less direct than that for an IC, which means that a manager’s org size is often deemed as a proxy to their scope and impact. Thus, while an IC has the incentives to collaborate on the most impactful projects, a manager has the incentives to protect their headcounts regardless of actual impact.

I didn’t fully agree with his second point because an engineer is often incentivized to take on the most visible project/role instead of the most impactful, and because a manager can work on impactful projects outside their organization. But both his points gave me a new perspective on the darkside into which other managers and I could fall.

Young Reacts #163

Since the reorg last October, I’ve been trying to define my team’s mission but couldn’t make satisfactory progress. I’d love to be able to tell a story of how all of us are building towards a single vision, unlocking unique values for Square sellers, but I don’t have that narrative yet. I can pull something out of thin air based on what we are doing now, but I am afraid it will not stand the test of time. Without this narrative, I find it challenging to hire the right people, motivate the team, make a trade-off between projects, and build towards the future-proof architecture. It’s a bit frustrating.


Software Engineering ⚙️

Roblox Return to Service 10/28-10/31 2021

Roblox posted a postmortem on its historic outage last October. It attributes the outage to two issues in its single-point-of-failure (the service discovery service) and will spread the load across multiple service discovery services. That solution will protect against similar outages but also add operational complexity. It makes me wonder how a team can prevent over-indexing from a recency bias.

From experiment to launch: how data shaped a new comments experience

This article explains how data analytics fits into product development, from setting goal metrics to guiding the design direction. It’d be interesting to see how it compares to your team’s process.

People ❤️

Setting Goals for a Life Worth Living

When the pandemic started, I felt like my life was on probation. While grateful that I had a good job, I couldn’t feel any sense of progress. I couldn’t go to a gym and beat my lifting records, nor travel and experience new things. After reading this article, I suspect this lack of progress outside work contributed to my burnout. What can I do to feel a sense of achievement in my life?

Cost of Attrition

Whenever there is a personnel change (either attrition or addition), it creates a whole new dynamic in the team. While a team competency matrix is a valuable tool, managers need to look at how the individuals are connected.

How to Write a Strategy Statement Your Team Will Actually Remember

To have and follow a strategy is to make hard choices consistently.

Tech’s big new ideas about work

Four-day work week and company-wide shutdowns are trending.

Business 💰

Welcoming the Incredible Teams and Legendary Franchises of Activision Blizzard to Microsoft Gaming

Microsoft agreed to acquire Activision Blizzard for $70B. The announcement says that this move is to bolster Microsoft’s metaverse, but I am not convinced as Microsoft’s metaverse concept focuses on the work environment. As a big fan of Blizzard IPs who’ve been disappointed by their recent work, I look forward to the changes they will bring.

Young Reacts #162

This time of the year provides a perfect excuse to reach out to now distant people. I checked it with a few of my former colleagues this week. It felt good to share stories and have a few laughs. Why not reach out to them?


Software Engineering ⚙️

MySQL Threads

Having too many threads in our MySQL DB is one of the top two reasons my team gets paged. So I wanted to understand what spins the threads up in the first place. In short, each DB connection reuses an existing thread or creates one new thread.

Chasing a Hung MySQL Transaction: InnoDB History Length Strikes Back

Growing InnoDB history length is the other top reason for our pages. When a long-running transaction prevents deletions from getting committed, the history length increases, and the DB slows down.

Could Rust be the Future of JavaScript Infrastructure?

This article surveys how JavaScript tools are adopting Rust to be faster.

People ❤️

A New Morality of Attainment (Goodhart’s Law)

I am excited that my org is putting more emphasis on setting success metrics on our projects and products. But I am also aware that metrics can be gamed and give a false sense of correctness. Goodhart’s Law states that

“When a measure becomes a target, it ceases to be a good measure.”

Business 💰

Calling the Shot: the Next Five Years at Riot Games

While I am not interested in working at Riot Games for the lack of diversity and inclusion (this post also calls it out), I did find Riot’s vision for the future compelling:

If we believe in a world where the biggest cinematic universes are rooted in games, the most popular music starts with games (K/DA anyone?), the most thriving sports are esports, and players are the driving force behind the biggest cultural moments, then what should we do over the next five years to get closer to that world?

Open source developer corrupts widely-used libraries, affecting tons of projects

The sustainability of open-source projects (especially, smaller projects with no backing company or foundation) has been questioned many times. This specific drama raises a louder alarm.

Interesting Finds 💡

Amazon Yankee Candle reviews are mirroring the COVID-19 surge — again

I heard about this study during my team chat. The study found a correlation between negative reviews with COVID cases in the US, which I found amusing.

Young Reacts #161

Some find fulfillment from work, and others don’t. The stereotype in tech, especially at startups, is the uber-driven people out to make a name for themselves. But we don’t have to live like that if we prefer not to. Plenty of people do solid work and go home and enjoy their lives outside work, and there is no shame in that. It’s ok to feel disinterests at work.


Software Engineering ⚙️

RFC: Fragment Modularity

Relay folks have pushed for this issue quite a few times, but I never understood the pain points. This GraphQL Spec RFC helped me know why by juxtaposing component-driven development with fragment-driven data fetching.

Bringing Features to Life in Legends of Runeterra

This article is outdated, but I enjoyed reading it, learning how games are developed. I found two points interesting: first, they use LFS to version control binary assets that Git cannot handle, and second, they also employ modern development methodology like feature branch deployments.

Apple’s use of Swift and SwiftUI in iOS 15

This article has some cool stats on how Apple adopted its language and framework, Swift and SwiftUI. Even though Swift is growing rapidly, Apple developers added more modules in Objective-C than in Swift (528 vs. 316), demonstrating the power of inertia and status quo.

People ❤️

New data: What developers look for in future job opportunities

I was surprised that the survey found salary transparency as the second most important factor for employers. I suspect that that’s another way for the developers to say they feel underpaid. I am curious how many decide on that factor.

Communities of Practice

Communities of Practice are similar to guilds that I shared a few issues ago. I found this article insightful as it defined the lifecycle of the communities, complete with shutting down. All good things have an end.

Problematic employers in tech

The author says

people choose to work for companies for many reasons, reasons that an outside observer may not be aware of (nor is it any of their damned business anyway).

While it is simple to argue this way, this position reminds me of Adolf Eichmann. I am not equating working at a tech company with war crimes. But there are shades of gray.