Pour kettle and let steep the gods of tea. I built NewsBlur and Turn Touch.
1144 stories
·
810 followers

Second-guessing the modern web

2 Shares

The emerging norm for web development is to build a React single-page application, with server rendering. The two key elements of this architecture are something like:

  1. The main UI is built & updated in JavaScript using React or something similar.
  2. The backend is an API that that application makes requests against.

This idea has really swept the internet. It started with a few major popular websites and has crept into corners like marketing sites and blogs.

I’m increasingly skeptical of it.

There is a sweet spot of React: in moderately interactive interfaces. Complex forms that require immediate feedback, UIs that need to move around and react instantly. That’s where it excels. I helped build the editors in Mapbox Studio and Observable and for the most part, React was a great choice.

But there’s a lot on either side of that sweet spot.

The high performance parts aren’t React. Mapbox GL, for example, is vanilla JavaScript and probably should be forever. The level of abstraction that React works on is too high, and the cost of using React - in payload, parse time, and so on - is too much for any company to include it as part of an SDK. Same with the Observable runtime, the juicy center of that product: it’s very performance-intensive and would barely benefit from a port.

The less interactive parts don’t benefit much from React. Listing pages, static pages, blogs - these things are increasingly built in React, but the benefits they accrue are extremely narrow. A lot of the optimizations we’re deploying to speed up these things, things like bundle splitting, server-side rendering, and prerendering, are triangulating what we had before the rise of React.

And they’re kind of messy optimizations. Here are some examples.

Bundle splitting.

As your React application grows, the application bundle grows. Unlike with a traditional multi-page app, that growth affects every visitor: you download the whole app the first time that you visit it. At some point, this becomes a real problem. Someone who lands on the About page is also downloading 20 other pages in the same application bundle. Bundle splitting ‘solves’ this problem by creating many JavaScript bundles that can lazily load each other. So you load the About page and what your browser downloads is an ‘index’ bundle, and then that ‘index’ bundle loads the ‘about page’ bundle.

This sort of solves the problem, but it’s not great. Most bundle splitting techniques require you to load that ‘index bundle’, and then only once that JavaScript is loaded and executed does your browser know which ‘page bundle’ it needs. So you need two round-trips to start rendering.

And then there’s the question of updating code-split bundles. User sessions are surprisingly long: someone might have your website open in a tab for weeks at a time. I’ve seen it happen. So if they open the ‘about page’, keep the tab open for a week, and then request the ‘home page’, then the home page that they request is dictated by the index bundle that they downloaded last week. This is a deeply weird and under-discussed situation. There are essentially two solutions to it:

  1. You keep all generated JavaScript around, forever, and people will see the version of the site that was live at the time of their first page request.
  2. You create a system that alerts users when you’ve deployed a new version of the site, and prompt them to reload.

The first solution has a drawback that might not be immediately obvious. In those intervening weeks between loading the site and clicking a link, you might’ve deployed a new API version. So the user will be using an old version of your JavaScript frontend with a new version of your API backend, and they’ll trigger errors that none of your testing knows about, because you’ll usually be testing current versions of each.

And the second solution, while it works (and is what we implemented for Mapbox Studio), is a bizarre way for a web application to behave. Prompting users to ‘update’ is something from the bad old days of desktop software, not from the shiny new days of the web.

Sure: traditional non-SPA websites are not immune to this pitfall. Someone might load your website, have a form open for many weeks, and then submit it after their session expired or the API changed. But that’s a much more limited exposure to failure than in the SPA case.

Server-Side Rendering

Okay, so the theory here is that SPAs are initially a blank page, which is then filled out by React & JavaScript. That’s bad for performance: HTML pages don’t need to be blank initially. So, Server-Side Rendering runs your JavaScript frontend code on the backend, creating a filled-out HTML page. The user loads the page, which now has pre-rendered content, and then the JavaScript loads and makes the page interactive.

A great optimization, but again, caveats.

The first is that the page you initially render is dead: you’ve created the Time To Interactive metric. It’s your startup’s homepage, and it has a “Sign up” button, but until the JavaScript loads, that button doesn’t do anything. So you need to compensate. Either you omit some interactive elements on load, or you try really hard to make sure that the JavaScript loads faster than users will click, or you make some elements not require JavaScript to work - like making them normal links or forms. Or some combination of those.

And then there’s the authentication story. If you do SSR on any pages that are custom to the user, then you need to forward any cookies or authentication-relevant information to your API backend and make sure that you never cache the server-rendered result. Your formerly-lightweight application server is now doing quite a bit of labor, running React & making API requests in order to do this pre-rendering.

APIs

The dream of APIs is that you have generic, flexible endpoints upon which you can build any web application. That idea breaks down pretty fast.

Most interactive web applications start to triangulate on “one query per page.” API calls being generic or reusable never seems to persist as a value in infrastructure. This is because a large portion of web applications are, at their core, query & transformation interfaces on top of databases. The hardest performance problems they tend to have are query problems and transfer problems.

For example: a generically-designed REST API that tries not to mix ‘concerns’ will produce a frontend application that has to make lots of requests to display a page. And then a new-age GraphQL application will suffer under the N+1 query problem at the database level until an optimization arrives. And a traditional “make a query and put it on a page” application will just, well, try to write some good queries.

None of these solutions are silver bullets: I’ve worked with overly-strict REST APIs, optimization-hungry GraphQL APIs, and hand-crafted SQL APIs. But no option really lets a web app be careless about its data-fetching layer. Web applications can’t sit on top of independently-designed APIs: to have a chance at performance, the application and its datasource need to be designed as one.

Data fetching

Speaking of data fetching. It’s really important and really bizarre in React land. Years ago, I expected that some good patterns would emerge. Frankly, they didn’t.

There are decent patterns in the form of GraphQL, but for a React component that loads data with fetch from an API, the solutions have only gotten weirder. There’s great documentation for everything else, but old-fashioned data loading is relegated to one example of how to mock out ‘fetch’ for testing, and lots of Medium posts of varying quality.


Don’t read this as anti-React. I still think React is pretty great, and for a particular scope of use cases it’s the best tool you can find. And I explicitly want to say that – from what I’ve seen – most other Single-Page-Application tools share most of these problems. They’re issues with the pattern, not the specific frameworks used to implement it. React alternatives have some great ideas, and they might be better, but they are ultimately really similar.

But I’m at the point where I look at where the field is and what the alternative patterns are – taking a second look at unloved, unpopular, uncool things like Django, Rails, Laravel – and think what the heck is happening. We’re layering optimizations upon optimizations in order to get the SPA-like pattern to fit every use case, and I’m not sure that it is, well, worth it.

And it should be easy to do a good job.

Frameworks should lure people into the pit of success, where following the normal rules and using normal techniques is the winning approach.

I don’t think that React, in this context, really is that pit of success. A naïvely implemented React SPA isn’t stable, or efficient, and it doesn’t naturally scale to significant complexity

You can add optimizations on top of it that fix those problems, or you can use a framework like Next.js that will include those optimizations by default. That’ll help you get pretty far. But then you’ll be lured by all of the easy one-click ways to add bloat and complexity. You’ll be responsible for keeping some of these complex, finicky optimizations working properly.

And for what? Again - there is a swath of use cases which would be hard without React and which aren’t complicated enough to push beyond React’s limits. But there are also a lot of problems for which I can’t see any concrete benefit to using React. Those are things like blogs, shopping-cart-websites, mostly-CRUD-and-forms-websites. For these things, all of the fancy optimizations are trying to get you closer to the performance you would’ve gotten if you just hadn’t used so much technology.

I can, for example, guarantee that this blog is faster than any Gatsby blog (and much love to the Gatsby team) because there is nothing that a React static site can do that will make it faster than a non-React static site.


But the cultural tides are strong. Building a company on Django in 2020 seems like the equivalent of driving a PT Cruiser and blasting Faith Hill’s “Breathe” on a CD while your friends are listening to The Weeknd in their Teslas. Swimming against this current isn’t easy, and not in a trendy contrarian way.

I don’t think that everyone’s using the SPA pattern for no reason. For large corporations, it allows teams to work independently: the “frontend engineers” can “consume” “APIs” from teams that probably work in a different language and can only communicate through the hierarchy. For heavily interactive applications, it has real benefits in modularity, performance, and structure. And it’s beneficial for companies to shift computing requirements from their servers to their customers browsers: a real win for reducing their spend on infrastructure.

But I think there are a lot of problems that are better solved some other way. There’s no category winner like React as an alternative. Ironically, backends are churning through technology even faster than frontends, which have been loyal to one programming language for decades. There are some age-old technologies like Rails, Django, and Laravel, and there are a few halfhearted attempts to do templating and “serve web pages” from Go, Node, and other new languages. If you go this way, you’re beset by the cognitive dissonance of following in the footsteps of enormous projects - Wikipedia rendering web pages in PHP, Craigslist rendering webpages in Perl - but being far outside the norms of modern web development. If Wikipedia were started today, it’d be React. Maybe?

What if everyone’s wrong? We’ve been wrong before.

Read the whole story
samuel
7 days ago
reply
Cambridge, Massachusetts
Share this story
Delete

Becky Hansmeyer’s WWDC 2020 Wishlist

1 Share

Good list from Becky Hansmeyer. Two of her wishlist items:

A complete redesign of Mail. There is no perfect e-mail client, but like, maybe Apple could try or something?

That’s a huge bite to chew. I don’t think a wholesale redesign is what’s called for, personally, but Mail could certainly use a lot of attention. I mean, just look at Mail on iPad. There’s no way to create a smart mailbox. How are we supposed to take iPad seriously as a computer when its built-in email client doesn’t even support smart mailboxes? Compare and contrast with Safari, which I think does an absolutely brilliant job of balancing features across iOS and MacOS. (Web inspector for iPad would be cool though.)

More home screen customization. Let us have an empty row at the top if we want. Give us some widgets. Allow for some chaos. Set us freeeeee.

Fiddling with the home screen on iOS is just awful. Whenever I sit down and try to clean it up — deleting apps I don’t use, moving apps into some semblance of order — it drives me insane. The 1984 Finder was awesome for rearranging icons, right on day one. Yet we’re 13 years into iOS and rearranging apps is still terrible, because the whole thing is based on a home screen design where there’s just one screen and no third-party apps. The concept worked fine when all you could do was rearrange 12 built-in apps on a single screen. It feels like a prank trying to use it today.

Read the whole story
samuel
9 days ago
reply
Cambridge, Massachusetts
Share this story
Delete

The Coronavirus Outbreak

1 Comment
Read the whole story
samuel
14 days ago
reply
There were two bags of whole wheat pasta!
Cambridge, Massachusetts
kazriko
14 days ago
There's a ton of flour in our stores here. I grabbed another 5 pound bag yesterday of the local brand we prefer.
Share this story
Delete

Bye, Amazon

2 Comments and 6 Shares

May 1st was my last day as a VP and Distinguished Engineer at Amazon Web Services, after five years and five months of rewarding fun. I quit in dismay at Amazon firing whistleblowers who were making noise about warehouse employees frightened of Covid-19.

What with big-tech salaries and share vestings, this will probably cost me over a million (pre-tax) dollars, not to mention the best job I’ve ever had, working with awfully good people. So I’m pretty blue.

What happened

Last year, Amazonians on the tech side banded together as Amazon Employees for Climate Justice (AECJ), first coming to the world’s notice with an open letter promoting a shareholders’ resolution calling for dramatic action and leadership from Amazon on the global climate emergency. I was one of its 8,702 signatories.

While the resolution got a lot of votes, it didn’t pass. Four months later, 3,000 Amazon tech workers from around the world joined in the Global Climate Strike walkout. The day before the walkout, Amazon announced a large-scale plan aimed at making the company part of the climate-crisis solution. It’s not as though the activists were acknowledged by their employer for being forward-thinking; in fact, leaders were threatened with dismissal.

Fast-forward to the Covid-19 era. Stories surfaced of unrest in Amazon warehouses, workers raising alarms about being uninformed, unprotected, and frightened. Official statements claimed every possible safety precaution was being taken. Then a worker organizing for better safety conditions was fired, and brutally insensitive remarks appeared in leaked executive meeting notes where the focus was on defending Amazon “talking points”.

Warehouse workers reached out to AECJ for support. They responded by internally promoting a petition and organizing a video call for Thursday April 16 featuring warehouse workers from around the world, with guest activist Naomi Klein. An announcement sent to internal mailing lists on Friday April 10th was apparently the flashpoint. Emily Cunningham and Maren Costa, two visible AECJ leaders, were fired on the spot that day. The justifications were laughable; it was clear to any reasonable observer that they were turfed for whistleblowing.

Management could have objected to the event, or demanded that outsiders be excluded, or that leadership be represented, or any number of other things; there was plenty of time. Instead, they just fired the activists.

Snap!

At that point I snapped. VPs shouldn’t go publicly rogue, so I escalated through the proper channels and by the book. I’m not at liberty to disclose those discussions, but I made many of the arguments appearing in this essay. I think I made them to the appropriate people.

That done, remaining an Amazon VP would have meant, in effect, signing off on actions I despised. So I resigned.

The victims weren’t abstract entities but real people; here are some of their names: Courtney Bowden, Gerald Bryson, Maren Costa, Emily Cunningham, Bashir Mohammed, and Chris Smalls.

I’m sure it’s a coincidence that every one of them is a person of color, a woman, or both. Right?

Let’s give one of those names a voice. Bashir Mohamed said “They fired me to make others scared.” Do you disagree?

Adjectives

Here are some descriptive phrases you might use to describe the activist-firing.

  1. “Chickenshit.”

  2. “Kill the messenger.”

  3. “Never heard of the Streisand effect.”

  4. “Designed to create a climate of fear.”

  5. “Like painting a sign on your forehead saying ‘Either guilty, or has something to hide.’”

Which do you like?

What about the warehouses?

It’s a matter of fact that workers are saying they’re at risk in the warehouses. I don’t think the media’s done a terribly good job of telling their stories. I went to the video chat that got Maren and Emily fired, and found listening to them moving. You can listen too if you’d like. Up on YouTube is another full-day videochat; it’s nine hours long, but there’s a table of contents, you can decide whether you want to hear people from Poland, Germany, France, or multiple places in the USA. Here’s more reportage from the NY Times.

It’s not just workers who are upset. Here are Attorneys-general from 14 states speaking out. Here’s the New York State Attorney-general with more detailed complaints. Here’s Amazon losing in French courts, twice.

On the other hand, Amazon’s messaging has been urgent that they are prioritizing this issue and putting massive efforts into warehouse safety. I actually believe this: I have heard detailed descriptions from people I trust of the intense work and huge investments. Good for them; and let’s grant that you don’t turn a supertanker on a dime.

But I believe the worker testimony too. And at the end of the day, the big problem isn’t the specifics of Covid-19 response. It’s that Amazon treats the humans in the warehouses as fungible units of pick-and-pack potential. Only that’s not just Amazon, it’s how 21st-century capitalism is done.

Amazon is exceptionally well-managed and has demonstrated great skill at spotting opportunities and building repeatable processes for exploiting them. It has a corresponding lack of vision about the human costs of the relentless growth and accumulation of wealth and power. If we don’t like certain things Amazon is doing, we need to put legal guardrails in place to stop those things. We don’t need to invent anything new; a combination of antitrust and living-wage and worker-empowerment legislation, rigorously enforced, offers a clear path forward.

Don’t say it can’t be done, because France is doing it.

Poison

Firing whistleblowers isn’t just a side-effect of macroeconomic forces, nor is it intrinsic to the function of free markets. It’s evidence of a vein of toxicity running through the company culture. I choose neither to serve nor drink that poison.

What about AWS?

Amazon Web Services (the “Cloud Computing” arm of the company), where I worked, is a different story. It treats its workers humanely, strives for work/life balance, struggles to move the diversity needle (and mostly fails, but so does everyone else), and is by and large an ethical organization. I genuinely admire its leadership.

Of course, its workers have power. The average pay is very high, and anyone who’s unhappy can walk across the street and get another job paying the same or better.

Spot a pattern?

At the end of the day, it’s all about power balances. The warehouse workers are weak and getting weaker, what with mass unemployment and (in the US) job-linked health insurance. So they’re gonna get treated like crap, because capitalism. Any plausible solution has to start with increasing their collective strength.

What’s next?

For me? I don’t know, genuinely haven’t taken time to think about it. I’m sad, but I’m breathing more freely.

Read the whole story
samuel
19 days ago
reply
An incredibly brave stance. Tim Bray's essay on resigning from Amazon is a moral lesson for how to stand up for what's right. Quite a lot of people are now focused on worker rights at this 21st century bohemeth.
Cambridge, Massachusetts
vitormazzi
19 days ago
reply
Brasil
Share this story
Delete
1 public comment
Belfong
14 days ago
reply
Dignity or $1M dollar... very difficult choice.
malaysia

Raspberry Pi High Quality Camera — 12 MP Sensor and Interchangeable Lenses

5 Comments

Les Pounder, writing for Tom’s Hardware:

The Raspberry Pi Camera Module is one of those add ons that we love to play with. Creating images and videos using a $35 Raspberry Pi in real time is still mind blowing for most. You can even use your Raspberry Pi as a PC webcam. But the two previous first-party camera modules have suffered with a fixed focus, albeit good quality, lens and fragile construction.

Enter the Raspberry Pi High Quality Camera, a new module that ups the image quality with a new 12-MP sensor and supports interchangeable lenses and tripod-mounting. The module is larger and, at $50 without any of the required lenses, quite a bit more expensive than prior models, but the increased resolution and flexibility make it a great choice for photography-intensive projects.

With so much of the computer industry moving away from hobbyist tinkering, Raspberry Pi is a delightful exception. I don’t know what I’d do with this but I want to do something.

Read the whole story
samuel
24 days ago
reply
My thesis uses a few dozen Raspberry PI cameras. I'm going to be showing it off in two weeks but if you want to take a peek it's at https://comfortmaps.com.
Cambridge, Massachusetts
Share this story
Delete
3 public comments
hansolosays
24 days ago
reply
Maybe its time to build that home photo booth...
Norfolk, Virginia
duerig
24 days ago
reply
I have lots of book scanner ideas. Pity it was sold out instantly. Hopefully I can get a couple some time in the next few months.

I keep hoping they will really jump in resolution. 20MP or higher cameras would be amazing.
jepler
24 days ago
reply
"I don’t know what I’d do with this but I want to do something."
Earth, Sol system, Western spiral arm

Will the post-plague world be a better place for rich people?

1 Comment and 2 Shares

A year ago I went to Disney World with a rich friend (post). He paid up $8,000 for two days of a VIP guide who enabled us to cut the lines. It still wasn’t that nice, however, due to the Times Square-ish crowds everywhere when we weren’t on a ride. I proposed the idea of a “crowd hater day” at each park each week where the ticket price would be 2X so that there would be some breathing room.

Every aspect of our Disney experience was ideal for spreading coronavirus. Even with the VIP guide we were jammed into crowds periodically. Dining was a mob scene. Shopping was mobbed.

I wonder if the parks will reopen with a capacity set by the tyrannical government at some level that seems unlikely to set off the next epidemic. If so, the most logical method for rationing the remaining tickets will be price. And Disney will have to raise the prices to get a similar level of revenue (since their overall expenses will be similar). So the parks will actually become a lot nicer for anyone who can afford $300/person for a ticket.

How about restaurants? If they have to cut capacity to 25 or 50 percent, as some of the reopened ones in various states are being ordered to do, again it seems as though though they’ll have to raise prices. So rich people will experience a negligible (to them) price penalty and a huge bonus in terms of peace and quiet for conversation, reduced waiting times for a table, etc.

Getting to Disney? I am dreaming that airlines won’t ever be able to sell the middle seats anymore! As long as they bump prices by 50 percent, though, they can still have the same revenue with somewhat richer customers. Spirit prices are essentially $0 from the perspective of a rich American. It shouldn’t be a problem to pay 1.5 * $0.

Readers: What do you think? If everything has to be de-crowded and prices consequently raised, isn’t that actually a good thing from the perspective of the richest 5-10 percent of Americans?

Read the whole story
samuel
25 days ago
reply
This is a near future I can see happening
Cambridge, Massachusetts
kazriko
25 days ago
I wonder if they could do capsule aircraft. Instead of having people sitting jammed together in seats, they could be in tubes in the wall, each with a separate air circulation system, sterilizing the air before it goes into each capsule. You could probably get the same density out of something like that, but people would be laying down with a book or laptop instead of sitting.
betajames
23 days ago
reply
Michigan
Share this story
Delete
Next Page of Stories