Archived: The tech behind Artifact, the newly launched news aggregator from Instagram's co-founders

Late last month, Artifact, a personalized news reader built by Instagram’s co-founders, opened to the public. The launch was a surprise to many consumers, who wondered why the team behind one of the world’s most iconic social apps would return to startups to focus on one of the toughest areas instead: news. It’s an ecosystem where publishers are failing left and right and misinformation is rampant, as the founders surely saw themselves while working at Facebook.

In an interview, Instagram co-founder Kevin Systrom explains what drew him to this space and how his new app’s underlying technology will serve to differentiate it from the competition.

The startup’s existence, in some ways, can be credited to the COVID pandemic, as that’s what brought the Instagram co-founders, Systrom and Mike Krieger, back together. During the early days of the pandemic in 2020, the co-founders teamed up for the first time since leaving Facebook to create a COVID-19 dashboard called Rt.live that tracked the spread of the virus across the U.S.

But in later months, other COVID trackers emerged and people were no longer as interested in tracking the virus’s spread on a state-by-state basis. Invigorated by the experience, the founders began then to think about returning to work on a “real company” instead of a public good side project.

Machine learning is the “future of social”

Deciding on their next act took time. The founders hacked around on stuff for a year and a half or so after their initial collaboration, Systrom says. The founder, who describes himself as a “very frameworks-driven person,” knew he wanted to do something that involved machine learning, having seen its power at Instagram.

“It was super fascinating to me to work on it at Instagram and watch it go from a company without machine learning to a company driven by machine learning,” he says. “I also witnessed the rise of TikTok and realized how important machine learning would be for the future of social.”

He himself observed how social networking’s underlying graph had changed a lot over the years, watching as Facebook invented what’s now known as the “friend graph” — a user’s personal social network of real-life connections. Later, he saw Twitter pioneer the “follow graph,” or a graph of connections based on the user’s explicit choices of who they want to follow on a service. Then, at Instagram, Systrom saw firsthand the shift from the “follow graph” to the “inferred graph” or, rather, the “interest graph.”

This, he explains, was basically a “follow graph” powered by machine learning, instead of by users clicking a button.

The potential to leverage machine learning and an interest graph within a new product appealed to him, he says.

“We looked for an area that was social in nature, but where we could apply 20% new techniques — and that would be the machine learning side of what we’re doing,” Systrom says, describing how the founders narrowed their focus.

Fixing a broken news ecosystem

The other consideration that prompted the founders’ interest in news, more specifically, was the potential impact if the app succeeded. They wanted to work on a startup they felt the world needed, Systrom says. No doubt, some of that interest may have been fueled by working at Facebook (now Meta), which had changed consumers’ news consumption behavior, impacting publishers as well as the spread of misinformation.

“It felt like our consumption of information — both factual, balanced, entertaining, etc. — had an existential crisis,” Systrom tells TechCrunch. Plus, he adds, “many of the people producing this content are going out of business.”

Meanwhile, on the consumer side of the news reading experience, there’s so much information swirling around that people don’t know what they can trust or which item to read. People are asking themselves if a link shared by a friend is actually legit and they’re wondering why they’re reading one article over the many others published on the same topic.

“It might be cheesy to say, because I’ve now said it a bunch of times, but I feel like the worst part about social media is that it’s social,” Systrom says. “I think the ‘social’ part of social media — for a long time, in terms of information consumption — has been a hack to filter for information that would be interesting to you. But we now don’t need that hack, because we can learn what’s interesting to you,” he continues. “We can quantify it. We can build profiles. And then we can serve you content that is both high-quality, balanced and interesting to you.”

This realization led to the creation of Artifact, a social news app powered by machine learning.

The app in some ways is very much like others that exist today, which have been founded in other countries, including ByteDance’s Toutiao in China, Japan’s SmartNews and News Break, another personalized news reader with Chinese roots. Like its rivals, Artifact learns from user behavior, engagement and other factors in order to personalize which headlines are presented and in which order.

Despite this competitive landscape, Systrom believes U.S. news consumers want an option that’s actually based in their home market.

“They want a domestic player with a team they trust,” Systrom says, speaking to the news aggregation landscape in the U.S.

His comment is a timely one, given how tensions in the U.S. have been growing over China’s grip on the social networking market with TikTok. The short video app, which is often used as a source for news by Gen Z, is now banned on government phones and bipartisan legislation to further police it is in the works.

Clearing out the clickbait

In addition to its locale, Systrom believes Artifact will be able to differentiate itself based on its unique combination of technology and taste — a directive that could also be used to describe Instagram’s founding for that matter.

Unlike Facebook — which became a platform by which any publisher could deliver news, and oftentimes clickbait — Artifact’s news sources are curated up front, the founder explains.

“When I say taste, I mean the top of the funnel in our system — the publishers we choose to distribute,” notes Systrom. “It’s not a free-for-all. We don’t crawl the entire web and just let everything go in.”

Instead, Artifact has selected the top publishers across different categories to fuel the content in the app. Customers can add their own paid subscriptions, as well. At this time, Artifact doesn’t sell those for a revenue share or involve itself in publishers’ ad sales, though one day that could change, depending on how the app chooses to monetize.

For now, however, the focus is on gaining traction with consumers and ensuring the app’s news sources are worth reading.

“The line, internally…is we want a balanced ideological corpus, subject to integrity and quality,” Systrom says. “And the idea is not that we only choose left-wing, or we only choose right-wing. We drew the line at quality and integrity subject to a bunch of the metrics that a lot of these third-party fact-checking services have. The third-party services basically rate the integrity of different publishers based on their research and based on public events — like how quickly they correct their stories, whether their funding is transparent, all that kind of stuff,” he notes.

“You can claim it’s editorial,” he continues. “But it’s more about making sure that the set that we’re going to distribute and pour into this powerful machine — that can distribute content widely — that we’re being responsible by giving it content that isn’t going to be misinformation.”

Beyond the integrity of reporting, Artifact aims to deliver a news experience that’s more fulfilling.

“If you log on to a lot of these other sources, you get pretty clickbaity-stuff,” Systrom points out. “I’m not trying to throw shade on folks working in this area, but we wouldn’t work on it if we thought that it was solved. We focus a lot in our system on a different objective, which is this idea of value…if we put in the top-quality sources in these categories and we have coverage — whether it’s parenting or mental wellness, or exercise — that we can create a differentiated content set that feels much different than the front page of a major newspaper but also feels very different than a lot of the other aggregators,” he says.

Yet, even as the app personalizes its content selection to the end user, it doesn’t leave them in so-called “filter bubbles,” necessarily, as Facebook did. Instead, when users click on a headline to read a story, they’re shown the entire coverage across sources, allowing them to peruse the story from different vantage points.

Artifact’s recommendation system

Systrom credits Toutiao for driving innovation in recommendation systems, noting that Toutiao essentially helped ByteDance give birth to TikTok. The technology developed for its news discovery was brought to its acquisition of Musical.ly, which became the Chinese app Douyin and its international counterpart TikTok.

But Systrom believes some of the machine learning that Artificat is doing is different.

“The machine learning that a lot of what we’re doing is based on was invented in 2017 at Google. It’s called the transformer…without that, GPT 3, 3.5 etc., wouldn’t exist. Without that, you wouldn’t have DALL-E. Without that, you wouldn’t have ChatGPT,” he says. “You’re inventing a core technology that can then be applied in many different ways.”

The app’s algorithms are focused on more than just tracking clicks and engagement. It weighs other factors, too, like dwell time, read time, shares, stories that get shared in DMs (private messages) and more.

“If you let your algorithm focus on clicks, it will end up serving clickbait. If you simply optimize for only what people have clicked on, you end up having tunnel vision,” Systrom explains.

The model additionally involves an algorithm called Epsilon-Greedy.

“That’s a technical term for you taking some small portion of your time — like 10% or 20% — and you explore. You do something different than you would normally do…you go to the outside of the recommendation spectrum rather than the core of it,” Systrom says. “It’s shown many, many times, especially in reinforcement learning, that having this ‘explore’ budget we’ll call it, actually is optimal for users.”

That’s the same reason why TikTok has begun testing tools that let users refresh their feeds. Without the added spice of unexpected content, the video app’s suggestions had grown stale for some users.

But because the news changes every day, Artifact’s use of this Epsilon-Greedy algorithm also has to adapt as users’ own interests in topics grow and wane. For example, someone might be very into reading about the upcoming elections up until Election Day has passed. Or a new story may immediately capture their attention when it comes out of nowhere, as the story about the Chinese spy balloon did.

Artifact’s editorial consideration, if you can call it that, will also come into how the app’s algorithm is programmed, not just its publisher selections.

“Actually, building the algorithm is enormously editorial,” Systrom says. “Because what you choose to train your algorithm on — the objective function, the data you put in, the data you include, the data you don’t include — is all in editorial judgment. The way you weight different objectives.”

“All of the edge a company has is in its ability to wrangle the data in such a way that produces an outcome that’s optimal,” Systrom says.

If anything is Artifact’s secret sauce, it’s that.

Does Artifact have a future?

Whether all this tech and “taste” is enough for Artifact to succeed remains to be seen. The U.S. news aggregation landscape is not like China’s. And here, Artifact will have to fight against the default ways people access news, including through the news apps that come bundled with their smartphones, as well as the large internet portals like Google and Facebook, and some extent, YouTube and TikTok too.

At launch, Artifact added new functionality, including a new feature that allows users to track how they’ve been engaging with the app and its content in a metrics section, which shows a list of publishers and topics they’ve been reading. Over time, Artifact plans to let users adjust which topics they want to see more and less of, or even block publishers.

They’ll also later be able to socialize through comment threads around the stories themselves, in what could be a stealth competitor to Twitter — an app that’s seen a small exodus in the wake of Elon Musk’s takeover. (In fact, Artifact just added a feature similar to Twitter’s with its ability to show you articles trending in your network, if you sync your contacts to the service.)

As for Artifact’s future, Systrom says he hopes it will eventually become a place where users can go, first and foremost, to discover content around their interests and be able to discuss them with others.

That said, it’s still immeasurably hard for a new consumer app to gain traction without fueling customer acquisition costs with buckets of money. But one thing the team learned from building Instagram, is that Facebook can be a useful tool for gaining adoption. Many of its first users found the app by way of Instagram photos posted to Facebook.

“It turns out that Artifact is actually very similar,” Systrom says. “People discover articles and they want to share them elsewhere…You can share an article from Artifact and it has our branding and it has our domain and URL. It works fairly well for top-of-mind awareness.”

He’s not all that worried that this would have him wading into Facebook’s territory, which to some represents an unbeatable giant.

“I might be a contrarian here, but I think the window has always been open [for new social experiences]. The question is whether or not people choose to attack it and if the right people choose to attack it. I could have never predicted that Snapchat would have risen during Instagram, which obviously had a stronghold on images and social — there was no way to predict there was an opening there. There’s no way to predict that TikTok would have come about when Snap and Instagram were so big. I think these things are far more random than you think.”

These changes tend to be around fundamental changes in technology, like the interest graph, Systrom says.

But, he adds, “I do think we see this wave of machine learning right now that opens up an enormous window to do new things in social.”