GioCities

blogs by Gio

Reddit: Your API *IS* Your Product

  • Posted in 🖱 cyber

Reddit is going the same route as Twitter by making “API access” prohibitively expensive. This is something they very famously, very vocally said they would not do, but they’re doing it anyway. This is very bad for Reddit, but what’s worse is it’s becoming clear that companies think that this is a remotely reasonable thing to do, when it’s very critically not.

It’s the same problem we see with Twitter and other late-capitalist hell websites: Reddit’s product is the service it provides, which is its API. The ability for users to interact with the service isn’t an auxiliary premium extra, it’s the whole caboodle!

I’ll talk about first principles first, and then get into what’s been going on with Reddit and Apollo. The Apollo drama is very useful in that it directly converts the corporate bullshit that sounds technical enough to make sense into something very easy to understand: a corporation hurting them, today, for money.

The API is the product🔗

Reddit and all these other companies who are making user-level API access prohibitively expensive have forgotten that the API is the product. - The API is the interface that lets you perform operations on the site. The operations a user can do are the product, they’re not auxiliary to it!

“Application programming interface” is a very formal, internal-sounding term for a system that is none of those things. The word “programming” in the middle comes from an age where using a personal computer at all was considered “programming” it.

What an API really is a high-level interface to the web application that is Reddit. Every action a user can take — viewing posts, posting, voting, commenting — goes from the app (which interfaces with the user) to the API (which interfaces with the Reddit server), gets processed by the server using whatever-they-use-it-doesn’t-matter, and the response is sent back to the user.

The API isn’t a god mode and it doesn’t provide any super-powers. It doesn’t let you do anything you can’t do as a user, as clearly evidenced by the fact that all the actions you do on the Reddit website go through the API too.

The Reddit website, the official Reddit app, and the Apollo app all interface with the user in different ways and on different platforms, but go through the same API to interact with what we understand as “Reddit”. The fact that the API is the machine interface without the human interface should also concisely explain why “API access” is all Apollo needs to build its own app.

Right now, you can view the announcement thread at https://www.reddit.com/r/apolloapp/comments/144f6xm/apollo_will_close_down_on_june_30th_reddits/, and you can view the “API” data for the same thread at https://www.reddit.com/r/apolloapp/comments/144f6xm/apollo_will_close_down_on_june_30th_reddits.json. It’s not very fun to look at, but it’s easy to tell what you’re looking at: the fundamental representation of the page without all the trappings of the interface.

Public APIs are good for both the user and the company. They’re a vastly more efficient way for people to interact with the service than by automating interaction (or “scraping”). Having an API cuts out an entire layer of expense that, without an API, Reddit would pay for.

The Reddit service is the application, and you interface with it through WHATEVER. Whatever browser you want, whatever browser extensions you want, whatever model phone you want, whatever app you want. This is fundamentally necessary for operability and accessibility.

The API is the service. The mechanical ability to post and view and organize is what makes Reddit valuable, not its frontend. Their app actually takes the core service offering and makes it less attractive to users, which is why they were willing to pay money for an alternative!

Premium APIs🔗

In the case of the “premium API”, the company (Reddit) provides access to its service, at a cost proportional to the service. This is solid stuff! Instead of being directly dependent on advertising revenue, Reddit can charge someone based on how much they use Reddit’s service (and therefore how much money they cost Reddit).

Usually this is the app developer, not the user. For example, you might pay Apollo $10/yr, and then Apollo pays Reddit based on Apollo’s users’ total usage (so individuals don’t have to worry about counting pages, and instead the math works out in the aggregate).

But that price has to be reasonable. It should approximate the opportunity cost of those users using Reddit’s interface and looking at Reddit’s ads, not twenty times that.

What Reddit (the company) is effectively doing is cutting off access from Reddit (the service/product) unless people go through Reddit (the aggressively bad app). It’s a policy decision that forgets that Reddit is infrastructure and instead tries to make it a nebulous app-product that monopolizes control over every facet of the interacting, no matter what sandpaper-bottleneck that means shoving their users through.

Net Neutrality🔗

This issue is somewhat parallel to the core issues of net neutrality: infrastructure providers should not be permitted to vastly advantage themselves over other players who want to use the same pipes. I bring this up here because Reddit, and CEO Steve Huffman in particular, are (on paper) strong advocates of net neutrality.

The argument goes that internet service providers who also sell services on that same network shouldn’t be allowed to force traffic to their own services by using their ownership of infrastructure components to discourage competition. Users can’t access services without network infrastructure that it’s infeasible for them to own themselves, so anyone with leverage over those key infrastructure components must be neutral rather than preferential.

But what Reddit’s doing here is the same abuse of the same advantage for the same profit. They control the API, so they own the pipes.

API bottleneck/Net Neutrality comparison

In the case of the API-as-gatekeeper, it’s even worse, because Reddit owns the client, the pipes, and the service. Even in AT&T’s anarchocapitalist fantasy where they can use their infrastructure ownership to discriminate against competing content, they still don’t own your web browser! But Reddit’s asserting that it does, it gets to, and it’s fine when it does it.

Why? AI🔗

But the reason Reddit is going this direction now isn’t just so it can inflate its value for its upcoming IPO by squeezing its users. In fact, I’d be willing to bet Reddit sees its users as “acceptable losses” rather than as its intended target. What Reddit is trying to cash in on here is the AI gold rush.

Generative text products like ChatGPT are based on huge corpuses of human conversational speech, and their quality directly depends on the quality of those sources. Reddit, meanwhile, is sitting on a treasure trove of real data on modern human conversations, and even metadata about what constitutes high-quality responses in the form of upvotes and downvotes.

Because of the enormous volumes of data required, text models have to use official, high-volume API endpoints in order to gather data in a reasonable manner. They can’t just “scrape” the site by browsing it at a comparable pace to a user, they need vast quantities of data in bulk. That’s exactly the kind of access a premium API is perfect for.

This is what Reddit is trying to cash in on; the value of AI products depend on their sources, so Reddit feels like if they can just hoover up all the data from Reddit for free, they’re being cheated. People have already posted all this information on Reddit, so it’s not a matter of violating users’ privacy; what Reddit sees as the problem is AI startups using Reddit’s product (access and interface to these communities) for profit while cutting Reddit out.

The problem is they’re being greedy and doing this at the direct expense of their users. In its haste to squeeze the AI industry for money, Reddit is starting with that defensive, feeling-cheated posture and lashing out at its entire ecosystem. All it would have taken was working with developers, tweaking scale parameters, and finding a reasonable cost point, and Reddit could have had everything. But it’s losing everything instead.

The story of Apollo (so far)🔗

I don’t want to focus too much on the story, because this story is just one example of a much more important truth, but here’s what’s been happening with Reddit:

Apollo is a (paid) third-party Reddit client for iPhone developed by Christian Selig, and it’s very good. It’s much better than Reddit’s app, and Reddit’s mobile website isn’t really an option so much as an advertisement for its mobile app. Apollo is the app I used.

On April 18, Reddit CTO Christopher Slowe posted An Update Regarding Reddit’s API, which included talk about a new premium API:

We are introducing a premium access point for third parties who require additional capabilities, higher usage limits, and broader usage rights. Our Data API will still be open for appropriate use cases and accessible via our Developer Platform.

Reddit will limit access to mature content via our Data API as part of an ongoing effort to provide guardrails to how sexually explicit content and communities on Reddit are discovered and viewed. (Note: This change should not impact any current moderator bots or extensions.)

Christian Selig immediately expressed concerns. How did the proposed pricing changes affect Apollo? He called Reddit, and got reassuring answers.

Christian Selig, “📣 Had a few calls with Reddit today about the announced Reddit API changes that they’re putting into place …”:

I had two calls with Reddit today where they explained things and answered my questions.

Here’s a bullet point synopsis of what was discussed that should answer a bunch of questions. Basically, changes be coming, but not necessarily for the worse in all cases, provided Reddit is reasonable.

  • Offering an API is expensive, third party app users understandably cause a lot of server traffic
  • Reddit is moving to a paid API model for apps. The goal is not to make this inherently a big profit center, but to cover both the costs of usage, as well as the opportunity costs of users not using the official app (lost ad viewing, etc.)
  • The API cost will be usage based, not a flat fee, and will not require Reddit Premium for users to use it, nor will it have ads in the feed. Goal is to be reasonable with pricing, not prohibitively expensive.
  • How much will this usage based API cost? It is not finalized yet, but plans are within 2-4 weeks
  • They seek to make these changes while in a dialog with developers
  • This is not an immediate thing rolling out tomorrow, but rather this is a heads up of changes to come

My thoughts: I think if done well and done reasonably, this could be a positive change (but that’s a big if). If Reddit provides a means for third party apps to have a stable, consistent, and future-looking relationship with Reddit that certainly has its advantages, and does not sound unreasonable, provided the pricing is reasonable.

I’m waiting for future communication and will obviously keep you all posted. If you have more questions that you think I missed, please post them and I’ll do my best to answer them and if I don’t have the answer I’ll ask Reddit.

  • Christian

There are a few ways this is framed (by Reddit) I object to. Third party app users don’t “cause” a lot of API traffic, because API use and vanilla application use aren’t two different things. Reddit has a pool of users, which are split between first-party and third-party apps. The API doesn’t somehow cause that traffic, it’s just where that traffic is allocated currently. Traffic isn’t caused by the API, it’s caused by people. Again, the API isn’t an extra thing, it is the product.

I actually think there’s something very clever in there, which is that the paid API would logically be ad-free. This makes complete sense: you’re paying for access to the content directly, so it doesn’t need to be ad-subsidized. But, more importantly, Reddit can’t control the user-interface: if the API pushed ads as part of the feed, Apollo could just choose not to display them. Instead of fighting that fight, Reddit is including that cost in the developer agreement, which I think is extremely solid.

The immediate fear was, of course, Reddit doing the same thing Twitter did under Elon Musk, and set API pricing to be prohibitively expensive, but on that same call Reddit specifically called this behaviour out as something it promised not to do:

Reddit: “I think one thing that we have tried to be very, very, very intentional about is we are not Elon, we’re not trying to be that, we’re not trying to go down that same path. […] We are trying to do is just use usage-based pricing, that will hopefully be very transparent to you, and very clear to you. Or we’re not trying to go down the same path that you may have seen some of our other peers go down.”

Everything “sounded pretty (in theory) reasonable”. The general sentiment continued to be “Reddit’s been great and continues to be great, they have a dedicated API team and calls with them have had very good vibes. They seem to have a genuine appreciation for developers, while also understanding screwing them and apps over is a loss for everyone, Reddit included.“

But this optimism only lasted a few weeks. When Reddit finally disclosed the price they were asking for that data, it turned out to be entirely unreasonable.

ChristianSelig, “📣 Had a call with Reddit to discuss pricing. Bad news for third-party apps, their announced pricing is close to Twitter’s pricing, and Apollo would have to pay Reddit $20 million per year to keep running as-is.” - May 31, 2023:

I’ll cut to the chase: 50 million requests costs $12,000, a figure far more than I ever could have imagined.

Apollo made 7 billion requests last month, which would put it at about 1.7 million dollars per month, or 20 million US dollars per year. Even if I only kept subscription users, the average Apollo user uses 344 requests per day, which would cost $2.50 per month, which is over double what the subscription currently costs, so I’d be in the red every month.

I’m deeply disappointed in this price. Reddit iterated that the price would be A) reasonable and based in reality, and B) they would not operate like Twitter. Twitter’s pricing was publicly ridiculed for its obscene price of $42,000 for 50 million tweets. Reddit’s is still $12,000. For reference, I pay Imgur (a site similar to Reddit in user base and media) $166 for the same 50 million API calls.

As for the pricing, despite claims that it would be based in reality, it seems anything but. …
For Apollo, the average user uses 344 requests daily, or 10.6K monthly. With the proposed API pricing, the average user in Apollo would cost $2.50, which is is 20x higher than a generous estimate of what each users brings Reddit in revenue.

While Reddit has been communicative and civil throughout this process with half a dozen phone calls back and forth that I thought went really well, I don’t see how this pricing is anything based in reality or remotely reasonable. I hope it goes without saying that I don’t have that kind of money or would even know how to charge it to a credit card.

So Reddit’s doing its heel turn and trying to squeeze Apollo out. Even if Apollo wanted to scale its subscription cost up to pay Reddit’s demands, the timetable Reddit is imposing doesn’t allow that.

And so Christian Selig announced Apollo will close down on June 30th. Not dead, but murdered.

Christian Selig Just to be clear about how wrong and out of touch that is, without naming names, a formerly very, very high up person at Twitter messaged me on Twitter and said:

“The Reddit api moves are crazy. I’m not sure what choices you have but to move to another network. […] That pricing is designed to prevent apps like yours forevermore.”

So to be clear, even this person thinks this pricing is unreasonable. I do too.

Apollo wasn’t the only one: this update killed all the major Reddit apps, including rif, ReddPlanet, and Sync.

lupeski The hurdles placed on third party apps by reddit just aren’t a feasible obstacle to overcome. With the removal of explicit content, the unreasonable pricing structure, short time period, lack of responsiveness, and unwillingness to come to a middle ground, reddit has shown that their intent is to extinguish all 3rd parties. As unfortunate as it is, I don’t want to be somewhere that I’m not welcome. Reddit has made it clear that I’m not welcome here.

Reddit directing extreme, overt hostility toward app developers🔗

But there’s money to be had at users’ expense. Blood money, sure, but corporate money, for the finances, which means anything goes.

Again, I really have to thank Reddit CEO Steve Huffman for this one, because without his down-to-earth lying, backstabbing, and cruelty, it’s hard to demonstrate the real ugliness of what’s going on here. But he came in and did the work for me.

file photo of big-boy CEO Steve Huffman Steve Huffman (file photo)

Attacking individuals🔗

Let’s start some stories from Christian Selig’s shutdown announcement:

Bizarre allegations by Reddit of Apollo “blackmailing” and “threatening” Reddit

About 24 hours after that call with Reddit, I received this odd message on Mastodon:

“Can you please comment publicly about the internal Reddit claim that you tried to “blackmail” them for a $10,000,000 payout to “stay quiet”?”

Then yesterday, moderators told me they were on a call with CEO Steve Huffman (spez), and he said the following per their transcript:

Steve: “Apollo threatened us, said they’ll “make it easy” if Reddit gave them $10 million.”

Steve: “This guy behind the scenes is coercing us. He’s threatening us.”

Wow. Because my memory is that you didn’t take it as a threat, and you even apologized profusely when you admitted you misheard it. It’s very easy to take a single line and make it look bad by removing all the rest of the context, so let’s look at the full context.

I can only assume you didn’t realize I was recording the call, because there’s no way you’d be so blatantly lying if you did.

As said, a common suggestion across the many threads on this topic was “If third-party apps are costing Reddit so much money, why don’t they just buy them out like they did Alien Blue?” That was the point I brought up. If running Apollo as it stands now would cost you $20 million yearly as you quote, I suggested you cut a check to me to end Apollo. I said I’d even do it for half that or six months worth: $10 million, what a deal!

The bizarre thing is - initially - on the call you interpreted that as a threat. Even giving you the benefit of the doubt that maybe my phrasing was confusing, I asked for you to elaborate on how you found what I said to be a threat, because I was incredibly confused how you interpreted it that way. You responded that I said “Hey, if you want this to go away…” Which is not at all what I said, so I reiterated that I said “If you want to Apollo to go quiet, as in it’s quite loud in terms of API usage”.

What did you then say?

Me: “I said ‘If you want Apollo to go quiet’. Like in terms of- I would say it’s quite loud in terms of its API usage.”

Reddit: “Oh. Go quiet as in that. Okay, got it. Got it. Sorry.”

Reddit: “That’s a complete misinterpretation on my end. I apologize. I apologize immediately.”

The admission that you mistook me, and the four subsequent apologies led me to believe that you acknowledged you mistook me and you were apologetic. The fact that you’re pretending none of this happened (or was recorded), and instead espousing a different reality where instead of apologizing for taking it as a threat, you’re instead going the complete opposite direction and saying “He threatened us!” is so low I almost don’t believe it.

But again, I’ve recorded all my calls with you just in case you tried something like this.

Transcript of this part of the call: https://gist.github.com/christianselig/fda7e8bc5a25aec9824f915e6a5c7014

Audio of this part of the call: http://christianselig.com/apollo-end/reddit-third-call-may-31-end.m4a

(If you take issue with the call being recorded please remember that I’m in Canada and so long as one participant in the call (me) consents to being recorded, it’s legal. If anyone would like the recording of the full call, I’m happy to provide.)

I bring this up for two reasons:

  • I don’t want Reddit slandering me to internal employees or public people by saying I threatened them when they reality is that they immediately apologized for misunderstanding me.
  • It shows why I’ve finally come to the conclusion that I don’t think this situation is recoverable. If Reddit is willing to stoop to such deep lows as to slander individuals with blatant lies to try to get community favor back, I no longer have any faith they want this to work, or ever did.

And one more:

Claims that Apollo has made no attempt to be a good user of the API

On the call with moderators, Steve Huffman said:

Steve: “I don’t use the app, so I’ll give you the best answer I can — he does scraping so that he can deliver notifications faster, but has done NO EFFORT to be a good citizen of the internet.”

First off, Apollo does no scraping, it’s purely through authenticated calls to the API and has checks in place to ensure it stays within Reddit’s API rate limits. I’ve open sourced the server code to show this.

Secondly, to say we have made no effort is categorically false. I have so many emails where I’ve reached out to Reddit expressing concerns about and bugs inefficiencies in the API, or ideas on how to improve things, or significant Reddit bugs that made things hard on us. When Reddit has had questions for us, as discussed above, we immediately jumped into action to get an answer as quickly as possible.
…
Up until a week ago, the stated Reddit API rate limits that apps were asked to operate within was 60 requests per minute per user. That works out to a total of 86,400 per day. Reddit stated that Apollo uses 345 requests per user per day on average, which is also in line with my findings. Thats 0.4% of the limit Reddit was previously imposing, which I would say is quite efficient.

This is the stuff that leaves me drooling at the mouth. CEO Steve told at least two specific, slanderous lies about Christian, all of which Christian was able to specifically, demonstrably disprove.

When confronted about this, Steve himself had this… exchange to have about it:

Artillect What were you thinking with your attempt to discredit Apollo by claiming that Christian threatened and blackmailed you? The confusion was sorted out during Christian’s call with Reddit, yet you proceeded to claim that he blackmailed Reddit the following week. To me (and the rest of Reddit) it comes across as a blatant attempt to pit us against him.

Steve Huffman: His “joke” is the least of our issues. His behavior and communications with us has been all over the place—saying one thing to us while saying something completely different externally; recording and leaking a private phone call—to the point where I don’t know how we could do business with him.

Christian Selig: Please feel free to give examples where I said something differently in public versus what I said to you. I give you full permission.

Absolutely incredible stuff. A oscillation between opacity and bullshit from Steve, and then when Christian has to bring out his records of conversations he’s being slandered about, Steve tries to twist his taking steps to defend himself from Steve as a justification for the attack in the first place. Vile behaviour.

And he’s still telling more lies, about everyone in spitting distance:

OkieWonBenobi Apologies if this has been asked already, but I know there’s been a request from the mods of some subreddits to delay the API pricing implementation by 90 days. It seems to me this would help developers and reddit both bridge some of the gaps between 3rd party apps and native reddit apps. This is a pretty big issue for mods and users on many fronts, and is leading to a good deal of pressure for subreddits to join the blackout. As a mod of AmItheAsshole in particular, I don’t know if we’ll be able to justify keeping our sub open without a clear commitment on a delay. Can you promise that?

spez We’re continuing to work with folks who want to work with us. For what it’s worth, this includes many of the apps that haven’t been taking the spotlight this week.

lupeski r/ReddPlanet developer here…I’ve attempted multiple times to get in contact with Reddit regarding these changes. Every attempt has been ignored. This is a blatant lie.

Setting infeasible timetables🔗

But it’s not just one person’s interpersonal ugliness, this same evil inherent to the API policy decision is clearly visible in Reddit’s policy decisions regarding the schedule, too.

On that first call, Reddit communicated that “this is not an immediate thing rolling out tomorrow, but rather this is a heads up of changes to come”.

But that’s not at all what happened. Reddit (while taking weeks to answer emails!) turned around and demanded Apollo implement and pay for the new $12,000 API within one month.

Christian Selig As a comparison, when Apple bought Dark Sky and announced a shut down of their API, knowing that this API was at the core of many businesses, they provided 18 months before the API would be turned off. When the 18 months came, they ultimately extended it another 12 months, resulting in a total transition period of 30 months. While I’m not asking for that much, Reddit’s in comparison is 30 days.

And, again, from Christian Selig’s shutdown announcement

I’ve seen a lot of questions along the lines of: “What if Reddit gives you a deadline extension because of this post and posts by other developers?” and that’s something I truly would have loved for them to have made an effort to communicate earlier. You can’t give developers 30 days between when the pricing is announced and when they will start incurring charges, and also wait a week (25% of the time we’re given) between replying to emails without so much as a “we hear you’re concerned about the short timeline and looking into what we can do”. In conjunction with your previous emails, it just appears like you’ve stopped any desire to communicate with developers, in a period where we have a serious, expensive deadline looming with not that much time to wind down our apps.

And I also just know if I sent another email saying “I’m going to post tomorrow that Apollo is shutting down unless you do something about the timeline”, it would be construed as a threat.

Even more than that, Reddit’s behavior has been so appalling that for any developer I’ve talked to it’s completely erased the indication that they even want us around.

It’s a true showcase of this intense hostility Reddit is eager to have towards someone who, if they were being remotely honest about the situation, should be one of their biggest customers! Imagine someone telling you they wanted to pay you cash, directly, in order to use your free website, and deciding that guy was public enemy number one? It’s utterly perverse.

Conclusion🔗

Just understand. Understand APIs as basic interfaces with your interaction with the web. Understand the user-agent and how the client isn’t the same thing as the service. Understand the power. It’s all, all about the power.

Howdy! If you found my writing worthwhile, the best thing you can do to support me is to share an article you found interesting somewhere you think people will appreciate it. Thanks as always for reading!

Comments

Loading...