The decision to default to the streaming engine is really interesting. My intuition is that this would be slower than other data frame operations that are more parallelizable with batch processing, because streaming engines necessarily process rows sequentially. Is my intuition off/am I overestimating how much auto-parallelization polars does?
Streaming here has a different meaning than perhaps what you're used to. It's not referring to online processing where you maintain aggregates/state while an endless stream of data comes in.
The name was chosen early on to contrast with the old execution model, which was essentially all-data-in-memory, column-at-a-time. That engine still exists, we use it as a fallback mechanism for things that aren't supported yet in the new engine (or if you explicitly ask for `engine="in-memory"`).
The new execution model first constructs a computational graph of nodes which communicate in streams of in-cache batches (morsels) of data, meaning the full dataset will never be held in memory if not necessary. This was called the streaming engine for that reason in an early prototype and the name stuck. In hindsight I do admit the naming choice is somewhat confusing.
Well... once my recent work on out-of-core lands the batch could be on disk when we run out of memory budget ;)
But no, that's not what I meant. I meant that the batch is meant to be of a size that fits in your CPU cache. This can be a huge throughput improvement as each bit of data stays in cache as it moves from data source to sink.
Compare this to column-at-a-time execution: by the time you start the next operation on this column the start of the column will be out of cache again, meaning you operate at RAM speed (or worse, disk speed) rather than cache speed.
I think you’ll also find that the best way to strongly pressure companies to do something is by law. The work that Right to Repair advocates have been doing across the industry is fantastic.
It was (my) main takeaway and I wanted it to be evident in the headline. I didn’t want to bury the lede; I’ll uneditorialize it because it seems it’s causing some fighting in the comments and I don’t really want to instigate anything, more just warn people using the distro. I don’t think it’ll reach the people who need to be warned if it’s not evident that there’s an issue from the start.
Appreciate it! Editorialised is fine but the post is mostly about the development practices leading to bugs & the lack of awareness of how dangerous that is, not about the current absolute number of security holes if that makes sense.
Everyone suggesting that they simply charge users for commits to drive off AI-heavy users forgets that Github is owned by Microsoft, who has a big incentive to keep having developers use AI.
I suspect that Microsoft would even prefer to have Github operate at a loss, if that loss were because all its users were using their models and paying for OpenAI subscriptions to generate the code.
Conversely, what suggests GitHub has a huge operating cost?
Running a GitHub clone at their same scale as a customer on cloud pricing would likely be insane. But y’all know infra is actually quite cheap when you run it yourself right?
It’s usually the case with these M&A deals that the profit just never quite makes sense to justify the purchase price, unless you can truly scale up the user base or revenue model. GitHub was already so mature as a solution when they bought it, I don’t know that they could have added that type of value just by slapping a Microsoft logo in the footer.
Absolute numbers aren’t useful in this analysis. We know capital is easily available for the expenditure. You have to consider it as a relative number to revenue or users otherwise big numbers always look scary.
I agree - however, I was responding to your direct question about whether GitHub's operating cost was high, and there's some data that suggests that it is!
If this is one of their major expenses and only represents 5% of annual revenue, it’s not. Often SaaS companies can run at 90%+ gross margins even with a highly utilized free user tier. Their selling expenses and labor/opex and SG&A run rates can be all over the map but these are extremely controllable and represent the operating strategy being executed more than the cost of service.
I don’t know that industry as well. But I can look at it through a similar lens and would guess that it’s high.
I say that because every thing that goes into the service is expensive. Large capital expenses (planes, probably a lot I don’t even know about), large variable operating costs (fuel, flight specific labor, airport fees, and all the other stuff that is required for every flight). A large portion of their expenses scale with revenue. Meanwhile, there is high revenue pressure (customers are price sensitive and competition is fierce and often a direct substitute).
SaaS is inherently more scalable than practically any other business/industry. Comparing to an industry that is exceedingly financially constrained, often to the point of requiring government subsidies, is frankly a bit silly if you’re trying to make any point here.
Let’s also look at it from a risk of growth perspective to highlight how different things are. If GitHub invests in a new server/rack/datacenter to support growth it likely breaks even once it’s at <20% capacity (or could, lots of variables). Even if it takes longer than they expected to use the new capacity it’s not too big of a deal. The unused part is bought but can be powered down until needed. So it’s not burning a hole in their P&L.
However, what does an airline have to do to rollout service to a new city? They need to buy new planes, sign contracts with the new airport, hire pilots and crew, launch a marketing campaign informing customers to generate demand, and probably some other sunk costs. Once service begins, they want to be at full capacity immediately. They can’t power down 80% of fuel or labor. It will burn cash until they reach a much higher utilization rate. Closing the city is a major public failure for the airline that will be discussed in media, among customers, and could easily lose people their jobs. It’s very risky in comparison is my point.
My point is very simple: operating costs are relative. But you didn't ask the relative question, you asked the absolute question ("what suggests GitHub has a huge operating cost?"). Reasonable people will differ on how to evaluate the answer to that. For example, to a wealthy person, $100 might not be a lot of money. To a poor person, that is a lot of money.
You yourself said: “Absolute numbers aren’t useful in this analysis.”
So perhaps you meant to ask a different question all along.
It used to be, and buying a few servers was a cheap way to get off the cloud (if you can manage them efficiently). But with RAM and other costs these days... I've had to rethink it a lot.
It's at worst ten times as expensive as it should be. But AWS was already ten to a hundred times as expensive, so even in the computer parts crisis, self hosting is still ahead.
Thinking more, the object of the "house" is to extract value, whatever it is by rent or letting them burn it down, as long as the correct dues are paid, the mission is accomplished.
The long view is that MS will sell those developers the tools that slow down or put band-aids on the damage they inflict on their own codebases by using AI.
They could do this in a way that lessens external pull requests.
For example, bolting co-pilot on to github, or a Codex in the web kind of thing that gives unlimited check ins.
It's like how Grok Heavy gives the user X premium or whatever. You charge for the tokens and give the unlimted premium access as a bonus. Basically, bundle it.
Right now anyone can publish to public repos in an unlimited manner. They could choose to limit that and elevate unlimited to a paid co-pilot of codex bundled plan.
The GPL is irrelevant nowadays anyway since everything is MIT (corporations won the license propaganda war). I think my computer has, like, five GPL programs? Ten? It's just that two of them are coreutils and Linux, and that basically holds up the GPL as a concept.
operating at a loss and non-operational because of outages are very different. If they can't maintain service levels nobody - AI super user or quant, old-fashioned human - will be happy.
There’s also an excellent paper called “Fair Benchmarking Considered Difficult:
Common Pitfalls In Database Performance Testing”.
Benchmarks are genuinely hard. Modern systems and hardware are notoriously complex and have different behaviors across runs and with minor tweaks to performance settings.
Yep, absolutely... and in the end benchmarks aren't going to sell a database anyways. Reliability and provable correctness and ease of integration are probably going to win in the end. It's a dubious marketing edge to lead with.
PostgreSQL itself is remarkably slow in many configurations and leaves a lot of performance on the table due to its architecture and assumptions. But it works and people trust it. So they use it.
As a SpacetimeDB dev, I absolutely agree. Reliability, provable correctness, and ease of integration are higher priority than performance. That doesn't mean we need to accept garbage performance though.
There’s definitely strategies here; A lot of the floating point operations use subnormals, and a lot of the worst instructions are slowed down by really, really fucking with MMIO.
reply