I don't think that hiring the engineers was ever an option. He said that he couldnt maintain six native apps before agentic coding. The business never even considered expanding headcount by 6X to do it, it was a complete nonstarter.
There is no fixed pie of work that AI is taking from us. In this circumstance, increased productivity caused the company to just make more stuff, not lay off people (or avoid planned hires).
To clarify, my point isn't that the market for software won't expand due to the greater productivity unlocked by AI.
It's that the benefits of that productivity won't go to the kinds of people that previously wanted to stay in engineering their whole career.
I'm not sure the Jevons Paradox applies because you're not just making engineering more efficient you're replacing it altogether with something much more commoditised.
People will gain from this but it won't be the same people.
Likewise, consumers will eventually reduce the amount they’re willing to pay for software so it will balance out. It will be a blip in time we look back on. Like, remember when elevators used to employ an operator. Now we just push a button and happily do it without another person’s labor.
Well at least in this particular case, it was the same engineers on the team. He specifically said he tried getting UX people to vibe code things and the results didnt work
Jevon's paradox has to do with consuming resources, not employment. There are plenty of examples of fields getting more efficient and employment falling. Farming and manufacturing being 2 examples.
You're conflating different categories of activities. Farming and manufacturing are amenable to automation precisely because not much real thinking and communication is required.
It's an interesting example in that a switch from web/React -> native seems like Jevons paradox from the perspective a developer considering "amount of effort I'd need to put in"
But Jevon's paradox is about consumption of a resource increasing. The resource being "developer effort" but the new supply all running inside GPUs and dev machines. He actually says he decreased their server deployment by 90%.
So on net, did they basically deliver their users a much nicer but functionally equivalent app, which in total now consumes less resources / spends less $ into the economy? And then the $10T question is to what degree this is representative of AI's effect.
I think a lot of people haven't actually watched his talk and don't understand what he is saying. If you watch the talk, he is not anti-rails. He is just very pro- agentic coding.
What he was saying in his talk was that his customers (people using Hey for email and calendar) never really wanted hybrid web-native apps, they want native apps (like most people prefer on mobile). His business could never afford to have a dev team that built and supported six different native apps, so they always built using hybrid web-native stuff using Turbo Native.
Now that his development team has access to agentic coding, they can build and maintain six mobile native apps. For his app, the back end part then becomes small and simple and they can reimplement it in rust to reduce server cost.
His point wasn't that rust is better than ruby, or anything like that. His point was that agentic coding allows smaller teams to do bigger things. For most people building on rails, their customers don't need native mobile apps (I know mine don't) and for them Rails is a fantastic choice for agentic coding.
Rails is so into agentic coding that they publish benchmarks of how effective different models are for building with rails:
TLDR: DHH isn't anti-rails, he is pro-agentic coding. Rails is great with agentic coding. AI doesn't mean layoffs, it means your teams can do much more than they could do before.
It seems to me, so far, everyone moved one level up.
- A layperson can now build one a solo developer can do
- A solo dev can now build what an agency of web development can do
- An agency of web development can now build native apps similar to what large corporation can do
- Large corporation can now tackle even more ambitious projects
Which makes sense given what got automated. This is with the current tech, people still exploring the full potential, but if that turns out to be the case then I'd argue the software market will grow substantially and we are just having an adjustment period.
There is nothing wrong with designing like in the year 1999. But claiming improvements when all great websites like Amazon in 2004, the original GitHub, Google etc. were before AI is a bit weak.
He is just promoting Shopify's vibe coding for merchants. I bet on Shopify pivoting to vibe coded Rust soon because of "security".
>It's interesting how this stuff gets spun on social media by people who don't watch the content.
I watched it live, and a few actually in the show have very different perspective to what you suggest, they were just keep it to themselves and not posting it on twitter. Some did, but then deleted it.
And no one is questioning he doesn't like ruby anymore. But it was obvious there was nothing much said about the future of Ruby and Rails.
It looks like he spent about 4 minutes out of the 1:03 making that point.
I can't but help think that he would have spent longer on it if that's what he really thought.
I don't think he believes in Rails any more.
It's clear that he was more excited about thing that are not ruby or rails and now considers rust better suited to the output of AI.
To be honest, I could see rust taking over from typescript/javascript as an output language for the web too. If no one needs to read source code any more why not use rust for everything?
I don't think anyone should really believe in Rails at this point. It was a monumental leap forward at the time but other stacks have shown that there are better ways.
When I started as rails dev in 2013, it was really the top solution for producing MVP fast. Now, I don’t think that’s the case and most of the incentives are gone if we are no longer writing code by hand
So what would you pick for producing an MVP web app fast that you also want to run in production, maintain, patch and scale that has a solid eco-system of pre-built libraries you can just plug in to solve problems?
At least within my circle, that’s not really the case. Instead, what I’m seeing is more of a hiring freeze, with the budget being used to empower existing employees through AI. Which I think is a more reasonable take.
That's often layoffs in disguise. If you want to cut 10%, but you don't want to do it as a single layoff, because that's expensive, can trigger certain laws depending where you are, and might disrupt teams, who do you cut?
Instead you can do layoff by attrition, you stop hiring, and you don't backfill seats, as people naturally leave or get pip-ed, the team shrink, you don't rehire into the position, and in a year or two, you cut down by 10%.
Also, I wouldn't trivialize a freeze in hiring growth, it means to get a job you have to wait for someone to retire, and also wait for people looking for jobs with more experience as they're more likely to fill the roll to also retire. Basically there's now way more demand for jobs than hiring, so that will drive compensation down for those currently employed.
as a solo: you can build and ship more in your own time
as a startup: you can iterate and scale more with more velocity and less growing pains
as an enterprise: people with ideas but not a technical background can put their tokens where their mouth is and build their own interfaces, personal apps, ask their own questions, ship their own features without having to navigate layer 8 and play games for resources
Why does it sound like layoffs? If you keep the team size the same and use the productivity increase to deliver more business value, how is that layoffs?
If I figure out a way to make my workers produce 10 times more widgets each, it doesn't automatically follow that there is a market for the extra 9 widgets. Otherwise I would have hired 9 extra workers before that invention to fulfill the market demand.
Making the widget cheaper doesn't necessarily create demand either. I will not buy 10 times more toothbrushes or writing pens if their prices suddenly decreased to 1 cent.
The demand for software or features will not increase at the speed of our ability to produce them.
If every engineer is 10-100x more productive with AI, and one company slashes its workforce to maintain the same level of productivity at reduced cost, it will lose out to its competitor who retains the same workforce and does far more with it.
Right now, the vibe at my company is not “we can save so much money with AI” it’s “here’s an infinite budget, use all the tokens you can to expand our moat because we’ve never been more sensitive to competitive disruption than we are now”
> Rails is so into agentic coding that they publish benchmarks of how effective different models are for building with rails
To each their own, but I think using Rails in the era of agents is probably a mistake. It has 2 big problems IMO.
(a) A ton of footguns that an agent can and will miss on occasion, and as we manually review less and less code, we will also miss it, and
(b) security on the web has become increasingly important due to hostile AI agents, and while rails has great secure conventions (see (a)), it's still easier to screw up something and not catch it until after it's deployed because there are simply less guardrails in place (dynamic types, no compile, slow tests, generally bad performance profile).
Rails dev here, still using it, still plan on using it, use agents, don't see them making food gun decisions with rails apis just code.
Rails comes with the console by default, so production inspection of a bug is something the ai can do without a ton of bs to be set up, and that console can be forced to be auditable (using standard gems) or read only (using flags).
I used to love rust, still do sorta, but yeah I think this is really not a good argument against a tool that operationally is better at most things at being observed or poked or prodded or tested in production
Not sure if you’re speaking from experience or just being theoretical here. We’re using Rails and it’s been great. Maintenance and upgrades have been a breeze.
All this static vs dynamic stuff doesn’t really matter anymore. AI doesn’t care. You just need to close the feedback loop with tests, linting, a compiler, or whatever else gives it a clear signal.
Rust is used a lot. Elixir/Phoenix comes up often. You’re looking for something statically typed with string guarantees and as FP as possible to make unit testing easier.
Rust for sure. Builds become a problem eventually though. Elixir is much more interesting now that they added types (I think they're doing more with this still?). I quite like Gleam from a human (non-agent) perspective.
In my experience, Go. It was designed to be idiot-proof. It's very verbose, not very expressive, compiles fast for quick feedback loops, strict autoformatter, plenty of examples in models' training data.
the only substantive difference I've noticed is between languages with a proof system and those without, where with a proof system is orders of magnitude better for agentic work.
One of the things I admire most about Basecamp is that they are a bootstrapped, profitable business. There's no shame in running a business this way. The world would be more resilient if more business owners grew sustainably instead of soaking up endless invester funding and losing money year after year before enshittifying in a desperate attempt to repay their overlords.
Here, I will give you my upvote! That's how Issuetrak is. The founder (me), still owns a majority of the stock, there's no outside investment, and we have customers who've been with us more than 20 years. Multiple employees as well!
It’s not his message, it’s the forum. Why host a rails conference, that he explicitly created because he was silenced from the main one…to them not talk about rails?
Why not adapt rails for this next generation?
Why do it this way? It’s like walking into a temple to yell about Jesus. It was absurdly obnoxious.
>His business could never afford to have a dev team that built and supported six different native apps
I have a hard time believing that, considering his hobby of racing cars and commissioning one-off hypercars. Unless it's that his business could never afford that dev team after DHH takes his salary.
> TLDR: DHH isn't anti-rails, he is pro-agentic coding.
That's an opinion, not a summary though. The way how DHH behaved is really awkward in the last years. See the "let's shoot down wolves" blog entry here:
That DHH is into AI agents everywhere is easy to see, but if you go to rails-con, would you not expect this to be about rails as such, and people? Now it's about rust - and AI. Controlled by big corporations. Yikes - some people must be upset about having gone there to listen to DHH preaching about an agentic future.
Edit: I also watched parts of the recent youtube video about rails. I could not focus on the content because DHH's style was so strange. I've seen that style before - nothingburger talks. Look at phrases he used such as "I have retired from being a professional programmer/software engineer". This is all aimed to hype up "agentic coding is the future". Rails is dead - that video is all the proof one needs. DHH shifted gears to AI, and it is quite similar to when he first promoted rails. But now it is about AI, not rails.
Do the teams fully understand the code that they are committing? Are the teams reviewing each others code or are agents? When agents do all the coding, and perhaps the reviewing, and the patching what is left of the team? People lose connection to their work, and the social bonds between workers begins to erode.(1) DHH's speech seems like a weird eulogy for the entire Rails community, although many of us saw this coming several years ago.
Also, I hate native apps. FWIW most devs I know actively try to avoid them.
Most devs I know who hate native apps are either web devs trying to embed web tech into everything they touch, or they just hate Apple.
Most users don’t care. What they do care about is performance, and native is usually the easiest way to get there.
For your AI workflow questions, it depends. With or without AI, those decisions depend on the organisation’s direction, the team’s capabilities, and their preferences. Did your org have a single answer for the best programming language, framework, or workflow even before AI? There was never a single right answer to any of that.
I hate native apps. There are use cases better suited to native apps but often it's just not what I want.
You have little to no control over privacy, tracking or advertising compared to the web. That's left to the original developer and there's nothing you can do about it.
You can't do things like disable autoplaying videos. Often you can't even copy text without taking a screen shot. On a phone it's also much easier to zoom random parts of the screen in a browser compared to a native app.
There are legitimate reasons to prefer the web to native apps.
Users absolutely care, people hate downloading random shit on their phone they don't want but are forced to.
What has always been true is that users often have no choice or voice to express their actual desires. It's not like there is a public alternative here (when there should be).
I agree wholeheartedly with the sentiment expressed in this post. Our attention is extremely valuable (both to us and to advertisers), and consumers today broadly squander it.
For consumer digital products, the only business model that seems to work is advertising, because consumers are cheap and refuse to pay money for software. But the result is that they instead trade something far more valuable for the software, which is their attention.
The products that get built, as a result, are supremely well-engineered to attract and retain attention, to the detriment of their users. Most of the "news" that reaches us these days is not at all an attempt to actually educate us about the state of the world. Instead its just a collection of inflammatory and enraging facts that are chosen to optimize for engagement (to hold your attention)
In theory, someone could rebuild an ecosystem where people stopped squandering their attention and monetized it with money, but this would require consumers to stop chronically undervaluing their time and overvaluing their money.
I think the funny (and somewhat scary) thing about ad-supported platforms is that both the creators of the platform and the customers implicitly agree that the content just isn't that valuable. They know that if they paywalled the platform and forced people to make that value judgement then barely anyone would pay.
Which is kind of scary where you realise people spend hours and hours each day watching youtube slop, tiktok, instagram reels, while also subconsciously feeling "this content is worthless and I wouldn't pay $10 a month to keep watching it".
Printing money causes inflation and inflation causes increasing interest rates. It is possible to enter a positive-feedback cycle, and has happened to countries before.
I'm not predicting that this will happen, but we do need to take the debt seriously and not assume we will be able to just print money to get rid of it.
It would be great if they could use the LiDAR scanner to sign a depth map of the scene in front of the camera. It would show if you photographed a flat image.
The sensor already uses phase detection autofocus. You can create a depth map out of it. iPhone X used its dual cameras and that phase data for depth maps.
How? Is this for every image taken, or does one need to take a second photo? Or is it "in theory phase detection hardware would let you make a depth map"
The sensor has special pixels which can detect phase differences in incoming light. Camera uses these pixels (aka embedded detectors on the image sensor) to be able to focus where you want. This is same from phones to professional cameras.
(DSLR cameras have independent phase detection arrays. This is why the mirror has a small mirror behind. To illuminate that section).
Lenses are tuned. They know how distant the thing they are focusing on, also photos' EXIF generally carries the focus point information. Side note: Apple Aperture used to be able to show where you focused with that info. Still no app does this. I'm still mad that we don't have Aperture anymore. Anyway...
So, if you collect this phasing information alongside the photo which you're taking, you are capturing the depth map of the photo. Its resolution will be lower, but not lower enough to be useless.
For example, Sony (and most probably all other big camera manufacturers) cameras use this phasing data throughout sensor in real-time for following moving subjects to keep the focus on them by predicting where they are going.
So, there's no "in theory". The phasing data is the depth map. Otherwise your camera can't focus on anything. It's there to focus, and it's done by reading the phasing data and knowing where to go by how much.
All fast AF systems are PDAF. When light is too low, then it's CDAF, which does way slower, without using any phase/depth data.
Images that are meant to be 2d (E.g. a photo of something on paper) isn't going to look 3d. I guess this approach will just have to let anything flat through?
"I took a real photo of this image printed on paper" seems possible to falsify, yeah, but that doesn't seem too surprising to me, given that anyone with a printer can print literally any possible image.
How would that work? I thought the premise here is that you can fool the apple camera by taking a (very carefully aligned) picture of a still image (printed out).
A depth map from the apple camera (again, signed) would show that the entire image had the same distance from the camera.
The photos are cryptographically signed in the apple image pipeline so it's not as simple as just AI generating something. That said, I can't see how this is any different to all the other times we have embedded crypto keys in consumer hardware where eventually someone finds a way to extract the key and the whole thing is busted open.
I think there's a chain of trust. The sensor signs raws, and the private cloud takes signed raws, does minimal processing so they're at least coherent, and re signs that output (maybe even including the original signed raw as well in the image file).
How would the server know that the request is coming from a real iPhone?
This is a pretty standard application of trusted computing and can be done entirely on the iPhone. A server would only possibly be needed for anonymization (while retaining key revocation capabilities if a key does end up leaking), but there are serverless ways to do even that (TPMs have supported these for a while now).
It wouldn't, but you could validate that a particular picture was created at a particular time, and had not changed, for example, especially with metadata that you may not want to share but that establishes certain parameters like gps coordinates. A lock, rather than an end-to-end pixel signature, which shows what was contemporaneous rather than exact provenance. If an event happened on day 0000-00-00 00:00:00am, but your photo was taken at some other time, it casts doubt.
I think a big part of validation for things like these are just "could it have been modified since Z event happened", because Z was not something people paid attention to before.
That's just a timestamping service then, not a content provenance/authentication scheme. Timestamping has been a solved problem for years; certificate authorities offer this, or you could just throw a hash onto any sufficiently trusted blockchain.
Nothing prevents anyone from opportunistically pre-generating and timestamping millions of permutations of fake kompromat and then selectively revealing the one that turns out to be useful after the fact.
You could charge per attestation, but the economics of that don't look great; you could demand publication of the image itself before attestation, but that would obviously not fly for most use cases out of privacy concerns.
> Nothing prevents anyone from opportunistically pre-generating and timestamping millions of permutations of fake kompromat and then selectively revealing the one that turns out to be useful after the fact.
If you're doing server-side timestamping and someone is sending millions of items, I think you just ban them. Apple accounts are free but not inexpensive.
That's a good point, You might still be able to trick the cloud to sign your photos, but that's something they could patch in updates without losing control of the key. They could have the server only sign photos taken on the latest ios version.
And honestly you could have a similar antitampering oracle that was at least obscured, in terms of "we've detected tampering but won't tell you how or why", which is frustrating but I have to imagine that 99.9%+ of images are clean.
... don't make it so shallow then. Perfectly possible in a consumer-friendly 3D printer for a face or even human body if you have lots of time for the prints.
You’re well on the way to 1:1 replicas at this point. Next you’ll need to match the thermal signature and the exact weather in the sky for the time at that location.
An important distinction but I have taken pics of pics, and sometimes its hard to tell I didn't take it directly. So now, its hard to tell I didn't directly take the photo AND its got the stamp of approval.
The credibility comes from who took the photo. The next logical and easy step is for this metadata to flow to the user agent. I'll know that it was taken by a legit journalist photographer. And id not, I can have my user agent make it fuzzy or replaced with
a kitten photo.
I think this is where we are headed. This feature seems useless on it's own since someone will eventually find a way to extract the key from the iphone and sign any image. But if they could make it so every iphone uses it's own key and the photos show "Taken by xyz" and it's linked to their icloud or identity somehow. That way images shared around will still be able to be linked to a source that you can choose to trust and the viewer can know it wasn't modified from what that person shot.
Then we might move to an age where photos which were not signed by someone will be treated as fake.
You may have forgotten that the world is driven by screenshots, and so a screenshot of a picture with the `(X) Verified by Apple to be Joe Schmoe` tacked on will be just as good as real verification for a lot of people.
What we need is almost something like the classic "press C+A+D to log in" or the idea of "above the browser pane"... something an image cannot show you unless it's legit. Perhaps a personal thumbprint icon that is different for each viewer, so you know its your device telling you that something is real and not just some mock fake thing?
I don't think we'll treat them as fake but they would probably undergo more scrutiny - perhaps in a crowd-sourced manner that provides an overall score of probable legitimacy.
People have constructed video walls so high resolution they can film TV series in front of them and to an audience it's indistinguishable from a real set. It's extremely cost-effective for things like space fantasy that needs lots of exotic-looking backdrops, apparently.
It may not be in reach for you and I - but within reach of anyone with the budget to run a 'bot farm'
The high resolution video wall is quite simple. The hard part and why this has not been done earlier is that for filming, they had to synchronize the rendered background with the movement of the camera.
Movement is solved by tracking the camera's position and projecting the background from there.
Proof of captured image with hardware-based-attestation can be increased by adding extra information besides the RGB channels, like depth mapping (which a projected screen wouldn't be able to fake), and eventually light field recording (plenoptic imaging, e.g. Lytro).
also no, even in the high rez video walls its still possible to tell. maybe to an "audience" no but there are lots of fundamental flaws with video walls that make them not the same as filming it for real. all of those flaws show up in the final image and could be detected.
things like color rendering, sharpness, screen door/morie, motion blur, and yup even good ole depth queues in the lens system all show up as artifacts.
case in point, outside of a few specific niche cases like the mandalorian, that virtual production video wall thing is not actually being used all that much because of the amount of post shoot cleanup required to fix all those issues, it wasnt actually that much cheaper and its not really better either. especially when you factor in how hard it is to shoot that way.
> things like color rendering, sharpness, screen door/morie, motion blur, and yup even good ole depth queues in the lens system all show up as artifacts.
Pre-AI fake videos sidestep this sort of issue by lowering the video quality.
Add some motion blur, some camera shake, some poor lighting, the camera being slightly out of focus, and make the video 720p instead of 4k.
Multiple good things would happen without prop 13:
People would move more instead of being locked in to a low effective property tax.
The tax burden would be much fairer: instead of the police and schools being funded by newcomers, everyone would be contributing their fair share to fund the city, instead of the longest tenure residents mooching off the rest of us.
Unproductive use of land would be discouraged. If a tract became highly desirable, it would be expensive to let it sit in an unproductive form
Cities could raise taxes as high as their residents wanted them to be, instead of being limited by the opinions of non-residents. Some people want to have better funded police and schools, and local government would actually be allowed to deliver that. Citizens who didnt want to live in expensive cities could vote with their feet to cheaper towns.
"People would move more instead of being locked in to a low effective property tax." - this just means that older and retired people would get kicked out of stable living situations, driven by property tax assessments which are often driven by real estate market boom/bust cycles. And housing prices would be just as high as with prop 13.
The bad part about prop 13, is that it never should have applied to anything other than primary residences of people. Having it apply to rental & commercial properties is what warps the market.
Cities should be cutting their police budgets - most small towns have nearly 50% of the budgets tied up in police. All while there has been a multi-decade decrease in crime.
Old people seem to be surviving just fine in areas where property tax is based on current market value? It sucks to move, but luckily they have an extremely valuable property to sell
Young people seem to be surviving just fine in areas where the market value is lower. Sucks to not live in the heart of the city but luckily they have their entire lives ahead of them to accumulate the wealth required.
I don't understand why you think old people would have to move. They have the ability to access the equity in their home at any time through HELOCs or mortgages. If they own a house that shoots up 1M in value, they have just had a massive windfall that they can access through financing to pay the taxes associated with that windfall.
Do you think cities would make the taxes fairer (e.g. spread them out over more residents) or just raise taxes on the these previously protected properties?
My money is on the latter. Nobody is getting a tax cut if Prop 13 is repealed. Everybody's taxes will just continue to go up.
... which is why it was, and still is, common economic knowledge that the government must control marketplaces. Meaning the physical areas where trading is taking place.
We've been through this rodeo before you know. Private ownership of markets, with the obvious result monopoly abuses. Hell we've had private money and that was a total disaster too. And now we have VISA as opposed to government controlled payments.
But then people stopped caring about that. With abuse upon abuse the result. Because it's not like this result, that only by buying a prominent position on a marketplace you can have success as a seller is only the case online. This is very much true in meatspace as well.
And now almost everything you can buy is effectively through a privately owned (semi-)monopoly rather than a marketplace. From supermarkets to app stores.
Addendum: and then governments must NOT let anyone, not even other governments, force "whoever pays most" rules (like the EU is forcing through in Europe)
100% not. To equate fun to play to people buying the game is 100% not how you should say making more money is more people buying the game. Money is money. If the game is mediocre but it is fun enough, it can get more money than a game that is more fun but doesn't have as many opportunities to spend.
This is why things like dark patterns exist that try to get you to spend money through FOMO and other mechanisms like loot boxes.
You can argue that all of these things are people having "fun," but I wouldn't define that as fun to play.
Finally, none of these necessitate people buying the game, but rather spending money in the game. Usually, it's free games where people buy a secondary currency to spend on virtual items. Mobile apps, for example, are huge with this.
So the real answer to your question on multiple grounds is no.
Edit: Now, I will give you that people do have to make a choice, but that's like saying that gambling addiction is not real and it's a choice, or drug addiction is real and it's a choice. The reason for alcoholism and Alcoholics Anonymous is bogus because it's just a choice, but it would be dangerous to classify it as just a choice.
Not necessarily. Some of the most profitable games out there are not very fun to play, but they have their monetization strategy nailed down.
People laughed at Blizzard when it came out with the mobile game Diablo Immortal, and I've never heard it listed among Blizzard's best games, but Diablo Immortal was one of, if not the, best earners the company ever made.
Anyone know the specifics of how this works under the hood? I assumed the people over at Twitter/X would aggressively block systems like this from working. Are they doing something creative to get around that?
Burner accounts are dirt, dirt cheap. I have several hundred I rotate out. I’m afraid to mention the library I’m using in case it’s not in their cross hairs, but it makes scraping trivial. At least at the scale I’m scraping I don’t even need a proxy. My residential up is enough to keep the blocks away.
On most of these sites you can just copy the requests the browser makes. There's a walls and ladders game going on, but all the easy to implement walls aren't very high so most sites don't bother building the tallest one possible.
You can start by making a burner account and viewing some pages in a browser and watching the developer network tab. Either you'll see some dynamic JSON request that fetches the main page content, or it'll be in the initial page request. In either case you can copy that request and parse it. If it's in the initial page then it's in HTML so parse it with BeautifulSoup or similar. A minority of sites are using TLS fingerprinting, so try curl-impersonate if plain curl doesn't work.
At least in Firefox I suggest right click "copy as cURL [command]" and then delete parts of the command to see what's actually needed to get the data, or copy the whole exact thing if you're worried about getting fingerprinted based on the parts you deleted.
There's certainly some kind of auth cookie or token. How long does it last? Maybe 15 minutes, an hour, a day, forever? If it's not forever there's a way to refresh it so look through the network requests again, or maybe the network requests when you log in.
If you get rate limited it's probably by IP address and you can try a few things. Rebooting your router to change IP? If you have IPv6 you can change your address arbitrarily within a /64 and if the site is really dumb it will consider that a new IP, but most know about that. If your ISP gives you a /48 or /56 you can change subnet. You can try running on cloudflare workers or Amazon lambda, which have big IP pools that continually churn. Some sites that use cloudflare may exempt all of cloudflare from rate limiting and you may greatly benefit from running on a cloudflare worker. You can try a mobile connection that may use CGNAT, forcing the site to choose between unblocking you or blocking your carrier. If all else fails, residential proxy access is either very cheap or very expensive to buy, depending on how much data you need to move.
I've found that working in a style similar to this is very useful for managing multiple threads of work at the same time.
I use virtual desktops, where each desktop contains a single thing that I am working on, and everything related to it. Then any good window manager can switch between them quickly and even give you a spatial sense of where you are. I organize them as a priority stack, so the topmost priority is in top position and then it goes downward from there
There's a lot that this "campus" product does that isn't present in such a set up (multi user, zooming, etc...) but virtual desktops give you a lot of the benefit and come built in to most of our operating systems
reply