This post is full of Claude's irritating verbal ticks ("It's not X, it's Y"). It is fine to use Claude (or another LLM) to help with this kind of reverse engineering, but I would prefer the write-up be done by hand.
This would have been more effective if written by hand. The issue content is typical LLM-generated markdown that has a lot of noise (formatting, headings, lists etc.) which makes it fatiguing to read. It comes across like a PowerPoint deck of dense slides. What is really needed is a clear, focused and well-written letter that effectively gets across a message and has the potential to be persuasive to the people within Microsoft who might be in a position to help. An additional problem with the LLM-written style is that it comes across as low-effort, which also lessens its impact.
The major constraint is that the compiler needs to guarantee that transformations produce semantically identical results to the unoptimized code, with the exception of undefined behavior or specific opt-outs (eg. `-ffast-math` rules).
An ML model can fit into existing compiler pipelines anywhere that heuristics are used though, as an alternative to PGO.
The custom scroll behavior on this page is infuriating and distracts from the content. It is like scrolling in treacle. If someone who works at Microsoft sees this and is able to file an issue, could you please do so.
I've seen talks on this topic at Rust conferences which seemed strongly influenced by Swift's approach, so that will probably be the direction this ends up going in.
I use fish + atuin. I leave the "Up" arrow set to use fish's default history search (see https://docs.atuin.sh/faq/#how-do-i-remove-the-default-up-ar...), which keeps the UI minimal when just going back one or two commands, then use atuin via Ctrl+R when I need to find a command from earlier in my history. At that point Atuin provides a nicer UI for searching the history.
Thanks! Would be interesting to see if Rust/LLVM folks can get the compiler to apply this optimization whenever possible, as Rust can be much more accurate w.r.t memory initialization.
I think rust may be able to get it by adding a `freeze` intrinsic to the codegen here. that would force LLVM to pick a deterministic value if there was poison, and should thus unblock the optimization (which is fine here because we know the value isn't poison)
I think in this case Rust and C code aren't equivalent which maybe caused this slow down. Union trick also affects the alignment. C side struct is 32 bit aligned, but Rust struct only has 16bit alignment because it only contains fields with 16bit alignment. In practice the fields are likely anyway correctly aligned to 32bits, but compiler optimizations may have hard time verifying that.
Have you tried manually defining alignment of Rust struct?
It varies. New public APIs or language features may take a long time, but changes to internals and missed optimizations can be fixed in days or weeks, in both LLVM and Rust.
The code for both the client and server are open source (https://github.com/hypothesis/h) so this is possible. The server is designed to support the needs of large scale deployments, so this does come with some complexity compared to a system you would design for smaller scale usage.
The text on https://web.hypothes.is/ mostly targets schools and universities, because Hypothesis pays for itself by selling integrations with online learning platforms (Canvas, D2L, Blackboard etc.) and associated support.
One interesting thing I discovered comparing various matrix multiplication implementations used in ML libraries is that several of them (ONNX Runtime, XNNPack, any others?) skip the step, from BLIS's textbook algorithm, of packing the LHS matrix. Instead they pack only the RHS. Since those are the weights, this can be done once ahead of time and then an inference pass does not need to do any packing at all.
From skimming various papers it seems like the motivation for packing the LHS originally, even though a single element is broadcast from it at a time (nb. this is opposite to the order in this post, where the row count in the microkernel is a multiple of the register size, rather than the column count), was to reduce TLB misses. Apparently this is not a problem in practice on modern CPUs and for problem sizes common in ML inference.
For context, ultralytics is the Python package for YOLO v8 and YOLO v11, two of the most widely used object detection models. The GitHub repo has 33K stars.
