I don't use Rust much, but I agree with the thrust of the article. However, I do think that the borrowchecker is the only reason Rust actually caught on. In my opinion, it's really hard for a new language to succeed unless you can point to something and say "You literally can't do this in your language"
Without something like that, I think it just would have been impossible for Rust to gain enough momentum, and also attract the sort of people that made its culture what it is.
Otherwise, IMO Rust would have ended up just like D, a language that few people have ever used, but most people who have heard of it will say "apparently it's a better safer C++, but I'm not going to switch because I can technically do all that stuff in C++"
Agreed. As a comparison Golang was sold as "CSP like Erlang without the weird syntax" but people realized channels kind of suck and goroutines are not really a lot better than threads in other languages. The actual core of OTP was the supervisor tree but that's too complicated so Golang is basically just more concise Java.
I don't think this is a bad thing but it's a funny consequence that to become mainstream you have to (1) announce a cool new feature that isn't in other languages (2) eventually accept the feature is actually pretty niche and your average developer won't get it (3) sand off the weird features to make another "C but slightly better/different"
Due to lack of many abstractions, and lack of exceptions, Go is a less concise Java. It's a language where the lack of expressiveness forces you to write simpler code. (Not that it helps too much.)
Go's selling points are different: it takes a weekend to learn, and a week to become productive, it has a well-stocked standard library, it compiles quickly, runs quickly enough, and produces a single self-contained executable.
I would say that Go is mostly a better Modula-2 (with bits of Oberon); it's only better from the language standpoint because now it has type parameters, but GC definitely helps make writing it simpler.
People sometimes say this like it's a dunk, but there's a finite amount of complexity any given programming task can shoulder, you have to allocate it somehow between the programming environment, the problem domain you're working on, and the algorithmic sophistication you bring to bear on that problem, and it's not clear to me what the benefit is in allocating more than you need to your programming language.
A lot of very smart stuff is written in Go by programmers who just want the language to get the hell out of their way. Go is some ways very good at that. Rough though if you want your programming language to model your problem domain or your algorithmic approach! TANSTAAFL.
Unfortunately, Golang has a whole lot of weird, pointless quirks in both its base language and standard library, compared to something with a more elegant and from-the-ground-up design, like Rust itself or perhaps OCaml/ReasonML. Not very good news if you want the language to just "get the hell out of your way". I suppose it's still way better than the "enterprise" favored alternative of Java/C# though!
> Golang has a whole lot of weird, pointless quirks in both its base language and standard library
Having used go in anger I don’t necessarily agree with this, could you point out an example. Maybe I just accepted it and work around it without paying much attention.
If you are referring to interface types being able to be null, well they are allocated on the heap and have dynamic dispatch, this isn’t particularly a surprise if you have worked in a lower level language but might be a surprise if you come from a language where that isn’t the case.
> Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.
I'm curious to know why you think so, I thought it was a great article, showcasing how simplicity in the language doesn't make complexity go away, it just moves it to programs written in it.
And the related read is purely a misunderstanding about how concurrency is modelled. Channels are not meant to be written from multiple writers, maybe this gets discussed in the next article. I can understand why it is confusing and you consider it unintuitive.
The read situation is literally not understanding or even looking up the interface, there is another return value that will tell you the channel is closed.
Nil channel reads make sense for it to do what it does if your familiar with the interface but at the same time you are literally holding it wrong. Vs a language where that would be UB its an improvement. Compared to a language like rust sure the information isn’t available at compile time but on the same not you would need to be developing in rust, and no offence but if you think golangs concurrency story is difficult to grasp just wait until you meet async rust…
These are the unintuitive, these are generally complaints of you failed to even begin to read the documentation and then complain when things aren’t doing what you expect, leave your preconceived notions at the door and you will be fine.
The faster than lime take needs to be updated, comparing golang from 5 years ago to modern golang is about as useful as comparing rust fron 5 years ago to rust now.
And you can happily make Cgo calls to the windows libraries if you want to access windows APIs, the language provides an abstraction for the normal use case not the specific use case, theres an escape hatch if you want it. And regarding the complaint about timeouts, context is a thing.
Because you don’t know how doesnt mean it’s unintuitive, it means you don’t know how.
I worked with C# for a decade and it's become a really great general purpose language. I'd still prefer never to work with it ever again after having worked with Go. This isn't for technical reasons at all, but because Golang is so easy to work with for "people reasons". There are brilliant parts of Go, but the only thing I find myself missing in other languages is the simplistic module isolation, where every folder is a module, every file within the folder is part of it and then you expose functions with capital letters at the beginning of their name. Holy hell did I wish Python had that. Anyway, the thing that makes Go nice to work with over time is the explicity of everything and a lot of the very opinionated decisions. With a piece of Go code I can jump in and immediately know what is going on regardless of who wrote it. With C# I'll often have to go down long "go to definition" paths. Often you will end up trying to figure out just how someone was trying to "fight" the implicit magic of the non-STL Microsoft dependencies they used. Usually because they didn't really understand what they were doing. All of these are human issues and no fault of C# or .Net as such.
Of the few technical advantages Go had for us is that we don't need a single dependency outside of the standard library, which can't live in isolation. We use SQLC and Goose, both are run in containers that only have rights and access on the development side of things.
I'm not sure I would say that Golang has a lot of weird, pointless quirks, but it has opinions and if you happen to dislike them, well... that sucks. I hate the fact that they didn't want runtime assertions as an example, so it's not like I don't understand why people dislike Go for various reasons. I've just accepted that those strong opinions is the reason Go is so productive.
The challenge for us is that it's not exactly as productive as Python. So while you'll need to do a lot of toolchain work to get Python anywhere near Go's opinionated stucture, that is often a better choice if you're not a hardcore software engineering team. At least for us, it's been much easier to get our business intelligence people to adopt UV, pyrefly, ruff and specific VSC configs for their work than to have them learn Go.
I suspect that is why Rust is also doing so well. Go is a better Java/C# for a lot of places, but are you really going to replace your Java/C# with Go if you have dacades worth? If you're not coming from Java/C# will you really pick Go over Rust? I'm not sure, but I do know, Go failed to become a Python replacement for us. It did replace our C#, but we didn't have a lot of C#. Eventually we'll likely replace our Go with C/Zig and Python to keep the language count lower.
> I hate the fact that they didn't want runtime assertions as an example,
So you hate writing:
if (something != 1) {
panic("oh no!")
}
vs
assert(something != 1, "oh no!")
? Reminds me of people complaining about Python's significant indentation. If that's what you're complaining about, you have nothing to complain about.
I think your point is fair as you can do runtime safety in Go by wrapping Panic, but that's not exactly what Panic is meant to be used for. I guess it's more of a intent vs syntax thing, aside from the part where Panic can be Recover()ed.
You're taking it out of a context of me praising Golang, however, and while I do hate the fact that they didn't just do runtime assertions, it's not like I dislike Go as a whole because of it.
From my point of view, the significant tradeoffs made in the design of go make it better suited toward software that has to change a lot: e.g., things closer to customer-facing that are constantly being iterated on.
Rust is more appropriate for areas where the problem boundaries are relatively fixed and slow-moving: libraries, backend services, and infrastructure.
By better allowing you to model your program domain, Rust actually lets you finish projects that are feature complete. I have multiple Rust projects in production for years that have needed at most one or two bugfix releases after their initial rollout on top of a half a dozen feature updates. For a very slight additional startup cost in doing that modeling, I’ve gotten massive dividends back in the volume of maintenance programming that hasn’t needed to be done.
But if your problem domain changes frequently enough that the model needs to be changed all the time, that extra inflexibility isn’t worth it. On the other hand, I don’t know much go software that isn’t beset by the same number of bugs as every other modern language.
There's no free lunches. But I could take a problem, make it arbitrarily harder, then take that arbitrary limit away. If someone thought they had to solve the harder problem, and found out they only have to solve the easier one, did they get a free lunch?
This isn't meant to be allegorical or anything specific. I guess it's just an observation that sometimes your lunch can be cheaper than you thought.
I can't substantiate your claim about Erlang or weird syntax, is that either a proper quote or some kind of paraphrasing because nothing remotely close to it comes up.
There are numerous interviews with Rob Pike about the design of Go from when Go was still being developed, and Erlang doesn't come up in anything that I can find other than this interview from 2010 where someone asks Rob Pike a question involving Erlang and Rob replies by saying he thinks the two languages have a different approach to are fairly different:
It's at the 32 minute mark, but once again this is in response to someone asking a question.
Here are other interviews about Go, and once again in almost every interview I'd say Rob kind of insinuates he was motivated by a dislike of using C++ within Google to write highly parallel services, but not once is Erlang ever mentioned:
The parent comment does not say that Go derives from Erlang, but that both Erlang and Go implement CSP (https://en.wikipedia.org/wiki/Communicating_sequential_proce...), with the advantage for Go over Erlang that its syntax is more familiar to programmers who know C.
Erlang does not implement CSP, and if you review the very Wikipedia link you presented it should be clear. Erlang implements the Actor model, which is more flexible for distributed, fault-tolerant systems, but lacks CSP's strict formalism and synchronization semantics. The very Wikipedia article you linked to has an entire section on how CSP differs from the Actor model. Similarly Go also does not implement CSP either, although it certainly is influenced by it.
The point of my comment is that to say that Go is basically Erlang style CSP without the weird syntax is not justified as an actual quote nor as a paraphrasing or summary of anything that anyone involved in the design or promotion of Go has ever said. It's best to reserve quotes for situations where someone actually said something or as a way to summarize something for which there are ample references available.
One should not use quotes as a way to present mostly original claims that are presented as if it's some kind of already well established knowledge.
The "X is like Y but Z" game w languages is pretty fun, and kind of illuminating as to what one thing you pick.
Go is like C but with concurrency/strings/GC/a good stdlib
Go is like C++ but simpler/fast compilation/no generics/a good stdlib
Go is like Python but statically typed/multithreaded/fast/single executable
Go is like Java but native/no OO
---
One thing Go haters rarely reckon with is that Go is the only popular modern language (ie from this millennium). Everything else is way older. Well, I would actually say that this is probably the cause of Go hate--if it weren't popular no one would care.
> One thing Go haters rarely reckon with is that Go is the only popular modern language (ie from this millennium).
This requires you to squint so that "popular" happens to identify only a handful of languages but conveniently catches Go and not say Swift or Rust. It's not difficult to do this, but it's not very honest to yourself.
"Go is on the frontier of recency vs popularity (i.e. newer than all more popular languages, and more popular than all newer languages)" is honestly an "interesting" statement in its own right, but would be a distinction it has to share with Python, Java, some of JS/C#/VB/PHP depending on your popularity (and age...) metrics, and yes, Swift and Rust. And a long tail of very new languages, I suppose.
Just searching for "programming language popularity":
- TIOBE [0]: Go 7 :: Rust n/a (> 10)
- IEEE Spectrum [1]: Go 8 :: Rust 11
- Stack Overflow [2]: Go 13 :: Rust 14 (they're pretty close here)
- GitHub [3]: Go 10 :: Rust n/a (> 10)
- PYPL [4]: Go 12 :: Rust 10
- HackerRank [5]: Go 8 :: Rust n/a (> 13)
- Pluralsight [6]: Go 9 :: Rust n/a (> 10)
- Redmonk [7]: Go 12 :: Rust 19
Although I admit we probably don't have great measures of this, there's clearly a sizable gap here.
Maybe you'd want to argue about TypeScript, Swift or Kotlin? I consider TypeScript JavaScript (because TypeScript wouldn't be popular unless all JavaScript programs were TypeScript programs), and I think Swift and Kotlin only survive because of their mobile platforms (they're also all below Go in these lists, on average).
You seem to be arguing that Go is more popular than Rust, but that's not what we're talking about.
The claim made was "Go is the only popular modern language (ie from this millennium)"
That's a binary, either a programming language is "modern" (from this millennium) or not and either it is "popular" (undefined) or it is not, Go is popular and modern according to the claim.
On your lists you find Go is anywhere from 7th to 13th in popularity. So apparently "popular" might mean 9th like Ada on TIOBE's list right? Or maybe "modern" includes Typescript, on several of these lists?
No, the whole contrivance is silly. Go is a relatively popular modern language, nobody is surprised to discover that. Is it the most popular? No. Is it the most modern? Also no.
TL;DR: mostly what I mean is Go is the only language from this millennium that's consistently in the top 10 most popular programming languages.
>> This requires you to squint so that "popular" happens to identify only a handful of languages but conveniently catches Go and not say Swift or Rust. It's not difficult to do this, but it's not very honest to yourself.
> You seem to be arguing that Go is more popular than Rust, but that's not what we're talking about.
Isn't this exactly what we're talking about? I don't think you have to squint too hard to invent the gap between Go and Rust (et al).
> So apparently "popular" might mean 9th like Ada on TIOBE's list right?
No because it's only on that list.
> Or maybe "modern" includes Typescript, on several of these lists?
No because TypeScript is JavaScript, which is from the last millennium.
> No, the whole contrivance is silly.
Emotionally I agree with you, but in practice you gotta pick a stack, and there's a lot of benefit to choosing Go over Rust/Kotlin/Swift/TypeScript/Java/C#/C/C++/Ada.
> Go is a relatively popular modern language, nobody is surprised to discover that.
Eh I'm making the Stroustrup "there's two kinds of programming languages" argument. IMO, at this point Go criticisms are just jeers from the cheap seats.
> TL;DR: mostly what I mean is Go is the only language from this millennium that's consistently in the top 10 most popular programming languages.
So, if you don't count the lists where it isn't in the top ten, and you don't count languages like Typescript? Does that feel like an important distinction to you with the exceptions, rather than an arbitrary post doc justification ?
> Isn't this exactly what we're talking about? I don't think you have to squint too hard to invent the gap between Go and Rust (et al).
Likewise for Go and Python or Javas, so why the "top 10" ? Arbitrary.
> TypeScript is JavaScript
No. Javascript is (very bad) Typescript, but Typescript is not Javascript. That's why they have a transpiler.
If you contend that the similarity means they're the same language that makes C++ also C and I don't think you want to start that fight.
My pithy response to the Stroustrup argument is a T-shirt I own which says "Haters gonna make some good points". Yes of course people will criticize your popular language, but this observation does not make the criticisms untrue, and resorting to Stroustrup's argument is best understood as an admission that he has no response to the actual criticism.
C++ is remarkably bad. You're looking at popularity lists. What else is on those lists which has similar levels of criticism? Go is a long way short of perfect but it's nowhere close to C++. If Go is the Stallone "Judge Dredd" then maybe C++ is "Batman & Robin".
(I've successfully resisted the urge to make my entire reply "Wait, you don't like Judge Dredd?")
I feel like we both understand each other at this point so I'll ask because I'm curious, what languages are you into? I'd really like a chance to dig into Erlang, OCaml, or Racket, but I can never really justify it. Mostly I'm a boring C/Python person (believe it or not, I don't like Go all that much)
I wrote a reply to a sibling comment here [0], but wanted to reemphasize that Rust and Swift are far from popular. I think it bears repeating because the gap between a Rust/Swift and Go is notable, but the gap between a Rust/Swift and Java is a chasm. I would guess this isn't the intuition of an HNer (it wasn't mine until I dug in for this discussion haha)
The median number of users across all programming languages is zero. Millions of people around the world use Rust and Swift, so it seems like a stretch to say they're far from popular. If Rust isn't a popular language compared to Java, we might just as well say Java is not a popular language compared to Excel.
I was an early Golang dev and people were _crazy_ with channels for a couple years. I remember the most popular Golang Kafka client was absolute spaghetti of channels and routines.
It's never been "safe C" because it's garbage collected. Java is truly the comp because it's a great Grug language.
I also wrote some Erlang in the past, I really enjoy it and I was sad that Go didn't borrow more.
This is "share by communicating, don't communicate by sharing". Pretty much everyone agrees it's a good thought. ~Most programs get there just fine with channels, because ~most code isn't so performance sensitive that channel limitations matter. Go leaves plenty of room for the rest of programs to do something else. Seems good.
I remember very well one of the first public presentations about Go. It focused heavily on goroutines and channels and included a live demonstration of pushing an element through one million channels. It also included a demo of spinning up three racing queries to the Google search engine using the select statement, and picking whoever returned first. it was all about the new cool feature. They also had TCP-over-channels and eventually had to remove that because the model didn’t fit.
Nobody may have known they cared about Erlang, but those features sure made people pay attention.
Just as a someone with a hammer could truthfully claim they don’t need a nail gun.
Not that I think Erlang manages to be a nail gun; it has enough idiosyncrasies that the comparison is not terribly accurate. Still, “need” is doing a lot of heavy lifting in that sentence.
> The actual core of OTP was the supervisor tree but that's too complicated so Golang is basically just more concise Java.
Which is ironic, maybe kinda funny. The first "larger" project I've tackled in Go was a chat server. I wanted a simple supervisor for each connected client; in case a goroutine encountered a recoverable error but needed to be restarted. I don't have any practical experience with OTP, but I've always been a big fan of daemontools/runit/etc: just let it crash, restart, and recover.
So in Go, you can't (easily) obtain a handle to a goroutine. The authors' entire argument seemed to be that this would allow developers to implement "bad" patterns, like thread-local storage. You can of course still come up with some wrapper code, like this:
type Service struct {
err <-chan error
run func(*Service)
}
s := &Service{err: make(chan error)}
realRun := func() error { ... }
s.run = func(s *Service) {
s.err <- realRun()
}
go s.run(s)
err := <-s.err
But what you really wanted was something like:
run := func() error { ... }
g := go run()
err := <-g
Now of course you still want some wrapper code to maintain client/connection state, etc. But if you wanted a "real" supervisor tree, you'd have to do this dance for every sub-goroutine you'd like to spawn. You'll soon end up with func(*Service, any) and throw away static typing along the way. Generics wouldn't be introduced until 18 releases after, and I don't think they would help all that much.
Definitely agree that goroutines don't suck; it makes go into one of the only languages without "function coloring" problem; True N:M multithreading without a separate sync and async versions of the IO libraries (thus everything else).
I think channels have too many footguns (what should its size be? closing without causing panics when there are multiple writers), thus it's definitely better "abstracted out" at the framework level. Most channels that developers interact with is the `Context.Done()` channel with <-chan struct{}.
Also, I'm not sure whether the go authors originally intended that closing a channel would effectively have a multicast semantics (all readers are notified, no matter how many are); everything else have pub-sub semantics, and turns out that this multicast semantics is much more interesting.
Ahh finally someone has said it. Unfortunately I can't seem to voice this opinion without getting the critique of you're just not smart enough to get it.
While I like the language, threads in Go are not any easier than any other language (which is to say, most devs can't use them correctly, and your program will have bugs), and suffer from a ton of ergonomic issues, like being hard to keep track of, difficult(ish) cancellation(how do you cancel a big synchronous I/O operation), and channels suffer from backpressure related hard-to-debug issues.
Most programmers are "lousy programmers" because most of the are not researchers and have no interest in theory.
I can write in Go or Rust (or anything else given some time) but I won't ever use Rust to write any kind of business logic for my company - and this is what I do most of the time I actually write code.
Why? Because Rust is terrible for the job (at least if we are talking about real life scenario where the job should be done in hours and not weeks)
I can do business logic, like complex CRUD, in Rust much faster than in other languages. I assume, «lousy programmers» have problems with separation of concerns, which then cause problems with borrow checker, because developers need to perform different things in different ways on different parts of data structures (AKA «god object» anti-pattern).
This has nothing to do with Rust vs Go vs Ruby or something else. Rust makes this even bigger problem, ok, but I don't think we should call people not using Rust (c\c++ etc) "lousy".
PS: tbh this kind of statements usually (from my experience) come from people who mainly write system level software or work with hardware directly.
I think that's harsh. IME Go excels in a business setting where the focus is on correct, performant, maintainable, business logic in larger organizations, that's easy to integrate with a bunch of other systems. You can't squeeze every last bit of low-level performance out of it but you can get ... 9x% of the way there with concurrent code that is easy to reason about.
What matters is being 100x faster than python, not 5x slower than C++ or 2x slower than Java. Performant enough that IO (or network) rather than CPU is the bottleneck (in my limited experience).
Personally I just far prefer to work in a GCed langauge. Much simpler mental model.
Flamebait? It's literally what the designers of Golang said publicly about the background of prospective developers, and how that constrained the language design:
"The key point here is our programmers are Googlers, they’re not researchers. They're typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They're not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt."
Nowhere in this quote are these fresh grads equated to "lousy programmers", though (which the flamebaity comment did).
And interpreting the quote charitably I'm going to have to agree with it - I don't think many of my coworkers care enough to get to the point where they'd appreciate everything something like Haskell can do for them.
Have you ever worked with fresh grads? They're definitionally lousy programmers, if we take lousy mean "having a propensity to inject bugs into programs".
The more time you spend in software engineering the more you realize that this is actually brilliant and helpful, not bad. And IME, the more expressivity I have fluently at my disposal, the more complexity I tend to create. I’m not saying I don’t enjoy more complex languages, but reducing cognitive load and easing collaboration with more junior engineers are easily the best features of Go, hands down (although I can’t say the new generics necessarily always help with that).
That's one interpretation but I think Pike was using a sarcastic meaning for brilliant. I think he's saying that he wants to mentor people to become programmers, not to learn a difficult, sublime language. It's like when a top scientist tells you they're not smart enough to understand what's going on in some part of his field. It's not necessarily a compliment.
Feel free to listen to the context at https://learn.microsoft.com/en-us/shows/lang-next-2014/from-... around 20:40 to 21:10. It's pretty clear that it's a serious quote and not "sarcastic" at all. Least of all about research languages since Pike mentions CSP at length and how to make concurrency be not "scary" to prospective users - overall, it seems that he's talking about a real constraint he's facing. In the same talk he's even apologetic for not preventing data races in the language, since it would have involved too much complexity.
There are a lot of those for sure. There are far more in Python and Java. Any popular, simple-enough language will attract a horde of jobseeking mouthbreathers, that’s kind of inevitable and normal really.
> goroutines are not really a lot better than threads in other languages
They are though. Even Java people agree, Java recently got their implementation of goroutines (green threads). It's a superior model to async for application level programming.
But gorutines are better than threads when you need you need many (say >100K) of them. Lightweight threads are not unique to Go but in Go they are easy to use and the default way to build network applications. Java is catching up with project Loom but it would never be as easy as Go to use.
> are better than threads when you need many (say >100K) of them
Stackless coroutines are even more efficient for that case, and Rust makes them comparatively easy. (And slated to get even easier in future versions, as the async support is improved further.)
Moreover stackful coroutines/fibers as used in Golang also makes it infeasible to have seamless FFI with the standard C API/ABI, which cuts you off from the bulk of the ecosystem. There are other issues too with having fibers in a C-like systems programming language - see https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p13... for a nice summary of them.
Alef (arguably one of the antecedent of Go) had stackful couroutines/fibers yet it could have had such a seamless FFI to the standard C API/ABI.
The plan9 libthread (which reimplemented the Alef concurrency model for C) did have seamless use of the C API/ABI.
The mechanism was that it had threads and coroutines, a function needing to make use of the C API in a seamless manner would simply run as a thread rather than a coroutine. It was then easy to share data as the CSP model (with channels) worked between both coroutines, threads, and coroutines within a thread.
So if one wishes to use stackful coroutines, and still have that seamless compatibility, an approach mixing the Go and Alef approaches would seem necessary. i.e. the Go migrating coroutines as a default, but with the option to use Alef like thread bound coroutines when necessary.
> The mechanism was that it had threads and coroutines, a function needing to make use of the C API in a seamless manner would simply run as a thread rather than a coroutine.
This reintroduces the colored functions that we were trying to get away from in the first place, by adopting stackful coroutines/fibers. Why not use async at that point? I can understand that Alef didn't, because stackless mechanisms were not well understood at the time. But it's plausible that we can do better.
Stackless coroutines are what every language that has async/await (Js, C#) uses. I've seen them getting a lot of hate here for needing a special kind of method to run in and being less elegant than green threads.
> but people realized channels kind of suck and goroutines are not really a lot better than threads in other languages. The actual core of OTP was the supervisor tree
Immutability is the actual core and power of Earlang
It was interesting towards the latter half of the article where the author talks about how much of the correctness may be culturally enforced:
>More amorphous, but not less important is Rust's strong cultural affinity for correctness. For example, go to YouTube and click on some Rust conference channel. You'll see that a large fraction of the talks are on correctness, in some way or another. That's not something I see in Julia or Python conference talks.
And it creates an interesting chicken and egg approach. The borrow checker may indeed be too strict (and of course, has its edge cases and outright bugs), but its existence (rather than the utility it brings) may have in fact attracted and amassed an audience who cares about correctness above all else. Even if we abolished the borrow checker tomorrow, this audience may still maintain a certain style based on such principles, party because the other utilities of Rust were built around it.
It's very intriguing. But like anything else trying to attract people, you need something new and flashy to get people in the door. Even for people who traditionally try to reject obvious sales pitches.
This is also somewhat backed up by the fact that OCaml (to my understanding) is basically GC Rust without a borrow checker, and yet it’s basically a hobby language.
Idiomatic programming in a functional language requires garbage collection. There is a reason languages like OCaml and Haskell have a garbage collector. Without it, programming in these languages would be completely different.
If you look at it from that perspective, then Rust is the hobby language.
> Idiomatic programming in a functional language requires garbage collection.
Rust has functional programming features and no garbage collection, because the borrow checker can tell when a closure will outlive the references in its captured environment. We used to think that would not be feasible other than perhaps in very special cases - hence the need for GC to keep that environment around - but Rust proved that wrong quite convincingly.
"Having FP features" does not mean "allow an idiomatic FP programming style". If this were the case, we'd be seeing a revolution in functional programming now. It's not like the FP community is slow to adopt good ideas. In fact, many innovations that lead to the invention of the borrow checker came from the functional programming world (linear, affine types, effect systems).
What's an "idiomatic FP programming style"? Is typical LISP code "FP" enough? That uses GC of course, but other than that it's often high-level enough to be quite comparable w/ modern Rust.
SML was a generation before ocaml. I would say the two languages from the same generation that competed for academia's mindshare were ocaml and Haskell.
I do not at all agree with this. Rust is by far the most complex language in terms of syntax that has ever become popular enough to compare it to anything.
I think some of that is just that C++ doesn’t explicitly express its semantic complexity in its syntax, while Rust does. In some ways this is an advantage for Rust, although yeah I agree it makes Rust really hard to use, and I kinda hate its syntax.
C++ is a stretch, but I sort of understand GP's point. I'm slowly learning Rust now and it feels like a deep evolution of C type syntax. Not dissimilar to Javascript in the most base sense of language constructs.
But you use it more and see actionscript types of function notation, funtional language semantics where you just "return" whatever was the last expression in a statement, how structs have no bodies (and classes aren't a thing) and instead everything is implemented externally, and it starts to really become its own beast.
I didn't think it was that controversial a statement. For example, Wikipedia says:[0]
> Rust's syntax is similar to that of C and C++,[43][44] although many of its features were influenced by functional programming languages such as OCaml.[45] Hoare has described Rust as targeted at frustrated C++ developers...[15]
I'm not sure what "actionscript types of function notation" means, but Rust's closures syntax (|x| ...) was probably inspired by Ruby and/or Smalltalk. Anyway, sure, the expression-orientedness, implicit return, etc., are very un-C++-like, but I do think the syntax was explicitly designed to be approachable to C++ developers.
In my eyes, Rust code is as simple as Python code or even simpler in most cases, but it also allows to create and maintain complex pieces of abstraction, when necessary, then hide them in a library or a macro. :-/
Can you show example of "written by random cat" code you often see?
I wouldn't read too much into its lacking the borrow checker.
It's not about not having a C-like syntax (huge mainstream points lost), good momentum, and not having the early marketing clout that came from Rust being Mozilla's "hot new language".
Facebook Messenger's backend was/is OCaml... React was originally written in SML, then OCaml, then whatever it is now. And a bunch of places use it for various things.
React was never written in SML or Ocaml. It was originally called FaxJS, and the source code is published online.
A version of React was built to run in ReasonML, which is a flavor of Ocaml for the web, but Reason didn't even exist before React was fairly well established.
That's a nonsensical point, though. Building a proof of concept in a language and then rebuilding the practical implementation of it in another language and runtime doesn't make the two the same thing. If Notch had built a proof of concept of Minecraft in Python before building the Java version, we wouldn't say Minecraft was originally written in Python. There wasn't even a robust way to compile OCaml for the web in 2010/2011 even if you wanted to try to use the same code. My understanding is that zero SML or Ocaml related to React ever ran in production, which makes the assertion that it was used in anything other than an academic capacity moot.
Hell, Facebook's own XHP's interface (plus PHP/Hack's execution model) is more conceptually relatable to React, and its initial development predates Jordan's time at Facebook. It wasn't JavaScript, but at the very least it defined rails for writing applications that used the DOM.
> Yes, the first prototype of React was written in SML; we then moved onto OCaml.
> Jordan transcribed the prototype into JS for adoption; the SML version of React, however great it might be, would have died in obscurity. The Reason project's biggest goal is to show that OCaml is actually a viable, incremental and familiar-looking choice. We've been promoting this a lot but I guess one blog post and testimonial helps way more.
Sure, a prototype of an idea that would eventually become React was rewritten into JS to create the initial seed of the software that would eventually be called React.
Realistically unless you want to work at Jane Street or Inria (the French computer science lab where Ocaml was made), if you want to use Ocaml, it's going to be as a hobby.
You can say that for almost any language that's not C/C++, C#, Java, Python and JS. Rust is just barely beginning to become "corporate". Even Ruby, which is pretty mainstream, has relatively few jobs compared to the big corporate languages.
Ummm Lua? It's a nice little scripting language, but literally never seen a job ad for a job using mostly Lua. It's almost the definition of hobby language...
OCaml runs software that billions use, is used by financial and defense firms, plus Facebook.
But Lua? By that metric I'm throwing in every language I've ever seen a job for...
R, Haskell, Odin, Lisp, etc...
Edit - this site is basically a meme at this point. Roblox is industrial strength but Facebook, Dassault and trading firms are "hobby". Lol.
Also, I'm not dissing Lua, there's just irony in calling Lua industrial but not OCaml...
Lua has petered out a bit but it has been used as a scripting and config language for a ton of games and commercial embedded. Not a hobby language, not typically a main implementation language but that doesn’t mean no commercial use. posix/bash shell isn’t a hobby language either, but unless you’re Tom Lord or something (RIP) you’re not doing the entire project in it.
Do realize that luajit for years was bankrolled by corporations.
Lua, Bash ... these are birds of a feather. They are the glue holding things together all over the place. No one thinks about them but if they disappeared over night a LOT of stuff would fall apart.
As others already pointed out, Lua is used in tons of video games as the scripting language.
The most famous example being World of Warcraft, but it's far from the only one. If you play, or have played, games, you almost certainly have run software built with Lua without realizing it.
It's not because a language isn't relevant in your personal coding niche that it's not industrially relevant.
I'm simply pointing out the irony in calling a (mostly game) scripting language like Lua "industrial" while calling a language used by FAANG, defense companies and finance companies a "hobby" language.
There's no irony, bash and VB6, no matter what you think about the quality of the said languages, are also scripting language and neither are in the “hobby language” category as their use is (or was) very broadly distributed.
OCaml's use is comparatively very, very narrow.
And, in case you are wondering, I have absolutely nothing again OCaml. In fact, my first ever programming language was, as a significant fraction of French engineer from my generation, the “Lite” dialect of Caml. And I suspect that its OCaml heritage is a significant fraction of the reason why I love Rust.
But being used by exactly one FANG company, a single finance one and allegedly a defense company (aren't you confusing Dassault System with Dassault Aviation ?) isn't enough to change its status, especially when it's not the dominant language in two of those (AFAIK Jane Street really is the only one where OCaml has such a central place).
The difference between academia languages such as ocaml or haskell and industry languages such as Java or C# is hundreds of millions of dollar in advertising. It's not limited to the academy: plenty of languages from other horizons failed, that weren't backed by companies with a vested interest in you using their language.
You should probably not infer too much from a language's success or failure.
The main difference is the ecosystem. The Haskell community has always focused primarily on the computer science part, so the developer experience has mostly been neglected. They have been unable to attract a large enough hobbyist community to develop the libraries and tooling you'd take for granted with any other language, and no company is willing to pay for it out of pocket either. Even load-bearing libraries feel like a half-finished master's thesis, because that's usually what it is.
No amount of advertising is going to propel Haskell to a mainstream language. If it wants to succeed (and let's be honest, it probably doesn't), it's going to need an investment of millions of developer-hours in libraries and tooling. No matter how pretty and elegant the language may be, if you have to reinvent the wheel every time you go beyond "hello world" you're going to think twice before considering it for production code.
C, C++, Python, Perl, Ruby, didn't have "millions of dollars in advertising", and yet.
Java and C# are the only one's that fit this. Go and Rust had some publicity from being associated with Google and Mozilla, but they both caught on without "millions of dollars in advertising" too. Endorsement by big companies like MS came much later for Rust, and Google only started devoting some PR to Go after several years of it already catching momentum.
I don't know about the others but C and C++ certainly did. They had a number of commercial compiler vendors advertising them in the 80s and 90s when they established themselves.
You're making it sound like the success of a language is determined purely by its advertising budget by pointing at languages that had financial backing, which disregards that financial backing allows for more resources to solve technical problems. Java and C# have excellent developer tools which wouldn't have existed in their current state without lots of money being thrown around, and the languages' adoption trajectory wouldn't have looked the way they did if their tooling hasn't been as good as it was. A new language with 3 people behind it can come up with great ideas and excellent execution, but if you can't get enough of the scaffolding built in order to gain development momentum and adoption, then it is very hard to become mainstream, and money can help with that.
> The difference between academia languages such as ocaml or haskell and industry languages such as Java or C# is hundreds of millions of dollar in advertising
About 20 years ago your choice of language basically boiled down to what you were going to pick for your web server. Your choices were
- Java (popular among people who went to college and learned all about OOP or places that had a lot of "enterprise" software development)
- Ruby on Rails (which was the hot new thing)
- Python or Perl to be the P in your LAMP stack
- C++ for "performance"
All of these were kitchen sink choices because they wound up needing to do everything. If you went back in time and said you were building a language that didn't do something incredibly common and got in the way of your work, no one would pick it up.
Not everybody is writing web applications. In fact, all of the languages have been invented years or even decades before the internet became popular. Except maybe for Perl, they have been designed as general purpose programming languages.
I'm not sure.. without the borrow checker you could have a pretty nice language that is like a "pro" version of golang, with better typing, concise error handling syntax, and sum types. If you only use things like String and Arc objects, you basically can do this, but it'd be nice to make that not required!
That's my whole point. Without the borrow checker it would have been a nice language, but I believe it would not have gotten popular, because being nice isnt enough to be popular in the current programming language landscape.
As a Rust fan, I 100% agree. I already know plenty of nice, "safe", "efficient" languages. I know only one language with a borrow checker, and that feature has honestly driven me to use it in excess.
Most of my smaller projects don't benefit so much from the statically proven compile time guarantees that e.g. Rust with it's borrow checker provide. They're simple enough to more-or-less exhaustively test. They also tend to have simple enough data models and/or lax enough latency requirements that garbage collectors aren't a drawback. C#? Kotlin? Java? Javascript? ??? Doesn't matter. I'm writing them in Rust now, and I'm comfortable enough with the borrow checker that I don't feel it slows me down, but I wouldn't have learned Rust in the first place without a borrow checker to draw me in, and I respect when people choose to pass on the whole circus for similar projects.
The larger projects... for me they tend to be C++, and haven't been rewritten in Rust, so I'm tormented with a stream of bugs, a large portion of which would've been prevented - or at least made shallow - by Rust's borrow checker. Every single one of them taunts me with how theoretically preventable they are.
You can use a garbage collector in Rust to circumvent borrow checker. You can use simple reference counting (Rc, Arc), or trace and sweep, arenas, or generation based garbage collectors. Even a simple .clone() can help a lot in many cases.
Borrow checker is my friend, it helps me write better code, but it doesn't stops me when I don't care about code quality and just want a task to be done.
> without the borrow checker ... golang... concise error handling syntax
Except both of these things are that way for a reason.
The author talks about the pain of having other refactor because of the borrow checker. Every one laments having to deal with errors in go. These are features, not bugs. They are forcing functions to get you to behave like an adult when you write code.
Dealing with error conditions at "google scale" means you need every one to be a good citizen to keep signal to noise down. GO solves a very google problem: don't let JR dev's leave trash on at the campsite, force them to be good boy scouts. It is Conways law in action (and it is a good thing).
Rust's forced refactors make it hard to leave things dangling. It makes it hard to have weak design. If you have something "stable", from a product, design and functionality standpoint then Rust is amazing. This is sort of antithetical to "go fast and break things" (use typescript, or python if you need this). It's antithetical to written in the stand up requirements, that change week to week where your artifacts are pantomime and post it notes.
Could the borrow checker be better, sure, and so could errors in go. But most people would still find them a reason to complain even after their improvement. The features are a product of design goals.
The lamentations I usually hear about errors in Go are that you have to use a product type where a sum type would be more appropriate, and that there isn't a concise syntax analogous to Rust's ? operator for the extremely common propagate-an-error-up-a-stack-frame operation, not that you have to declare errors in your API.
Also, in my experience, the Rust maintainers generally err on the side of pragmatism rather than opinionatedness; language design decisions generally aren't driven by considerations like "this will force junior developers to adhere to the right discipline". Rust tries to be flexible, because people's requirements are flexible, especially in the domain of low-level programming. In general, they try to err on the side of letting you write your code however you want, subject to the constraints of the language's two overriding design goals (memory safety and precise programmer control over runtime behavior). The resulting language is in many ways less flexible than some more opinionated languages, but that's because meeting those design goals is inherently hard and forces compromises elsewhere (and because the language has limited development resources and a large-but-finite complexity budget), not because anyone views this as a positive in and of itself.
(The one arguable exception to this that I can think of is the lack of syntactic sugar for features like reference counting and fallible operations that are syntactically invisible in some other languages. That said, this is not just because some people are ideologically against them; they've been seriously considered and haven't been rejected outright, it's just that a new feature requires consensus in favor and dedicated resources to make it happen. "You can do the thing but it requires syntactic salt" is the default in Rust, because of its design, and in these cases the default has prevailed for now.)
You can absolutely "go fast and break things" (i.e. write prototype-quality code) in Rust, but it requires a lot of boilerplate that will be very visible in the code, and will also make it comparatively easy to refactor the prototype into a real production-quality implementation. You can't really say this of any other languages AIUI. What often happens instead is that the prototype is put in production more or less as-is, without comprehensively fixing the breakage. Rust makes it very clear how to avoid that.
Java, especially after generics were introduced, was a pain to use because of the type system. That’s not my opinion, I always found that claim a bit overwrought but it’s true that it was fairly widespread. Dealing with the type system got progressively a bit easier as the language evolved and certain types could be inferred. From the release notes I’ve seen I’ve gotten the impression that similar things have happened with the borrow checker. But people who have gone through the gauntlet have trouble reframing that experience and it’s always difficult to tell if the reputation is still accurate or not.
I am curious what the second language with a borrow checker will look like.
There are some artificial limitations, but I love the upside: I don't need defensive programming!
When my function gets an exclusive reference to an object, I know for sure that it won't be touched by the caller while I use it, but I can still mutate it freely. I never need to make deep copies of inputs defensively just in case the caller tries to keep a reference to somewhere in the object they've passed to my function.
And conversely, as a user of libraries, I can look at an API of any function and know whether it will only temporarily look at its arguments (and I can then modify or destroy them without consequences), or whether it keeps them, or whether they're shared between the caller and the callee.
All of this is especially important in multi-threaded code where a function holding on to a reference for too long, or mutating something unexpectedly, can cause painful-to-debug bugs. Once you know the limitations of the borrow checker, and how to work with or around them, it's not that hard. Dealing with a picky compiler is IMHO still preferable to dealing with mysterious bugs from unexpectedly-mutated state.
In a way, borrow checker also makes interfaces simpler. The rules may be restrictive, but the same rules apply to everything everywhere. I can learn them once, and then know what to expect from every API using references. There are no exceptions in libraries that try to be clever. There are no exceptions for single-threaded programs. There are no exceptions for DLLs. There are no exceptions for programs built with -fpointers-go-sideways. It may be tricky like a game of chess, but I only need to consider the rules of the game, and not odd stuff like whether my opponent glued pieces to the chessboard.
Yes! One of the worst bugs to debug in my entire career boiled down to a piece of Java mutating a HashSet that it received from another component. That other component had independently made the decision to cache these HashSet instances. Boom! Spooky failure scenarios where requests only start to fail if you previously made an unrelated request that happened to mutate the cached object.
This is an example where ownership semantics would have prevented that bug. (references to the cached HashSets could have only been handed out as shared/immutable references; the mutation of the cached HashSet could not have happened).
The ownership model is about much more than just memory safety. This is why I tell people: spending a weekend to learn rust will make you a better programmer in any language (because you will start thinking about proper ownership even in GC-ed languages).
> This is an example where ownership semantics would have prevented that bug.
It’s also a bug prevented by basic good practices in Java. You can’t cache copies of mutable data and you can’t mutate shared data. Yes it’s a shame that Java won’t help you do that but I honestly never see mistakes like this except in code review for very junior developers.
The whole point is that languages like Java won't keep track of what's "shared" or "mutable" for you. And no, it doesn't just trip up "very junior developers in code review", quite the opposite. It typically comes up as surprising cross-module interactions in evolving code bases, that no "code review" process can feasibly catch.
Speak for yourself. I haven't seen any bug like this in Java for years. You think you know better and my experience is not valid? Ha. Ok. Keep living in your dreams.
> When my function gets an exclusive reference to an object, I know for sure that it won't be touched by the caller while I use it, but I can still mutate it freely.
I love how this very real problem can be solved in two ways:
1. Avoid non-exclusive mutable references to objects
2. Avoid mutable objects
Former approach results in pervasive complexity and rigidity (Rust), latter results in pervasive simplicity and flexibility (Clojure).
Shared mutable state is the root of all evil, and it can be solved either by completely banning sharing (actors) or by banning mutation (functional), but Rust gives fine-grained control that lets you choose on case-by-case basis, without completely giving up either one. In Rust, immutability is not a property of an object in Rust, but a mode of access.
It's also silly to blame Rust for not having flexibility of a high-level GC-heavy VM-based language. Rust deliberately focuses on the extreme opposite of that: low-level high-performance systems programming niche, where Clojure isn't an option.
This reminds me of something that was popular in some bioinformatics circles years ago. People claimed that Java was faster than C++. To "prove" that, they wrote reasonably efficient Java code for some task, and then rewrote it in C++. Using std::shared_ptr extensively to get something resembling garbage collection. No wonder the real Java code was faster than the Java code written in C++.
I've been writing C++ for almost 30 years, and a few years of Rust. I sometimes struggle with the Rust borrow checker, and it's almost always my fault. I keep trying to write C++ in Rust, because I'm thinking in C++ instead of Rust.
The lesson is always the same. If you want to use language X, you must learn to write X, instead of writing language Y in X.
Using indexes (or node ids or opaque handles) in graph/tree implementations is a good idea both in C++ and in Rust. It makes serialization easier and faster. It allows you to use data structures where you can't have a pointer to a node. And it can also save memory, as pointers and separate memory allocations take a lot of space when you have billions of them. Like when working with human genomes.
If using indices is going to be your answer, then it seems to me you should at least contend with the OP's argument that this approach violates the very reason the borrowchecker was introduced in the first place.
From the post:
"The Rust community's whole thing is commitment to compiler-enforced correctness, and they built the borrowchecker on the premise that humans can't be trusted to handle references manually. When the same borrowchecker makes references unworkable, their solution is to... recommend that I manually manage them, with zero safety and zero language support?!? The irony is unreal."
That seems rather easy? There's still plenty of safety. It's not an `unsafe` trick or worse, you still have bounds checking and safe concurrency and well-defined behavior, and you can trivially guarantee that (while mutating the vec) it cannot be changed unexpectedly while you hold ownership of the vec. The extreme ease of making that guarantee is kinda the core of their complaint.
The dangling-pointer equivalent is an issue, but it's still safe (unlike in C-like langs) and there are ways to mitigate the risk of accidental misbehavior (e.g. generational pointers, or simply an ID check if you have a convenient ID).
That's quite a bit different than what you get in almost any other widely-used language - e.g. some will at best be able to claim "concurrent access won't lead to undefined behavior" via e.g. the GIL, but not prevent unexpected modification (e.g. "freeze" doesn't deep-freeze in the vast majority of languages).
>OP's argument that this approach violates the very reason the borrowchecker was introduced in the first place.
No it doesn't. I just don't think author understands the pitfalls of implementing something like a graph structure in a memory unsafe language. The author doesn't write C so I don't believe he has struggled with the pain of chasing a dangling pointer with valgrind.
There are plenty of libraries in C that eventually decided to use indexes instead of juggling pointers around because it's much harder to eventually introduce a use-after-free when dereferencing nodes this way.
Entity component systems were invented in 1998 which essentially implement this pattern. I don't find it ironic that the Rust compiler herds people towards a safe design that has been rediscovered again and again.
The borrow checker was introduced to statically verify memory safety. Using indices into graphs has been a memory safe option in languages like C for decades. I find his argument as valid as if someone said "I can't use goto? you expect me to manually run my cleanup code before I return?" Just because I took away your goto to make control flow easier it doesn't make it "ironic" if certain legitimate uses of goto are harder. Surely you wouldn't accept his argument for someone arguing for the return of goto in mainstream languages?
Whoah, hold on, the author isn't comparing writing graph structures in Rust to writing it in memory-unsafe languages --- they're comparing it to writing it in other memory-safe languages. You can't force a false dichotomy between Rust and C to rebut them.
The comparison is contrived precisely because he's comparing it to other memory-safe languages. The borrow checker was introduced because the goal of the language was a memory safe language without a runtime. Falling back to indices isn't "ironic", its exactly how you would solve the problem in C/C++.
If your argument is "well Rust should be like Julia and have a GC", well thats not Rust. That language also exists, its possibly called OCaml, but its not Rust.
The author is also suggesting that a data structure consisting of a sea of objects referencing each other works perfectly in Python. It doesn’t — Python’s GC can only collect it as a cycle, and this path is not immediate the way the refcount is. And you’re paying for refcounts. Even in a language with full tracing GC (Java, etc), you’re not winning any performance points by making a ton of objects all referencing each other.
Indices or arenas or such will result in better code in all these languages.
I think it's reasonable enough. The author already argued that there are reasons for non GC languages to exist, even if the performance doesn't matter to them.
One interpretation of the article is just the author doesn't personally like the borrow checker, but another interpretation is the author saying the borrow checker is just a bad abstraction.
So under the assumption that we don't have a GC available, what else can we compare the borrow checker against?
Author here. Yeah, I don't like the borrowchecker. But the motivation for me writing the article is the almost religious zeal with which many Rustaceans refuse to even acknowledge that the borrowchecker has a cost in terms of ergonomics, and that this translates to e.g. iteration speed.
You encounter borrowchecker issues? Well, you're just a beginner or not skilled enough. Rust makes you jump through hoops? No it doesn't, it just makes you pay upfront what you otherwise would have. It slows development? No, studies show it doesn't, you must be imagining it.
This is extremely annoying to be on the receiving end of. Even though it comes from a good place (mostly just excitement), it can feel like gaslighting.
Indices can be dangling in almost exactly the same way as pointers. Worse, it's easier to accidentally use-after-free clobber some other item in the same structure, because allocations are "dense." (Pointer designs on systems where malloc/free is ~LIFO experience similar problems.)
This is well known to be solved by using generational indexes. Its not a big deal. The author's entire post is an overreaction/rage-bait.
Basically any data structure like this where you want it relocatable in memory is going to use indirection like indexes or something instead of pointers. Its a very common use case outside of rust.
Might I suggest that the scpptool-enforced safe subset of C++ has a better solution for such data structures with cyclic or complex reference graphs, which is run-time checked non-owning pointers [1] that impose no restrictions on how or where the target objects are allocated. Unlike indices, they are safe against use-after-destruction, and they don't require the additional level of indirection either.
It’s pretty common to implement graphs in terms of arrays, not because of indices but because of cache locality.
So your “UB” and “non-UB” code would look effectively identical to the CPU and would take the same amount of debugging.
The reality is whether an index was tombstones and referenced or “deallocated” and referenced it is still a programmer fault that is a bug that the compiler could not catch
Ummm. Yeah it is. I'm sure you can come up with some cases where the impact of the bugs is roughly equivalent, but generally speaking, UB is a terrible class of bug.
For example, if the aho-corasick crate accidentally tries to look up a dangling state node, you'll get a panic. But if this were UB instead, that could easily lead to a security problem or just pretty much anything... because behavior is undefined.
So generally speaking, yes, this is absolutely a major difference and it matters in practice. You even have other people in this thread saying this technique is used in C for this exact reason. Leaving out this very obvious difference and pretending like these are the same thing is extremely misleading.
Author here. In scientific computing, there isn't much of a security implication of UB, and the most dangerous kind of error are when your program silently computes the wrong thing. And that is especially likely when you use indices manually.
You could say that UB enables all behaviour, including silently wrong answers, and you'd be right. But it's more likely to crash your program and therefore be caught.
Most importantly, the comparison to a raw pointer is not relevant. My blog post states that integers come with zero safety (as in: preventing bugs, not risk of UB) and zero language support and that is true. My blog post compares Rust with GC languages, not with raw pointer arithmetic. And it's clear that, when you compare to using GC references, manual indices are horribly unsafe.
You're engaging in a motte-and-bailey fallacy. Your motte is your much more narrow claim about Rust's advantages in a context where you claim not to care about undefined behavior as a class of bugs more severe than logic errors. And you specifically make this claim in an additional context where a GC language is appropriate. That's an overall pretty easy position to defend, and if that were clearly the extent of it, I probably wouldn't have responded at all.
But, that context has been dropped in this thread, and instead folks are making very general claims outside of that very restricted context. Moreover, your bailey is that you don't do a good job outlining that context either. Your blog's opening paragraphs mention nothing about that more constrained context and instead seem to imply a very general context. This is much harder to defend. You go on to mention scientific computing, but instead of it being a centerpiece of the context of your claim, it's just mentioned as aside. Instead, your blog appears to be making very broad claims. But you've jumped in here to narrow them significantly, to the point that it materially changes your point IMO. Let's just look at what you said here:
> The first time someone gave be this advice, I had to do a double take. The Rust community's whole thing is commitment to compiler-enforced correctness, and they built the borrowchecker on the premise that humans can't be trusted to handle references manually. When the same borrowchecker makes references unworkable, their solution is to... recommend that I manually manage them, with zero safety and zero language support?!? The irony is unreal. Asking people to manually manage references is so hilariously unsafe and unergonomic, the suggestion would be funny if it wasn't mostly sad.
There's no circumspection about the context. You're just generally and broadly dismissing this entirely as if it weren't a valid thing ever. But it absolutely is a valid technique and it has real practical differences with an approach that uses raw pointers. If the comparison is with a GC and that context is made clear, then yes, absolutely, the comparison point changes entirely! If you can abide a GC, then a whole bunch of things get easier... at some cost. For example, I don't think it's possible to write a tool like ripgrep with its performance profile in a GC language. At least, I've never seen it done.
> I don't think it's possible to write a tool like ripgrep with its performance profile in a GC language.
I think it is possible to make a language that has both a GC and a borrow checker, treating the GC types as a third level next to the stack and the heap, where complex referencial cycles can bé promoted to the GC, but the defaults push you towards fast execution patterns. Don't know if such a language would be successful in finding its niche. The only way I could see that, is of a non-gc mode could be enforced so that libraries can be written in the more restrictive, faster by default mode, while being consumed by application developers that have less stringent restrictions. This is no different in concept than Python libraries implemented in native languages. Making it the mode be part of the same language could help with prototyping pains: write with the GC and then refactor once at the end after the general design is mostly found.
> I think it is possible to make a language that has both a GC and a borrow checker, treating the GC types as a third level next to the stack and the heap, where complex referencial cycles can be 'promoted to the GC'
It's doable but there are a few issues with that whole idea. You need the ability to safely promote objects to GC-roots whenever they're being referenced by GC-unaware code, and demote them again afterwards. And if any GC object happens to control the lifecycle of any heap-allocated objects, it must have a finalizer that cleans them up RAII style to avoid resource leaks.
> And that is especially likely when you use indices manually.
But that is never the recommendation Rust practitioners would give for graph data structures. One would instead recommend using a generational arena, where each index holds their "generation". The arena can be growable or not. When an element is removed from the arena it gets tombstoned, marked as no longer valid. If a new value reuses a tombstoned position, its generation changes. This shifts the cost of verifying the handle is correct at the read point: if the handle corresponds to an index with a tombstone sentinel or has a different generation, the result of the read operation is None, meaning the handle is no longer valid. This is much better than the behavior of pointers.
For the record, I completely agree that using GC is appropriate for problems that deal with possibly-cyclic general graphs, and maybe even wrt. DAG structures whenever one wishes to maximize throughput and the use of simpler refcounting would involve heavy overhead. (This is a well-known pitfall wrt. languages like Swift, that enforce pervasive use of atomic reference counting.)
Since the use of "pluggable" GC in C-like or Rust-like languages is still uncommon, this generally means resorting to a language which happens to inherently rely on GC, such as Golang.
C doesn’t have objects, which are great for encapsulating data structures, but in C++ it’s not at all hard to write a graph data structure.
One wraps it behind a reliable interface, writes automated test and runs them under valgrind/sanitizers and that’s pretty much it.
It’s normal for life and software development to have a certain degree of risk and to spend some effort solving problems.
Too many HN comments make it sound like it’s a jungle out there and a use-after-free will chop your head clean off.
I don't mean to imply that writing a graph in C++ is impossible. It's clearly possible in modern C++.
My contention is treating indicies-based management as some tedious, manual workaround. Unity's ECS is written in C#, which has both a GC and the language expressiveness for an actual graph objects, but has adopted an indicies based system. It works, and is performant, so it isn't a mistake for the compiler to herd users down that path.
In a similar vein, the Linux codebase is full of completely legitimate and readable uses of goto, but we are perfectly happy with languages that force us to use structured control flow.
The OP's argument is bunk. It's been said many times too over the years. The fact is that the index approach does not give up everything. The obvious thing it doesn't give up is safety. It's true you can still get bugs via out of bounds accesses, but it won't result in undefined behavior. You get a panic instead.
This is how the regex crate works internally and uses almost no `unsafe`.
An important saving grace that `unsafe` has is that it's local and clearly demarcated. If a core data structure of your program can be compared to `unsafe` and has to be manually managed for correctness, it's very valid to ask whether the hoops Rust makes you jump through are actually gaining you anything.
There is such a thing of languages that align with human intuition. C++ and Rust are not these languages so you have to really learn these languages in depth. Languages like typescript or python or go align more with intuition and you don't really need to learn as much about the details or patterns as these just naturally flow from your intuition. This is a huge huge thing as it makes the language literally take about a week to develop proficiency and two weeks to develop mastery. A language like C++... you can't even develop mastery in a year.
That is not to say these languages are better. Intuition is just one trade off.
Typescript is not a language that matches intuition. Typing complex code while avoiding the any hatch resembles fighting limitations of the borrow checker in Rust.
I find it highly highly intuitive. But my background is haskell like languages.
I feel with practice basic type checking is something that helps you rather then hinders you. It can be learned easily imo. People coming from js tend to have a hard time but that's understandable.
The borrow checker is not easily learned imo. It's always me running into a wall.
For me the problem with TypeScript or Flow when the latter was a thing was that the syntax/semantics of the sub language of types was extremely ad-hoc with so many idiosyncrasies. Maybe if I programmed it all the time I would learned it. But I had to change the relevant code only occasionally and typing helpers to access DOM required constant look at the spec and StackOverflow.
With Rust the rules at least are simple. While following them can be a struggle the compiler errors at least are much more helpful and points to the problem with the design or the checker limitations.
> [The pain of the borrow checker is felt] when your existing project requires a small modification to ownership structure, and the borrowchecker then refuses to compile your code. Then, once you pull at the tiny loose fiber in your code's fabric, you find you have to unspool half your code before the borrowchecker is satisfied.
Probably I just haven't been writing very "advanced" rust programs in the sense of doing complicated things that require advanced usages of lifetimes and references. But having written rust professionally for 3 years now, I haven't encountered this once. Just putting this out there as another data point.
Of course, partial borrows would make things nicer. So would polonius (which I believe is supposed to resolve the "famous" issue the post mentions, and maybe allow self-referential structs a long way down the road). But it's very rare that I encounter a situation where I actually need these. (example: a much more common need for me is more powerful consteval.)
Before writing Rust professionally, I wrote OCaml professionally. To people who wish for "rust, but with a garbage collector", I suggest you use OCaml! The languages are extremely similar.
I guess I'm a bit confused how you can write rust professionally dor 3 years and never encounter this. When I started writing rust in ~2020/2021 i already had issues with the brorow checker.
Maybe its an idiom you already picked up in OCaml and did it mostly right in rust too?
I think I don't end up doing very complicated things most of the time. If you're writing a zero-copy deserialization crate or an ECS framework or something, I'm sure you're bound to run into this issue. But I almost never even have to explicitly write lifetimes. I rarely even see borrowck errors for code I intended to write (usually when I see borrowck errors, it's because I made an error in copy-pasting that resulted in me using a variable after it's been moved, or something like that).
You might have a point with my OCaml background though. I rarely use mutable references, since I prefer to write code in a functional style. That means I rarely am in a situation where I want to create a mutable reference but already have other references floating around.
I believe it. I experienced this once, as I tried to have everything owned. Now I just clone around as if there's no tomorrow and tell myself I'll optimize later.
I've mostly experienced it when moving from borrowing to ownership and vice versa. E.g. having a struct that takes ownership over its fields, and then moving it to a borrow with a lifetime.
It's not super common though, especially if the code is not in the hot path which means you can just keep things simple and clone.
For the disjoint field issues raised, it’s not that the borrow checker can’t “reason across functions,” it’s that the field borrows are done through getter functions which themselves borrow the whole struct mutably. This could be avoided by making the fields public so they can be referenced directly, or if the fields needs to be passed to other functions, just pass the the field references rather than passing the whole struct.
There are open ideas for how to handle “view types” that express that you’re only borrowing specific fields of a struct, including Self, but they’re an ergonomic improvement, not a semantic power improvement.
> For the disjoint field issues raised, it’s not that the borrow checker can’t “reason across functions,” it’s that the field borrows are done through getter functions which themselves borrow the whole struct mutably
Right, and even more to the point, there's another important property of Rust at play here: a function's signature should be the only thing necessary to typecheck the program; changes in the body of a function should not cause a caller to fail. This is why you can't infer types in function signatures and a variety of other restrictions.
Exactly. We've talked about fixing this, but doing so without breaking this encapsulation would require being able to declare something like (syntax is illustrative only) `&mut [set1] self` and `&mut [set2] self`, where `set1` and `set2` are defined as non-overlapping sets of fields in the definition of the type. (A type with private fields could declare semantic non-overlapping subsets without actually exposing which fields those subsets consist of.)
This seems to be a golden rule of many languages? `return 3` in a function with a signature that says it's going to return a string is going to fail in a lot of places, especially once you exclude bolted-on-after-the-fact type hinting like what Python has.
It's easier to "abuse" in some languages with casts, and of course borrow checking is not common, but it also seems like just "typed function signatures 101".
Are there common exceptions to this out there, where you can call something that says it takes or returns one type but get back or send something entirely different?
Many functional and ML-based languages, such as Haskell, OCaml, F#, etc. allow the signature of a function to be inferred, and so a change in the implementation of a function can change the signature.
In C++, the signature of a function template doesn't necessarily tell you what types you can successfully call it with, nor what the return type is.
Much analysis is delayed until all templates are instantiated, with famously terrible consequences for error messages, compile times, and tools like IDEs and linters.
By contrast, rust's monomorphization achieves many of the same goals, but is less of a headache to use because once the signature is satisfied, codegen isn't allowed to fail.
Concepts are basically a half solution - they check that a type has some set of properties, but they don't check that the implementation only uses those properties. As a result, even with concepts you can't know what types will work in a template without looking at the implementation as well.
Example [0]:
#include <concepts>
template<typename T>
concept fooable = requires(T t) {
{ t.foo() } -> std::same_as<int>;
};
struct only_foo {
int foo();
};
struct foo_and_bar {
int foo();
int bar();
};
template<fooable T>
int do_foo_bar(T t) {
t.bar(); // Compiles despite fooable not specifying the presence of bar()
return t.foo();
}
// Succeeds despite fooable only requiring foo()
template int do_foo_bar<foo_and_bar>(foo_and_bar t);
// Fails even though only_foo satisfies fooable
template int do_foo_bar<only_foo>(only_foo t);
> I'd say that's a mistake of the person who wrote the template then.
The fact that it's possible to make that mistake is basically the point! If "the whole point of concepts" were to "tell you what types you can successfully call it with" then that kind of mistake should not be possible.
It's true that there are certain cases where you know the full set of types you can use, but I'd argue that those are the less interesting/useful cases, anyways.
There are languages with full inference that break this rule.
Moreover, this rule is more important for Rust than other languages because Rust makes a lot of constraints visible in function signatures.
But the most important purpose of the rule is communicating that this is a deliberate design decision and a desireable property of code. Unfortunately, there's an overwhelming lack of taste and knowledge when it comes to language design, often coming from the more academic types. The prevailing tasteless idea is that "more is better" and therefore "more type inference is better", so surely full type inference is just better than the "limited" inference Rust does! Bleh.
My interpretation of the post is that the rule is deeper than that. This is the most important part:
> Here is the most famous implication of this rule: Rust does not infer function signatures. If it did, changing the body of the function would change its signature. While this is convenient in the small, it has massive ramifications.
Many languages violate this. As another commenter mentioned, C++ templates are one example. Rust even violates it a little - lifetime variance is inferred, not explicitly stated.
Lifetimes for a function signature in Rust are never inferred from the function code. Rather Rust has implicit lifetime specs with straightforward rules to recover the explicit full signature.
I was speaking about variance specifically. They are not inferred from function bodies, but I think it's fair to say that it's a soft violation of the golden rule because variance has a lot of spooky action at a distance (changing a struct definition can change its variance requirements, which then has ripple effects over all type signatures that mention that struct)
> Are there common exceptions to this out there, where you can call something that says it takes or returns one type but get back or send something entirely different?
I would personally consider null in Java to be an exception to this.
It's super easy to demonstrate your point with the first example the article gives as well; instead of separate methods, nothing prevents defining a method `fn x_y_mut(&mut self) -> (&mut f64, &mut 64)` to return both and use that in place of separate methods, and everything works! This obviously doesn't scale super well, but it's also not all that common to need to structure this way in the first place.
struct Id(u32);
fn main() {
let id = Id(5);
let mut v = vec![id];
println!("{}", id.0);
}
isn't even legit in modern C++. That's just move semantics. When you move it, it's gone at the old name.
He does point out two significant problems in Rust. When you need to change a program, re-doing the ownership plumbing can be quite time-consuming. Losing a few days on that is a routine Rust experience. Rust forces you to pay for your technical debt up front in that area.
The other big problem is back references. Rust still lacks a good solution in that area. So often, you want A to own B, and B to be able to reference A. Rust will not allow that directly.
There are three workarounds commonly used.
- Put all the items in an array and refer to them by index. Then write run-time code to manage all that. The Bevy game engine is an example of a large Rust system which does this. The trouble is that you've re-created dangling pointers, in the form of indices kept around after they are invalid. Now you have most of the problems of raw pointers. They will at least be an index to some structure of the right type, but that's all the guarantee you get. I've found bugs in that approach in Rust crates.
- Unsafe code with raw pointers. That seldom ends well. Crates which do that are almost the only time I've had to use a debugger on Rust code.
- Rc/RefCell/run-time ".borrow()". This moves all the checking to run time. It's safe, but you panic at run time if two things borrow the same item.
This is a fundamental problem in Rust. I've mentioned this before. What's needed to fix this is an analyzer that checks the scope of explicit .borrow() and .borrow_mut() calls, and determines that all scopes for the same object are disjoint. This is not too hard conceptually if all the .borrow() calls produce locally scoped results. It does mean a full call chain analysis. It's a lot like static detection of deadlock, which is a known area of research [1] but something not seen in production yet.
I've discussed this with some of the Rust developers. The problem is generics. When you call a generic, the calling code has no idea what code the generic is going to generate. You don't know what it's going to borrow. You'd have to do this static analysis after generic expansion. Rust avoids that; generics either compile for all cases, or not at all. Such restricted generic expansion avoids the huge compile error messages from hell associated with C++ template instantiation fails. Post template expansion static analysis is thus considered undesirable.
Fixing that could be done with annotation, along the lines of "this function might borrow 'foo'". That rapidly gets clunky. People hate doing transitive closure by hand. Remember Java checked exceptions.
This is a good PhD topic for somebody in programming language theory. It's a well-known hard problem for which a solution would be useful. There's no easy general fix.
> isn't even legit in modern C++. That's just move semantics. When you move it, it's gone at the old name.
Exactly the opposite actually.
Rust has destructive move while modern C++ has nondestructive move.
So in Rust, an object is dead after you move out of it, and any further attempts to use it are a compiler diagnosed error. In contrast, a C++ object is remains alive after the move, and further use of it isn't forbidden by the language, although some or all uses might be forbidden by the specific user provided move function - you'll have to reference the documentation for that move function to find out.
Indeed, this is strictly worse than rust. The object is alive but in an invalid state, so using it is a bug but not one the compiler catches. In the worse case the move is only destructive for larger objects (like SSO), so your tests can pass and you've still got a bug.
C++ teachers like Herb Sutter like to jump in here and make a correction: A moved-from object is in a valid state, you just might not know which state. You can still call methods on it that are valid in any state. ("What is your length?") But you shouldn't fall methods that are only valid in some states. ("Give me your first element.")
Or when you enable optimizations and since accessing an object with invalid state is UB, the compiler helpfully decides it is now permitted to format your hard drive.
Rust borrow checker is designed to enforce "one owner" model (a tree).
When you need to have more than one reference, you can use Rc + Weak[0]. Example DoubleLinkedList implementation:
Moreover, if you have cycles instead of trees, you can use a garbage collector with support for cycles, like rust-cc[1].
So yes, it's cannot be done statically, because Rust is not designed for that.
However, problem disappears when 'static lifetime is used (or arenas). Nodes can be marked as deleted, instead of dropping them, so pointers are always valid.
In same vein, when nodes are deleted rarely, they can be simply marked as deleted, without dropping them completely (until sibling nodes are updated, at least):
> adding a generational counter (maybe only in debug builds) to allocations can catch use-after-frees.
At the cost of making the use of the resulting heap significantly slower and larger than if you just wrote the thing in Java to begin with, though! The resulting instrumentation is likely to be isomorphic to GC's latency excursions, even.
This is the biggest issue that bugs me about Rust. It starts from a marketing position of "Safety With No Compromises" on runtime metrics like performance or whatever, then when things get hairy it's always "Well, here's a very reasonable compromise". We know how to compromise! The world is filled with very reasonably compromised memory-safe runtimes that are excellent choices for your new system.
>It starts from a marketing position of "Safety With No Compromises" on runtime metrics like performance or whatever, then when things get hairy it's always "Well, here's a very reasonable compromise".
Well, yes. The other compromise is the one Java gives you: write a state of the art garbage collector and include it in your program.
This complaint is very annoying because it assumes a garbage collector is "free" and Rust decided to just not give you one. If you want memory safe trees your options are
* A slow, simple, reference counted garbage collector
* A fast, complex, garbage collector
both are compromises! You are just ignoring the second one.
Well, in some sense. But in practice, come on: "write your own slow simple reference-counted GC" is a rather more expensive compromise than "just use Java or Go or whatever, it's faster and simpler for your problem".
Aside from all the nitpickery about runtime implementation, the rustacean community has a serious problem with compromise in general. If you whine in a python/Go/Java/whatever forum about performance, they'll point you to their FFI and show you how to get what you want via C++ integration, because clearly no environment is going to be perfect for everyone. But come at rust with a use case (cyclic data structures here) for which it's a poor fit and everyone goes blue in the face explaining how it's not really a problem. It's exhausting.
> At the cost of making the use of the resulting heap significantly slower and larger than if you just wrote the thing in Java to begin with, though!
Lower throughput, probably. But it introduces constant latency. It has some advantages over doing it in Java:
* You're never going to get latency spikes by adding a counter to each allocation slot.
* If you really want to, you can disable them in release builds and still not give up memory-safety, although you might get logical use-after-frees.
* You don't need to use such "compromises" for literally everything, just where it's needed.
> It starts from a marketing position of "Safety With No Compromises"
I haven't seen that marketing, but if it exists, sure, it's misleading. Yes, you have to compromise. But in my opinion, the compromises that Rust lets you make are meaningfully different from the compromises in other mainstream languages. Sometimes better, sometimes worse. Probably worse for most applications than a GC language, tbh.
> * You're never going to get latency spikes by adding a counter to each allocation slot.
The suggestion wasn't just the counter though. A counter by itself does nothing. At some point you need to iterate[1] through your set to identify[2] the unreferenced[3] blocks. And that has to be done with some kind of locking vs. the unrestricted contexts elsewhere trying to do their own allocation work. And that has costs.
Bottom line is that the response was isomorphic to "That's OK, you can work around it by writing a garbage collector". And... yeah. We have that, and it's better than this nonsense.
No, sorry, in case I wasn't clear, I was talking about manual deallocation. I wasn't talking about a garbage collector. You still allocate and free cells. Here's an example of what I am talking about:
> It starts from a marketing position of "Safety With No Compromises"
Since such a tradeoff is not possible (full safety for free?!) I doubt there's any such marketing.
> The world is filled with very reasonably compromised memory-safe runtimes that are excellent choices for your new system.
Bringing a whole managed runtime just to handle a single structure with cycles in your program is not reasonable.
There's no problem with using a simple GC for a tiny part of an otherwise manually managed program just how there's no issue in managing memory manually for a small, but performance sensitive part of your GC managed program.
I always say for people coming from C++ ... just imagine a std::move is there around everything (well, except for things that are Copy) ... then it will all make sense.
The problem is this mental model is entirely foreign to people who have worked in literally every other language where pass by value (copy or pass by reference are the way things work, always.
I just had a thought! It might make languages like Rust more ergonomic if movement could also update the reference to the new location so that moving a local variable into a container could also update the variable to reference the target also.
The "broken" example can be trivially fixed:
fn main() {
let id = Id(5);
let mut v = vec![id];
let id = v[0].0; // just use the new name!
println!("{}", id );
}
But what if the language supported "move and in-place update the reference"?
Something like:
let mut v = vec![@id]; // Some new symbol
Where '@' (or whatever) is a new operator that says: move the object and update the reference `id` to point to the moved value. This could only be used for parameters marked with some sort of attribute indicating that this is possible.
On its own, that doesn't sound like the worst idea. But at least in Rust, one problem is that further modifying the container will immediately invalidate the new reference, as will reading from the container if you get a mutable reference. References into containers are temperamental in general, unless you can handle them in smaller scopes that are unaware of the larger container.
But why add a language feature (and a new symbol, even!) when by definition you can always fix the issue by shadowing the original variable and pointing it at the new location? At most, this calls for a change in compiler diagnostics to add a hint in cases that are trivially fixable.
There are scenarios where retrieving the just-inserted value without first cloning it isn't possible. E.g.: when inserting a value into a non-empty hashset and then needing it again immediately.
But yeah... that's a bit of a contrived example and can be solved by a simple change to the insert function without specialised support from the language.
This post pretty much completely ignores the advantages of the borrow checker. I'm not talking about memory safety, which is it's original purpose. I'm talking about the fact that code that follows Rust's tree-style ownership pattern and doesn't excessively circumvent the borrow checker is more likely to be correct.
I don't think that was ever the intent behind the borrow checker but it is definitely an outcome.
So yes, the borrow checker makes some code more awkward than it would be in GC languages, but the benefits are easily worth it and they stretch far beyond memory safety.
He's not ignoring them. The point of the article is that the author doesn't experience those things as concrete advantages for them. Like sure, there are advantages to those things, but the author says he doesn't feel it's worth the trouble in his experience for the sorts of code he's writing.
The author is a bioinformatician writing scientific software, and often switching back and forth between Rust, Julia, and Python. His concerns and priorities are not the same as people doing systems-level programming.
Maybe Rust, a systems language, is just a wrong tool for bioinformatic tasks. Go, Java, Typescript, Ocaml, Scala, Haskell easily offer a spectrum from extreme simplicity to extreme expressiveness, with good performance and library support, but without needing to care about memory allocation and deallocation. (Python, if you use it as the frontend to pandas / polars, also counts.)
I know you're not supposed to berate people here for not reading TFA, but this really feels like a case where it's very frustrating to engage with you because you really should read TFA.
The author may have a point in the idea of borrowing record fields separately. It is possible if we assume that the fields are completely orthogonal and can be mutated independently without representing an incorrect state. It would be a good option to have.
But a doubly-linked list (or graph) just can't be safely represented in the existing reference semantics. Dropping a node would lead to a dangling pointer, or several. An RDBMS can handle that ("on delete set null"), because it requires a specific way to declare all such links, so that they can be updated (aka "foreign keys"). A program usually does not (Rc / Arc or shared_ptr provide a comparable capability though).
Of course a bidirectional link is a special case, because it always provides a backlink to the object that would have a dangling pointer. The problem is that the borrow checker does not know that these two pointers belonging to different structs are a pair. I wish Rust had direct support for such things, then, when one end of the bidirectional link dies, the borrow checker would unset the pointer on the reciprocal end. Linking the objects together would also be a special operation that atomically sets both pointers.
In a more general case, it would be interesting to have a way to declare some invariants over fields of a struct. A mutual pair of pointers would be one case, allowing / forbidding to borrow two fields at once would be another. But we're far from that.
Special-casing some kinds of pointers is not unheard of; almost every GC-based language offers weak references (and Java, also Soft and Phantom references). I don't see why Rust could not get a BidirectionalRef of sorts.
Until then, Arc or array with indexes seem to be the only guaranteed memory-safe approaches.
Also, in the whole article I could not find a single reason why the author chose Rust, but I suppose it's because of its memory efficiency, considering the idea of keeping large graphs in RAM. Strictly speaking, Go could be about as efficient, but it has other inflammation points. C++... well, I hope Rust is not painful enough to resort to that.
Rust is widely used in bioinformatics, as is C++. Largely because people developing new tools often have to write their own low-level algorithms and data structures.
Many tools deal with sequencing data, which means collections of strings that cannot be parsed or tokenized in any meaningful way. The collections are often very large, and the strings can also be very long. The standard algorithmic toolkit you learn when doing a CS degree (or a PhD in algorithms) is inadequate with such data. Hence, if you go to a CS conference focused on combinatorial algorithms and data structures, the presented work is often motivated by applications in bioinformatics.
There's the practical end goal benefit of safer and more robust programs, but I think there's also the piece that pg talks about in Beating The Averages which is that learning how to cooperate with these conventions and think like there's the borrow checker there makes you a better programmer even when you return to languages that don't have it.
If a language is bad, but you must use it, then yes learn it. But, if the borrowchecker is a source of pain in Rust, why not andmit it needs work instead of saying that “it makes you better”?
I’m not going to start writing brainfuck because it makes me a better programmer.
We do admit it needs work. The issues the author highlights can be annoying, a smarter borrow checker could maybe solve them.
The point is the borrow checker has already gone beyond the point where the benefits outweigh those annoyances.
It's like... Static typing. Obviously there are cases where you're like "I know the types are correct! Get out of my way compiler!" but static types are still vastly superior because of all the benefits they convey in spite of those occasional times when they get in the way.
> The issues the author highlights can be annoying, a smarter borrow checker could maybe solve them
I don't think a smarter borrow checker could solve most of the issues the author raises. The author wants borrow checking to be an interprocedural analysis, but it isn't one by design. Everything the borrow checker knows about a function is in its signature.
Making partial borrows to be expressable in method definitions would allow the design pattern to be expressed without breaking the current lifetime evaluation boundary.
Allowing the borrow checker to peek inside of the body of local methods for the purposes of identifying partial borrows would fundamentally break the locality of the borrow checker, but I think that as long as that analysis is only extended to methods on the local trait impl, it could be done without too much fanfare. These two things would be relaxations of the borrow checker rules, making it smarter, if you will.
I believe that codebases written in Rust, with borrow checking in mind, are often very readable and allow local reasoning better than most other languages. The potential hardness might not stem from "making people better programmers" but from "making programmers write better code, perhaps at the cost of some level of convenience".
> code that follows Rust's tree-style ownership pattern
This is a pretty important qualification. Most low-level systems code doesn't and can't have this ownership structure. It provides a reason why Rust has more traction replacing code that could have been written in Java rather than e.g. C++ in the domains where C++ excels (like database engines).
This is a moot statement. Here is a thought experiment that demonstrates the pointlessness of languages like Rust in terms of correctness.
Lets say your goal is ultimate correctness - i.e for any possible input/inital state, the program produces a known and deterministic output.
You can chose 1 of 2 languages to write your program in:
First is standard C
Second is an absolutely strict programming language, that incorporates not only memory membership Rust style, but every single object must have a well defined type that determines not only the set of values that the object can have, but the operations on that object, which produce other well defined types. Basically, the idea is that if your program compiles, its by definition correct.
The issue is, the time it takes to develop the program to be absolutely correct is about the same. In the first case with C, you would write your program with carefully designed memory allocation (something like mempool that allocates at the start), you would design unit tests, you would run valgrind, and so on.
In the second case, you would spend a lot more time carefully designing types and operations, leading to a lot of churn of code-compile-fix error-repeat, taking you way longer to develop the program.
You could argue that the programmer is somewhat incompetent (for example, forgets to run valgrind), so the second language will have a higher change of being absolutely correct. However the argument still holds - in the second language, a slightly incompetent programmer can be lazy and define wide ranging types (similar to `any` in languages like typescript), leading to technical correctness, but logic bugs.
So in the end, it really doesn't matter which language you chose if you want ultimate correctness, because its all up to the programmer. However, if your goal is rapid prototyping, and you can guarantee that your input is constrained to a certain range, and even though out of range program will lead to a memory bug or failure of some sort, programming in something like C is going to be more efficient, whereas the second language will force you write a lot more code for basic things.
This claim makes some sense to me if your development life cycle is: write and compile once, never touch again.
Working at a company with lots of systems written by former employees running in production… the advantages of Rust become starkly obvious. If it’s C++, I walk on eggshells. I have to become a Jedi master of the codebase before I can make any meaningful change, lest I become responsible for some disaster. If it’s Rust, I can just do stuff and I’ve never broken anything. Unit tests of business logic are all the QA I need. Other than that, if it compiles it works.
I have two primary complaints about Java, relative to Rust:
1. NullPointerException. I get some object with a bunch of fields and I don’t know which of them are null. In Rust I am given a struct, and usually the fields aren’t an Option unless they need to be.
2. Complicated design patterns with inheritance. Maybe it’s more of a problem with the culture/ecosystem than Java the language. But Rust doesn’t have inheritance, and traits are less complicated. So you rarely get the same smells.
Compared to C++, Java is easier to debug, but I still have a lot of “wtf” moments trying to understand what the author was thinking. It is almost like the language is so simple to write, people are making the program more complicated on purpose just to make it interesting.
The point is that programmers are ultimately responsible for clean code. You can write very good C code that is very easy to debug. Languages don't really matter for this.
Also practice, most of Rust codebases are filled with `unsafe`, for interface with system libraries.
Also the argument of forcing a language onto a project based on the lowest common denominator of programmers never plays out - this is how you get insanely messy Java codebases. Language choice will never solve poor programming style.
>Like how string manipulation is so much simpler and easier in C compared to Rust? Hmm.
Your example is of a greenfield project rather than living code that is constantly updated, in some cases by many people simultaneously. The compiler being a gate for correctness is far superior in the latter case.
If you enforce very strict interfaces through language, no matter how many people work on them, the tradeoff still applies.
For example, you can have multiple people working on a code base for the second case, and some sub team has a new requirement for added functionality. Now they have to go refactor a whole bunch of the codebase to make all the types coherent. And consequently, shortcuts happen here, which leads to shit codebases.
The author's motivation for writing this is well-founded. However, the author doesn't take into account the full spirit of rust and the un-constructive conclusion doesn't really help anyone.
A huge part of the spirit of rust is fearless concurrency. The simple seeming false positive examples become non-trivial in concurrent code.
The author admits they don't write large concurrent - which clearly explains why they don't find much use in the borrow checker. So the problem isn't that the rust doesn't work for them - it's that a central language feature of rust hampers them instead of helping them.
The conclusion for this article should have been: if you're like me and don't write concurrent programs, enums and matches are great. The language would be work better for me if the arc/box syntax spam went away.
As a side note, if your code is a house of cards, it's probably because you prematurely optimized. A good way to get around this problem is to arc/box spam upfront with as little abstraction as possible, then profile, then optimize.
Yeah, the author's theoretical "Rust but with garbage collector" would gain a whole bunch of concurrency bugs. It wouldn't be Rust anymore, just c# with a more functional syntax.
"Fearless concurrency" is one of the best things the borrow checker gives us, and I think a lot of people undervalue it.
I think there's a lot love for the borrowchecker because a lot of people in the Rust community are working on ecosystems (eg https://github.com/linebender) which means they are building up an api over many years. In that case having a very restrictive language is really great, because it kinda defines the shape the api can have at the language level, meaning that familiarity with Rust also means quick familiarity with your api. In that sense it doesn't matter if the restrictions are "arbitrary" or useful.
The other end of the spectrum is something like gamedev: you write code that pretty explicitly has an end-date, and the actual shape of the program can change drastically during development (because it's a creative thing) so you very much don't want to slowly build up rigidity over time.
To the author, I would be a borrow checker apologist or perhaps extremist. I will take that mantle gladly: I am very much of the opinion that a systems programming language without a borrow checker[^1] will not find itself holding C++-like hegemony anymore (once/if C++ releases the scepter, that is). I guess I would even be sad if C++ kept the scepter for the rest of my life, or was replaced by another language that didn't have something like a borrow checker.
It doesn't need to be Rust: Rust's borrow checker has (mostly reasonable) limitations that eg. make some interprocedural things impossible while being possible within a single function (eg. &mut Vec<u32> and &mut u32 derived from it, both being used at the same time as shared references, and then one or the other being used as exclusive later). Maybe some other language will come in with a more powerful and omniscient borrow checker[^1], and leave Rust in the dust. It definitely can happen, and if it does then I suppose we'll enjoy that language then.
But: it is my opinion that a borrow checker is an absolutely massive thing in a (non-GC) programming language, and one that cannot be ignored in the future. (Though, Zig is proving me wrong here and it's doing a lot of really cool things. What memory safety vulnerabilities in the Ziglang world end up looking like remains to be seen.) Memory is always owned by some_one_, its validity is always determined by some_one_, and having that validity enforced by the language is absolutely priceless.
Wanting GC for some things is of course totally valid; just reach for a GC library for those cases, or if you think it's the right tool for the job then use a GC language.
[^1]: Or something even better that can replace the borrow checker; maybe Graydon Hoare's original idea of path based aliasing analysis would've been that? Who knows.
Imo a GC needs some cooperation from the language implementation, at least to find the rootset. Workarounds are either inefficient or unergonomic. I guess inefficient GC is fine in plenty of scenarios, though.
> In that sense, Rust enables escapism: When writing Rust, you get to solve lots of 'problems' - not real problems, mind you, but fun problems.
If this is true for Rust, it's 10x more true for C++!
Lifetime issues are puzzles, yes, but boring and irritating ones.
But in C++? Select an appetizer, entree, and desert (w/ bottomless breadsticks) from the Menu of Meta Programming. An endless festival of computer science sideshows living _in the language itself_ that juices the dopamine reward of figuring out a clever way of doing something.
Came here to comment on the same thing. I've never been able to articulate this as well as the author did, and it is so true! Every programming language requires you to solve some puzzles that are just in the way of the real problems you are trying to solve, but some much more than others.
People have compared Rust to C++ and others have argued that they really aren't alike, but I think it's in these puzzles that they are more alike than any other two languages. Even just reading rust code is a brain teaser for me!
I think this is why C and Zig get compared too. They apparently have roughly the same level of "fun problems" to solve.
the returned references are, for the purposes of aliasing rules, references to the entire struct rather than to pieces of it. `x` and `y` are implementation details of the struct and not part of its public API. Yes, this is occasionally annoying but I think the inverse (the borrow checker looking into the implementations of functions, rather than their signature, and reasoning about private API details) would be more confusing.
I also disagree with the author that his rejected code:
fn main() {
let mut point = Point { x: 1.0, y: 2.0 };
let x_ref = point.x_mut();
let y_ref = point.y_mut();
*x_ref *= 2.0;
*y_ref *= 2.0;
}
"doesn't even violate the spirit of Rust's ownership rules."
I think the spirit of Rust's ownership rules is quite clear that when calling a function whose signature is
fn f<'a>(param: &'a mut T1) -> &'a mut T2;
`param` is "locked" (i.e., no other references to it may exist) for the lifetime of the return value. This is clear once you start to think of Rust borrow-checking as compile-time reader-writer locks.
This is often necessary for correctness (because there are many scenarios where you need to be guaranteed exclusive access to an object beyond just wanting to satisfy the LLVM "noalias" rules) and is not just an implementation detail: the language would be fundamentally different if instead the borrow checker tried to loosen this requirement as much as it could while still respecting the aliasing rules at a per-field level.
It would not just be "confusing". It would be fundamentally unacceptable because there would just be no local reasoning anymore, and a single private field change might trigger a whole cascade of nonlocal borrowing errors.
Unfortunately, this behavior does sometimes occur with Send bounds in deeply nested async code, which is why I mostly restrain from using colored-function style asynchronous code at all in favor of explicit threadpool management which the borrow checker excels at compared to every other language I used.
I found the arguments in this article disingenuous. First, the author complains that borrowchecker examples are toys, then proceeds to support their case with rather contrived examples themselves. For instance, the map example is not using the entry api. They’d be better served by offering up some real world examples.
The author explained why he used contrived examples. It's because the pain arises most acutely only after your project has become large and mature but demands a small ownership-impacting change. The toy examples demonstrate the problem in the small, but they generalize to larger and more complex scenarios.
He's basically talking about the rigidity that Rust's borrow checking imposes on a program's data design. Once you've got the program following all the rules, it can be extraordinarily difficult to make even a minor change without incurring a time-consuming and painful refactor.
This is an argument about the language's ergonomics, so it seems like a fair criticism.
Regarding Indexes: "When the same borrowchecker makes references unworkable, their solution is to... recommend that I manually manage them, with zero safety and zero language support?!?"
Language support: You can implement extension traits on an integer so you can do things like current_node.next(v) (like if you have an integer named 'current_node' which is an index into a vector v of nodes) and customize how your next() works.
Also, I disagree there is 'zero safety', since the indexes are into a Rust vector, they are bounds checked by default when "dereferencing" the index into the vector (v[i]), and the checking is not that slow for vast majority of use cases. If you go out of bounds, Rust will panic and tell you exactly where it panicked. If panicking is a problem you could theoretically have custom deference code that does something more graceful than panic.
But with using indexes there is no corruption of memory outside of the vector where you are keeping your data, in other words there isn't a buffer overflow attack that allows for machine instructions to be overwritten with data, which is where a huge amount of vulnerabilities and hacks have come from over the past few decades. That's what is meant by 'safety' in general.
I know people stick in 'unsafe' to gain a few percent speed sometimes, but then it's unsafe rust by definition. I agree that unsafe rust is unsafe.
Also you can do silly optimization tricks like if you need to perform a single operation on the entire collection of nodes, you can parallelize it easily by iterating thru the vector without having to iterate through the data structure using next/prev leaf/branch whatever.
> If you go out of bounds, Rust will panic and tell you exactly where it panicked
This arguement has a long history.
It is a widely used pattern in rust.
It is true that panics are memory safe, and there is nothing unsafe about having your own ref ids.
However, I believe thats its both fair and widely acknowledged that in general this approach is prone to bugs that cause panics for exactly this reason, and thats bad.
Just use Arc or Rc.
Or, an existing crate that implements a wrapper around it.
Its enormously unlikely that most applications need the performance of avoiding them, and very likely that if you are rolling your own, youll get caught up by edge cases.
This is a prime example of a rust antipattern.
You shouldnt be implementing it in your application code.
Indices are the way to go for a whole range of programs - compilers (IR instructions), GUI (widgets), web browser (UI elements), databases, games (ECS), simulations, etc. It is however without borrowcheck guarantees.
Rc and Arc are definitely a bad way to avoid borrowchecker. GC is used because it is much faster than reference counting. OCaml and Go are experimenting with smarter local variable handling without GC. At that point they may outperform Arc and Rc heavy Rust code.
Interesting. My projects would benefit from those more convenient clones into closure captures. The other point where Rc is uglier than, say, Swift, is the explicit `.borrow()` and `.borrow_mut()` everywhere. I wonder if that could also be made more convenient/high-level without sacrificing the control over high performance (like C++) that got me to use Rust in the first place.
I've recently wondered if it's possible to extract a subset of Rust without references and borrow checking by using macros (and a custom stdlib).
In principle, the language already has raw pointers with the same expressive power as in C, and unlike references they don't have aliasing restrictions. That is, so long as you only use pointers to access data, this should be fine (in the sense of, it's as safe as doing the same thing in C or Zig).
Note that this last point is not the same as "so long as you don't use references" though! The problem is that aliasing rules apply to variables themselves - e.g. in safe rust taking a mutable reference to, say, local variable and then writing directly to that variable is forbidden, so doing the same with raw pointers is UB. So if you want to be on the safe side, you must never work with variables directly - you must always take a pointer first and then do all reads and writes through it, which guarantees that it can be aliased.
However, this seems something that could be done in an easy mechanical transform. Basically a macro that would treat all & as &raw, and any `let mut x = ...` as something like `let mut x_storage = ...; let x = &raw mut x_storage` and then patch up all references to `x` in scope to `*x`.
The other problem is that stdlib assumes references, but in principle it should be possible to mechanically translate the whole thing as well...
And if you make it into a macro instead of patching the compiler directly, you can still use all the tooling, Cargo, LSP(?) etc.
I've similarly thought about building a language that compiles to Rust, but handles everything around references and borrowing and abstracts that away from the user. Then you get a language where you don't have to think about memory at all, but the resulting code "should" still be fairly fast because Rust is fast (kind of ending up in the same place as Go).
I haven't written a ton of Rust so maybe my assumptions of what's possible are wrong, but it is an idea I've come back to a few times.
Think of Rust as a kind of kernel guaranteeing correctness of your program, the rules of which your transpiler should not have to reimplement. This may be compared to how proof assistants are able to implement all sorts of complicated simplification and resolution techniques while not endangering correctness of the generated proofs at all due to them having a small kernel that implements all of verification, and as long as that kernel is satisfied with your chain of reasoning, the processes behind its generation can be entirely disregarded.
Arc<Box<T>> is redundant, for the contents of the Arc are already stored on the heap. You may be thinking of Arc<Mutex<T>> for multithreaded access or Rc<RefCell<T>> for singlethreaded access. Both enable the same "feature" of moving the compile-time borrow checking to runtime (Mutex/RefCell) and using reference-counting instead of direct ownership (Arc/Rc).
You can freely alias &Cell<T>, and this would give you memory safety compared to raw pointers. AIUI, &Cell<T> is effectively the moral equivalent (but safe) to T* in C/C++.
It works if you only ever need to reference the outer value, but it doesn't allow you to e.g. get a working mutable reference to a field inside a struct - even if you declare the field as another Cell<T>, when you read the value from the outer cell, you get a copy, not a reference to original. And the same thing goes for arrays, which is especially inconvenient. So no, not quite equivalent.
If you don't want memory safety, it seems like it'd be easier to use C++ with a static analyzer to disallow the parts you don't like. I suppose the lack of a good package manager would still be a problem.
The whole point of TFA (with which I fully agree) is that Rust is just generally a much nicer language than C++, once borrow checker is out of the picture. Discriminated unions (enums) with pattern matching alone are worth their weight in gold.
I personally don't think I would want to use a language that doesnt have a borrow checker ever again. I would like a Rust-like language with GC, but I still want the borrow checker.
Every time I write Go, I find it so annoying all the defensive deep copying I see. In JS I always find myself getting confused on whether a mutation is safe (as in my program won't break some assumption) to do or not. Marking arguments as shared or exclusive is really great for me to know what kind of access I can have. It needs to be enforced so that the owner also doesn't accidentally mutate while a shared borrow is still active (again, not just for memory safety but for my own invariants). The classic example could be inserting into a collection while iterating over it
I think the borrow checker is necessary if you have ADTs like Rust and you want memory safety. You could pattern match a union into one of its variants and get a pointer to one of the fields, but without a borrow checker there's nothing to stop you from changing the variant stored in the union. This would obviously cause issues and the only way you'd solve this with GC alone is by allocating each variant individually where the union is just a tagged pointer.
> Use fewer references and copy data. Or: "Just clone".
> This is generally good advice. Usually, extra allocations are fine, and the resulting performance degradation is not an issue. But it is a little strange that it allocations are encouraged in an otherwise performance-forcused language, not because the program logic demands it, but because the borrowchecker does.
I often end up writing code that (seems) to do a million tiny clones. I've always been a little worried about fragmentation and such but it's never been that much of an issue -- I'm sure one day it will be. I've often wanted a dynamically scoped allocator for that reason.
If you notice a rule for the first time, restricting what you want to do and making you jump through a hoop, it can be hard to see what the rule is actually for. The thrust of the piece is 'there should not be so many rules, let me do whatever I want to do if the code would make sense to me'. This does not survive contact with (say) an Enterprise Java codebase filled with a billion cases of defensive strict immutability and defensive copies, because the language lacks Rust's rules and constructs about shared mutability and without them you have to use constant discipline to prevent bugs. `derive(Copy)` as One More Pointless Thing You Have To Do only makes sense if you haven't spent much time code-reviewing someone's C++.
If you try to write Java in Rust, you will fail. Rust is no different in this regard from Haskell, but method syntax feels so friendly that it doesn't register that this is a language you genuinely have to learn instead of picking up the basics in a couple hours and immediately start implementing `increment_counter`-style interfaces.
And this is an inexperienced take, no matter how eloquently it's written. You can see it immediately from the complaint about CS101 pointer-chasing graph structures, and apoplexy at the thought of index-based structures, when any serious graph should be written with an index-based adjacency list and writing your own nonintrusive collection types is pretty rare in normal code. Just Use Petgraph.
A beginner is told to 'Just' use borrow-splitting functions, and this feels like a hoop to jump through. This is because it's not the real answer. The real answer is that once you have properly learned Rust, once your reflexes are procedural instead of object-oriented, you stop running into the problem altogether; you automatically architect code so it doesn't come up (as often). The article mentions this point and says 'nuh uh', but everyone saying it is personally at this level; 'intermittent' Rust usage is not really a good Learning Environment.
The borrow checker is also what I like the least about Rust, but only because I like pattern matching, zero-cost abstractions, the type system, fearless concurrency, algebraic data types, and Cargo even more.
This isn't really right; Pony does static concurrency safety without borrow checking, and while Swift sort of has borrow checking it's mostly orthogonal to how static concurrency safety works there. The relationship between borrow checking and concurrency safety in Rust is closer to "they rhyme" than "they use the exact same mechanism".
You've provided reasons why other languages have other properties, but Rust's fearless concurrency requires borrow checking.
The concurrency focused trait Send only makes sense because we have control over references, through the borrow checker. Thread #1 can give Thread #2 this Doodad because it no longer has any references to it, if you try to give it a Doodad you're still referring to, then the borrow checker will reject your code. The Send trait guarantees that this (giving the Doodad to a different thread) is an OK thing to do - but only if you don't have outstanding borrows so you won't be able to look at the Doodad once you give it to Thread #2
I choose a language that is as ergonomic as possible, but as performant as necessary. If e.g. Kotlin is fine, there is no way I will choose Rust.
Many projects are written in Rust that would absolutely be fine in Go, Swift or a JVM language. And I don't understand: it is nicer to write in those other languages, why choose Rust?
On the other hand, Rust is a lot nicer than C/C++, so I see it as a valid alternative there: I'm a lot happier having to make the borrow-checker happy than tracking tricky memory errors in C.
Personally I like my programs to be as performant as is reasonable, not just as is necessary. Rust's low-cost abstractions strike the right balance for me where I feel like I'm getting pretty solid ergonomics (most of the time) while also enjoying near-peak performance as the default.
> It is nicer to write in those other languages, why choose Rust?
Honestly I don't think it is nicer to write in those other languages you mention. I might still prefer Rust if performance was removed from the equation entirely. That is just to say I think preference and experience matters just as much, if not more, than the language's memory model.
For a sufficiently large program, i am faster at writing a correct rust implementation than I am with Go. I find myself a lot more able to reason about the program thanks to the work the compiler makes me do upfront.
I think this is a matter of preference. Nowadays I cannot stand environments like Java (or especially Kotlin). "Tricky memory errors" is in my opinion nicer than a borrow-checker refusing sound code. I guess I really hate 'magic'...
I mostly agree with you, but ironically the article's own conclusion answers this: the language's general ethos is designed for correctness-oriented programming in the large, and this gives rise to lots of other features that garbage-collected languages could adopt but mostly don't. Like, it's way harder to avoid runtime panics in Go. (Swift does okay but people don't have faith that its custodians will do an adequate job of prioritizing the needs of developers using it for anything other than client-side apps on Apple platforms, and also it's slightly too low-level in that it doesn't have a tracing garbage collector either.)
I find Rust quite ergonomic in most cases. Yes there is more code, but in terms of thinking about it, it's usually "I have this and I want that", and the middle almost fills itself out.
What's the alternative though? If you're fine with garbage collection, just use garbage collection. If you're _not_ fine with garbage collection (because you want deterministic performance, or you have resources that aren't just memory) then Rust's borrow checker seems like the best thing going.
You can use Zig, a faster, safer C with best-in-class metaprogramming for a systems-level language. It doesn't guarantee safety to the same extent as Rust but gets you 80% of the benefit with 20% of the pain.
I think under the constrained of not using GC and not defaulting to unsafe memory that the borrow checker is a decent design.
The constraints mean that you need some form of formal verification of your lifetimes.
These can get very complex and difficult to use. So it might make sense to limit their expressiveness and instead rely on unsafe escape hatches.
I think Rust's borrow checker provides a decent tradeoff between easy of use and how often unsafe is needed. There are rough edges and progress has been slow, but I have yet to see anything radically better. Other system PLs require a lot more unsafe usage (e.g. Zig) and I'm not aware of any mainstream system PLs that offer much more expressive verification.
I'm far from a Rust pro, but I think the dismissal of alternatives like Polonius seems too shallow. Yes, it is still in the works, but there's nothing fundamentally wrong about the idea of a borrow checker.
This is true both in theory and in practice, as you can write any program with a borrow checker as you can without it.
TFA also dismisses all the advantages of the borrow checker and focuses on a narrow set of pain points of which every Rust developer is already aware. We still prefer those borrowing pain points over what we believe to be the much greater pain inflicted by other languages.
Polonius will not fix the "issues" the author is complaining about, because contrary to his assertion, they are actual fundamental properties of how the Rust ownership/borrowing model is supposed to work, not shortcomings of an insufficiently smart implementation.
I used Rust for about an hour and immediately ran into an issue I found amusing.
The compiler changed the type of my variable based on its usage. Usage in code I didn't write. There was no warning about this (even with clippy). The program crashed at runtime.
I found this amusing because it doesn't happen in dynamic languages, and it doesn't happen in languages where you have to specify the types. But Rust, with its emphasis on safety, somehow lured me into this trap within the first 15 minutes of programming.
I found it more amusing because in my other attempts at Rust, the compiler rejected my code constantly (which was valid and worked fine), but then also silently modified my program without warning to crash at runtime.
I saw an article by the developers of the Flow language, which suffered from a similar issue until it was fixed. They called it Spooky Action at a Distance.
This being said, I like Rust and its goals overall. I just wish it was a little more explicit with the types, and a little more configurable on the compiler strictness side. Many of its errors are actually just warnings, depending on your program. It feels disrespectful for a compiler to insist it knows better than the programmer, and to refuse to even compile the program.
The author does point out that these problems come up when implementing various data structures.
It might be surprising to some folks, but there is a lot of unsafe code in Rust, and a lot of that is in the standard’s data structure implementations.
Also —
Common in network programming, the pain of lifetimes, is in async.
The model sort of keels over and becomes obtuse when every task requires ownership of its data with static lifetimes.
I think the borrow checker doesn't get enough credit for supporting one of rusts other biggest selling points - it's ecosystem. In C and C++ libraries often go through pains to pass as little allocated memory over the API barrier as possible, communicating about lifetime constraints and ownership is flimsy and frequently causes crashes. In Rust if a function returns Vec<Foo> then every Foo is valid until dropped, and if it's not someone did something unsafe.
> Data races in multithreaded code are elegantly and statically prevented by Rust. I have never written a large concurrent Rust program, and I'll grant the possibility that the borrowchecker is miraculous for that use case and easily pays for its own clunkiness.
Data races prevented not just in concurrent programs. I personally was hit once by a panic from RefCell. I have coded a data race, but I wouldn't notice it without RefCell. I borrowed the value and then called some function, and a several stack frames deeper I tried to borrow it mutably.
- Marking and sweeping cause latency spikes which may be unacceptable if your program must have millisecond responsiveness.
- GC happens intermittently, which means garbage accumulates until each collection, and so your program is overall less memory efficient.
With modern concurrent collectors like Java's ZGC, that's not the case any longer. They show sub-millisecond pause times and run concurrently. The trade-off is a higher CPU utilization and thus reduced overall throughput, which if and when it is a problem can oftentimes be mitigated by scaling out to more compute nodes.
> They might argue that the snippets don't show there is any real ergonomic problem, because the solutions to make the snippets compile are completely trivial. In the last example, I could just derive Clone + Copy for Id.
I’ll bite a little bit. If I write:
struct Thing(u32);
Then I need to make a choice: is Thing a plain value that its holders can freely clone or is Thing an affine object that is gone when consumed? I think it’s great that Rust makes this explicit, even if it’s a tiny bit odd that one option is a default and the other option is rather verbose. If I could have a unicorn, too, I’d also like to be able to request “linear” behavior, i.e. disallow code that fails to consume the object.
Sure, I suppose someone could invent a language where an LLM reads the type definitions and fills in the blanks as to what uses are valid, but this seems like a terrible idea. Or one could use a language without affine types (most languages, and even C++’s move feature entirely fails to enforce any sort of good behavior).
I'd argue the very reason Rust was created to get rid of the aliasing problem that's plaguing C-family languages, (meaning that there's always a chance 2 pointers refer to the same piece of memory, meaning all writes might potentially invalidate all variables).
This isn't really solvable in C/C++, and its worked around with a bunch of hacks, which might be overly convervative at times, and at others, generates buggy code.
Rust managed to fix this issue elegantly, but I'm wondering if the solution might be worse than the problem.
It's essentially impossible to write a Rust program without relying on many of its escape hatches like RefCell and unsafe, that make the borrow checker go away.
Essentially any program, that needs to communicate or store data in some sort of persistent structure, which is then accessed in some other part of the program, has multiple mutable references to said abstract central point, which is not allowed in Rust, without the aforementioned workarounds.
> It's essentially impossible to write a Rust program without relying on many of its escape hatches like RefCell and unsafe, that make the borrow checker go away.
I get this may be hyperbole but it's also just factually incorrect. Not only that, why is this even a goal? RefCell isn't a wart. It is part of Rust and meant to be used where it makes sense.
Not really hyperbole - I can't iamgine how one would build a fully borrow-checked app, that doesn't use these escape hatches (either directly or through a library). The only thing you could do maybe is to pass down everything that could possible be mutable as parameters everywhere, but that would not exactly be a nice way to program.
And what's wrong with RefCell (and friends)? It's extra complexity, memory footprint, and runtime cost. Also you gave up on the ability of the compiler to check you program's correctness. Granted it's not much, but neither is std::shared_ptr or Swift's automatic reference counting. Once you go from zero overhead to some overhead, there's a ton of quality of life features you can offer to the programmer.
If Rust's borrow checker was smart enough to allow multiple mutable borrows of the same variable, provided some conditions are met (like if you're borrowing struct S, you could borrow it's fields), it would eliminate the need for much of RefCell hacks, as well as solve the gripes in the article.
The author's first example seems to undermine the thesis. Their Point class allows two separate locations to independently update the X and Y fields, leaving the object in an inconsistent state.
It seems to me that this is exactly the sort of thing that Rust is intended to prevent, and it makes complete sense to reject the code.
Rust is pain when you want to be super basic with your types.
This is actually a learning lesson for the user to understand that the bugs one has seen in languages like c++ are inherent to using simple types.
The author goes about mentioning python. If you do change all your types to python equivalents, ref counted etc. Rust becomes as easy. But you don’t want to do that and so it becomes pain, but pain with a gain.
You must decide if that gain is worth it.
From my point of view the issue is that rust defaults to be a system programming language. Meaning, simple types are written simple (i32, b32, mut ..), complex types are written complex (ref, arc, etc.).
And because of that one wants to use the simple types, which makes the solutions complex.
Let’s imagine a rust dialect, where every type without annotation is ref counted and if you want the simple type you would have to annotate your types, the situation would change.
What one must realize is that verifiable correctness is hard , the simplicity of the given problematic examples is a clear indication of how close those screw ups are even with very simple code. And exactly why we are still seeing issues in core c libs after decades of fixing them.
If a friend told me they liked Rust but didn't like the borrow checker, I'd probably point them to Gleam and Moonbit, which both seem awesome in their own niches.
Both have rust-like flavor and neither has a borrow checker.
I can't really get over Gleam's position that nobody really needs type-based polymorphism, real programmers write their own vtables by hand.
(It also needs some kind of reflection-like thing, either compile-time or runtime, so that there can be an equivalent of Rust's Serde, but at least they admit that that needs doing.)
Someone should create a DAG of programming languages with edges denoting contextual influence and changes in design and philosophy, such that every time a PL is critized for a feature (or lack thereof), the relevant alternatives exactly considering this would be readily available. It could even have a great interactive visualization.
> My examples code above may not be persuasive to experienced Rustaceans. They might argue that the snippets don't show there is any real ergonomic problem, because the solutions to make the snippets compile are completely trivial. In the last example, I could just derive Clone + Copy for Id.
No, you could use destructuring. This doesn't work for all cases but it does for your examples without needing to derive copy or clone. Here's a more complex but also compelling example of the problem:
struct Graph {
nodes: BTreeMap<u32, Node>,
}
struct Node {
edges: Vec<u32>,
}
impl Graph {
fn visit_mut(&mut self, visit: impl Fn(&mut Node, &mut Node)) {
let mut visited = BTreeSet::new();
let mut stack = vec![0];
while let Some(id) = stack.pop() {
if !visited.insert(id) { continue; }
let curr = self.nodes.get_mut(&id);
for id in source.edges.clone() {
let next = self.nodes.get_mut(&id);
visit(curr, next);
stack.push(id);
}
}
}
}
We're doing everything in the "Rust" way here. We're using IDs instead of pointers. We're cloning a vec even if it's a bit excessive. But the bigger problem is we actually _do_ need to have multiple mutable references to two values owned by a collection that we know don't transitively reference the collection. We need to wrap these in an RefCell or UnsafeCell and unsafe { } block to actually get mutable references to the underlying data to correctly implement visit_mut().
This is a problem that shows up all the time when using collections, which Rust encourages within the ecosystem.
Your example has an undefined 'source' variable - you likely meant 'curr' - but more importantly, this pattern can be solved with indices + interior mutability or split_at_mut() rather than unsafe, giving you safe simultaneous mutable access to different nodes.
That requires you use a Vec or similar data structure which may not be appropriate/forces you to reinvent garbage collection or deal with tombstoning/generations/etc for unused slots.
And I pointed out you can use interior mutability. Still sucks because the code is guaranteed sound, the compiler just can't prove it. IMO the correct choice is UnsafeCell and unsafe {}.
The point about inter-procedural borrow checking is fair -- and in fact not a matter of "sufficiently smart" borrow checker, but a rather fundamental type system limitation. But the flaw here is pretty manageable -- sometimes you can get away with calling .split_at_mut(...), sometimes you can borrow the field directly instead of going through a wrapper method, sometimes you have to be mindful to provide a split_at_mut equivalent yourself. Last I checked there was some work being done on "partial borrows", which could solve this.
Most of the other criticisms are pretty disappointing, though.
> The following example is a famous illustration of how it can't properly reason across branches, either:
Except that function body would have been better rewritten as map.entry(key).or_default() -- which passes the borrow checker just fine and is more performant as it avoids multiple lookups. I suspect many other examples would benefit from being re-written into higher-level primitives that can be easier to borrow-check in this manner.
> But what's the point of the rules in this case, though? Here, the ownership rules does not prevent use after free, or double free, or data races, or any other bug. It's perfectly clear to a human that this code is fine and doesn't have any actual ownership issues.
What if Id represents a file descriptor that ought to be closed when the value ceases to exist? Or some other type of handle? This is an ownership problem: these are not limited to memory safety issues. For very good API stability reasons, without an explicit #[derive(Clone,Copy)] the compiler is not free to assume the type represents pure information that can be copied at will, but can only treat it as one potentially containing owned resources, that just happens not to include any at this time.
> References to temporary values, e.g. values created in a closure, are forbidden even though it's obvious to a human that the solution is simply to extend the lifetime of the value to its use outside the closure.
Which is to say, to what? The type signature does not say whether a closure will be called immediately or in another thread after two hours. And in which stack frame should the value be stored, the soon-to-be-exiting closure's?
I don't agree with the examples in the post. To me, they all seem to support the case that the compiler is doing the right thing and flagging potential issues. In a larger and more complex program (or a library to be used by others), it's a lot harder to reason about such things. Frankly, why should I be keeping all that in my mind when the compiler can do it for me and warn when I'm about to do something that can't verified as safe.
Of course, designing for safety is quite complex and easy to get wrong. For example, Swift's "structured concurrency" is an attempt to provide additional abstractions to try to hide some complexity around life times and synchronization... but (personally) I think the results are even more confusing and volatile.
The closest language to "rust without borrowchecker" is probably MoonBit [0] - weirdly niche, practical, beautifully designed language.
When I was going through its docs I was impressed with all those good ideas one after the other. Docs itself are really good (high information density that reads itself).
Rust is pretty good target for Claude Code and the like. I don't write much Rust but I have to say of the langs I used Claude Code with Rust experience was among the best. If default async runtime was not work stealing I prob would use Rust way more.
Someone once told me "easy things are hard in rust, and hard things are easy". There's some truth to that and the author has highlighted borrowchecker as probably the best example of this.
> Borrowchecker frustration is like being brokenhearted - you can't easily demonstrate it, you have to suffer it yourself to understand what people are talking about. Real borrowchecker pain is not felt when your small, 20-line demonstration snippet fails to compile.
As someone who writes Rust professionally this sentence is sus. Typically, the borrow checker is somewhere between 10th and 100th in the list with regards to things I think about when programming. At the end of the day, you could in theory just wrap something in a reference counter if needed, but even that hasn't happened to me yet.
>The first time someone gave be this advice, I had to do a double take. The Rust community's whole thing is commitment to compiler-enforced correctness, and they built the borrowchecker on the premise that humans can't be trusted to handle references manually. When the same borrowchecker makes references unworkable, their solution is to... recommend that I manually manage them, with zero safety and zero language support?!? The irony is unreal. Asking people to manually manage references is so hilariously unsafe and unergonomic, the suggestion would be funny if it wasn't mostly sad.
Indices aren't simply "references but worse". There are some advantages:
- they are human readable
- they are just data, so can be trivially serialized/deserialized and retain their meaning
- you can make them smaller than 64 bits, saving memory and letting you keep more in cache
Also I don't see how they're unsafe. The array accesses are still bounds-checked and type-checked. Logical errors, sure I can see that. But where's the unsafety?
If you start making assumptions based on indices, you can turn logical errors into memory safety errors. ie. whenever you use unsafe with the SAFETY comment above it mentioning an index, you'd better be damn sure that index is valid.
This goes for not only unchecked indexing but also eg. transmuting based on a checked index into a &[u8] or such. If those indexes move in and out of your API and you do some kind of GC on your arrays / vectors, then you might run into indices being use-after-free and now those SAFETY comments that previously felt pretty obvious, even trivial, may no longer be quite so safe to be around of.
I've actually written about this previously w.r.t. the borrow checker and implementing a GC system based on indices / handles. My opinion was that unless you're putting in ironclad lifetimes on your indices, all assumptions based on indices must be always checked before use.
Different meanings of "unsafety". A cool thing about Rust's references, when used idiomatically, is that not only are you guaranteed no memory corruption, you're also guaranteed no runtime panics. These are not "unsafe" by Rust's definition, but they're still a correctness violation and it's good to prevent them statically when you can. Indices don't give you this protection.
> In that sense, Rust enables escapism: When writing Rust, you get to solve lots of 'problems' - not real problems, mind you, but fun problems.
This is a real problem across the entire industry, and Rust is a particularly egregious example because you get to justify playing with the fun stimulating puzzle machine because safety—you don't want unsafe code, do you? Meanwhile there's very little consideration to whether the level of rigidity is justified in the problem domain. And Rust isn't alone here, devs snort lines of TypeScript rather than do real work for weeks on end.
Typescript has escape hatches so you can just say "I don't care, or don't know."
With Rust, you're battling a compiler that has a very restrictive model, that you can't shut up. You will end up performing major refactors to implement what seem like trivial additions.
There's no avoiding that in a language that's designed to offer low-level control of runtime behavior, regardless of whether it's memory-safe or not. You have to tell the compiler something about how you want the data to be laid out in memory; otherwise it wouldn't know what code to generate. If you don't want to do that, use an interpreted language that doesn't expose those details.
Have you tried to assure yourself that this or that piece of software (your primary text editor for example) doesn't need to be memory safe because it won't ever receive as input any data that might have been crafted by an attacker? In my experience, doing that is harder than satisfying the borrow checker.
Yes, and you can choose to use any language with a garbage collector and get the same benefit. The list of memory safe languages at your disposal is endless and they come in every flavor you can imagine.
The cost of it is spending more CPU and more RAM on the GC. Often it's the cost you don't mind paying; a ton of good software is written in Java, Kotlin, TS/JS, OCaml, etc.
Sometimes you can't afford that though, from web browsers to MCUs to hardware drivers to HFT.
That's true (with some qualifications), but everyone seems to continue using C and C++ for everything, even for applications like text editors, where the performance of a GC language would presumably be good enough. I wonder why.
One day I will write a blog post called “The Rust borrow checker is overrated, kinda”.
The borrow checker is certainly Rust’s claim to fame. And a critical reason why the language got popular and grew. But it’s probably not in my Top 10 favorite things about using Rust. And if Rust as it exists today existed without the borrow checker it’d be a great programming experience. Arguably even better than with the borrow checker.
Rust’s ergonomics, standardized cargo build system, crates.io ecosystem, and community community to good API design are probably my favorite things about Rust.
The borrow checker is usually fine. But does require a staunch commitment to RAII which is not fine. Rust is absolute garbage at arenas. No bumpalo doesn’t count. So Rust w/ borrow checker is not strictly better than C. A Rust without a borrow checker would probably be strictly better than C and almost C++. Rust generics are mostly good, and C++ templates are mostly bad, but I do badly wish at times that Rust just had some damn template notation.
This is something I've been thinking about lately. I do think memory safety is an important trait that rust has over c and other languages with manual memory management. However, I think Rust also has other attractive features that those older languages don't have:
* a very nice package manager
* Libraries written in it tend to be more modular and composable.
* You can more confidently compile projects without worrying too much about system differences or dependencies.
I think this is because:
* It came out during the Internet era.
* It's partially to do with how cargo by default encourages more use of existing libraries rather than reinventing the wheel or using custom/vendored forks of them.
* It doesn't have dynamic linking unless you use FFI. So rust can still run into issues here but only when depending on non-rust libraries.
Everytime I try to use bumpalo I get frustrated, give up, and fallback to RAII allocation bullshit.
My last attempt is I had a text file with a custom DSL. Pretend it’s JSON. I was parsing this into a collection of nodes. I wanted to dump the file into an arena. And then have all the nodes have &str living in and tied to the arena. I wanted zero unnecessary copies. This is trivially safe code.
I’m sure it’s possible. But it required an ungodly amount of ugly lifetime 'a lifetime markers and I eventually hit a wall where I simply could not get it to compile. It’s been awhile so I forget the details.
I love Rust. But you really really have to embrace the RAII or your life is hell.
To make sure I understand correctly: did you want to read a `String` and have lots of references to slices within the same string without having to deal with lifetimes? If so, would another variant of `Rc<str>` which supports substrings that also update the same reference count have worked for you? Looking through crates.io, I see multiple libraries that seem to offer this functionality:
Let’s pretend I was in C. I would allocate one big flat segment of memory. I’d read the “JSON” text file into this block. Then I’d build an AST of nodes. Each node would be appended into the arena. Object nodes would container a list of pointers to child nodes.
Once I built the AST of nested nodes of varying type I would treat it as constant. I’d use it for a few purposes. And then at some point I would free the chunk of memory in one go.
In C this is trivial. No string copies. No duplicated data. Just a bunch of dirty unsafe pointers. Writing this “safely” is very easy.
In Rust this is… maybe possible. But brutally difficult. I’m pretty good at Rust. I gave up. I don’t recall what exact what wall I hit.
I’m not saying it can’t be done. But I am saying it’s really hard and really gross. It’s radically easier to allocate lots of little Strings and Vecs and Box each nested value. And then free them all one-by-one.
oxc_parser uses bumpalo (IIRC) to compile an AST into arena from a string. I think the String is outside the arena though, but their lifetimes are "mixed together" into a single 'a, so lifetime-wise it's the same horror to manage. But manage they did.
Very rarely, but it depends on the domain and/or the part of the code I'm writing.
For context, I mostly write GUI apps using `iced` which is inspired by Elm, so the separation between reading and writing state is front and center and makes it easy for me to avoid a whole set of issues.
I really struggle to understand the PoV of the author in his The rules themselves are unergonomical section:
> But what's the point of the rules in this case, though? Here, the ownership rules does not prevent use after free, or double free, or data races, or any other bug. It's perfectly clear to a human that this code is fine and doesn't have any actual ownership issues
I mean, of course there is an obvious ownership issue with the code above, how are the destructors supposed to be ran without freeing the Id object twice?
The whole point is that `Id` doesn't have a destructor (it's purely stack-allocated); that is, conceptually it _could_ be `Copy`.
A more precise way to phrase what he's getting at would be something like "all types that _can_ implement `Copy` should do so automatically unless you opt out", which is not a crazy thing to want, but also not very important (the ergonomic effect of this papercut is pretty close to zero).
I think the primary reason Rust doesn't do this is because it's a semver hazard. I.e., adding a non-Copy field would silently break downstream dependents. Yeah, this is already a problem with the existing auto traits, but types that don't implement those are rarer than types that don't implement Copy.
Actually; I'm not sure I'm wrong. If Copy was automatically derived based on fields of a struct (without the user explicitly asking for it with `#[derive(Copy)]` that is, as the parent comment suggested the OP is asking for), then your example S and the std Vec would both automatically derive Copy. Then, implementing Drop on them would become a compile error that you would have to silence by using the escape hatch to "un-derive" Copy from S/Vec.
So, whenever you wanted to implement Drop you'd need to engage the escape hatch.
So if you have some struct that you use extensively through an application and you need to extend it by adding a vector you are stuck because the change would need to touch so much code.
> A more precise way to phrase what he's getting at would be something like "all types that _can_ implement `Copy` should do so automatically unless you opt out", which is not a crazy thing to want,
From a memory safety PoV it's indeed entirely valid, but from a programming logic standpoint it sounds like a net regression. Rust's move semantics are such a bliss compared to the hidden copies you have in Go (Go not having pointer semantics by default is one of my biggest gripe with the language).
I think you are misunderstanding what Copy means, and perhaps confusing it with Clone. A type being Copy has no effect on what machine code is emitted when you write "x = y". In either case, it is moved by copying the bit pattern.
The only thing that changes if the type is Copy is that after executing that line, you are still allowed to use y.
I'm not misunderstanding. Ore confusing the two. Copy: Clone.
Yes when an item is Copy-ed, you are still allowed to use it, but it means that you now have two independent copies of the same thing, and you may edit one, then use the other, and be surprised that it hasn't been updated. (When I briefly worked with Go, junior developers with mostly JavaScript or Python experience would fall into this trap all the time). And given that most languages nowadays have pointer semantics, having default copy types would lead to a very confusing situation: people would need to learn about value semantics AND about move semantics for objects with a destructor (including all collections).
No thanks. Rust is already complex enough for beginners to grasp.
Is it a particularly terrible thing, in and of itself, to pass structs by value? Less implicit aliasing seems less bug-prone. Note that Go only has implicit shallow copies (i.e., this only affects the direct fields of a struct or array); all other builtin types either are deeply immutable (booleans, numbers, strings), can point to other variables (pointers, slices, interfaces, functions), or are implicit references to something that can't be passed by value (maps, channels).
I'd argue that move semantics is strictly superior to value semantics.
But more importantly, my point is that in a world where pretty much all modern languages except Go have pointer semantics by default, and when you language needs to have move semantics for memory safety in many cases, having yet another alien (to most developers) behavior is really pushing the complexity bar up for no particular gains.
It is true that any sufficiently complicated technology requires a certain skill level to use it adequately. The question remains whether the complexity of the technology is justified, and the author presents an argument why this might not be the case. Remarking their supposed lack of skill does not seem particularly productive.
I don't use Rust much, but I agree with the thrust of the article. However, I do think that the borrowchecker is the only reason Rust actually caught on. In my opinion, it's really hard for a new language to succeed unless you can point to something and say "You literally can't do this in your language"
Without something like that, I think it just would have been impossible for Rust to gain enough momentum, and also attract the sort of people that made its culture what it is.
Otherwise, IMO Rust would have ended up just like D, a language that few people have ever used, but most people who have heard of it will say "apparently it's a better safer C++, but I'm not going to switch because I can technically do all that stuff in C++"
Agreed. As a comparison Golang was sold as "CSP like Erlang without the weird syntax" but people realized channels kind of suck and goroutines are not really a lot better than threads in other languages. The actual core of OTP was the supervisor tree but that's too complicated so Golang is basically just more concise Java.
I don't think this is a bad thing but it's a funny consequence that to become mainstream you have to (1) announce a cool new feature that isn't in other languages (2) eventually accept the feature is actually pretty niche and your average developer won't get it (3) sand off the weird features to make another "C but slightly better/different"
Due to lack of many abstractions, and lack of exceptions, Go is a less concise Java. It's a language where the lack of expressiveness forces you to write simpler code. (Not that it helps too much.)
Go's selling points are different: it takes a weekend to learn, and a week to become productive, it has a well-stocked standard library, it compiles quickly, runs quickly enough, and produces a single self-contained executable.
I would say that Go is mostly a better Modula-2 (with bits of Oberon); it's only better from the language standpoint because now it has type parameters, but GC definitely helps make writing it simpler.
People sometimes say this like it's a dunk, but there's a finite amount of complexity any given programming task can shoulder, you have to allocate it somehow between the programming environment, the problem domain you're working on, and the algorithmic sophistication you bring to bear on that problem, and it's not clear to me what the benefit is in allocating more than you need to your programming language.
A lot of very smart stuff is written in Go by programmers who just want the language to get the hell out of their way. Go is some ways very good at that. Rough though if you want your programming language to model your problem domain or your algorithmic approach! TANSTAAFL.
Unfortunately, Golang has a whole lot of weird, pointless quirks in both its base language and standard library, compared to something with a more elegant and from-the-ground-up design, like Rust itself or perhaps OCaml/ReasonML. Not very good news if you want the language to just "get the hell out of your way". I suppose it's still way better than the "enterprise" favored alternative of Java/C# though!
> Golang has a whole lot of weird, pointless quirks in both its base language and standard library
Having used go in anger I don’t necessarily agree with this, could you point out an example. Maybe I just accepted it and work around it without paying much attention.
If you are referring to interface types being able to be null, well they are allocated on the heap and have dynamic dispatch, this isn’t particularly a surprise if you have worked in a lower level language but might be a surprise if you come from a language where that isn’t the case.
Here’s a great read on its quirks, and where it really isn’t “simple”: https://fasterthanli.me/articles/i-want-off-mr-golangs-wild-...
https://dave.cheney.net/2014/03/19/channel-axioms
Your "great read" is horrible.
From HN's guidelines:
> Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.
I'm curious to know why you think so, I thought it was a great article, showcasing how simplicity in the language doesn't make complexity go away, it just moves it to programs written in it.
> A send to a closed channel panics
And the related read is purely a misunderstanding about how concurrency is modelled. Channels are not meant to be written from multiple writers, maybe this gets discussed in the next article. I can understand why it is confusing and you consider it unintuitive.
The read situation is literally not understanding or even looking up the interface, there is another return value that will tell you the channel is closed.
Nil channel reads make sense for it to do what it does if your familiar with the interface but at the same time you are literally holding it wrong. Vs a language where that would be UB its an improvement. Compared to a language like rust sure the information isn’t available at compile time but on the same not you would need to be developing in rust, and no offence but if you think golangs concurrency story is difficult to grasp just wait until you meet async rust…
These are the unintuitive, these are generally complaints of you failed to even begin to read the documentation and then complain when things aren’t doing what you expect, leave your preconceived notions at the door and you will be fine.
The faster than lime take needs to be updated, comparing golang from 5 years ago to modern golang is about as useful as comparing rust fron 5 years ago to rust now.
And you can happily make Cgo calls to the windows libraries if you want to access windows APIs, the language provides an abstraction for the normal use case not the specific use case, theres an escape hatch if you want it. And regarding the complaint about timeouts, context is a thing.
Because you don’t know how doesnt mean it’s unintuitive, it means you don’t know how.
> Rust itself or perhaps OCaml/ReasonML
So compared to... very niche languages? Go has exceptionally few quirks compared to mainstream languages.
Rust is a mainstream language
I worked with C# for a decade and it's become a really great general purpose language. I'd still prefer never to work with it ever again after having worked with Go. This isn't for technical reasons at all, but because Golang is so easy to work with for "people reasons". There are brilliant parts of Go, but the only thing I find myself missing in other languages is the simplistic module isolation, where every folder is a module, every file within the folder is part of it and then you expose functions with capital letters at the beginning of their name. Holy hell did I wish Python had that. Anyway, the thing that makes Go nice to work with over time is the explicity of everything and a lot of the very opinionated decisions. With a piece of Go code I can jump in and immediately know what is going on regardless of who wrote it. With C# I'll often have to go down long "go to definition" paths. Often you will end up trying to figure out just how someone was trying to "fight" the implicit magic of the non-STL Microsoft dependencies they used. Usually because they didn't really understand what they were doing. All of these are human issues and no fault of C# or .Net as such.
Of the few technical advantages Go had for us is that we don't need a single dependency outside of the standard library, which can't live in isolation. We use SQLC and Goose, both are run in containers that only have rights and access on the development side of things.
I'm not sure I would say that Golang has a lot of weird, pointless quirks, but it has opinions and if you happen to dislike them, well... that sucks. I hate the fact that they didn't want runtime assertions as an example, so it's not like I don't understand why people dislike Go for various reasons. I've just accepted that those strong opinions is the reason Go is so productive.
The challenge for us is that it's not exactly as productive as Python. So while you'll need to do a lot of toolchain work to get Python anywhere near Go's opinionated stucture, that is often a better choice if you're not a hardcore software engineering team. At least for us, it's been much easier to get our business intelligence people to adopt UV, pyrefly, ruff and specific VSC configs for their work than to have them learn Go.
I suspect that is why Rust is also doing so well. Go is a better Java/C# for a lot of places, but are you really going to replace your Java/C# with Go if you have dacades worth? If you're not coming from Java/C# will you really pick Go over Rust? I'm not sure, but I do know, Go failed to become a Python replacement for us. It did replace our C#, but we didn't have a lot of C#. Eventually we'll likely replace our Go with C/Zig and Python to keep the language count lower.
> I hate the fact that they didn't want runtime assertions as an example,
So you hate writing:
vs ? Reminds me of people complaining about Python's significant indentation. If that's what you're complaining about, you have nothing to complain about.I think your point is fair as you can do runtime safety in Go by wrapping Panic, but that's not exactly what Panic is meant to be used for. I guess it's more of a intent vs syntax thing, aside from the part where Panic can be Recover()ed.
You're taking it out of a context of me praising Golang, however, and while I do hate the fact that they didn't just do runtime assertions, it's not like I dislike Go as a whole because of it.
From my point of view, the significant tradeoffs made in the design of go make it better suited toward software that has to change a lot: e.g., things closer to customer-facing that are constantly being iterated on.
Rust is more appropriate for areas where the problem boundaries are relatively fixed and slow-moving: libraries, backend services, and infrastructure.
By better allowing you to model your program domain, Rust actually lets you finish projects that are feature complete. I have multiple Rust projects in production for years that have needed at most one or two bugfix releases after their initial rollout on top of a half a dozen feature updates. For a very slight additional startup cost in doing that modeling, I’ve gotten massive dividends back in the volume of maintenance programming that hasn’t needed to be done.
But if your problem domain changes frequently enough that the model needs to be changed all the time, that extra inflexibility isn’t worth it. On the other hand, I don’t know much go software that isn’t beset by the same number of bugs as every other modern language.
There's no free lunches. But I could take a problem, make it arbitrarily harder, then take that arbitrary limit away. If someone thought they had to solve the harder problem, and found out they only have to solve the easier one, did they get a free lunch?
This isn't meant to be allegorical or anything specific. I guess it's just an observation that sometimes your lunch can be cheaper than you thought.
I can't substantiate your claim about Erlang or weird syntax, is that either a proper quote or some kind of paraphrasing because nothing remotely close to it comes up.
There are numerous interviews with Rob Pike about the design of Go from when Go was still being developed, and Erlang doesn't come up in anything that I can find other than this interview from 2010 where someone asks Rob Pike a question involving Erlang and Rob replies by saying he thinks the two languages have a different approach to are fairly different:
https://www.youtube.com/watch?v=3DtUzH3zoFo
It's at the 32 minute mark, but once again this is in response to someone asking a question.
Here are other interviews about Go, and once again in almost every interview I'd say Rob kind of insinuates he was motivated by a dislike of using C++ within Google to write highly parallel services, but not once is Erlang ever mentioned:
https://www.informit.com/articles/article.aspx?p=1623555
https://www.infoq.com/interviews/pike-google-go
https://go.dev/blog/waza-talk
The parent comment does not say that Go derives from Erlang, but that both Erlang and Go implement CSP (https://en.wikipedia.org/wiki/Communicating_sequential_proce...), with the advantage for Go over Erlang that its syntax is more familiar to programmers who know C.
Erlang does not implement CSP, and if you review the very Wikipedia link you presented it should be clear. Erlang implements the Actor model, which is more flexible for distributed, fault-tolerant systems, but lacks CSP's strict formalism and synchronization semantics. The very Wikipedia article you linked to has an entire section on how CSP differs from the Actor model. Similarly Go also does not implement CSP either, although it certainly is influenced by it.
The point of my comment is that to say that Go is basically Erlang style CSP without the weird syntax is not justified as an actual quote nor as a paraphrasing or summary of anything that anyone involved in the design or promotion of Go has ever said. It's best to reserve quotes for situations where someone actually said something or as a way to summarize something for which there are ample references available.
One should not use quotes as a way to present mostly original claims that are presented as if it's some kind of already well established knowledge.
The "X is like Y but Z" game w languages is pretty fun, and kind of illuminating as to what one thing you pick.
Go is like C but with concurrency/strings/GC/a good stdlib
Go is like C++ but simpler/fast compilation/no generics/a good stdlib
Go is like Python but statically typed/multithreaded/fast/single executable
Go is like Java but native/no OO
---
One thing Go haters rarely reckon with is that Go is the only popular modern language (ie from this millennium). Everything else is way older. Well, I would actually say that this is probably the cause of Go hate--if it weren't popular no one would care.
> One thing Go haters rarely reckon with is that Go is the only popular modern language (ie from this millennium).
This requires you to squint so that "popular" happens to identify only a handful of languages but conveniently catches Go and not say Swift or Rust. It's not difficult to do this, but it's not very honest to yourself.
"Go is on the frontier of recency vs popularity (i.e. newer than all more popular languages, and more popular than all newer languages)" is honestly an "interesting" statement in its own right, but would be a distinction it has to share with Python, Java, some of JS/C#/VB/PHP depending on your popularity (and age...) metrics, and yes, Swift and Rust. And a long tail of very new languages, I suppose.
Swift isn't a general purpose language and Rust is not suitable for general application development; it's also much less popular.
> Swift isn't a general purpose language and Rust is not suitable for general application development
How do you figure Swift is not a general purpose language ?
Just searching for "programming language popularity":
- TIOBE [0]: Go 7 :: Rust n/a (> 10)
- IEEE Spectrum [1]: Go 8 :: Rust 11
- Stack Overflow [2]: Go 13 :: Rust 14 (they're pretty close here)
- GitHub [3]: Go 10 :: Rust n/a (> 10)
- PYPL [4]: Go 12 :: Rust 10
- HackerRank [5]: Go 8 :: Rust n/a (> 13)
- Pluralsight [6]: Go 9 :: Rust n/a (> 10)
- Redmonk [7]: Go 12 :: Rust 19
Although I admit we probably don't have great measures of this, there's clearly a sizable gap here.
Maybe you'd want to argue about TypeScript, Swift or Kotlin? I consider TypeScript JavaScript (because TypeScript wouldn't be popular unless all JavaScript programs were TypeScript programs), and I think Swift and Kotlin only survive because of their mobile platforms (they're also all below Go in these lists, on average).
[0]: https://www.tiobe.com/tiobe-index/
[1]: https://spectrum.ieee.org/top-programming-languages-2024
[2]: https://survey.stackoverflow.co/2024/technology
[3]: https://github.blog/news-insights/octoverse/octoverse-2024/#...
[4]: https://pypl.github.io/PYPL.html
[5]: https://www.hackerrank.com/blog/most-popular-languages-2024/
[6]: https://www.pluralsight.com/resources/blog/upskilling/top-pr...
[7]: https://redmonk.com/sogrady/2025/06/18/language-rankings-1-2...
You seem to be arguing that Go is more popular than Rust, but that's not what we're talking about.
The claim made was "Go is the only popular modern language (ie from this millennium)"
That's a binary, either a programming language is "modern" (from this millennium) or not and either it is "popular" (undefined) or it is not, Go is popular and modern according to the claim.
On your lists you find Go is anywhere from 7th to 13th in popularity. So apparently "popular" might mean 9th like Ada on TIOBE's list right? Or maybe "modern" includes Typescript, on several of these lists?
No, the whole contrivance is silly. Go is a relatively popular modern language, nobody is surprised to discover that. Is it the most popular? No. Is it the most modern? Also no.
TL;DR: mostly what I mean is Go is the only language from this millennium that's consistently in the top 10 most popular programming languages.
>> This requires you to squint so that "popular" happens to identify only a handful of languages but conveniently catches Go and not say Swift or Rust. It's not difficult to do this, but it's not very honest to yourself.
> You seem to be arguing that Go is more popular than Rust, but that's not what we're talking about.
Isn't this exactly what we're talking about? I don't think you have to squint too hard to invent the gap between Go and Rust (et al).
> So apparently "popular" might mean 9th like Ada on TIOBE's list right?
No because it's only on that list.
> Or maybe "modern" includes Typescript, on several of these lists?
No because TypeScript is JavaScript, which is from the last millennium.
> No, the whole contrivance is silly.
Emotionally I agree with you, but in practice you gotta pick a stack, and there's a lot of benefit to choosing Go over Rust/Kotlin/Swift/TypeScript/Java/C#/C/C++/Ada.
> Go is a relatively popular modern language, nobody is surprised to discover that.
Eh I'm making the Stroustrup "there's two kinds of programming languages" argument. IMO, at this point Go criticisms are just jeers from the cheap seats.
> TL;DR: mostly what I mean is Go is the only language from this millennium that's consistently in the top 10 most popular programming languages.
So, if you don't count the lists where it isn't in the top ten, and you don't count languages like Typescript? Does that feel like an important distinction to you with the exceptions, rather than an arbitrary post doc justification ?
> Isn't this exactly what we're talking about? I don't think you have to squint too hard to invent the gap between Go and Rust (et al).
Likewise for Go and Python or Javas, so why the "top 10" ? Arbitrary.
> TypeScript is JavaScript
No. Javascript is (very bad) Typescript, but Typescript is not Javascript. That's why they have a transpiler.
If you contend that the similarity means they're the same language that makes C++ also C and I don't think you want to start that fight.
My pithy response to the Stroustrup argument is a T-shirt I own which says "Haters gonna make some good points". Yes of course people will criticize your popular language, but this observation does not make the criticisms untrue, and resorting to Stroustrup's argument is best understood as an admission that he has no response to the actual criticism.
C++ is remarkably bad. You're looking at popularity lists. What else is on those lists which has similar levels of criticism? Go is a long way short of perfect but it's nowhere close to C++. If Go is the Stallone "Judge Dredd" then maybe C++ is "Batman & Robin".
(I've successfully resisted the urge to make my entire reply "Wait, you don't like Judge Dredd?")
I feel like we both understand each other at this point so I'll ask because I'm curious, what languages are you into? I'd really like a chance to dig into Erlang, OCaml, or Racket, but I can never really justify it. Mostly I'm a boring C/Python person (believe it or not, I don't like Go all that much)
> One thing Go haters rarely reckon with is that Go is the only popular modern language (ie from this millennium). Everything else is way older.
Rust. TypeScript. Swift.
I wrote a reply to a sibling comment here [0], but wanted to reemphasize that Rust and Swift are far from popular. I think it bears repeating because the gap between a Rust/Swift and Go is notable, but the gap between a Rust/Swift and Java is a chasm. I would guess this isn't the intuition of an HNer (it wasn't mine until I dug in for this discussion haha)
The median number of users across all programming languages is zero. Millions of people around the world use Rust and Swift, so it seems like a stretch to say they're far from popular. If Rust isn't a popular language compared to Java, we might just as well say Java is not a popular language compared to Excel.
I don't think millions of people use either Rust or Swift.
Millions of people don't use Go either.
You got me
Kotlin as well, although I think this is mostly because of Android?
> Golang is basically just more concise Java.
That is exactly how it was sold.
A safe C, or a nicer simpler Java.
Nobody cared about Erlang back then and nobody does today.
I write Erlang for a living.
I was an early Golang dev and people were _crazy_ with channels for a couple years. I remember the most popular Golang Kafka client was absolute spaghetti of channels and routines.
It's never been "safe C" because it's garbage collected. Java is truly the comp because it's a great Grug language.
I also wrote some Erlang in the past, I really enjoy it and I was sad that Go didn't borrow more.
This is "share by communicating, don't communicate by sharing". Pretty much everyone agrees it's a good thought. ~Most programs get there just fine with channels, because ~most code isn't so performance sensitive that channel limitations matter. Go leaves plenty of room for the rest of programs to do something else. Seems good.
> nicer simpler Java
Ironically, Java was sold as a "nicer simpler C++".
Both can be true at the same time.
I remember very well one of the first public presentations about Go. It focused heavily on goroutines and channels and included a live demonstration of pushing an element through one million channels. It also included a demo of spinning up three racing queries to the Google search engine using the select statement, and picking whoever returned first. it was all about the new cool feature. They also had TCP-over-channels and eventually had to remove that because the model didn’t fit.
Nobody may have known they cared about Erlang, but those features sure made people pay attention.
> Nobody cared about Erlang back then and nobody does today. > > I write Erlang for a living.
I think this is incredibly correct and obviously personally true for you but I'd like to add one more thing from the peanut gallery.
No one really needs Erlang either. Turns out most problems are just fine not being modeled in the way that Erlang wants to model problems.
Just as a someone with a hammer could truthfully claim they don’t need a nail gun.
Not that I think Erlang manages to be a nail gun; it has enough idiosyncrasies that the comparison is not terribly accurate. Still, “need” is doing a lot of heavy lifting in that sentence.
Writing Elixir for a living is seemingly a growing trend.
> The actual core of OTP was the supervisor tree but that's too complicated so Golang is basically just more concise Java.
Which is ironic, maybe kinda funny. The first "larger" project I've tackled in Go was a chat server. I wanted a simple supervisor for each connected client; in case a goroutine encountered a recoverable error but needed to be restarted. I don't have any practical experience with OTP, but I've always been a big fan of daemontools/runit/etc: just let it crash, restart, and recover.
So in Go, you can't (easily) obtain a handle to a goroutine. The authors' entire argument seemed to be that this would allow developers to implement "bad" patterns, like thread-local storage. You can of course still come up with some wrapper code, like this:
But what you really wanted was something like: Now of course you still want some wrapper code to maintain client/connection state, etc. But if you wanted a "real" supervisor tree, you'd have to do this dance for every sub-goroutine you'd like to spawn. You'll soon end up with func(*Service, any) and throw away static typing along the way. Generics wouldn't be introduced until 18 releases after, and I don't think they would help all that much.Correct me if I'm missing anything obvious.
I don’t think channels and goroutines suck. For example their usage in golang’s ssh server implementation is eminently readable.
And performant enough.
Engineers ship with them, and do not care if there’s a purer system elsewhere.
Definitely agree that goroutines don't suck; it makes go into one of the only languages without "function coloring" problem; True N:M multithreading without a separate sync and async versions of the IO libraries (thus everything else).
I think channels have too many footguns (what should its size be? closing without causing panics when there are multiple writers), thus it's definitely better "abstracted out" at the framework level. Most channels that developers interact with is the `Context.Done()` channel with <-chan struct{}.
Also, I'm not sure whether the go authors originally intended that closing a channel would effectively have a multicast semantics (all readers are notified, no matter how many are); everything else have pub-sub semantics, and turns out that this multicast semantics is much more interesting.
The answer to your question is zero, unless there is a compelling reason for it to buffer, and if there is you’ll know it.
Non-buffering channels are much simpler to reason about and have very useful semantics in pipelines.
Ahh finally someone has said it. Unfortunately I can't seem to voice this opinion without getting the critique of you're just not smart enough to get it.
While I like the language, threads in Go are not any easier than any other language (which is to say, most devs can't use them correctly, and your program will have bugs), and suffer from a ton of ergonomic issues, like being hard to keep track of, difficult(ish) cancellation(how do you cancel a big synchronous I/O operation), and channels suffer from backpressure related hard-to-debug issues.
In the case of golang, all you need is hordes of lousy programmers that can't understand anything serious.
Serious being what?
Most programmers are "lousy programmers" because most of the are not researchers and have no interest in theory.
I can write in Go or Rust (or anything else given some time) but I won't ever use Rust to write any kind of business logic for my company - and this is what I do most of the time I actually write code.
Why? Because Rust is terrible for the job (at least if we are talking about real life scenario where the job should be done in hours and not weeks)
I can do business logic, like complex CRUD, in Rust much faster than in other languages. I assume, «lousy programmers» have problems with separation of concerns, which then cause problems with borrow checker, because developers need to perform different things in different ways on different parts of data structures (AKA «god object» anti-pattern).
> separation of concerns
but this would be a problem almost anywhere, no?
This has nothing to do with Rust vs Go vs Ruby or something else. Rust makes this even bigger problem, ok, but I don't think we should call people not using Rust (c\c++ etc) "lousy".
PS: tbh this kind of statements usually (from my experience) come from people who mainly write system level software or work with hardware directly.
I think that's harsh. IME Go excels in a business setting where the focus is on correct, performant, maintainable, business logic in larger organizations, that's easy to integrate with a bunch of other systems. You can't squeeze every last bit of low-level performance out of it but you can get ... 9x% of the way there with concurrent code that is easy to reason about.
What matters is being 100x faster than python, not 5x slower than C++ or 2x slower than Java. Performant enough that IO (or network) rather than CPU is the bottleneck (in my limited experience).
Personally I just far prefer to work in a GCed langauge. Much simpler mental model.
> correct, performant, maintainable, business logic
That's the requirement for 100% if professional software development.
I don’t like writing golang, but I sure like reading it. It’s nearly impossible not to understand.
Yikes, language flamebait in 2025?
Flamebait? It's literally what the designers of Golang said publicly about the background of prospective developers, and how that constrained the language design:
"The key point here is our programmers are Googlers, they’re not researchers. They're typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They're not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt."
Nowhere in this quote are these fresh grads equated to "lousy programmers", though (which the flamebaity comment did).
And interpreting the quote charitably I'm going to have to agree with it - I don't think many of my coworkers care enough to get to the point where they'd appreciate everything something like Haskell can do for them.
Have you ever worked with fresh grads? They're definitionally lousy programmers, if we take lousy mean "having a propensity to inject bugs into programs".
The more time you spend in software engineering the more you realize that this is actually brilliant and helpful, not bad. And IME, the more expressivity I have fluently at my disposal, the more complexity I tend to create. I’m not saying I don’t enjoy more complex languages, but reducing cognitive load and easing collaboration with more junior engineers are easily the best features of Go, hands down (although I can’t say the new generics necessarily always help with that).
That's one interpretation but I think Pike was using a sarcastic meaning for brilliant. I think he's saying that he wants to mentor people to become programmers, not to learn a difficult, sublime language. It's like when a top scientist tells you they're not smart enough to understand what's going on in some part of his field. It's not necessarily a compliment.
Feel free to listen to the context at https://learn.microsoft.com/en-us/shows/lang-next-2014/from-... around 20:40 to 21:10. It's pretty clear that it's a serious quote and not "sarcastic" at all. Least of all about research languages since Pike mentions CSP at length and how to make concurrency be not "scary" to prospective users - overall, it seems that he's talking about a real constraint he's facing. In the same talk he's even apologetic for not preventing data races in the language, since it would have involved too much complexity.
There are a lot of those for sure. There are far more in Python and Java. Any popular, simple-enough language will attract a horde of jobseeking mouthbreathers, that’s kind of inevitable and normal really.
> goroutines are not really a lot better than threads in other languages
They are though. Even Java people agree, Java recently got their implementation of goroutines (green threads). It's a superior model to async for application level programming.
> but people realized channels kind of suck and goroutines are not really a lot better than threads in other languages
Are these people here in the room with us right now?
But gorutines are better than threads when you need you need many (say >100K) of them. Lightweight threads are not unique to Go but in Go they are easy to use and the default way to build network applications. Java is catching up with project Loom but it would never be as easy as Go to use.
> are better than threads when you need many (say >100K) of them
Stackless coroutines are even more efficient for that case, and Rust makes them comparatively easy. (And slated to get even easier in future versions, as the async support is improved further.)
Moreover stackful coroutines/fibers as used in Golang also makes it infeasible to have seamless FFI with the standard C API/ABI, which cuts you off from the bulk of the ecosystem. There are other issues too with having fibers in a C-like systems programming language - see https://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p13... for a nice summary of them.
Alef (arguably one of the antecedent of Go) had stackful couroutines/fibers yet it could have had such a seamless FFI to the standard C API/ABI.
The plan9 libthread (which reimplemented the Alef concurrency model for C) did have seamless use of the C API/ABI.
The mechanism was that it had threads and coroutines, a function needing to make use of the C API in a seamless manner would simply run as a thread rather than a coroutine. It was then easy to share data as the CSP model (with channels) worked between both coroutines, threads, and coroutines within a thread.
So if one wishes to use stackful coroutines, and still have that seamless compatibility, an approach mixing the Go and Alef approaches would seem necessary. i.e. the Go migrating coroutines as a default, but with the option to use Alef like thread bound coroutines when necessary.
> The mechanism was that it had threads and coroutines, a function needing to make use of the C API in a seamless manner would simply run as a thread rather than a coroutine.
This reintroduces the colored functions that we were trying to get away from in the first place, by adopting stackful coroutines/fibers. Why not use async at that point? I can understand that Alef didn't, because stackless mechanisms were not well understood at the time. But it's plausible that we can do better.
Stackless coroutines are what every language that has async/await (Js, C#) uses. I've seen them getting a lot of hate here for needing a special kind of method to run in and being less elegant than green threads.
Why do you need that many threads? Linux doesn't even allow you that many file handles.
> but people realized channels kind of suck and goroutines are not really a lot better than threads in other languages. The actual core of OTP was the supervisor tree
Immutability is the actual core and power of Earlang
if err != nil every few lines is hardly "more concise"...
It was interesting towards the latter half of the article where the author talks about how much of the correctness may be culturally enforced:
>More amorphous, but not less important is Rust's strong cultural affinity for correctness. For example, go to YouTube and click on some Rust conference channel. You'll see that a large fraction of the talks are on correctness, in some way or another. That's not something I see in Julia or Python conference talks.
And it creates an interesting chicken and egg approach. The borrow checker may indeed be too strict (and of course, has its edge cases and outright bugs), but its existence (rather than the utility it brings) may have in fact attracted and amassed an audience who cares about correctness above all else. Even if we abolished the borrow checker tomorrow, this audience may still maintain a certain style based on such principles, party because the other utilities of Rust were built around it.
It's very intriguing. But like anything else trying to attract people, you need something new and flashy to get people in the door. Even for people who traditionally try to reject obvious sales pitches.
This is also somewhat backed up by the fact that OCaml (to my understanding) is basically GC Rust without a borrow checker, and yet it’s basically a hobby language.
Idiomatic programming in a functional language requires garbage collection. There is a reason languages like OCaml and Haskell have a garbage collector. Without it, programming in these languages would be completely different.
If you look at it from that perspective, then Rust is the hobby language.
> Idiomatic programming in a functional language requires garbage collection.
Rust has functional programming features and no garbage collection, because the borrow checker can tell when a closure will outlive the references in its captured environment. We used to think that would not be feasible other than perhaps in very special cases - hence the need for GC to keep that environment around - but Rust proved that wrong quite convincingly.
"Having FP features" does not mean "allow an idiomatic FP programming style". If this were the case, we'd be seeing a revolution in functional programming now. It's not like the FP community is slow to adopt good ideas. In fact, many innovations that lead to the invention of the borrow checker came from the functional programming world (linear, affine types, effect systems).
What's an "idiomatic FP programming style"? Is typical LISP code "FP" enough? That uses GC of course, but other than that it's often high-level enough to be quite comparable w/ modern Rust.
Not necessarily GC, but any kind of automated memory management that allows composition.
> Without it, programming in these languages would be completely different.
How different?
I believe you meant "Different how?"
They addressed that: completely.
Rather, an academia language.
Also, OCaml had trouble with multithreading for quite some time, which was a limiting factor for many applications.
Facebook made a large effort to thrust OCaml into the limelight, and even wrote a nice alternative frontend (Reason). Sadly, it did not stick.
I had the impression that SML was more popular in academia, and OCaml in industry.
Old but funny comparison: http://adam.chlipala.net/mlcomp/
SML was a generation before ocaml. I would say the two languages from the same generation that competed for academia's mindshare were ocaml and Haskell.
Timeline-wise, sure, but I was referring to their present-day use.
If by "popular in industry" you mean a handful of companies use it, but has 1/100th the momentum and community and resources of Go or Rust, then yes.
I meant among ML-family languages.
> 1/100th the momentum and community and resources of Go or Rust
I think even 1/100 would be pretty generous.
I think there are two other big differences that also helped Rust become popular:
* Rust has a C++-flavored syntax, but OCaml has a relatively alien ML-flavored syntax.
* Rust has the backing of Mozilla, but I don't think OCaml had comparable industry backing. (Jane Street, maybe?)
> rust has a C++-flavored syntax
I do not at all agree with this. Rust is by far the most complex language in terms of syntax that has ever become popular enough to compare it to anything.
I think some of that is just that C++ doesn’t explicitly express its semantic complexity in its syntax, while Rust does. In some ways this is an advantage for Rust, although yeah I agree it makes Rust really hard to use, and I kinda hate its syntax.
C++ is a stretch, but I sort of understand GP's point. I'm slowly learning Rust now and it feels like a deep evolution of C type syntax. Not dissimilar to Javascript in the most base sense of language constructs.
But you use it more and see actionscript types of function notation, funtional language semantics where you just "return" whatever was the last expression in a statement, how structs have no bodies (and classes aren't a thing) and instead everything is implemented externally, and it starts to really become its own beast.
I didn't think it was that controversial a statement. For example, Wikipedia says:[0]
> Rust's syntax is similar to that of C and C++,[43][44] although many of its features were influenced by functional programming languages such as OCaml.[45] Hoare has described Rust as targeted at frustrated C++ developers...[15]
I'm not sure what "actionscript types of function notation" means, but Rust's closures syntax (|x| ...) was probably inspired by Ruby and/or Smalltalk. Anyway, sure, the expression-orientedness, implicit return, etc., are very un-C++-like, but I do think the syntax was explicitly designed to be approachable to C++ developers.
[0]: https://en.wikipedia.org/w/index.php?title=Rust_(programming...
I 100% disagree with this. Typescript is syntactically far more complex.
Complex isn't the same thing as hard. The rust syntax often times looks like a cat walked across the keyboard.
And typescript's doesn't? They're incredibly similar syntactically except Typescripts type system is turing complete and unsound.
In my eyes, Rust code is as simple as Python code or even simpler in most cases, but it also allows to create and maintain complex pieces of abstraction, when necessary, then hide them in a library or a macro. :-/
Can you show example of "written by random cat" code you often see?
Facebook and their Reason?
The first version of Rust compiler, I think, was written in OCaml.
It was a bootstrap implementation.
I wouldn't read too much into its lacking the borrow checker.
It's not about not having a C-like syntax (huge mainstream points lost), good momentum, and not having the early marketing clout that came from Rust being Mozilla's "hot new language".
Hobby language? Plenty of commercial and important software has been written in OCaml.
Hell, the early versions of the Rust compiler were written in OCaml...
How may be the wrong word, but it’s definitely a niche language and significantly less software is written in it
Maybe I’m wrong, but I only really know of Jane Street for OCaml, meanwhile FAANG all has at least some rust code.
Also I would argue the rust compiler started as a hobby project
Also Bloomberg: https://news.ycombinator.com/item?id=28278553
Facebook Messenger's backend was/is OCaml... React was originally written in SML, then OCaml, then whatever it is now. And a bunch of places use it for various things.
https://ocaml.org/industrial-users
React was never written in SML or Ocaml. It was originally called FaxJS, and the source code is published online.
A version of React was built to run in ReasonML, which is a flavor of Ocaml for the web, but Reason didn't even exist before React was fairly well established.
https://news.ycombinator.com/item?id=15209814
That's a nonsensical point, though. Building a proof of concept in a language and then rebuilding the practical implementation of it in another language and runtime doesn't make the two the same thing. If Notch had built a proof of concept of Minecraft in Python before building the Java version, we wouldn't say Minecraft was originally written in Python. There wasn't even a robust way to compile OCaml for the web in 2010/2011 even if you wanted to try to use the same code. My understanding is that zero SML or Ocaml related to React ever ran in production, which makes the assertion that it was used in anything other than an academic capacity moot.
Hell, Facebook's own XHP's interface (plus PHP/Hack's execution model) is more conceptually relatable to React, and its initial development predates Jordan's time at Facebook. It wasn't JavaScript, but at the very least it defined rails for writing applications that used the DOM.
React is and has always been javascript…
Not what the author of React says:
> Yes, the first prototype of React was written in SML; we then moved onto OCaml.
> Jordan transcribed the prototype into JS for adoption; the SML version of React, however great it might be, would have died in obscurity. The Reason project's biggest goal is to show that OCaml is actually a viable, incremental and familiar-looking choice. We've been promoting this a lot but I guess one blog post and testimonial helps way more.
https://news.ycombinator.com/item?id=15209814
Sure, a prototype of an idea that would eventually become React was rewritten into JS to create the initial seed of the software that would eventually be called React.
Realistically unless you want to work at Jane Street or Inria (the French computer science lab where Ocaml was made), if you want to use Ocaml, it's going to be as a hobby.
There is also Ahrefs.
You can say that for almost any language that's not C/C++, C#, Java, Python and JS. Rust is just barely beginning to become "corporate". Even Ruby, which is pretty mainstream, has relatively few jobs compared to the big corporate languages.
Speaking of Ruby, its usage has cratered down in the past decade: https://madnight.github.io/githut/#/pull_requests/2024/1
Well, it was hyped beyond belief at one point... Kind of nice that it's back to being niche, I'd hate for it to become Python or TS.
I kind of like that Ruby is still focusing on single developer/small team productivity.
Non-hobby languages is a narrow club, yes.
Your list is at least missing PHP, Typescript, Swift, Go, Lua, Ruby and Rust though.
But Ocaml really doesn't belong anywhere close to this list.
Ummm Lua? It's a nice little scripting language, but literally never seen a job ad for a job using mostly Lua. It's almost the definition of hobby language...
OCaml runs software that billions use, is used by financial and defense firms, plus Facebook.
But Lua? By that metric I'm throwing in every language I've ever seen a job for...
R, Haskell, Odin, Lisp, etc...
Edit - this site is basically a meme at this point. Roblox is industrial strength but Facebook, Dassault and trading firms are "hobby". Lol.
Also, I'm not dissing Lua, there's just irony in calling Lua industrial but not OCaml...
Lua is widely used for scripting in games. It might not be the main language of the project, but it's still very common.
Lua has petered out a bit but it has been used as a scripting and config language for a ton of games and commercial embedded. Not a hobby language, not typically a main implementation language but that doesn’t mean no commercial use. posix/bash shell isn’t a hobby language either, but unless you’re Tom Lord or something (RIP) you’re not doing the entire project in it.
Do realize that luajit for years was bankrolled by corporations.
LuaJIT is widely used for writing Nginx plugins.
> But Lua?
Lua, Bash ... these are birds of a feather. They are the glue holding things together all over the place. No one thinks about them but if they disappeared over night a LOT of stuff would fall apart.
As others already pointed out, Lua is used in tons of video games as the scripting language.
The most famous example being World of Warcraft, but it's far from the only one. If you play, or have played, games, you almost certainly have run software built with Lua without realizing it.
It's not because a language isn't relevant in your personal coding niche that it's not industrially relevant.
I'm simply pointing out the irony in calling a (mostly game) scripting language like Lua "industrial" while calling a language used by FAANG, defense companies and finance companies a "hobby" language.
There's no irony, bash and VB6, no matter what you think about the quality of the said languages, are also scripting language and neither are in the “hobby language” category as their use is (or was) very broadly distributed.
OCaml's use is comparatively very, very narrow.
And, in case you are wondering, I have absolutely nothing again OCaml. In fact, my first ever programming language was, as a significant fraction of French engineer from my generation, the “Lite” dialect of Caml. And I suspect that its OCaml heritage is a significant fraction of the reason why I love Rust.
But being used by exactly one FANG company, a single finance one and allegedly a defense company (aren't you confusing Dassault System with Dassault Aviation ?) isn't enough to change its status, especially when it's not the dominant language in two of those (AFAIK Jane Street really is the only one where OCaml has such a central place).
Roblox apps are built in Lua.
> and yet it’s basically a hobby language.
The difference between academia languages such as ocaml or haskell and industry languages such as Java or C# is hundreds of millions of dollar in advertising. It's not limited to the academy: plenty of languages from other horizons failed, that weren't backed by companies with a vested interest in you using their language.
You should probably not infer too much from a language's success or failure.
The main difference is the ecosystem. The Haskell community has always focused primarily on the computer science part, so the developer experience has mostly been neglected. They have been unable to attract a large enough hobbyist community to develop the libraries and tooling you'd take for granted with any other language, and no company is willing to pay for it out of pocket either. Even load-bearing libraries feel like a half-finished master's thesis, because that's usually what it is.
No amount of advertising is going to propel Haskell to a mainstream language. If it wants to succeed (and let's be honest, it probably doesn't), it's going to need an investment of millions of developer-hours in libraries and tooling. No matter how pretty and elegant the language may be, if you have to reinvent the wheel every time you go beyond "hello world" you're going to think twice before considering it for production code.
C, C++, Python, Perl, Ruby, didn't have "millions of dollars in advertising", and yet.
Java and C# are the only one's that fit this. Go and Rust had some publicity from being associated with Google and Mozilla, but they both caught on without "millions of dollars in advertising" too. Endorsement by big companies like MS came much later for Rust, and Google only started devoting some PR to Go after several years of it already catching momentum.
I don't know about the others but C and C++ certainly did. They had a number of commercial compiler vendors advertising them in the 80s and 90s when they established themselves.
You're making it sound like the success of a language is determined purely by its advertising budget by pointing at languages that had financial backing, which disregards that financial backing allows for more resources to solve technical problems. Java and C# have excellent developer tools which wouldn't have existed in their current state without lots of money being thrown around, and the languages' adoption trajectory wouldn't have looked the way they did if their tooling hasn't been as good as it was. A new language with 3 people behind it can come up with great ideas and excellent execution, but if you can't get enough of the scaffolding built in order to gain development momentum and adoption, then it is very hard to become mainstream, and money can help with that.
> which disregards that financial backing allows for more resources to solve technical problems.
No, I'm not talking about financial backing in general, I'm talking specifically about advertising the language.
There's no doubt that a big ecosystem helps a lot, Python is a good proof of that ; but this is not what i was talking about.
> hundreds of millions of dollar
Yes.
> in advertising
No, in hiring 500 compiler and tool developers, developing and supporting libraries, optimizing it for niche use cases.
The Java ecosystem has benefited from way more than that. I was specifically refering to advertising. For instance this kind of headlines [1].
[1]: https://www.theregister.com/2003/06/09/sun_preps_500m_java_b...
> The difference between academia languages such as ocaml or haskell and industry languages such as Java or C# is hundreds of millions of dollar in advertising
Rust is a significant counterexample
About 20 years ago your choice of language basically boiled down to what you were going to pick for your web server. Your choices were
- Java (popular among people who went to college and learned all about OOP or places that had a lot of "enterprise" software development)
- Ruby on Rails (which was the hot new thing)
- Python or Perl to be the P in your LAMP stack
- C++ for "performance"
All of these were kitchen sink choices because they wound up needing to do everything. If you went back in time and said you were building a language that didn't do something incredibly common and got in the way of your work, no one would pick it up.
Not everybody is writing web applications. In fact, all of the languages have been invented years or even decades before the internet became popular. Except maybe for Perl, they have been designed as general purpose programming languages.
I'm not sure.. without the borrow checker you could have a pretty nice language that is like a "pro" version of golang, with better typing, concise error handling syntax, and sum types. If you only use things like String and Arc objects, you basically can do this, but it'd be nice to make that not required!
That's my whole point. Without the borrow checker it would have been a nice language, but I believe it would not have gotten popular, because being nice isnt enough to be popular in the current programming language landscape.
As a Rust fan, I 100% agree. I already know plenty of nice, "safe", "efficient" languages. I know only one language with a borrow checker, and that feature has honestly driven me to use it in excess.
Most of my smaller projects don't benefit so much from the statically proven compile time guarantees that e.g. Rust with it's borrow checker provide. They're simple enough to more-or-less exhaustively test. They also tend to have simple enough data models and/or lax enough latency requirements that garbage collectors aren't a drawback. C#? Kotlin? Java? Javascript? ??? Doesn't matter. I'm writing them in Rust now, and I'm comfortable enough with the borrow checker that I don't feel it slows me down, but I wouldn't have learned Rust in the first place without a borrow checker to draw me in, and I respect when people choose to pass on the whole circus for similar projects.
The larger projects... for me they tend to be C++, and haven't been rewritten in Rust, so I'm tormented with a stream of bugs, a large portion of which would've been prevented - or at least made shallow - by Rust's borrow checker. Every single one of them taunts me with how theoretically preventable they are.
You can use a garbage collector in Rust to circumvent borrow checker. You can use simple reference counting (Rc, Arc), or trace and sweep, arenas, or generation based garbage collectors. Even a simple .clone() can help a lot in many cases.
Borrow checker is my friend, it helps me write better code, but it doesn't stops me when I don't care about code quality and just want a task to be done.
Rust's original author agrees with you: https://graydon2.dreamwidth.org/307291.html
> without the borrow checker ... golang... concise error handling syntax
Except both of these things are that way for a reason.
The author talks about the pain of having other refactor because of the borrow checker. Every one laments having to deal with errors in go. These are features, not bugs. They are forcing functions to get you to behave like an adult when you write code.
Dealing with error conditions at "google scale" means you need every one to be a good citizen to keep signal to noise down. GO solves a very google problem: don't let JR dev's leave trash on at the campsite, force them to be good boy scouts. It is Conways law in action (and it is a good thing).
Rust's forced refactors make it hard to leave things dangling. It makes it hard to have weak design. If you have something "stable", from a product, design and functionality standpoint then Rust is amazing. This is sort of antithetical to "go fast and break things" (use typescript, or python if you need this). It's antithetical to written in the stand up requirements, that change week to week where your artifacts are pantomime and post it notes.
Could the borrow checker be better, sure, and so could errors in go. But most people would still find them a reason to complain even after their improvement. The features are a product of design goals.
The lamentations I usually hear about errors in Go are that you have to use a product type where a sum type would be more appropriate, and that there isn't a concise syntax analogous to Rust's ? operator for the extremely common propagate-an-error-up-a-stack-frame operation, not that you have to declare errors in your API.
Also, in my experience, the Rust maintainers generally err on the side of pragmatism rather than opinionatedness; language design decisions generally aren't driven by considerations like "this will force junior developers to adhere to the right discipline". Rust tries to be flexible, because people's requirements are flexible, especially in the domain of low-level programming. In general, they try to err on the side of letting you write your code however you want, subject to the constraints of the language's two overriding design goals (memory safety and precise programmer control over runtime behavior). The resulting language is in many ways less flexible than some more opinionated languages, but that's because meeting those design goals is inherently hard and forces compromises elsewhere (and because the language has limited development resources and a large-but-finite complexity budget), not because anyone views this as a positive in and of itself.
(The one arguable exception to this that I can think of is the lack of syntactic sugar for features like reference counting and fallible operations that are syntactically invisible in some other languages. That said, this is not just because some people are ideologically against them; they've been seriously considered and haven't been rejected outright, it's just that a new feature requires consensus in favor and dedicated resources to make it happen. "You can do the thing but it requires syntactic salt" is the default in Rust, because of its design, and in these cases the default has prevailed for now.)
You can absolutely "go fast and break things" (i.e. write prototype-quality code) in Rust, but it requires a lot of boilerplate that will be very visible in the code, and will also make it comparatively easy to refactor the prototype into a real production-quality implementation. You can't really say this of any other languages AIUI. What often happens instead is that the prototype is put in production more or less as-is, without comprehensively fixing the breakage. Rust makes it very clear how to avoid that.
Java, especially after generics were introduced, was a pain to use because of the type system. That’s not my opinion, I always found that claim a bit overwrought but it’s true that it was fairly widespread. Dealing with the type system got progressively a bit easier as the language evolved and certain types could be inferred. From the release notes I’ve seen I’ve gotten the impression that similar things have happened with the borrow checker. But people who have gone through the gauntlet have trouble reframing that experience and it’s always difficult to tell if the reputation is still accurate or not.
I am curious what the second language with a borrow checker will look like.
> but I'm not going to switch because I can technically do all that stuff in C++"
That's exactly what people say about Rust: just get good, use some tools, and be careful - and you can achieve the same memory safety.
There are some artificial limitations, but I love the upside: I don't need defensive programming!
When my function gets an exclusive reference to an object, I know for sure that it won't be touched by the caller while I use it, but I can still mutate it freely. I never need to make deep copies of inputs defensively just in case the caller tries to keep a reference to somewhere in the object they've passed to my function.
And conversely, as a user of libraries, I can look at an API of any function and know whether it will only temporarily look at its arguments (and I can then modify or destroy them without consequences), or whether it keeps them, or whether they're shared between the caller and the callee.
All of this is especially important in multi-threaded code where a function holding on to a reference for too long, or mutating something unexpectedly, can cause painful-to-debug bugs. Once you know the limitations of the borrow checker, and how to work with or around them, it's not that hard. Dealing with a picky compiler is IMHO still preferable to dealing with mysterious bugs from unexpectedly-mutated state.
In a way, borrow checker also makes interfaces simpler. The rules may be restrictive, but the same rules apply to everything everywhere. I can learn them once, and then know what to expect from every API using references. There are no exceptions in libraries that try to be clever. There are no exceptions for single-threaded programs. There are no exceptions for DLLs. There are no exceptions for programs built with -fpointers-go-sideways. It may be tricky like a game of chess, but I only need to consider the rules of the game, and not odd stuff like whether my opponent glued pieces to the chessboard.
Yes! One of the worst bugs to debug in my entire career boiled down to a piece of Java mutating a HashSet that it received from another component. That other component had independently made the decision to cache these HashSet instances. Boom! Spooky failure scenarios where requests only start to fail if you previously made an unrelated request that happened to mutate the cached object.
This is an example where ownership semantics would have prevented that bug. (references to the cached HashSets could have only been handed out as shared/immutable references; the mutation of the cached HashSet could not have happened).
The ownership model is about much more than just memory safety. This is why I tell people: spending a weekend to learn rust will make you a better programmer in any language (because you will start thinking about proper ownership even in GC-ed languages).
A weekend?
> This is an example where ownership semantics would have prevented that bug.
It’s also a bug prevented by basic good practices in Java. You can’t cache copies of mutable data and you can’t mutate shared data. Yes it’s a shame that Java won’t help you do that but I honestly never see mistakes like this except in code review for very junior developers.
The whole point is that languages like Java won't keep track of what's "shared" or "mutable" for you. And no, it doesn't just trip up "very junior developers in code review", quite the opposite. It typically comes up as surprising cross-module interactions in evolving code bases, that no "code review" process can feasibly catch.
Speak for yourself. I haven't seen any bug like this in Java for years. You think you know better and my experience is not valid? Ha. Ok. Keep living in your dreams.
> When my function gets an exclusive reference to an object, I know for sure that it won't be touched by the caller while I use it, but I can still mutate it freely.
I love how this very real problem can be solved in two ways:
1. Avoid non-exclusive mutable references to objects
2. Avoid mutable objects
Former approach results in pervasive complexity and rigidity (Rust), latter results in pervasive simplicity and flexibility (Clojure).
Shared mutable state is the root of all evil, and it can be solved either by completely banning sharing (actors) or by banning mutation (functional), but Rust gives fine-grained control that lets you choose on case-by-case basis, without completely giving up either one. In Rust, immutability is not a property of an object in Rust, but a mode of access.
It's also silly to blame Rust for not having flexibility of a high-level GC-heavy VM-based language. Rust deliberately focuses on the extreme opposite of that: low-level high-performance systems programming niche, where Clojure isn't an option.
This reminds me of something that was popular in some bioinformatics circles years ago. People claimed that Java was faster than C++. To "prove" that, they wrote reasonably efficient Java code for some task, and then rewrote it in C++. Using std::shared_ptr extensively to get something resembling garbage collection. No wonder the real Java code was faster than the Java code written in C++.
I've been writing C++ for almost 30 years, and a few years of Rust. I sometimes struggle with the Rust borrow checker, and it's almost always my fault. I keep trying to write C++ in Rust, because I'm thinking in C++ instead of Rust.
The lesson is always the same. If you want to use language X, you must learn to write X, instead of writing language Y in X.
Using indexes (or node ids or opaque handles) in graph/tree implementations is a good idea both in C++ and in Rust. It makes serialization easier and faster. It allows you to use data structures where you can't have a pointer to a node. And it can also save memory, as pointers and separate memory allocations take a lot of space when you have billions of them. Like when working with human genomes.
If using indices is going to be your answer, then it seems to me you should at least contend with the OP's argument that this approach violates the very reason the borrowchecker was introduced in the first place.
From the post:
"The Rust community's whole thing is commitment to compiler-enforced correctness, and they built the borrowchecker on the premise that humans can't be trusted to handle references manually. When the same borrowchecker makes references unworkable, their solution is to... recommend that I manually manage them, with zero safety and zero language support?!? The irony is unreal."
That seems rather easy? There's still plenty of safety. It's not an `unsafe` trick or worse, you still have bounds checking and safe concurrency and well-defined behavior, and you can trivially guarantee that (while mutating the vec) it cannot be changed unexpectedly while you hold ownership of the vec. The extreme ease of making that guarantee is kinda the core of their complaint.
The dangling-pointer equivalent is an issue, but it's still safe (unlike in C-like langs) and there are ways to mitigate the risk of accidental misbehavior (e.g. generational pointers, or simply an ID check if you have a convenient ID).
That's quite a bit different than what you get in almost any other widely-used language - e.g. some will at best be able to claim "concurrent access won't lead to undefined behavior" via e.g. the GIL, but not prevent unexpected modification (e.g. "freeze" doesn't deep-freeze in the vast majority of languages).
>OP's argument that this approach violates the very reason the borrowchecker was introduced in the first place.
No it doesn't. I just don't think author understands the pitfalls of implementing something like a graph structure in a memory unsafe language. The author doesn't write C so I don't believe he has struggled with the pain of chasing a dangling pointer with valgrind.
There are plenty of libraries in C that eventually decided to use indexes instead of juggling pointers around because it's much harder to eventually introduce a use-after-free when dereferencing nodes this way.
Entity component systems were invented in 1998 which essentially implement this pattern. I don't find it ironic that the Rust compiler herds people towards a safe design that has been rediscovered again and again.
The borrow checker was introduced to statically verify memory safety. Using indices into graphs has been a memory safe option in languages like C for decades. I find his argument as valid as if someone said "I can't use goto? you expect me to manually run my cleanup code before I return?" Just because I took away your goto to make control flow easier it doesn't make it "ironic" if certain legitimate uses of goto are harder. Surely you wouldn't accept his argument for someone arguing for the return of goto in mainstream languages?
Whoah, hold on, the author isn't comparing writing graph structures in Rust to writing it in memory-unsafe languages --- they're comparing it to writing it in other memory-safe languages. You can't force a false dichotomy between Rust and C to rebut them.
The comparison is contrived precisely because he's comparing it to other memory-safe languages. The borrow checker was introduced because the goal of the language was a memory safe language without a runtime. Falling back to indices isn't "ironic", its exactly how you would solve the problem in C/C++.
If your argument is "well Rust should be like Julia and have a GC", well thats not Rust. That language also exists, its possibly called OCaml, but its not Rust.
Rust have few garbage collectors[1] to chose from. For example, rust-cc is simple to use, just put #[derive(Trace, Finalize)] on struct.
[1]: https://crates.io/keywords/garbage-collector
The author is also suggesting that a data structure consisting of a sea of objects referencing each other works perfectly in Python. It doesn’t — Python’s GC can only collect it as a cycle, and this path is not immediate the way the refcount is. And you’re paying for refcounts. Even in a language with full tracing GC (Java, etc), you’re not winning any performance points by making a ton of objects all referencing each other.
Indices or arenas or such will result in better code in all these languages.
I think it's reasonable enough. The author already argued that there are reasons for non GC languages to exist, even if the performance doesn't matter to them.
One interpretation of the article is just the author doesn't personally like the borrow checker, but another interpretation is the author saying the borrow checker is just a bad abstraction.
So under the assumption that we don't have a GC available, what else can we compare the borrow checker against?
Author here. Yeah, I don't like the borrowchecker. But the motivation for me writing the article is the almost religious zeal with which many Rustaceans refuse to even acknowledge that the borrowchecker has a cost in terms of ergonomics, and that this translates to e.g. iteration speed.
You encounter borrowchecker issues? Well, you're just a beginner or not skilled enough. Rust makes you jump through hoops? No it doesn't, it just makes you pay upfront what you otherwise would have. It slows development? No, studies show it doesn't, you must be imagining it.
This is extremely annoying to be on the receiving end of. Even though it comes from a good place (mostly just excitement), it can feel like gaslighting.
Indices can be dangling in almost exactly the same way as pointers. Worse, it's easier to accidentally use-after-free clobber some other item in the same structure, because allocations are "dense." (Pointer designs on systems where malloc/free is ~LIFO experience similar problems.)
This is well known to be solved by using generational indexes. Its not a big deal. The author's entire post is an overreaction/rage-bait.
Basically any data structure like this where you want it relocatable in memory is going to use indirection like indexes or something instead of pointers. Its a very common use case outside of rust.
Might I suggest that the scpptool-enforced safe subset of C++ has a better solution for such data structures with cyclic or complex reference graphs, which is run-time checked non-owning pointers [1] that impose no restrictions on how or where the target objects are allocated. Unlike indices, they are safe against use-after-destruction, and they don't require the additional level of indirection either.
[1] https://github.com/duneroadrunner/SaferCPlusPlus#norad-point...
Except for the fact that one leads to undefined behavior and the other doesn't. This is a massive difference.
Non-UB data structure corruption and other incorrect behavior isn't like, super obviously better than UB corruption and other incorrect behavior.
The obvious upside is that it's so much easier to debug when there's no UB. Debugging UB is never enjoyable.
It’s pretty common to implement graphs in terms of arrays, not because of indices but because of cache locality.
So your “UB” and “non-UB” code would look effectively identical to the CPU and would take the same amount of debugging.
The reality is whether an index was tombstones and referenced or “deallocated” and referenced it is still a programmer fault that is a bug that the compiler could not catch
The practical ramifications are often similar, even if UB can in theory do anything.
Ummm. Yeah it is. I'm sure you can come up with some cases where the impact of the bugs is roughly equivalent, but generally speaking, UB is a terrible class of bug.
For example, if the aho-corasick crate accidentally tries to look up a dangling state node, you'll get a panic. But if this were UB instead, that could easily lead to a security problem or just pretty much anything... because behavior is undefined.
So generally speaking, yes, this is absolutely a major difference and it matters in practice. You even have other people in this thread saying this technique is used in C for this exact reason. Leaving out this very obvious difference and pretending like these are the same thing is extremely misleading.
Author here. In scientific computing, there isn't much of a security implication of UB, and the most dangerous kind of error are when your program silently computes the wrong thing. And that is especially likely when you use indices manually.
You could say that UB enables all behaviour, including silently wrong answers, and you'd be right. But it's more likely to crash your program and therefore be caught.
Most importantly, the comparison to a raw pointer is not relevant. My blog post states that integers come with zero safety (as in: preventing bugs, not risk of UB) and zero language support and that is true. My blog post compares Rust with GC languages, not with raw pointer arithmetic. And it's clear that, when you compare to using GC references, manual indices are horribly unsafe.
You're engaging in a motte-and-bailey fallacy. Your motte is your much more narrow claim about Rust's advantages in a context where you claim not to care about undefined behavior as a class of bugs more severe than logic errors. And you specifically make this claim in an additional context where a GC language is appropriate. That's an overall pretty easy position to defend, and if that were clearly the extent of it, I probably wouldn't have responded at all.
But, that context has been dropped in this thread, and instead folks are making very general claims outside of that very restricted context. Moreover, your bailey is that you don't do a good job outlining that context either. Your blog's opening paragraphs mention nothing about that more constrained context and instead seem to imply a very general context. This is much harder to defend. You go on to mention scientific computing, but instead of it being a centerpiece of the context of your claim, it's just mentioned as aside. Instead, your blog appears to be making very broad claims. But you've jumped in here to narrow them significantly, to the point that it materially changes your point IMO. Let's just look at what you said here:
> The first time someone gave be this advice, I had to do a double take. The Rust community's whole thing is commitment to compiler-enforced correctness, and they built the borrowchecker on the premise that humans can't be trusted to handle references manually. When the same borrowchecker makes references unworkable, their solution is to... recommend that I manually manage them, with zero safety and zero language support?!? The irony is unreal. Asking people to manually manage references is so hilariously unsafe and unergonomic, the suggestion would be funny if it wasn't mostly sad.
There's no circumspection about the context. You're just generally and broadly dismissing this entirely as if it weren't a valid thing ever. But it absolutely is a valid technique and it has real practical differences with an approach that uses raw pointers. If the comparison is with a GC and that context is made clear, then yes, absolutely, the comparison point changes entirely! If you can abide a GC, then a whole bunch of things get easier... at some cost. For example, I don't think it's possible to write a tool like ripgrep with its performance profile in a GC language. At least, I've never seen it done.
> I don't think it's possible to write a tool like ripgrep with its performance profile in a GC language.
I think it is possible to make a language that has both a GC and a borrow checker, treating the GC types as a third level next to the stack and the heap, where complex referencial cycles can bé promoted to the GC, but the defaults push you towards fast execution patterns. Don't know if such a language would be successful in finding its niche. The only way I could see that, is of a non-gc mode could be enforced so that libraries can be written in the more restrictive, faster by default mode, while being consumed by application developers that have less stringent restrictions. This is no different in concept than Python libraries implemented in native languages. Making it the mode be part of the same language could help with prototyping pains: write with the GC and then refactor once at the end after the general design is mostly found.
> I think it is possible to make a language that has both a GC and a borrow checker, treating the GC types as a third level next to the stack and the heap, where complex referencial cycles can be 'promoted to the GC'
It's doable but there are a few issues with that whole idea. You need the ability to safely promote objects to GC-roots whenever they're being referenced by GC-unaware code, and demote them again afterwards. And if any GC object happens to control the lifecycle of any heap-allocated objects, it must have a finalizer that cleans them up RAII style to avoid resource leaks.
> And that is especially likely when you use indices manually.
But that is never the recommendation Rust practitioners would give for graph data structures. One would instead recommend using a generational arena, where each index holds their "generation". The arena can be growable or not. When an element is removed from the arena it gets tombstoned, marked as no longer valid. If a new value reuses a tombstoned position, its generation changes. This shifts the cost of verifying the handle is correct at the read point: if the handle corresponds to an index with a tombstone sentinel or has a different generation, the result of the read operation is None, meaning the handle is no longer valid. This is much better than the behavior of pointers.
For the record, I completely agree that using GC is appropriate for problems that deal with possibly-cyclic general graphs, and maybe even wrt. DAG structures whenever one wishes to maximize throughput and the use of simpler refcounting would involve heavy overhead. (This is a well-known pitfall wrt. languages like Swift, that enforce pervasive use of atomic reference counting.)
Since the use of "pluggable" GC in C-like or Rust-like languages is still uncommon, this generally means resorting to a language which happens to inherently rely on GC, such as Golang.
This might not seem obvious. So allow me an attempt at expanding a bit.
The difference is that a dangling raw pointer to the heap will point to anything that can be modified at any time.
But indexing a dynamic array guarantees that every elements is always well formed and safe to use.
It's true that the array approach cannot suffer from type confusion.
C doesn’t have objects, which are great for encapsulating data structures, but in C++ it’s not at all hard to write a graph data structure.
One wraps it behind a reliable interface, writes automated test and runs them under valgrind/sanitizers and that’s pretty much it.
It’s normal for life and software development to have a certain degree of risk and to spend some effort solving problems. Too many HN comments make it sound like it’s a jungle out there and a use-after-free will chop your head clean off.
I don't mean to imply that writing a graph in C++ is impossible. It's clearly possible in modern C++.
My contention is treating indicies-based management as some tedious, manual workaround. Unity's ECS is written in C#, which has both a GC and the language expressiveness for an actual graph objects, but has adopted an indicies based system. It works, and is performant, so it isn't a mistake for the compiler to herd users down that path.
In a similar vein, the Linux codebase is full of completely legitimate and readable uses of goto, but we are perfectly happy with languages that force us to use structured control flow.
The OP's argument is bunk. It's been said many times too over the years. The fact is that the index approach does not give up everything. The obvious thing it doesn't give up is safety. It's true you can still get bugs via out of bounds accesses, but it won't result in undefined behavior. You get a panic instead.
This is how the regex crate works internally and uses almost no `unsafe`.
I think this is like unsafe - most of your code won’t have it, so you get the benefits of borrow checker (memory safety and race freedom) elsewhere.
An important saving grace that `unsafe` has is that it's local and clearly demarcated. If a core data structure of your program can be compared to `unsafe` and has to be manually managed for correctness, it's very valid to ask whether the hoops Rust makes you jump through are actually gaining you anything.
There is such a thing of languages that align with human intuition. C++ and Rust are not these languages so you have to really learn these languages in depth. Languages like typescript or python or go align more with intuition and you don't really need to learn as much about the details or patterns as these just naturally flow from your intuition. This is a huge huge thing as it makes the language literally take about a week to develop proficiency and two weeks to develop mastery. A language like C++... you can't even develop mastery in a year.
That is not to say these languages are better. Intuition is just one trade off.
Typescript is not a language that matches intuition. Typing complex code while avoiding the any hatch resembles fighting limitations of the borrow checker in Rust.
I find it highly highly intuitive. But my background is haskell like languages.
I feel with practice basic type checking is something that helps you rather then hinders you. It can be learned easily imo. People coming from js tend to have a hard time but that's understandable.
The borrow checker is not easily learned imo. It's always me running into a wall.
For me the problem with TypeScript or Flow when the latter was a thing was that the syntax/semantics of the sub language of types was extremely ad-hoc with so many idiosyncrasies. Maybe if I programmed it all the time I would learned it. But I had to change the relevant code only occasionally and typing helpers to access DOM required constant look at the spec and StackOverflow.
With Rust the rules at least are simple. While following them can be a struggle the compiler errors at least are much more helpful and points to the problem with the design or the checker limitations.
> [The pain of the borrow checker is felt] when your existing project requires a small modification to ownership structure, and the borrowchecker then refuses to compile your code. Then, once you pull at the tiny loose fiber in your code's fabric, you find you have to unspool half your code before the borrowchecker is satisfied.
Probably I just haven't been writing very "advanced" rust programs in the sense of doing complicated things that require advanced usages of lifetimes and references. But having written rust professionally for 3 years now, I haven't encountered this once. Just putting this out there as another data point.
Of course, partial borrows would make things nicer. So would polonius (which I believe is supposed to resolve the "famous" issue the post mentions, and maybe allow self-referential structs a long way down the road). But it's very rare that I encounter a situation where I actually need these. (example: a much more common need for me is more powerful consteval.)
Before writing Rust professionally, I wrote OCaml professionally. To people who wish for "rust, but with a garbage collector", I suggest you use OCaml! The languages are extremely similar.
I guess I'm a bit confused how you can write rust professionally dor 3 years and never encounter this. When I started writing rust in ~2020/2021 i already had issues with the brorow checker.
Maybe its an idiom you already picked up in OCaml and did it mostly right in rust too?
I think I don't end up doing very complicated things most of the time. If you're writing a zero-copy deserialization crate or an ECS framework or something, I'm sure you're bound to run into this issue. But I almost never even have to explicitly write lifetimes. I rarely even see borrowck errors for code I intended to write (usually when I see borrowck errors, it's because I made an error in copy-pasting that resulted in me using a variable after it's been moved, or something like that).
You might have a point with my OCaml background though. I rarely use mutable references, since I prefer to write code in a functional style. That means I rarely am in a situation where I want to create a mutable reference but already have other references floating around.
Here's an example of some of my code: https://github.com/not-pizza/tysm/blob/main/src/chat_complet... . I wouldn't be surprised if there's not a mutable reference or lifetime specifier in this whole project
I believe it. I experienced this once, as I tried to have everything owned. Now I just clone around as if there's no tomorrow and tell myself I'll optimize later.
I need to do this, but I get fixated on interesting but premature optimizations.
Just clone everything, profile, and remove the clones that take significant time.
I've mostly experienced it when moving from borrowing to ownership and vice versa. E.g. having a struct that takes ownership over its fields, and then moving it to a borrow with a lifetime.
It's not super common though, especially if the code is not in the hot path which means you can just keep things simple and clone.
OCaml lacks Rust's ecosystem support. Also, I personally found it ugly, although this is admittedly subjective and also kind of petty.
For the disjoint field issues raised, it’s not that the borrow checker can’t “reason across functions,” it’s that the field borrows are done through getter functions which themselves borrow the whole struct mutably. This could be avoided by making the fields public so they can be referenced directly, or if the fields needs to be passed to other functions, just pass the the field references rather than passing the whole struct.
There are open ideas for how to handle “view types” that express that you’re only borrowing specific fields of a struct, including Self, but they’re an ergonomic improvement, not a semantic power improvement.
> For the disjoint field issues raised, it’s not that the borrow checker can’t “reason across functions,” it’s that the field borrows are done through getter functions which themselves borrow the whole struct mutably
Right, and even more to the point, there's another important property of Rust at play here: a function's signature should be the only thing necessary to typecheck the program; changes in the body of a function should not cause a caller to fail. This is why you can't infer types in function signatures and a variety of other restrictions.
Exactly. We've talked about fixing this, but doing so without breaking this encapsulation would require being able to declare something like (syntax is illustrative only) `&mut [set1] self` and `&mut [set2] self`, where `set1` and `set2` are defined as non-overlapping sets of fields in the definition of the type. (A type with private fields could declare semantic non-overlapping subsets without actually exposing which fields those subsets consist of.)
See Rust's golden rule: https://steveklabnik.com/writing/rusts-golden-rule
This seems to be a golden rule of many languages? `return 3` in a function with a signature that says it's going to return a string is going to fail in a lot of places, especially once you exclude bolted-on-after-the-fact type hinting like what Python has.
It's easier to "abuse" in some languages with casts, and of course borrow checking is not common, but it also seems like just "typed function signatures 101".
Are there common exceptions to this out there, where you can call something that says it takes or returns one type but get back or send something entirely different?
Many functional and ML-based languages, such as Haskell, OCaml, F#, etc. allow the signature of a function to be inferred, and so a change in the implementation of a function can change the signature.
In C++, the signature of a function template doesn't necessarily tell you what types you can successfully call it with, nor what the return type is.
Much analysis is delayed until all templates are instantiated, with famously terrible consequences for error messages, compile times, and tools like IDEs and linters.
By contrast, rust's monomorphization achieves many of the same goals, but is less of a headache to use because once the signature is satisfied, codegen isn't allowed to fail.
> In C++, the signature of a function template doesn't necessarily tell you what types you can successfully call it with, nor what the return type is.
That's the whole point of Concepts, though.
Concepts are basically a half solution - they check that a type has some set of properties, but they don't check that the implementation only uses those properties. As a result, even with concepts you can't know what types will work in a template without looking at the implementation as well.
Example [0]:
[0]: https://cpp.godbolt.org/z/jh6vMnajj> they check that a type has some set of properties, but they don't check that the implementation only uses those properties.
I'd say that's a mistake of the person who wrote the template then.
Also, there are Concepts where you absolutely know which types are allowed, e.g. std::same_as, std::integral, std::floating_point, etc.
> I'd say that's a mistake of the person who wrote the template then.
The fact that it's possible to make that mistake is basically the point! If "the whole point of concepts" were to "tell you what types you can successfully call it with" then that kind of mistake should not be possible.
It's true that there are certain cases where you know the full set of types you can use, but I'd argue that those are the less interesting/useful cases, anyways.
There are languages with full inference that break this rule.
Moreover, this rule is more important for Rust than other languages because Rust makes a lot of constraints visible in function signatures.
But the most important purpose of the rule is communicating that this is a deliberate design decision and a desireable property of code. Unfortunately, there's an overwhelming lack of taste and knowledge when it comes to language design, often coming from the more academic types. The prevailing tasteless idea is that "more is better" and therefore "more type inference is better", so surely full type inference is just better than the "limited" inference Rust does! Bleh.
My interpretation of the post is that the rule is deeper than that. This is the most important part:
> Here is the most famous implication of this rule: Rust does not infer function signatures. If it did, changing the body of the function would change its signature. While this is convenient in the small, it has massive ramifications.
Many languages violate this. As another commenter mentioned, C++ templates are one example. Rust even violates it a little - lifetime variance is inferred, not explicitly stated.
Lifetimes for a function signature in Rust are never inferred from the function code. Rather Rust has implicit lifetime specs with straightforward rules to recover the explicit full signature.
I was speaking about variance specifically. They are not inferred from function bodies, but I think it's fair to say that it's a soft violation of the golden rule because variance has a lot of spooky action at a distance (changing a struct definition can change its variance requirements, which then has ripple effects over all type signatures that mention that struct)
> Are there common exceptions to this out there, where you can call something that says it takes or returns one type but get back or send something entirely different?
I would personally consider null in Java to be an exception to this.
It's super easy to demonstrate your point with the first example the article gives as well; instead of separate methods, nothing prevents defining a method `fn x_y_mut(&mut self) -> (&mut f64, &mut 64)` to return both and use that in place of separate methods, and everything works! This obviously doesn't scale super well, but it's also not all that common to need to structure this way in the first place.
One of his examples of a borrow checker excess:
isn't even legit in modern C++. That's just move semantics. When you move it, it's gone at the old name.He does point out two significant problems in Rust. When you need to change a program, re-doing the ownership plumbing can be quite time-consuming. Losing a few days on that is a routine Rust experience. Rust forces you to pay for your technical debt up front in that area.
The other big problem is back references. Rust still lacks a good solution in that area. So often, you want A to own B, and B to be able to reference A. Rust will not allow that directly. There are three workarounds commonly used.
- Put all the items in an array and refer to them by index. Then write run-time code to manage all that. The Bevy game engine is an example of a large Rust system which does this. The trouble is that you've re-created dangling pointers, in the form of indices kept around after they are invalid. Now you have most of the problems of raw pointers. They will at least be an index to some structure of the right type, but that's all the guarantee you get. I've found bugs in that approach in Rust crates.
- Unsafe code with raw pointers. That seldom ends well. Crates which do that are almost the only time I've had to use a debugger on Rust code.
- Rc/RefCell/run-time ".borrow()". This moves all the checking to run time. It's safe, but you panic at run time if two things borrow the same item.
This is a fundamental problem in Rust. I've mentioned this before. What's needed to fix this is an analyzer that checks the scope of explicit .borrow() and .borrow_mut() calls, and determines that all scopes for the same object are disjoint. This is not too hard conceptually if all the .borrow() calls produce locally scoped results. It does mean a full call chain analysis. It's a lot like static detection of deadlock, which is a known area of research [1] but something not seen in production yet.
I've discussed this with some of the Rust developers. The problem is generics. When you call a generic, the calling code has no idea what code the generic is going to generate. You don't know what it's going to borrow. You'd have to do this static analysis after generic expansion. Rust avoids that; generics either compile for all cases, or not at all. Such restricted generic expansion avoids the huge compile error messages from hell associated with C++ template instantiation fails. Post template expansion static analysis is thus considered undesirable.
Fixing that could be done with annotation, along the lines of "this function might borrow 'foo'". That rapidly gets clunky. People hate doing transitive closure by hand. Remember Java checked exceptions.
This is a good PhD topic for somebody in programming language theory. It's a well-known hard problem for which a solution would be useful. There's no easy general fix.
[1] https://dl.acm.org/doi/10.1145/3540250.3549110
> isn't even legit in modern C++. That's just move semantics. When you move it, it's gone at the old name.
Exactly the opposite actually.
Rust has destructive move while modern C++ has nondestructive move.
So in Rust, an object is dead after you move out of it, and any further attempts to use it are a compiler diagnosed error. In contrast, a C++ object is remains alive after the move, and further use of it isn't forbidden by the language, although some or all uses might be forbidden by the specific user provided move function - you'll have to reference the documentation for that move function to find out.
This article explains the difference well: https://www.foonathan.net/2017/09/destructive-move/
Indeed, this is strictly worse than rust. The object is alive but in an invalid state, so using it is a bug but not one the compiler catches. In the worse case the move is only destructive for larger objects (like SSO), so your tests can pass and you've still got a bug.
C++ teachers like Herb Sutter like to jump in here and make a correction: A moved-from object is in a valid state, you just might not know which state. You can still call methods on it that are valid in any state. ("What is your length?") But you shouldn't fall methods that are only valid in some states. ("Give me your first element.")
Or when you enable optimizations and since accessing an object with invalid state is UB, the compiler helpfully decides it is now permitted to format your hard drive.
https://gcc.gnu.org/legacy-ml/gcc/2016-02/msg00381.html
"The fact is, undefined compiler behavior is never a good idea. Not for serious projects."
Rust borrow checker is designed to enforce "one owner" model (a tree). When you need to have more than one reference, you can use Rc + Weak[0]. Example DoubleLinkedList implementation:
Moreover, if you have cycles instead of trees, you can use a garbage collector with support for cycles, like rust-cc[1].So yes, it's cannot be done statically, because Rust is not designed for that.
However, problem disappears when 'static lifetime is used (or arenas). Nodes can be marked as deleted, instead of dropping them, so pointers are always valid.
In same vein, when nodes are deleted rarely, they can be simply marked as deleted, without dropping them completely (until sibling nodes are updated, at least):
When node is deleted (its payload is dropped), linked list is still walkable.[0]: https://doc.rust-lang.org/std/rc/struct.Weak.html
[1]: https://github.com/frengor/rust-cc
> The trouble is that you've re-created dangling pointers
That's true, but as a runtime mitigation, adding a generational counter (maybe only in debug builds) to allocations can catch use-after-frees.
And at least it's less likely to be a security vulnerability, unless you put sensitive information inside one of these arrays.
> adding a generational counter (maybe only in debug builds) to allocations can catch use-after-frees.
At the cost of making the use of the resulting heap significantly slower and larger than if you just wrote the thing in Java to begin with, though! The resulting instrumentation is likely to be isomorphic to GC's latency excursions, even.
This is the biggest issue that bugs me about Rust. It starts from a marketing position of "Safety With No Compromises" on runtime metrics like performance or whatever, then when things get hairy it's always "Well, here's a very reasonable compromise". We know how to compromise! The world is filled with very reasonably compromised memory-safe runtimes that are excellent choices for your new system.
>It starts from a marketing position of "Safety With No Compromises" on runtime metrics like performance or whatever, then when things get hairy it's always "Well, here's a very reasonable compromise".
Well, yes. The other compromise is the one Java gives you: write a state of the art garbage collector and include it in your program.
This complaint is very annoying because it assumes a garbage collector is "free" and Rust decided to just not give you one. If you want memory safe trees your options are
* A slow, simple, reference counted garbage collector
* A fast, complex, garbage collector
both are compromises! You are just ignoring the second one.
Well, in some sense. But in practice, come on: "write your own slow simple reference-counted GC" is a rather more expensive compromise than "just use Java or Go or whatever, it's faster and simpler for your problem".
Aside from all the nitpickery about runtime implementation, the rustacean community has a serious problem with compromise in general. If you whine in a python/Go/Java/whatever forum about performance, they'll point you to their FFI and show you how to get what you want via C++ integration, because clearly no environment is going to be perfect for everyone. But come at rust with a use case (cyclic data structures here) for which it's a poor fit and everyone goes blue in the face explaining how it's not really a problem. It's exhausting.
> At the cost of making the use of the resulting heap significantly slower and larger than if you just wrote the thing in Java to begin with, though!
Lower throughput, probably. But it introduces constant latency. It has some advantages over doing it in Java:
* You're never going to get latency spikes by adding a counter to each allocation slot.
* If you really want to, you can disable them in release builds and still not give up memory-safety, although you might get logical use-after-frees.
* You don't need to use such "compromises" for literally everything, just where it's needed.
> It starts from a marketing position of "Safety With No Compromises"
I haven't seen that marketing, but if it exists, sure, it's misleading. Yes, you have to compromise. But in my opinion, the compromises that Rust lets you make are meaningfully different from the compromises in other mainstream languages. Sometimes better, sometimes worse. Probably worse for most applications than a GC language, tbh.
> * You're never going to get latency spikes by adding a counter to each allocation slot.
The suggestion wasn't just the counter though. A counter by itself does nothing. At some point you need to iterate[1] through your set to identify[2] the unreferenced[3] blocks. And that has to be done with some kind of locking vs. the unrestricted contexts elsewhere trying to do their own allocation work. And that has costs.
Bottom line is that the response was isomorphic to "That's OK, you can work around it by writing a garbage collector". And... yeah. We have that, and it's better than this nonsense.
[1] "sweep", in the vernacular
[2] "collect", in some idioms
[3] Yup, "garbage"
No, sorry, in case I wasn't clear, I was talking about manual deallocation. I wasn't talking about a garbage collector. You still allocate and free cells. Here's an example of what I am talking about:
https://docs.rs/generational-arena
If you're implementing a tracing garbage collector you obviously don't need any such counters to detect use-after-frees.
This is clearly a different compromise entirely to the one made by tracing garbage collection. I'm actually not sure how you confused the two.
> It starts from a marketing position of "Safety With No Compromises"
Since such a tradeoff is not possible (full safety for free?!) I doubt there's any such marketing.
> The world is filled with very reasonably compromised memory-safe runtimes that are excellent choices for your new system.
Bringing a whole managed runtime just to handle a single structure with cycles in your program is not reasonable.
There's no problem with using a simple GC for a tiny part of an otherwise manually managed program just how there's no issue in managing memory manually for a small, but performance sensitive part of your GC managed program.
I always say for people coming from C++ ... just imagine a std::move is there around everything (well, except for things that are Copy) ... then it will all make sense.
The problem is this mental model is entirely foreign to people who have worked in literally every other language where pass by value (copy or pass by reference are the way things work, always.
> When you move it, it's gone at the old name.
(Emphasis mine.)
I just had a thought! It might make languages like Rust more ergonomic if movement could also update the reference to the new location so that moving a local variable into a container could also update the variable to reference the target also.
The "broken" example can be trivially fixed:
But what if the language supported "move and in-place update the reference"?Something like:
Where '@' (or whatever) is a new operator that says: move the object and update the reference `id` to point to the moved value. This could only be used for parameters marked with some sort of attribute indicating that this is possible.On its own, that doesn't sound like the worst idea. But at least in Rust, one problem is that further modifying the container will immediately invalidate the new reference, as will reading from the container if you get a mutable reference. References into containers are temperamental in general, unless you can handle them in smaller scopes that are unaware of the larger container.
But why add a language feature (and a new symbol, even!) when by definition you can always fix the issue by shadowing the original variable and pointing it at the new location? At most, this calls for a change in compiler diagnostics to add a hint in cases that are trivially fixable.
There are scenarios where retrieving the just-inserted value without first cloning it isn't possible. E.g.: when inserting a value into a non-empty hashset and then needing it again immediately.
But yeah... that's a bit of a contrived example and can be solved by a simple change to the insert function without specialised support from the language.
It sounds like you'd want the entry API: https://doc.rust-lang.org/std/collections/struct.HashMap.htm...
This post pretty much completely ignores the advantages of the borrow checker. I'm not talking about memory safety, which is it's original purpose. I'm talking about the fact that code that follows Rust's tree-style ownership pattern and doesn't excessively circumvent the borrow checker is more likely to be correct.
I don't think that was ever the intent behind the borrow checker but it is definitely an outcome.
So yes, the borrow checker makes some code more awkward than it would be in GC languages, but the benefits are easily worth it and they stretch far beyond memory safety.
He's not ignoring them. The point of the article is that the author doesn't experience those things as concrete advantages for them. Like sure, there are advantages to those things, but the author says he doesn't feel it's worth the trouble in his experience for the sorts of code he's writing.
One good RCE in production could alter this perception quite a bit. "The mosquito repellent is useless, I see too few mosquitos around me anyway."
The author is a bioinformatician writing scientific software, and often switching back and forth between Rust, Julia, and Python. His concerns and priorities are not the same as people doing systems-level programming.
Maybe Rust, a systems language, is just a wrong tool for bioinformatic tasks. Go, Java, Typescript, Ocaml, Scala, Haskell easily offer a spectrum from extreme simplicity to extreme expressiveness, with good performance and library support, but without needing to care about memory allocation and deallocation. (Python, if you use it as the frontend to pandas / polars, also counts.)
I know you're not supposed to berate people here for not reading TFA, but this really feels like a case where it's very frustrating to engage with you because you really should read TFA.
Well, I have read TFA.
The author may have a point in the idea of borrowing record fields separately. It is possible if we assume that the fields are completely orthogonal and can be mutated independently without representing an incorrect state. It would be a good option to have.
But a doubly-linked list (or graph) just can't be safely represented in the existing reference semantics. Dropping a node would lead to a dangling pointer, or several. An RDBMS can handle that ("on delete set null"), because it requires a specific way to declare all such links, so that they can be updated (aka "foreign keys"). A program usually does not (Rc / Arc or shared_ptr provide a comparable capability though).
Of course a bidirectional link is a special case, because it always provides a backlink to the object that would have a dangling pointer. The problem is that the borrow checker does not know that these two pointers belonging to different structs are a pair. I wish Rust had direct support for such things, then, when one end of the bidirectional link dies, the borrow checker would unset the pointer on the reciprocal end. Linking the objects together would also be a special operation that atomically sets both pointers.
In a more general case, it would be interesting to have a way to declare some invariants over fields of a struct. A mutual pair of pointers would be one case, allowing / forbidding to borrow two fields at once would be another. But we're far from that.
Special-casing some kinds of pointers is not unheard of; almost every GC-based language offers weak references (and Java, also Soft and Phantom references). I don't see why Rust could not get a BidirectionalRef of sorts.
Until then, Arc or array with indexes seem to be the only guaranteed memory-safe approaches.
Also, in the whole article I could not find a single reason why the author chose Rust, but I suppose it's because of its memory efficiency, considering the idea of keeping large graphs in RAM. Strictly speaking, Go could be about as efficient, but it has other inflammation points. C++... well, I hope Rust is not painful enough to resort to that.
Rust is widely used in bioinformatics, as is C++. Largely because people developing new tools often have to write their own low-level algorithms and data structures.
Many tools deal with sequencing data, which means collections of strings that cannot be parsed or tokenized in any meaningful way. The collections are often very large, and the strings can also be very long. The standard algorithmic toolkit you learn when doing a CS degree (or a PhD in algorithms) is inadequate with such data. Hence, if you go to a CS conference focused on combinatorial algorithms and data structures, the presented work is often motivated by applications in bioinformatics.
There's the practical end goal benefit of safer and more robust programs, but I think there's also the piece that pg talks about in Beating The Averages which is that learning how to cooperate with these conventions and think like there's the borrow checker there makes you a better programmer even when you return to languages that don't have it.
> makes you a better programmer
If a language is bad, but you must use it, then yes learn it. But, if the borrowchecker is a source of pain in Rust, why not andmit it needs work instead of saying that “it makes you better”?
I’m not going to start writing brainfuck because it makes me a better programmer.
We do admit it needs work. The issues the author highlights can be annoying, a smarter borrow checker could maybe solve them.
The point is the borrow checker has already gone beyond the point where the benefits outweigh those annoyances.
It's like... Static typing. Obviously there are cases where you're like "I know the types are correct! Get out of my way compiler!" but static types are still vastly superior because of all the benefits they convey in spite of those occasional times when they get in the way.
> The issues the author highlights can be annoying, a smarter borrow checker could maybe solve them
I don't think a smarter borrow checker could solve most of the issues the author raises. The author wants borrow checking to be an interprocedural analysis, but it isn't one by design. Everything the borrow checker knows about a function is in its signature.
Making partial borrows to be expressable in method definitions would allow the design pattern to be expressed without breaking the current lifetime evaluation boundary.
Allowing the borrow checker to peek inside of the body of local methods for the purposes of identifying partial borrows would fundamentally break the locality of the borrow checker, but I think that as long as that analysis is only extended to methods on the local trait impl, it could be done without too much fanfare. These two things would be relaxations of the borrow checker rules, making it smarter, if you will.
Fixing the get_default example wouldn't require interprocedural analysis. (It requires Polonius, which, yeah, has taken a long time to ship.)
The get_default example requires only one to use the stdlib API: https://doc.rust-lang.org/std/collections/hash_map/enum.Entr...
I believe that codebases written in Rust, with borrow checking in mind, are often very readable and allow local reasoning better than most other languages. The potential hardness might not stem from "making people better programmers" but from "making programmers write better code, perhaps at the cost of some level of convenience".
> code that follows Rust's tree-style ownership pattern
This is a pretty important qualification. Most low-level systems code doesn't and can't have this ownership structure. It provides a reason why Rust has more traction replacing code that could have been written in Java rather than e.g. C++ in the domains where C++ excels (like database engines).
> is more likely to be correct.
This is a moot statement. Here is a thought experiment that demonstrates the pointlessness of languages like Rust in terms of correctness.
Lets say your goal is ultimate correctness - i.e for any possible input/inital state, the program produces a known and deterministic output.
You can chose 1 of 2 languages to write your program in:
First is standard C
Second is an absolutely strict programming language, that incorporates not only memory membership Rust style, but every single object must have a well defined type that determines not only the set of values that the object can have, but the operations on that object, which produce other well defined types. Basically, the idea is that if your program compiles, its by definition correct.
The issue is, the time it takes to develop the program to be absolutely correct is about the same. In the first case with C, you would write your program with carefully designed memory allocation (something like mempool that allocates at the start), you would design unit tests, you would run valgrind, and so on.
In the second case, you would spend a lot more time carefully designing types and operations, leading to a lot of churn of code-compile-fix error-repeat, taking you way longer to develop the program.
You could argue that the programmer is somewhat incompetent (for example, forgets to run valgrind), so the second language will have a higher change of being absolutely correct. However the argument still holds - in the second language, a slightly incompetent programmer can be lazy and define wide ranging types (similar to `any` in languages like typescript), leading to technical correctness, but logic bugs.
So in the end, it really doesn't matter which language you chose if you want ultimate correctness, because its all up to the programmer. However, if your goal is rapid prototyping, and you can guarantee that your input is constrained to a certain range, and even though out of range program will lead to a memory bug or failure of some sort, programming in something like C is going to be more efficient, whereas the second language will force you write a lot more code for basic things.
This claim makes some sense to me if your development life cycle is: write and compile once, never touch again.
Working at a company with lots of systems written by former employees running in production… the advantages of Rust become starkly obvious. If it’s C++, I walk on eggshells. I have to become a Jedi master of the codebase before I can make any meaningful change, lest I become responsible for some disaster. If it’s Rust, I can just do stuff and I’ve never broken anything. Unit tests of business logic are all the QA I need. Other than that, if it compiles it works.
>If it’s Rust, I can just do stuff and I’ve never broken anything.
This is not true. Case and point- Java. Many times simpler than Rust, and large codebases are as horrible as C++ ones.
I have two primary complaints about Java, relative to Rust:
1. NullPointerException. I get some object with a bunch of fields and I don’t know which of them are null. In Rust I am given a struct, and usually the fields aren’t an Option unless they need to be.
2. Complicated design patterns with inheritance. Maybe it’s more of a problem with the culture/ecosystem than Java the language. But Rust doesn’t have inheritance, and traits are less complicated. So you rarely get the same smells.
Compared to C++, Java is easier to debug, but I still have a lot of “wtf” moments trying to understand what the author was thinking. It is almost like the language is so simple to write, people are making the program more complicated on purpose just to make it interesting.
The point is that programmers are ultimately responsible for clean code. You can write very good C code that is very easy to debug. Languages don't really matter for this.
This isn't true in practice. People don't write impossibly comprehensive test suites in C, and they don't use extremely loose types in Rust either.
It really does matter which language you choose if you want correct code.
> programming in something like C is going to be more efficient, whereas the second language will force you write a lot more code for basic things.
Like how string manipulation is so much simpler and easier in C compared to Rust? Hmm.
Also practice, most of Rust codebases are filled with `unsafe`, for interface with system libraries.
Also the argument of forcing a language onto a project based on the lowest common denominator of programmers never plays out - this is how you get insanely messy Java codebases. Language choice will never solve poor programming style.
>Like how string manipulation is so much simpler and easier in C compared to Rust? Hmm.
Plenty of libraries for C for this.
Your example is of a greenfield project rather than living code that is constantly updated, in some cases by many people simultaneously. The compiler being a gate for correctness is far superior in the latter case.
If you enforce very strict interfaces through language, no matter how many people work on them, the tradeoff still applies.
For example, you can have multiple people working on a code base for the second case, and some sub team has a new requirement for added functionality. Now they have to go refactor a whole bunch of the codebase to make all the types coherent. And consequently, shortcuts happen here, which leads to shit codebases.
The author's motivation for writing this is well-founded. However, the author doesn't take into account the full spirit of rust and the un-constructive conclusion doesn't really help anyone.
A huge part of the spirit of rust is fearless concurrency. The simple seeming false positive examples become non-trivial in concurrent code.
The author admits they don't write large concurrent - which clearly explains why they don't find much use in the borrow checker. So the problem isn't that the rust doesn't work for them - it's that a central language feature of rust hampers them instead of helping them.
The conclusion for this article should have been: if you're like me and don't write concurrent programs, enums and matches are great. The language would be work better for me if the arc/box syntax spam went away.
As a side note, if your code is a house of cards, it's probably because you prematurely optimized. A good way to get around this problem is to arc/box spam upfront with as little abstraction as possible, then profile, then optimize.
Yeah, the author's theoretical "Rust but with garbage collector" would gain a whole bunch of concurrency bugs. It wouldn't be Rust anymore, just c# with a more functional syntax.
"Fearless concurrency" is one of the best things the borrow checker gives us, and I think a lot of people undervalue it.
I think there's a lot love for the borrowchecker because a lot of people in the Rust community are working on ecosystems (eg https://github.com/linebender) which means they are building up an api over many years. In that case having a very restrictive language is really great, because it kinda defines the shape the api can have at the language level, meaning that familiarity with Rust also means quick familiarity with your api. In that sense it doesn't matter if the restrictions are "arbitrary" or useful.
The other end of the spectrum is something like gamedev: you write code that pretty explicitly has an end-date, and the actual shape of the program can change drastically during development (because it's a creative thing) so you very much don't want to slowly build up rigidity over time.
To the author, I would be a borrow checker apologist or perhaps extremist. I will take that mantle gladly: I am very much of the opinion that a systems programming language without a borrow checker[^1] will not find itself holding C++-like hegemony anymore (once/if C++ releases the scepter, that is). I guess I would even be sad if C++ kept the scepter for the rest of my life, or was replaced by another language that didn't have something like a borrow checker.
It doesn't need to be Rust: Rust's borrow checker has (mostly reasonable) limitations that eg. make some interprocedural things impossible while being possible within a single function (eg. &mut Vec<u32> and &mut u32 derived from it, both being used at the same time as shared references, and then one or the other being used as exclusive later). Maybe some other language will come in with a more powerful and omniscient borrow checker[^1], and leave Rust in the dust. It definitely can happen, and if it does then I suppose we'll enjoy that language then.
But: it is my opinion that a borrow checker is an absolutely massive thing in a (non-GC) programming language, and one that cannot be ignored in the future. (Though, Zig is proving me wrong here and it's doing a lot of really cool things. What memory safety vulnerabilities in the Ziglang world end up looking like remains to be seen.) Memory is always owned by some_one_, its validity is always determined by some_one_, and having that validity enforced by the language is absolutely priceless.
Wanting GC for some things is of course totally valid; just reach for a GC library for those cases, or if you think it's the right tool for the job then use a GC language.
[^1]: Or something even better that can replace the borrow checker; maybe Graydon Hoare's original idea of path based aliasing analysis would've been that? Who knows.
> just reach for a GC library for those cases
Imo a GC needs some cooperation from the language implementation, at least to find the rootset. Workarounds are either inefficient or unergonomic. I guess inefficient GC is fine in plenty of scenarios, though.
> In that sense, Rust enables escapism: When writing Rust, you get to solve lots of 'problems' - not real problems, mind you, but fun problems.
If this is true for Rust, it's 10x more true for C++!
Lifetime issues are puzzles, yes, but boring and irritating ones.
But in C++? Select an appetizer, entree, and desert (w/ bottomless breadsticks) from the Menu of Meta Programming. An endless festival of computer science sideshows living _in the language itself_ that juices the dopamine reward of figuring out a clever way of doing something.
You're right about C++. A fairer comparison would be to a simpler garbage-collected language like Go.
Came here to comment on the same thing. I've never been able to articulate this as well as the author did, and it is so true! Every programming language requires you to solve some puzzles that are just in the way of the real problems you are trying to solve, but some much more than others.
People have compared Rust to C++ and others have argued that they really aren't alike, but I think it's in these puzzles that they are more alike than any other two languages. Even just reading rust code is a brain teaser for me!
I think this is why C and Zig get compared too. They apparently have roughly the same level of "fun problems" to solve.
IMO, it is reasonable that in the example given:
the returned references are, for the purposes of aliasing rules, references to the entire struct rather than to pieces of it. `x` and `y` are implementation details of the struct and not part of its public API. Yes, this is occasionally annoying but I think the inverse (the borrow checker looking into the implementations of functions, rather than their signature, and reasoning about private API details) would be more confusing.I also disagree with the author that his rejected code:
"doesn't even violate the spirit of Rust's ownership rules."I think the spirit of Rust's ownership rules is quite clear that when calling a function whose signature is
`param` is "locked" (i.e., no other references to it may exist) for the lifetime of the return value. This is clear once you start to think of Rust borrow-checking as compile-time reader-writer locks.This is often necessary for correctness (because there are many scenarios where you need to be guaranteed exclusive access to an object beyond just wanting to satisfy the LLVM "noalias" rules) and is not just an implementation detail: the language would be fundamentally different if instead the borrow checker tried to loosen this requirement as much as it could while still respecting the aliasing rules at a per-field level.
It would not just be "confusing". It would be fundamentally unacceptable because there would just be no local reasoning anymore, and a single private field change might trigger a whole cascade of nonlocal borrowing errors.
Unfortunately, this behavior does sometimes occur with Send bounds in deeply nested async code, which is why I mostly restrain from using colored-function style asynchronous code at all in favor of explicit threadpool management which the borrow checker excels at compared to every other language I used.
I found the arguments in this article disingenuous. First, the author complains that borrowchecker examples are toys, then proceeds to support their case with rather contrived examples themselves. For instance, the map example is not using the entry api. They’d be better served by offering up some real world examples.
The author explained why he used contrived examples. It's because the pain arises most acutely only after your project has become large and mature but demands a small ownership-impacting change. The toy examples demonstrate the problem in the small, but they generalize to larger and more complex scenarios.
He's basically talking about the rigidity that Rust's borrow checking imposes on a program's data design. Once you've got the program following all the rules, it can be extraordinarily difficult to make even a minor change without incurring a time-consuming and painful refactor.
This is an argument about the language's ergonomics, so it seems like a fair criticism.
Regarding Indexes: "When the same borrowchecker makes references unworkable, their solution is to... recommend that I manually manage them, with zero safety and zero language support?!?"
Language support: You can implement extension traits on an integer so you can do things like current_node.next(v) (like if you have an integer named 'current_node' which is an index into a vector v of nodes) and customize how your next() works.
Also, I disagree there is 'zero safety', since the indexes are into a Rust vector, they are bounds checked by default when "dereferencing" the index into the vector (v[i]), and the checking is not that slow for vast majority of use cases. If you go out of bounds, Rust will panic and tell you exactly where it panicked. If panicking is a problem you could theoretically have custom deference code that does something more graceful than panic.
But with using indexes there is no corruption of memory outside of the vector where you are keeping your data, in other words there isn't a buffer overflow attack that allows for machine instructions to be overwritten with data, which is where a huge amount of vulnerabilities and hacks have come from over the past few decades. That's what is meant by 'safety' in general.
I know people stick in 'unsafe' to gain a few percent speed sometimes, but then it's unsafe rust by definition. I agree that unsafe rust is unsafe.
Also you can do silly optimization tricks like if you need to perform a single operation on the entire collection of nodes, you can parallelize it easily by iterating thru the vector without having to iterate through the data structure using next/prev leaf/branch whatever.
> If you go out of bounds, Rust will panic and tell you exactly where it panicked
This arguement has a long history.
It is a widely used pattern in rust.
It is true that panics are memory safe, and there is nothing unsafe about having your own ref ids.
However, I believe thats its both fair and widely acknowledged that in general this approach is prone to bugs that cause panics for exactly this reason, and thats bad.
Just use Arc or Rc.
Or, an existing crate that implements a wrapper around it.
Its enormously unlikely that most applications need the performance of avoiding them, and very likely that if you are rolling your own, youll get caught up by edge cases.
This is a prime example of a rust antipattern.
You shouldnt be implementing it in your application code.
Indices are the way to go for a whole range of programs - compilers (IR instructions), GUI (widgets), web browser (UI elements), databases, games (ECS), simulations, etc. It is however without borrowcheck guarantees.
Rc and Arc are definitely a bad way to avoid borrowchecker. GC is used because it is much faster than reference counting. OCaml and Go are experimenting with smarter local variable handling without GC. At that point they may outperform Arc and Rc heavy Rust code.
I wish Rust had syntax or a directive to make Rcs a bit less obtrusive.
This is being worked on: https://rust-lang.github.io/rust-project-goals/2025h1/ergono...
Interesting. My projects would benefit from those more convenient clones into closure captures. The other point where Rc is uglier than, say, Swift, is the explicit `.borrow()` and `.borrow_mut()` everywhere. I wonder if that could also be made more convenient/high-level without sacrificing the control over high performance (like C++) that got me to use Rust in the first place.
I've recently wondered if it's possible to extract a subset of Rust without references and borrow checking by using macros (and a custom stdlib).
In principle, the language already has raw pointers with the same expressive power as in C, and unlike references they don't have aliasing restrictions. That is, so long as you only use pointers to access data, this should be fine (in the sense of, it's as safe as doing the same thing in C or Zig).
Note that this last point is not the same as "so long as you don't use references" though! The problem is that aliasing rules apply to variables themselves - e.g. in safe rust taking a mutable reference to, say, local variable and then writing directly to that variable is forbidden, so doing the same with raw pointers is UB. So if you want to be on the safe side, you must never work with variables directly - you must always take a pointer first and then do all reads and writes through it, which guarantees that it can be aliased.
However, this seems something that could be done in an easy mechanical transform. Basically a macro that would treat all & as &raw, and any `let mut x = ...` as something like `let mut x_storage = ...; let x = &raw mut x_storage` and then patch up all references to `x` in scope to `*x`.
The other problem is that stdlib assumes references, but in principle it should be possible to mechanically translate the whole thing as well...
And if you make it into a macro instead of patching the compiler directly, you can still use all the tooling, Cargo, LSP(?) etc.
I've similarly thought about building a language that compiles to Rust, but handles everything around references and borrowing and abstracts that away from the user. Then you get a language where you don't have to think about memory at all, but the resulting code "should" still be fairly fast because Rust is fast (kind of ending up in the same place as Go).
I haven't written a ton of Rust so maybe my assumptions of what's possible are wrong, but it is an idea I've come back to a few times.
Why compile to Rust for this? Many people that build transpilation languages target C directly.
Think of Rust as a kind of kernel guaranteeing correctness of your program, the rules of which your transpiler should not have to reimplement. This may be compared to how proof assistants are able to implement all sorts of complicated simplification and resolution techniques while not endangering correctness of the generated proofs at all due to them having a small kernel that implements all of verification, and as long as that kernel is satisfied with your chain of reasoning, the processes behind its generation can be entirely disregarded.
A C compiler won't complain if your generated code does certain horrible things.
Why, macros that put Arc<Box<T>> everywhere might just be it.
Arc<Box<T>> is redundant, for the contents of the Arc are already stored on the heap. You may be thinking of Arc<Mutex<T>> for multithreaded access or Rc<RefCell<T>> for singlethreaded access. Both enable the same "feature" of moving the compile-time borrow checking to runtime (Mutex/RefCell) and using reference-counting instead of direct ownership (Arc/Rc).
Very tangential, but I couldn't help but remember Crust [1]. This tsoding madlad even wrote a B compiler [2] using these... rules. Or lack thereof?
[1]: https://github.com/tsoding/Crust
[2]: https://github.com/tsoding/b
Crust is exactly what I had in mind, but enforced on language level basically.
You can freely alias &Cell<T>, and this would give you memory safety compared to raw pointers. AIUI, &Cell<T> is effectively the moral equivalent (but safe) to T* in C/C++.
It works if you only ever need to reference the outer value, but it doesn't allow you to e.g. get a working mutable reference to a field inside a struct - even if you declare the field as another Cell<T>, when you read the value from the outer cell, you get a copy, not a reference to original. And the same thing goes for arrays, which is especially inconvenient. So no, not quite equivalent.
If you don't want memory safety, it seems like it'd be easier to use C++ with a static analyzer to disallow the parts you don't like. I suppose the lack of a good package manager would still be a problem.
The whole point of TFA (with which I fully agree) is that Rust is just generally a much nicer language than C++, once borrow checker is out of the picture. Discriminated unions (enums) with pattern matching alone are worth their weight in gold.
I personally don't think I would want to use a language that doesnt have a borrow checker ever again. I would like a Rust-like language with GC, but I still want the borrow checker.
Every time I write Go, I find it so annoying all the defensive deep copying I see. In JS I always find myself getting confused on whether a mutation is safe (as in my program won't break some assumption) to do or not. Marking arguments as shared or exclusive is really great for me to know what kind of access I can have. It needs to be enforced so that the owner also doesn't accidentally mutate while a shared borrow is still active (again, not just for memory safety but for my own invariants). The classic example could be inserting into a collection while iterating over it
I think the borrow checker is necessary if you have ADTs like Rust and you want memory safety. You could pattern match a union into one of its variants and get a pointer to one of the fields, but without a borrow checker there's nothing to stop you from changing the variant stored in the union. This would obviously cause issues and the only way you'd solve this with GC alone is by allocating each variant individually where the union is just a tagged pointer.
> Use fewer references and copy data. Or: "Just clone".
> This is generally good advice. Usually, extra allocations are fine, and the resulting performance degradation is not an issue. But it is a little strange that it allocations are encouraged in an otherwise performance-forcused language, not because the program logic demands it, but because the borrowchecker does.
I often end up writing code that (seems) to do a million tiny clones. I've always been a little worried about fragmentation and such but it's never been that much of an issue -- I'm sure one day it will be. I've often wanted a dynamically scoped allocator for that reason.
If you notice a rule for the first time, restricting what you want to do and making you jump through a hoop, it can be hard to see what the rule is actually for. The thrust of the piece is 'there should not be so many rules, let me do whatever I want to do if the code would make sense to me'. This does not survive contact with (say) an Enterprise Java codebase filled with a billion cases of defensive strict immutability and defensive copies, because the language lacks Rust's rules and constructs about shared mutability and without them you have to use constant discipline to prevent bugs. `derive(Copy)` as One More Pointless Thing You Have To Do only makes sense if you haven't spent much time code-reviewing someone's C++.
If you try to write Java in Rust, you will fail. Rust is no different in this regard from Haskell, but method syntax feels so friendly that it doesn't register that this is a language you genuinely have to learn instead of picking up the basics in a couple hours and immediately start implementing `increment_counter`-style interfaces.
And this is an inexperienced take, no matter how eloquently it's written. You can see it immediately from the complaint about CS101 pointer-chasing graph structures, and apoplexy at the thought of index-based structures, when any serious graph should be written with an index-based adjacency list and writing your own nonintrusive collection types is pretty rare in normal code. Just Use Petgraph.
A beginner is told to 'Just' use borrow-splitting functions, and this feels like a hoop to jump through. This is because it's not the real answer. The real answer is that once you have properly learned Rust, once your reflexes are procedural instead of object-oriented, you stop running into the problem altogether; you automatically architect code so it doesn't come up (as often). The article mentions this point and says 'nuh uh', but everyone saying it is personally at this level; 'intermittent' Rust usage is not really a good Learning Environment.
The borrow checker is also what I like the least about Rust, but only because I like pattern matching, zero-cost abstractions, the type system, fearless concurrency, algebraic data types, and Cargo even more.
Fearless concurrency is only possible because of the borrow checker.
This isn't really right; Pony does static concurrency safety without borrow checking, and while Swift sort of has borrow checking it's mostly orthogonal to how static concurrency safety works there. The relationship between borrow checking and concurrency safety in Rust is closer to "they rhyme" than "they use the exact same mechanism".
You've provided reasons why other languages have other properties, but Rust's fearless concurrency requires borrow checking.
The concurrency focused trait Send only makes sense because we have control over references, through the borrow checker. Thread #1 can give Thread #2 this Doodad because it no longer has any references to it, if you try to give it a Doodad you're still referring to, then the borrow checker will reject your code. The Send trait guarantees that this (giving the Doodad to a different thread) is an OK thing to do - but only if you don't have outstanding borrows so you won't be able to look at the Doodad once you give it to Thread #2
Thanks for all your clarifying comments. I appreciate them and have learned lots. If you have a blog, I'd love to read whatever you write.
I choose a language that is as ergonomic as possible, but as performant as necessary. If e.g. Kotlin is fine, there is no way I will choose Rust.
Many projects are written in Rust that would absolutely be fine in Go, Swift or a JVM language. And I don't understand: it is nicer to write in those other languages, why choose Rust?
On the other hand, Rust is a lot nicer than C/C++, so I see it as a valid alternative there: I'm a lot happier having to make the borrow-checker happy than tracking tricky memory errors in C.
Personally I like my programs to be as performant as is reasonable, not just as is necessary. Rust's low-cost abstractions strike the right balance for me where I feel like I'm getting pretty solid ergonomics (most of the time) while also enjoying near-peak performance as the default.
> It is nicer to write in those other languages, why choose Rust?
Honestly I don't think it is nicer to write in those other languages you mention. I might still prefer Rust if performance was removed from the equation entirely. That is just to say I think preference and experience matters just as much, if not more, than the language's memory model.
For a sufficiently large program, i am faster at writing a correct rust implementation than I am with Go. I find myself a lot more able to reason about the program thanks to the work the compiler makes me do upfront.
> it is nicer to write in those other languages
I think this is a matter of preference. Nowadays I cannot stand environments like Java (or especially Kotlin). "Tricky memory errors" is in my opinion nicer than a borrow-checker refusing sound code. I guess I really hate 'magic'...
I mostly agree with you, but ironically the article's own conclusion answers this: the language's general ethos is designed for correctness-oriented programming in the large, and this gives rise to lots of other features that garbage-collected languages could adopt but mostly don't. Like, it's way harder to avoid runtime panics in Go. (Swift does okay but people don't have faith that its custodians will do an adequate job of prioritizing the needs of developers using it for anything other than client-side apps on Apple platforms, and also it's slightly too low-level in that it doesn't have a tracing garbage collector either.)
I find Rust quite ergonomic in most cases. Yes there is more code, but in terms of thinking about it, it's usually "I have this and I want that", and the middle almost fills itself out.
What's the alternative though? If you're fine with garbage collection, just use garbage collection. If you're _not_ fine with garbage collection (because you want deterministic performance, or you have resources that aren't just memory) then Rust's borrow checker seems like the best thing going.
You can use Zig, a faster, safer C with best-in-class metaprogramming for a systems-level language. It doesn't guarantee safety to the same extent as Rust but gets you 80% of the benefit with 20% of the pain.
I think under the constrained of not using GC and not defaulting to unsafe memory that the borrow checker is a decent design. The constraints mean that you need some form of formal verification of your lifetimes. These can get very complex and difficult to use. So it might make sense to limit their expressiveness and instead rely on unsafe escape hatches. I think Rust's borrow checker provides a decent tradeoff between easy of use and how often unsafe is needed. There are rough edges and progress has been slow, but I have yet to see anything radically better. Other system PLs require a lot more unsafe usage (e.g. Zig) and I'm not aware of any mainstream system PLs that offer much more expressive verification.
I'm far from a Rust pro, but I think the dismissal of alternatives like Polonius seems too shallow. Yes, it is still in the works, but there's nothing fundamentally wrong about the idea of a borrow checker.
This is true both in theory and in practice, as you can write any program with a borrow checker as you can without it.
TFA also dismisses all the advantages of the borrow checker and focuses on a narrow set of pain points of which every Rust developer is already aware. We still prefer those borrowing pain points over what we believe to be the much greater pain inflicted by other languages.
Polonius will not fix the "issues" the author is complaining about, because contrary to his assertion, they are actual fundamental properties of how the Rust ownership/borrowing model is supposed to work, not shortcomings of an insufficiently smart implementation.
That's not true of the get_default example, which is fixable without fundamental type-system changes but requires Polonius.
I used Rust for about an hour and immediately ran into an issue I found amusing.
The compiler changed the type of my variable based on its usage. Usage in code I didn't write. There was no warning about this (even with clippy). The program crashed at runtime.
I found this amusing because it doesn't happen in dynamic languages, and it doesn't happen in languages where you have to specify the types. But Rust, with its emphasis on safety, somehow lured me into this trap within the first 15 minutes of programming.
I found it more amusing because in my other attempts at Rust, the compiler rejected my code constantly (which was valid and worked fine), but then also silently modified my program without warning to crash at runtime.
I saw an article by the developers of the Flow language, which suffered from a similar issue until it was fixed. They called it Spooky Action at a Distance.
This being said, I like Rust and its goals overall. I just wish it was a little more explicit with the types, and a little more configurable on the compiler strictness side. Many of its errors are actually just warnings, depending on your program. It feels disrespectful for a compiler to insist it knows better than the programmer, and to refuse to even compile the program.
Did you file a bug?
[dead]
The author does point out that these problems come up when implementing various data structures.
It might be surprising to some folks, but there is a lot of unsafe code in Rust, and a lot of that is in the standard’s data structure implementations.
Also —
Common in network programming, the pain of lifetimes, is in async.
The model sort of keels over and becomes obtuse when every task requires ownership of its data with static lifetimes.
I think the borrow checker doesn't get enough credit for supporting one of rusts other biggest selling points - it's ecosystem. In C and C++ libraries often go through pains to pass as little allocated memory over the API barrier as possible, communicating about lifetime constraints and ownership is flimsy and frequently causes crashes. In Rust if a function returns Vec<Foo> then every Foo is valid until dropped, and if it's not someone did something unsafe.
> Data races in multithreaded code are elegantly and statically prevented by Rust. I have never written a large concurrent Rust program, and I'll grant the possibility that the borrowchecker is miraculous for that use case and easily pays for its own clunkiness.
Data races prevented not just in concurrent programs. I personally was hit once by a panic from RefCell. I have coded a data race, but I wouldn't notice it without RefCell. I borrowed the value and then called some function, and a several stack frames deeper I tried to borrow it mutably.
GC also has its downsides:
- Marking and sweeping cause latency spikes which may be unacceptable if your program must have millisecond responsiveness.
- GC happens intermittently, which means garbage accumulates until each collection, and so your program is overall less memory efficient.
With modern concurrent collectors like Java's ZGC, that's not the case any longer. They show sub-millisecond pause times and run concurrently. The trade-off is a higher CPU utilization and thus reduced overall throughput, which if and when it is a problem can oftentimes be mitigated by scaling out to more compute nodes.
> They might argue that the snippets don't show there is any real ergonomic problem, because the solutions to make the snippets compile are completely trivial. In the last example, I could just derive Clone + Copy for Id.
I’ll bite a little bit. If I write:
Then I need to make a choice: is Thing a plain value that its holders can freely clone or is Thing an affine object that is gone when consumed? I think it’s great that Rust makes this explicit, even if it’s a tiny bit odd that one option is a default and the other option is rather verbose. If I could have a unicorn, too, I’d also like to be able to request “linear” behavior, i.e. disallow code that fails to consume the object.Sure, I suppose someone could invent a language where an LLM reads the type definitions and fills in the blanks as to what uses are valid, but this seems like a terrible idea. Or one could use a language without affine types (most languages, and even C++’s move feature entirely fails to enforce any sort of good behavior).
I'd argue the very reason Rust was created to get rid of the aliasing problem that's plaguing C-family languages, (meaning that there's always a chance 2 pointers refer to the same piece of memory, meaning all writes might potentially invalidate all variables).
This isn't really solvable in C/C++, and its worked around with a bunch of hacks, which might be overly convervative at times, and at others, generates buggy code.
Rust managed to fix this issue elegantly, but I'm wondering if the solution might be worse than the problem.
It's essentially impossible to write a Rust program without relying on many of its escape hatches like RefCell and unsafe, that make the borrow checker go away.
Essentially any program, that needs to communicate or store data in some sort of persistent structure, which is then accessed in some other part of the program, has multiple mutable references to said abstract central point, which is not allowed in Rust, without the aforementioned workarounds.
> It's essentially impossible to write a Rust program without relying on many of its escape hatches like RefCell and unsafe, that make the borrow checker go away.
I get this may be hyperbole but it's also just factually incorrect. Not only that, why is this even a goal? RefCell isn't a wart. It is part of Rust and meant to be used where it makes sense.
Not really hyperbole - I can't iamgine how one would build a fully borrow-checked app, that doesn't use these escape hatches (either directly or through a library). The only thing you could do maybe is to pass down everything that could possible be mutable as parameters everywhere, but that would not exactly be a nice way to program.
And what's wrong with RefCell (and friends)? It's extra complexity, memory footprint, and runtime cost. Also you gave up on the ability of the compiler to check you program's correctness. Granted it's not much, but neither is std::shared_ptr or Swift's automatic reference counting. Once you go from zero overhead to some overhead, there's a ton of quality of life features you can offer to the programmer.
If Rust's borrow checker was smart enough to allow multiple mutable borrows of the same variable, provided some conditions are met (like if you're borrowing struct S, you could borrow it's fields), it would eliminate the need for much of RefCell hacks, as well as solve the gripes in the article.
Try out iced https://iced.rs next weekend
No need for RefCell anywhere
I'm active on the Discord if you need help getting started
The author's first example seems to undermine the thesis. Their Point class allows two separate locations to independently update the X and Y fields, leaving the object in an inconsistent state.
It seems to me that this is exactly the sort of thing that Rust is intended to prevent, and it makes complete sense to reject the code.
Rust is pain when you want to be super basic with your types.
This is actually a learning lesson for the user to understand that the bugs one has seen in languages like c++ are inherent to using simple types.
The author goes about mentioning python. If you do change all your types to python equivalents, ref counted etc. Rust becomes as easy. But you don’t want to do that and so it becomes pain, but pain with a gain. You must decide if that gain is worth it.
From my point of view the issue is that rust defaults to be a system programming language. Meaning, simple types are written simple (i32, b32, mut ..), complex types are written complex (ref, arc, etc.). And because of that one wants to use the simple types, which makes the solutions complex.
Let’s imagine a rust dialect, where every type without annotation is ref counted and if you want the simple type you would have to annotate your types, the situation would change.
What one must realize is that verifiable correctness is hard , the simplicity of the given problematic examples is a clear indication of how close those screw ups are even with very simple code. And exactly why we are still seeing issues in core c libs after decades of fixing them.
[dead]
If a friend told me they liked Rust but didn't like the borrow checker, I'd probably point them to Gleam and Moonbit, which both seem awesome in their own niches.
Both have rust-like flavor and neither has a borrow checker.
I can't really get over Gleam's position that nobody really needs type-based polymorphism, real programmers write their own vtables by hand.
(It also needs some kind of reflection-like thing, either compile-time or runtime, so that there can be an equivalent of Rust's Serde, but at least they admit that that needs doing.)
Someone should create a DAG of programming languages with edges denoting contextual influence and changes in design and philosophy, such that every time a PL is critized for a feature (or lack thereof), the relevant alternatives exactly considering this would be readily available. It could even have a great interactive visualization.
> My examples code above may not be persuasive to experienced Rustaceans. They might argue that the snippets don't show there is any real ergonomic problem, because the solutions to make the snippets compile are completely trivial. In the last example, I could just derive Clone + Copy for Id.
No, you could use destructuring. This doesn't work for all cases but it does for your examples without needing to derive copy or clone. Here's a more complex but also compelling example of the problem:
We're doing everything in the "Rust" way here. We're using IDs instead of pointers. We're cloning a vec even if it's a bit excessive. But the bigger problem is we actually _do_ need to have multiple mutable references to two values owned by a collection that we know don't transitively reference the collection. We need to wrap these in an RefCell or UnsafeCell and unsafe { } block to actually get mutable references to the underlying data to correctly implement visit_mut().This is a problem that shows up all the time when using collections, which Rust encourages within the ecosystem.
Your example has an undefined 'source' variable - you likely meant 'curr' - but more importantly, this pattern can be solved with indices + interior mutability or split_at_mut() rather than unsafe, giving you safe simultaneous mutable access to different nodes.
That requires you use a Vec or similar data structure which may not be appropriate/forces you to reinvent garbage collection or deal with tombstoning/generations/etc for unused slots.
And I pointed out you can use interior mutability. Still sucks because the code is guaranteed sound, the compiler just can't prove it. IMO the correct choice is UnsafeCell and unsafe {}.
What stops you from writing such data container with interior mutability, like vec_cell[0], which then enforce borrowing rules at runtime?
[0]: https://github.com/alexanderved/vec_cell
The point about inter-procedural borrow checking is fair -- and in fact not a matter of "sufficiently smart" borrow checker, but a rather fundamental type system limitation. But the flaw here is pretty manageable -- sometimes you can get away with calling .split_at_mut(...), sometimes you can borrow the field directly instead of going through a wrapper method, sometimes you have to be mindful to provide a split_at_mut equivalent yourself. Last I checked there was some work being done on "partial borrows", which could solve this.
Most of the other criticisms are pretty disappointing, though.
> The following example is a famous illustration of how it can't properly reason across branches, either:
Except that function body would have been better rewritten as map.entry(key).or_default() -- which passes the borrow checker just fine and is more performant as it avoids multiple lookups. I suspect many other examples would benefit from being re-written into higher-level primitives that can be easier to borrow-check in this manner.
> But what's the point of the rules in this case, though? Here, the ownership rules does not prevent use after free, or double free, or data races, or any other bug. It's perfectly clear to a human that this code is fine and doesn't have any actual ownership issues.
What if Id represents a file descriptor that ought to be closed when the value ceases to exist? Or some other type of handle? This is an ownership problem: these are not limited to memory safety issues. For very good API stability reasons, without an explicit #[derive(Clone,Copy)] the compiler is not free to assume the type represents pure information that can be copied at will, but can only treat it as one potentially containing owned resources, that just happens not to include any at this time.
> References to temporary values, e.g. values created in a closure, are forbidden even though it's obvious to a human that the solution is simply to extend the lifetime of the value to its use outside the closure.
Which is to say, to what? The type signature does not say whether a closure will be called immediately or in another thread after two hours. And in which stack frame should the value be stored, the soon-to-be-exiting closure's?
Allowing this code:
would break ‘zero cost abstractions’.I don't agree with the examples in the post. To me, they all seem to support the case that the compiler is doing the right thing and flagging potential issues. In a larger and more complex program (or a library to be used by others), it's a lot harder to reason about such things. Frankly, why should I be keeping all that in my mind when the compiler can do it for me and warn when I'm about to do something that can't verified as safe.
Of course, designing for safety is quite complex and easy to get wrong. For example, Swift's "structured concurrency" is an attempt to provide additional abstractions to try to hide some complexity around life times and synchronization... but (personally) I think the results are even more confusing and volatile.
The syntax is what I don't like. You can create some intensely cursed 3 line segments.
The closest language to "rust without borrowchecker" is probably MoonBit [0] - weirdly niche, practical, beautifully designed language.
When I was going through its docs I was impressed with all those good ideas one after the other. Docs itself are really good (high information density that reads itself).
[0] https://www.moonbitlang.com
Rust is pretty good target for Claude Code and the like. I don't write much Rust but I have to say of the langs I used Claude Code with Rust experience was among the best. If default async runtime was not work stealing I prob would use Rust way more.
Someone once told me "easy things are hard in rust, and hard things are easy". There's some truth to that and the author has highlighted borrowchecker as probably the best example of this.
Funny that, because I really like the borrow checker. In my opinion Rust only has 2 problems:
1. The culture of licensing stuff under MIT by default rather than (A)GPL.
2. The syntax. I'd much rather have S-expressions.
I suspect that people like and choose Rust mostly because of its modern build and package system which are clear improvements over C++.
again https://www.moonbitlang.com/
That is the whole point of using a language like Rust, otherwise use OCaml, Haskell, F#, Scala, Standard ML.
> Borrowchecker frustration is like being brokenhearted - you can't easily demonstrate it, you have to suffer it yourself to understand what people are talking about. Real borrowchecker pain is not felt when your small, 20-line demonstration snippet fails to compile.
As someone who writes Rust professionally this sentence is sus. Typically, the borrow checker is somewhere between 10th and 100th in the list with regards to things I think about when programming. At the end of the day, you could in theory just wrap something in a reference counter if needed, but even that hasn't happened to me yet.
>The first time someone gave be this advice, I had to do a double take. The Rust community's whole thing is commitment to compiler-enforced correctness, and they built the borrowchecker on the premise that humans can't be trusted to handle references manually. When the same borrowchecker makes references unworkable, their solution is to... recommend that I manually manage them, with zero safety and zero language support?!? The irony is unreal. Asking people to manually manage references is so hilariously unsafe and unergonomic, the suggestion would be funny if it wasn't mostly sad.
Indices aren't simply "references but worse". There are some advantages:
- they are human readable
- they are just data, so can be trivially serialized/deserialized and retain their meaning
- you can make them smaller than 64 bits, saving memory and letting you keep more in cache
Also I don't see how they're unsafe. The array accesses are still bounds-checked and type-checked. Logical errors, sure I can see that. But where's the unsafety?
If you start making assumptions based on indices, you can turn logical errors into memory safety errors. ie. whenever you use unsafe with the SAFETY comment above it mentioning an index, you'd better be damn sure that index is valid.
This goes for not only unchecked indexing but also eg. transmuting based on a checked index into a &[u8] or such. If those indexes move in and out of your API and you do some kind of GC on your arrays / vectors, then you might run into indices being use-after-free and now those SAFETY comments that previously felt pretty obvious, even trivial, may no longer be quite so safe to be around of.
I've actually written about this previously w.r.t. the borrow checker and implementing a GC system based on indices / handles. My opinion was that unless you're putting in ironclad lifetimes on your indices, all assumptions based on indices must be always checked before use.
My comment was implicitly about safe Rust. Obviously if you're using `unsafe`, you have to deal with unsafety ...
Different meanings of "unsafety". A cool thing about Rust's references, when used idiomatically, is that not only are you guaranteed no memory corruption, you're also guaranteed no runtime panics. These are not "unsafe" by Rust's definition, but they're still a correctness violation and it's good to prevent them statically when you can. Indices don't give you this protection.
> In that sense, Rust enables escapism: When writing Rust, you get to solve lots of 'problems' - not real problems, mind you, but fun problems.
This is a real problem across the entire industry, and Rust is a particularly egregious example because you get to justify playing with the fun stimulating puzzle machine because safety—you don't want unsafe code, do you? Meanwhile there's very little consideration to whether the level of rigidity is justified in the problem domain. And Rust isn't alone here, devs snort lines of TypeScript rather than do real work for weeks on end.
Typescript has escape hatches so you can just say "I don't care, or don't know."
With Rust, you're battling a compiler that has a very restrictive model, that you can't shut up. You will end up performing major refactors to implement what seem like trivial additions.
You can always use `Box<dyn Any>` to get the same result in Rust :)
It's not the same because putting it in a box semantically changes the program, adds a level of indirection. It's not just telling it to go away.
There's no avoiding that in a language that's designed to offer low-level control of runtime behavior, regardless of whether it's memory-safe or not. You have to tell the compiler something about how you want the data to be laid out in memory; otherwise it wouldn't know what code to generate. If you don't want to do that, use an interpreted language that doesn't expose those details.
Or use clone everywhere. I am not ashamed of having lots of clones everywhere outside of inner loops.
Have you tried to assure yourself that this or that piece of software (your primary text editor for example) doesn't need to be memory safe because it won't ever receive as input any data that might have been crafted by an attacker? In my experience, doing that is harder than satisfying the borrow checker.
Yes, and you can choose to use any language with a garbage collector and get the same benefit. The list of memory safe languages at your disposal is endless and they come in every flavor you can imagine.
The cost of it is spending more CPU and more RAM on the GC. Often it's the cost you don't mind paying; a ton of good software is written in Java, Kotlin, TS/JS, OCaml, etc.
Sometimes you can't afford that though, from web browsers to MCUs to hardware drivers to HFT.
That's true (with some qualifications), but everyone seems to continue using C and C++ for everything, even for applications like text editors, where the performance of a GC language would presumably be good enough. I wonder why.
> Yes, and you can choose to use any language with a garbage collector
Uh, no thanks.
> and get the same benefit.
Not quite.
One day I will write a blog post called “The Rust borrow checker is overrated, kinda”.
The borrow checker is certainly Rust’s claim to fame. And a critical reason why the language got popular and grew. But it’s probably not in my Top 10 favorite things about using Rust. And if Rust as it exists today existed without the borrow checker it’d be a great programming experience. Arguably even better than with the borrow checker.
Rust’s ergonomics, standardized cargo build system, crates.io ecosystem, and community community to good API design are probably my favorite things about Rust.
The borrow checker is usually fine. But does require a staunch commitment to RAII which is not fine. Rust is absolute garbage at arenas. No bumpalo doesn’t count. So Rust w/ borrow checker is not strictly better than C. A Rust without a borrow checker would probably be strictly better than C and almost C++. Rust generics are mostly good, and C++ templates are mostly bad, but I do badly wish at times that Rust just had some damn template notation.
This is something I've been thinking about lately. I do think memory safety is an important trait that rust has over c and other languages with manual memory management. However, I think Rust also has other attractive features that those older languages don't have:
* a very nice package manager
* Libraries written in it tend to be more modular and composable.
* You can more confidently compile projects without worrying too much about system differences or dependencies.
I think this is because:
* It came out during the Internet era.
* It's partially to do with how cargo by default encourages more use of existing libraries rather than reinventing the wheel or using custom/vendored forks of them.
* It doesn't have dynamic linking unless you use FFI. So rust can still run into issues here but only when depending on non-rust libraries.
Agree on all points
> No bumpalo doesn't count.
Mind explaining why? I have made good experiences with bumpalo.
Everytime I try to use bumpalo I get frustrated, give up, and fallback to RAII allocation bullshit.
My last attempt is I had a text file with a custom DSL. Pretend it’s JSON. I was parsing this into a collection of nodes. I wanted to dump the file into an arena. And then have all the nodes have &str living in and tied to the arena. I wanted zero unnecessary copies. This is trivially safe code.
I’m sure it’s possible. But it required an ungodly amount of ugly lifetime 'a lifetime markers and I eventually hit a wall where I simply could not get it to compile. It’s been awhile so I forget the details.
I love Rust. But you really really have to embrace the RAII or your life is hell.
To make sure I understand correctly: did you want to read a `String` and have lots of references to slices within the same string without having to deal with lifetimes? If so, would another variant of `Rc<str>` which supports substrings that also update the same reference count have worked for you? Looking through crates.io, I see multiple libraries that seem to offer this functionality:
[1]: https://crates.io/crates/arcstr [2]: https://crates.io/crates/imstr
It’s deeper than that.
Let’s pretend I was in C. I would allocate one big flat segment of memory. I’d read the “JSON” text file into this block. Then I’d build an AST of nodes. Each node would be appended into the arena. Object nodes would container a list of pointers to child nodes.
Once I built the AST of nested nodes of varying type I would treat it as constant. I’d use it for a few purposes. And then at some point I would free the chunk of memory in one go.
In C this is trivial. No string copies. No duplicated data. Just a bunch of dirty unsafe pointers. Writing this “safely” is very easy.
In Rust this is… maybe possible. But brutally difficult. I’m pretty good at Rust. I gave up. I don’t recall what exact what wall I hit.
I’m not saying it can’t be done. But I am saying it’s really hard and really gross. It’s radically easier to allocate lots of little Strings and Vecs and Box each nested value. And then free them all one-by-one.
Something like the following? I am trying and failing to reproduce the issue, even with mutable AST nodes.
I checked my repo history and never committed because I failed to get it working. I don’t recall my issues.
If you can get a full JSON parser working then maybe I’m just wrong. Arrays, objects with keys/values, etc.
I’d like to think I’m a decent Rust programmer. Maybe I just need to give it another crack and if I fail again turn it into a blog post…
oxc_parser uses bumpalo (IIRC) to compile an AST into arena from a string. I think the String is outside the arena though, but their lifetimes are "mixed together" into a single 'a, so lifetime-wise it's the same horror to manage. But manage they did.
For those who program a lot (daily) in Rust, how often do you run into borrow checker issues? Or are lifetime scopes second nature at this point?
Very rarely, but it depends on the domain and/or the part of the code I'm writing.
For context, I mostly write GUI apps using `iced` which is inspired by Elm, so the separation between reading and writing state is front and center and makes it easy for me to avoid a whole set of issues.
I really struggle to understand the PoV of the author in his The rules themselves are unergonomical section:
> But what's the point of the rules in this case, though? Here, the ownership rules does not prevent use after free, or double free, or data races, or any other bug. It's perfectly clear to a human that this code is fine and doesn't have any actual ownership issues
I mean, of course there is an obvious ownership issue with the code above, how are the destructors supposed to be ran without freeing the Id object twice?
The whole point is that `Id` doesn't have a destructor (it's purely stack-allocated); that is, conceptually it _could_ be `Copy`.
A more precise way to phrase what he's getting at would be something like "all types that _can_ implement `Copy` should do so automatically unless you opt out", which is not a crazy thing to want, but also not very important (the ergonomic effect of this papercut is pretty close to zero).
I think the primary reason Rust doesn't do this is because it's a semver hazard. I.e., adding a non-Copy field would silently break downstream dependents. Yeah, this is already a problem with the existing auto traits, but types that don't implement those are rarer than types that don't implement Copy.
Auto-deriving Copy would also mean that there needs to be an escape-hatch: eg. Vec would auto-derive Copy.
Yes you would need an escape hatch, but your example is wrong. Vec can't be Copy, because it has a destructor.
This program fails to compile:
Actually; I'm not sure I'm wrong. If Copy was automatically derived based on fields of a struct (without the user explicitly asking for it with `#[derive(Copy)]` that is, as the parent comment suggested the OP is asking for), then your example S and the std Vec would both automatically derive Copy. Then, implementing Drop on them would become a compile error that you would have to silence by using the escape hatch to "un-derive" Copy from S/Vec.
So, whenever you wanted to implement Drop you'd need to engage the escape hatch.
What I suggested OP was asking for was:
> all types that _can_ implement `Copy` should do so automatically unless you opt out
, which was explicitly intended to exclude types with destructors, not
> types should auto-derive `Copy` based purely on an analysis of their fields.
So if you have some struct that you use extensively through an application and you need to extend it by adding a vector you are stuck because the change would need to touch so much code.
Oh, good point yeah; I wasn't thinking of Drop clashing with Copy, but just about the fields that make up a `Vec`.
Copy is already banned for any type that directly or indirectly contains a non-Copy type, and Vec contains a `*const T`, which is not Copy.
const T is Copy, actually.
https://doc.rust-lang.org/std/primitive.pointer.html#impl-Co...
Ah I see.
> A more precise way to phrase what he's getting at would be something like "all types that _can_ implement `Copy` should do so automatically unless you opt out", which is not a crazy thing to want,
From a memory safety PoV it's indeed entirely valid, but from a programming logic standpoint it sounds like a net regression. Rust's move semantics are such a bliss compared to the hidden copies you have in Go (Go not having pointer semantics by default is one of my biggest gripe with the language).
I think you are misunderstanding what Copy means, and perhaps confusing it with Clone. A type being Copy has no effect on what machine code is emitted when you write "x = y". In either case, it is moved by copying the bit pattern.
The only thing that changes if the type is Copy is that after executing that line, you are still allowed to use y.
I'm not misunderstanding. Ore confusing the two. Copy: Clone.
Yes when an item is Copy-ed, you are still allowed to use it, but it means that you now have two independent copies of the same thing, and you may edit one, then use the other, and be surprised that it hasn't been updated. (When I briefly worked with Go, junior developers with mostly JavaScript or Python experience would fall into this trap all the time). And given that most languages nowadays have pointer semantics, having default copy types would lead to a very confusing situation: people would need to learn about value semantics AND about move semantics for objects with a destructor (including all collections).
No thanks. Rust is already complex enough for beginners to grasp.
Got it. Indeed, I misunderstood your point. I agree with you now that you clarified.
Is it a particularly terrible thing, in and of itself, to pass structs by value? Less implicit aliasing seems less bug-prone. Note that Go only has implicit shallow copies (i.e., this only affects the direct fields of a struct or array); all other builtin types either are deeply immutable (booleans, numbers, strings), can point to other variables (pointers, slices, interfaces, functions), or are implicit references to something that can't be passed by value (maps, channels).
I'd argue that move semantics is strictly superior to value semantics.
But more importantly, my point is that in a world where pretty much all modern languages except Go have pointer semantics by default, and when you language needs to have move semantics for memory safety in many cases, having yet another alien (to most developers) behavior is really pushing the complexity bar up for no particular gains.
lol, it’s the one thing that keeps me coming back!
[dead]
Skill issue
It is true that any sufficiently complicated technology requires a certain skill level to use it adequately. The question remains whether the complexity of the technology is justified, and the author presents an argument why this might not be the case. Remarking their supposed lack of skill does not seem particularly productive.