I'm a little worried on behalf of the "Python Language Tooling Team" at Meta, because uv has been so popular, and I wouldn't be surprised if ty wins out in this space.
So watch out, or this will become like Atom or Flow, an internal competitor of a technology that is surpassed by the more popular external open source version, leaving the directors/vps muttering to themselves "It's too bad that this team exists at all. Could we get rid of them and just switch to the open source stuff?"
Perhaps just something for the manager (Aaron Pollack?) to keep an eye on....
Hey Kevin, we overlapped for a bit during your time at FB when I was working on Flow. Nice to hear from you!
I’m working on Pyrefly now, but worked on Flow for many years before. For what it’s worth, we are taking a different approach compared to Flow and have explicitly prioritized open source and community building, something I know we both care a lot about.
Of course, nothing is guaranteed and we’ve seen plenty of volatility in bigco investments to infra lately, but I do believe we’re starting this journey on the right foot.
Meta seems to place a pretty high premium on controlling its open source projects, especially dev tooling. I guess dating back at least to the git maintainers telling them they were doing things wrong with their monorepo and refusing to upstream scale fixes, which precipitated their migration to mercurial (who were more than happy to take the contributions).
Given the change velocity of internal tooling you can understand why owning your own project makes sense here.
I believe the argument was that jsx doesn't support that either, which should be correct as far as I know.
They're all features in the jsx ecosystem that are provided by other libs, which are often preconfigured with projects utilizing jsx.
I did miss a lot of the tooling I've gotten used to the last time I made a weekend project utilizing lit webcomponents for sure, but I believe the people advocating for it usually rejoice in having less tooling in their development pipeline - it's just a fundamental disconnect between people that have learned and gotten used to current frontend development and people that don't really want to bother with it, looking for simpler solutions with less moving parts.
Hi folks, I work on the Pyrefly team at Meta. Our FAQ covers a good number of the questions raised here: https://pyrefly.org/en/docs/pyrefly-faq/. I can also try to answer some of your questions. Thanks for taking a look!
There's nothing magical to type-checking Python. You can write this in any programming language. TypeScript is actually a pretty nice language for writing static analysis tools.
Yes. If you want runtime validation of data you’re taking in people recommended pydantic. If you’re looking for runtime validation within your own code I’ve seen people use beartype, though to be honest I don’t personally understand the value added from it
On one hand, I feel like I've been in a coma since covid because I've just been coasting along with Marshmallow and jsonschema, but on the other hand it's like a lot of the major advances have been in the past couple years. Apparently pydantic got a big version update in 2023? And now all these competing static type checkers?
"Today we’re releasing Pyrefly as an alpha. At the same time, we’re busy burning down the long-tail of bugs and features aiming to remove the alpha label this Summer. "
Why is "written in Rust" a feature to be mentioned? Who cares? So my type checker has memory protection and is compiled. I'm not running my type checker on an embedded system or in a mission critical service. It seems kind of like "written in Erlang". I'd prefer to have non-performance critical code for Python written in Python. That way the broader community can support and extend it.
Have you used Rust before? As a user, the speed and safety is nice. But as a developer, Rust projects are easier to hack on and contribute to.
That's kind of the whole appeal of Astral. I know Python better than Rust, but it's a lot easier for me to hack on Rust projects. The whole appeal of Astral is that they want to bring Rust-quality tooling to Python.
Rust has very arcane syntax and a lot of rules that developers coming from interpreted / garbage-collected languages (like the ones using these tools) would have a hard time grasping. It’s easy for people who are already familiar with it, but isn’t that always the case?
Static typing is a big one, but I've been so steeped in Python that I don't appreciate it as much as maybe I should.
The big thing for me is that most Rust projects are statically(ish) compiled (like go) and only need a `cargo build`. No venvs or pip commands or node/npm/nvm or make, etc.
Also, the Zen of Python is supposed to be one obvious way to do something but ddg "python dependency manager" and have a good laugh. That actually becomes triple lol for when uv steps in and says "uh, your pip requirements.txt is unsatisifyable, what are you even?!"
And let me just say what an absolutely stellar job the rustc authors have done with the error messages. Just out of this world thoughtful. I have not yet gotten on board with their compile time vs safety and insight tradeoff, but I can see why they chose those two axes
Type inference is handy, but as an observation source code is 90% written for humans - I believe that CPython knows what types they are, that's how TypeError even exists, but seeing "def doit(erthang):" doesn't help me in the slightest. Often PyCharm can figure it out, because bless their hearts, but then developers whine about "wwwaaa, pycharm too slo grug use ed" and then one gets AttributeError in production because who could have foreseen
Pyright being so fast has gotten my coworkers (as well as myself) who dare to code using tooling not made by JetBrains an easy to use instant-feedback type checker that can be used in CI. So I think it's a huge win.
It makes it easy to find performant and quality software by searching for "[insert tool description] rust", I personally don't mind! Seeing how 95% of the tools I use on the daily are written in Rust, I love finding new ones and am rarely disappointed.
I feel like the likelihood that a project will say what language it is written in is much higher if that language is Rust. I like Rust but I do find this trend a little annoying. (Even though I acknowledge that "written in Rust" probably means the tool is relatively new, not buggy, and easy to use.)
> I'd prefer to have non-performance critical code for Python written in Python
A type checker is performance critical code. You can watch how Pylint, just a linter, written in Python, lints your source code line by line. It's so slow it can take 30 seconds to update the linting after you change some lines.
The Rust code written here is so easy to follow but all these new Python tooling being written in Rust worries me, it adds yes another vector to the N-language problem.
I'm curious to know more about the Pyre to Pyrefly transition, specifically the rewrite in Rust. Was that merely a case of trading in a lesser-known language for the language du jour? Were there specific advantages they wanted to get out of Rust?
> Not only is Pyrefly written in a new language (Rust instead of OCaml), but its design deviates in a major way from Pyre.
I'm sure you had reasons to do it this way. But given sufficient time to market, implementing the algorithm in pyre and then tooling/llm assisted conversion to pyrefly would've been preferable.
May be you'd have had some humans in the loop initially. But that tech is getting better and aligned with the direction Meta and the rest of the industry are taking.
pyrefly is not tied to vscode? Also please try to be more considerate of people preferences, and pycharm is not strictly better. Remote dev on vscode is very convenient for me, should I go on the Internet saying that pycharm is trash? No
I don't know the differences between the two well enough to know if it was the case here, but in my experience sometimes you need to innovate on a fork, or from scratch in order to create the space/freedom to do so.
Once a project is popular, it's harder to justify and be confident about major changes (aka https://m.xkcd.com/1172/)
I think astral and meta were both working on their own type-checkers independently. My current understanding is that meta released so they could preempt the initial release of ty. It seems like they're a bit further ahead in development. Not sure if there are going to be any real differences between the two down the line.
Sure, but in this case they are both implementations of a spec defined by PEPs, so a bit more like gcc vs clang (less tightly bound than those, of course, in design decisions). Neither company is trying to invent a new language here.
To repeat an earlier comment of mine from the launch of uv on hn (tl; dr: these new type checkers never support django):
The way these type checkers get fast is usually by not supporting the crazy rich reality of realworld python code.
The reason we're stuck on mypy at work is because it's the only type checker that has a plugin for Django that properly manages to type check its crazy runtime generated methods.
I wish more python tooling took the TS approach of "what's in the wild IS the language", as opposed to a "we only typecheck the constructs we think you SHOULD be using".
1. Maybe it's time to drop the crazy runtime generation and have something statically discoverable, or at least a way to annotate the typing statically.
2. Astral indicated already they plan to just add direct support for Django and other popular languages.
3. As people replied to similar comments on the previous threads (maybe to you?): that's not why ty is fast and why mypy is slow. It's also an easy claim to disprove: run both without plugins and you'll see ty is still 100x+ faster.
> 1. Maybe it's time to drop the crazy runtime generation and have something statically discoverable, or at least a way to annotate the typing statically.
That, and duck typing, are one of the biggest things that make Python what it is. If I have to drop all that for type checking and rewrite my code, why would I rewrite it in Python?
Having used Python for many years, it’s the least interesting aspect of the language. Almost all such tricks can be done with compile time meta programming, often even without API changes.
TS has the luxury of being its own distinct language, defining its own semantics and having its own compiler. You could have something like that targeting Python.
The only type checker that fully works (meaning it successfully performs the necessary type inference all for inherited objects) for our large and highly modular python codebase, is Pycharm (I'm guessing it's their own custom tool from the ground up? Not really sure, actually.)
I'm a little worried on behalf of the "Python Language Tooling Team" at Meta, because uv has been so popular, and I wouldn't be surprised if ty wins out in this space.
So watch out, or this will become like Atom or Flow, an internal competitor of a technology that is surpassed by the more popular external open source version, leaving the directors/vps muttering to themselves "It's too bad that this team exists at all. Could we get rid of them and just switch to the open source stuff?"
Perhaps just something for the manager (Aaron Pollack?) to keep an eye on....
Hey Kevin, we overlapped for a bit during your time at FB when I was working on Flow. Nice to hear from you!
I’m working on Pyrefly now, but worked on Flow for many years before. For what it’s worth, we are taking a different approach compared to Flow and have explicitly prioritized open source and community building, something I know we both care a lot about.
Of course, nothing is guaranteed and we’ve seen plenty of volatility in bigco investments to infra lately, but I do believe we’re starting this journey on the right foot.
Cheers, Sam
Best of luck!
Meta seems to place a pretty high premium on controlling its open source projects, especially dev tooling. I guess dating back at least to the git maintainers telling them they were doing things wrong with their monorepo and refusing to upstream scale fixes, which precipitated their migration to mercurial (who were more than happy to take the contributions).
Given the change velocity of internal tooling you can understand why owning your own project makes sense here.
JSX is my favorite thing to come out of Facebook (also the only good thing).
I feel bad for that people that love JSX and don't know about lit-html yet.
JSX supports
* Autocompletion
* Type checking
* Syntax highlighting
* Lack of runtime string parsing
Tagged template literals don't.
JSX does not support any of these, it was coded by others.
tagged template literals can have all of these, some already exist¹ and doesn't need a build step.
1. https://marketplace.visualstudio.com/items?itemName=bierner....
The only inherent feature of JSX is compile time "parsing" whereas tagged literals inherently require runtime string parsing.
But yeah other than that you're right. I'm just talking about first-class VS Code support. Which to me means a lot.
Matt, who works pretty high level on VS Code, has written some amazing plugins.
But they still don't have all the first-class features of JSX.
I don't think this plugin works with type checking for instance.
I believe the argument was that jsx doesn't support that either, which should be correct as far as I know.
They're all features in the jsx ecosystem that are provided by other libs, which are often preconfigured with projects utilizing jsx.
I did miss a lot of the tooling I've gotten used to the last time I made a weekend project utilizing lit webcomponents for sure, but I believe the people advocating for it usually rejoice in having less tooling in their development pipeline - it's just a fundamental disconnect between people that have learned and gotten used to current frontend development and people that don't really want to bother with it, looking for simpler solutions with less moving parts.
Hi folks, I work on the Pyrefly team at Meta. Our FAQ covers a good number of the questions raised here: https://pyrefly.org/en/docs/pyrefly-faq/. I can also try to answer some of your questions. Thanks for taking a look!
Seems there are at least three Rust-based competitors for type checkers in Python now (Microsoft, Facebook, Astral), and of course there's still mypy.
Close, Microsoft’s type checker Pyright is Typescript. Its still faster than mypy for me though.
Pls forgive my ignorance, but how is Typescript (a superset of Javascript) used to type-check Python?
You can write a parser and type checker for pretty much any language in pretty much any language. It's just text files as input and text as output.
They're saying pyright is a Python type checker, but it's written in Typescript, not Rust.
There's nothing magical to type-checking Python. You can write this in any programming language. TypeScript is actually a pretty nice language for writing static analysis tools.
just like the Python compiler/interpreter is written in C.
They're all static type checkers right? None for runtime?
Yes. If you want runtime validation of data you’re taking in people recommended pydantic. If you’re looking for runtime validation within your own code I’ve seen people use beartype, though to be honest I don’t personally understand the value added from it
...or Marshmallow, which allows one to do many complex validations in a relatively trivial manner.
On one hand, I feel like I've been in a coma since covid because I've just been coasting along with Marshmallow and jsonschema, but on the other hand it's like a lot of the major advances have been in the past couple years. Apparently pydantic got a big version update in 2023? And now all these competing static type checkers?
Pydantic got the re write in rust treatment so de/serialization is crazy fast now.
msgspec must be insanely fast then: https://jcristharif.com/msgspec/benchmarks.html
But of course unless parsing and manipulating JSON is your bottleneck, Pydantic is great, too.
this is the official announcement, but pyrefly was previously discuted a few weeks ago: https://news.ycombinator.com/item?id=43831524
"Today we’re releasing Pyrefly as an alpha. At the same time, we’re busy burning down the long-tail of bugs and features aiming to remove the alpha label this Summer. "
Happy to see instructions for integrating into Vim/Neovim: https://pyrefly.org/en/docs/IDE/#other-editors
Why is "written in Rust" a feature to be mentioned? Who cares? So my type checker has memory protection and is compiled. I'm not running my type checker on an embedded system or in a mission critical service. It seems kind of like "written in Erlang". I'd prefer to have non-performance critical code for Python written in Python. That way the broader community can support and extend it.
Have you used Rust before? As a user, the speed and safety is nice. But as a developer, Rust projects are easier to hack on and contribute to.
That's kind of the whole appeal of Astral. I know Python better than Rust, but it's a lot easier for me to hack on Rust projects. The whole appeal of Astral is that they want to bring Rust-quality tooling to Python.
Rust has very arcane syntax and a lot of rules that developers coming from interpreted / garbage-collected languages (like the ones using these tools) would have a hard time grasping. It’s easy for people who are already familiar with it, but isn’t that always the case?
> but it's a lot easier for me to hack on Rust projects
That static typing is nice, I wonder if it's going to catch on one day.
The amount of energy spent trying to bend dynamically types languages into being real ones is just comical. Even the standard library is barely typed, so they give no fucks https://github.com/python/cpython/blob/v3.13.3/Lib/re/__init...
What does it accept? Who knows. What does it return? Don't worry about it
Static typing is a big one, but I've been so steeped in Python that I don't appreciate it as much as maybe I should.
The big thing for me is that most Rust projects are statically(ish) compiled (like go) and only need a `cargo build`. No venvs or pip commands or node/npm/nvm or make, etc.
Also, the Zen of Python is supposed to be one obvious way to do something but ddg "python dependency manager" and have a good laugh. That actually becomes triple lol for when uv steps in and says "uh, your pip requirements.txt is unsatisifyable, what are you even?!"
And let me just say what an absolutely stellar job the rustc authors have done with the error messages. Just out of this world thoughtful. I have not yet gotten on board with their compile time vs safety and insight tradeoff, but I can see why they chose those two axes
It’s not a rule. It’s a guide to being idiomatic.
It’s like laughing at photographers for not using the “rule of thirds” sometimes.
Hindly Milner type inference needs to catch on
Type inference is handy, but as an observation source code is 90% written for humans - I believe that CPython knows what types they are, that's how TypeError even exists, but seeing "def doit(erthang):" doesn't help me in the slightest. Often PyCharm can figure it out, because bless their hearts, but then developers whine about "wwwaaa, pycharm too slo grug use ed" and then one gets AttributeError in production because who could have foreseen
Pyright being so fast has gotten my coworkers (as well as myself) who dare to code using tooling not made by JetBrains an easy to use instant-feedback type checker that can be used in CI. So I think it's a huge win.
regex.match takes strings and returns a match object. There are most likely stubs, if you are new to it and need support.
It makes it easy to find performant and quality software by searching for "[insert tool description] rust", I personally don't mind! Seeing how 95% of the tools I use on the daily are written in Rust, I love finding new ones and am rarely disappointed.
Shortcut for "noticeably fast."
Open source Rust is still review able.
I feel like 70% of open source projects on GitHub say written in the language that they were written in
I feel like the likelihood that a project will say what language it is written in is much higher if that language is Rust. I like Rust but I do find this trend a little annoying. (Even though I acknowledge that "written in Rust" probably means the tool is relatively new, not buggy, and easy to use.)
> Even though I acknowledge that "written in Rust" probably means the tool is relatively new, not buggy, and easy to use.
I genuinely chose the language for one of my projects based on this reasoning. I want to be in the nice UX gang.
That's because the point of writing something in Rust is virtue signaling just as often as it's about the software itself.
> I'd prefer to have non-performance critical code for Python written in Python
A type checker is performance critical code. You can watch how Pylint, just a linter, written in Python, lints your source code line by line. It's so slow it can take 30 seconds to update the linting after you change some lines.
Or entire projects abandoning checking because mypy is so damned slow for anything non-trivial
Many of these make the mistake of running against an entire codebase instead of checking vcs first and only running against changed files.
The Rust code written here is so easy to follow but all these new Python tooling being written in Rust worries me, it adds yes another vector to the N-language problem.
I hope Mojo can offer something here
I'm curious to know more about the Pyre to Pyrefly transition, specifically the rewrite in Rust. Was that merely a case of trading in a lesser-known language for the language du jour? Were there specific advantages they wanted to get out of Rust?
Hi! We address this question in our FAQ and probably could do a longer blog post about our experience after we are further along: https://pyrefly.org/en/docs/pyrefly-faq/#why-rust
> Not only is Pyrefly written in a new language (Rust instead of OCaml), but its design deviates in a major way from Pyre.
I'm sure you had reasons to do it this way. But given sufficient time to market, implementing the algorithm in pyre and then tooling/llm assisted conversion to pyrefly would've been preferable.
May be you'd have had some humans in the loop initially. But that tech is getting better and aligned with the direction Meta and the rest of the industry are taking.
Yes, I'm biased :)
I lost all interest when I saw VS Code. I don’t get why people consider this a suitable IDE for python when you can have a real IDE like PyCharm.
pyrefly is not tied to vscode? Also please try to be more considerate of people preferences, and pycharm is not strictly better. Remote dev on vscode is very convenient for me, should I go on the Internet saying that pycharm is trash? No
Sounds like a project that is trying to solve too many things at once ...
This is very cool but why wouldn’t they just contribute to uv and ruff and ty https://github.com/astral-sh/ty
That's sort of how I felt about things before, but the reality I believe is we wouldn't have uv if they 'just contributed to poetry'.
I tend to agree.
I don't know the differences between the two well enough to know if it was the case here, but in my experience sometimes you need to innovate on a fork, or from scratch in order to create the space/freedom to do so.
Once a project is popular, it's harder to justify and be confident about major changes (aka https://m.xkcd.com/1172/)
I think astral and meta were both working on their own type-checkers independently. My current understanding is that meta released so they could preempt the initial release of ty. It seems like they're a bit further ahead in development. Not sure if there are going to be any real differences between the two down the line.
I think they've mentioned earlier that it's really just because PyCon is this week (so a good time to announce new Python tooling).
Sounds a lot like TypeScript and Flow.
Sure, but in this case they are both implementations of a spec defined by PEPs, so a bit more like gcc vs clang (less tightly bound than those, of course, in design decisions). Neither company is trying to invent a new language here.
Typescript was Microsoft though. Meta might have the edge here based on brand awareness, but who knows for sure.
Right, and Flow was FB
ty is so new right now - it only got its current name a few weeks ago!
That’s not true, they have been developing it as red knot for a good while :)
I just ran ty and it can't resolve any imports whereas pyrefly passes. Why would that be? I hate Python so much.
NIH + copyright
Isn't both this and ty MIT license?
Because this has been tested at Meta / Facebook scale which means it's faster for any Python codebase massive and small.
Since Meta built this, I have confidence this will be maintained more than others and I will use this and ask for Pyrefly experience in the future.
I suggest you try uv and ruff and then see if you still think only companies the size of meta can provide quality tooling.
This feels like a somewhat closed minded approach given both tools are in their infancy
To repeat an earlier comment of mine from the launch of uv on hn (tl; dr: these new type checkers never support django):
The way these type checkers get fast is usually by not supporting the crazy rich reality of realworld python code.
The reason we're stuck on mypy at work is because it's the only type checker that has a plugin for Django that properly manages to type check its crazy runtime generated methods.
I wish more python tooling took the TS approach of "what's in the wild IS the language", as opposed to a "we only typecheck the constructs we think you SHOULD be using".
1. Maybe it's time to drop the crazy runtime generation and have something statically discoverable, or at least a way to annotate the typing statically.
2. Astral indicated already they plan to just add direct support for Django and other popular languages.
3. As people replied to similar comments on the previous threads (maybe to you?): that's not why ty is fast and why mypy is slow. It's also an easy claim to disprove: run both without plugins and you'll see ty is still 100x+ faster.
> 1. Maybe it's time to drop the crazy runtime generation and have something statically discoverable, or at least a way to annotate the typing statically.
That, and duck typing, are one of the biggest things that make Python what it is. If I have to drop all that for type checking and rewrite my code, why would I rewrite it in Python?
Having used Python for many years, it’s the least interesting aspect of the language. Almost all such tricks can be done with compile time meta programming, often even without API changes.
I'd like to read a blog post on this subject, if anyone knows one.
The problem is not with the type checkers.
https://code.djangoproject.com/ticket/32759
Similar (but lesser) problems exist with pydantic and sqlmodel. They're both fine projects except for:
https://www.reddit.com/r/Python/comments/1i5atpy/fquery_meet...
This is a long winded way of saying type checkers will deal with:
a lot better. Move what doesn't fit here to dataclass metadata.TS has the luxury of being its own distinct language, defining its own semantics and having its own compiler. You could have something like that targeting Python.
The only type checker that fully works (meaning it successfully performs the necessary type inference all for inherited objects) for our large and highly modular python codebase, is Pycharm (I'm guessing it's their own custom tool from the ground up? Not really sure, actually.)
how does this compare to mypy
non AI IDEs are going to have a hard time in the future.
Unsure how this comment is relevant
Its probably cool n' all but fb isnt getting any of my attention. They'd need to come up with AGI for that to happen, and even then I'd shrug it off.
I agree. I simply can't support anything Mark Zuckerberg does at this point.
This is an open source developer tooling project, not a product which Zuckerberg had anything to do with. What's the point of ignoring it?
> Why we built Pyrefly: Back in 2017, we embarked on a mission to create a type checker that could handle Instagram’s massive codebase of typed Python
They're saying this on fb.com. How does it not have anything to do with fb?
The feedback section takes you to fb's github.
Guilt by association is very in vogue these days
These talented engineers could take their skills elsewhere. That's the message.
what if they pull off the metaverse? ;)