Beau­tiful Racket / appendix: Thoughts on Racket & Chez Scheme

Beautiful Racket: Thoughts on Racket & Chez Scheme

Beautiful Racket: Thoughts on Racket & Chez Scheme

 

Beau­tiful Racket / appendix



Thoughts on Racket & Chez Scheme



Chez Scheme is a well-respected imple­men­ta­tion of the Scheme program­ming language that was bought by Cisco in 2012 and released under an open-source license for the first time in 2016. The following year, the Racket team announced that Racket would abandon its own C-based foun­da­tion in favor of Chez Scheme. That is, the parts of Racket that were written in C would be rewritten in Racket (where possible) or Chez Scheme (where neces­sary). In so doing, Racket would essen­tially itself become a DSL running on Chez Scheme. In 2021, that project was mean­ing­fully finished, as the new Racket based on Chez Scheme (working title “Racket CS”) became the default version of Racket in release 8.0.



Though Chez Scheme is one of the fastest Scheme imple­men­ta­tions, the primary moti­va­tion for the move was not perfor­mance so much as main­tain­ability. Instead of relying on a custom pile of C code, and the ongoing main­te­nance costs thereof, Racket planned to hop on the back of Chez Scheme and float merrily down the river. Further­more, because Chez Scheme is owned by Cisco—and employs Chez Scheme’s prin­cipal archi­tect, Kent Dybvig—there’s been a sense that Chez Scheme’s long-term survival is already assured. So by lashing Racket to Chez Scheme, Racket would indi­rectly benefit from this insti­tu­tional under­pin­ning.



Looking back



When I first heard about this “Racket CS” project in 2017, I didn’t spend too long thinking about it. I trusted that the Racket team had good reasons for making the switch. What­ever adjust­ments that would be needed down the road from Racket devel­opers would be worth­while.



In hind­sight, I misun­der­stood the project. I thought the goal was to move Racket onto Chez Scheme and accept what­ever losses to back­ward compat­i­bility might arise, on the idea that the future bene­fits were so terrific. This is a tradeoff that happens all the time in program­ming.



But the actual project under­taken by the Racket team was to preserve the existing behavior of Racket and all its libraries.  +  On the one hand, this is osten­sibly a more ambi­tious & accom­mo­dating goal. On the other hand, 4+ years later, it has left the Racket team in a puddle of unin­tended conse­quences.



Fore­most among these: though the project is notion­ally complete, the main mission was arguably never accom­plished, because Racket doesn’t run on the main branch of Chez Scheme. Rather, it runs on a special Racket-specific fork of Chez Scheme that’s had certain patches applied to preserve compat­i­bility with every­thing already in Racket.



Let’s rewind for a moment. It was always apparent to the Racket team that certain discrep­an­cies between the Racket view of the world and the Chez Scheme view would need to be smoothed over. How? By patching the main branch of Chez Scheme through pull requests, they predicted. The Racket team got some of their pull requests accepted into the main branch of Chez Scheme, because these patches were broadly useful.



But not all. Some patches that Racket needed were detri­mental for Chez Scheme at large. In an “expe­ri­ence report” near the end of the project, the Racket team admitted that “[t]he least predictable part of our effort was how big the differ­ences between Racket and Chez Scheme would turn out to be”. And instead of capit­u­lating to the Chez Scheme way—say, by removing or changing certain features of Racket—these Racket-compat­i­bility patches became the basis of the Racket-specific fork of Chez Scheme.



“But that’s close enough, right?” As a Racket devel­oper, I hope that turns out to be true. But I have doubts that it will. The main point of moving to Chez Scheme was to harvest certain main­tain­ability bene­fits. It stands to reason—if your project is based on Chez Scheme, then as Chez Scheme is improved, your project is thereby also improved.



But that dream is already over. As of today, the Racket fork of Chez Scheme is “1249 commits ahead, 617 commits behind” the main branch of Chez Scheme. As time passes, the two will natu­rally tend to diverge further. Maybe some of those later commits to Chez Scheme can be applied as patches to Racket’s fork. But that process will be neither easy nor auto­matic.



In their expe­ri­ence report, the Racket team is blasé about the risks of this diver­gence. Maybe that’s fair. Even with the diver­gence factored in, maybe the bene­fits of using a fork of Chez Scheme is better than no Chez Scheme at all.



Still, as a Racket devel­oper, it seems like an oppor­tu­nity has been missed. I thought the broader objec­tive was to perma­nently combine the greatest assets of Chez Scheme (= its speed and stability) with the greatest assets of Racket (= its language-oriented program­ming facil­i­ties, including its macro system and module system). This didn’t happen. As the Racket team turns its atten­tion to other projects, it doesn’t look like it ever will.



Looking ahead



The Racket CS effort has also had the perhaps unin­tended conse­quence of encour­aging me to take a closer look at Chez Scheme for my own future projects.



Inter­est­ingly, both Racket and Chez Scheme began as acad­emic projects. Both have been contin­u­ously devel­oped over a long time—Racket since 1995; Chez Scheme since 1984. (There’s also Gambit Scheme, a Scheme-to-C compiler with acad­emic roots, devel­oped since 1988.)



At times, the Racket team has claimed to want more users outside of acad­emia. After eight years working with Racket, it’s hard to take them seri­ously on that point. (See also: the ongoing strug­gles with the Rhombus project.) I still like the language. But despite occa­sional gestures other­wise, it was and is a project led by five CS profes­sors. Their devel­op­ment prior­i­ties are set largely by their research objec­tives. To be clear, there’s nothing wrong with this: it’s their project and they can do what­ever they want.



But Chez Scheme has the much stronger story beyond acad­emia. In 2012, Kent Dybvig left his CS profes­sor­ship to work at Cisco, which (reput­edly) uses Chez Scheme as an embedded language for networking equip­ment.  +  That is, Chez Scheme mean­ing­fully tran­si­tioned from an acad­emic project to an indus­trial project. To be fair, Cisco’s patronage isn’t neces­sarily perma­nent. Big tech compa­nies are famously capable of pivoting to new tech­nolo­gies and dumping the earlier ones with zero senti­men­tality.



Racket has a larger & richer set of libraries than Chez Scheme, which makes getting things done in Racket easier at times. But that’s largely because Racket has been open source since the begin­ning. Chez Scheme has only been open source since 2016. Given its tech­nical supe­ri­ority as a Scheme imple­men­ta­tion, it seems inevitable that its library ecosystem will grow.



Will an indus­trial user be better off investing in Racket or Chez Scheme? As one of those users, I’m not sure yet. In terms of engi­neering, the Racket team has conceded that Chez Scheme is supe­rior. In terms of orien­ta­tion to indus­trial users, Chez Scheme has also been supe­rior (keeping in mind that before it was open-sourced, it was licensed as propri­etary soft­ware to customers largely outside acad­emia). Racket has barely any use outside acad­emia, and no demon­strated interest in getting there. As for the libraries, though Racket has an advan­tage for now, it seems likely that many of these libraries could be ported to Chez Scheme if needed—Racket & Chez Scheme are under compat­ible open-source licenses—though this will require more study.



In sum, I think the possi­bil­i­ties of Chez Scheme are inter­esting enough that I plan to spend more time exper­i­menting with it, and using it where possible.



[July 2021]← prev