An Opinionated Guide to Technology Frontiers

TECHNOLOGY RADAR An opinionated guide to technology frontiers

Volume 24

#TWTechRadar thoughtworks.com/radar The Technology Advisory Board (TAB) is a group of 20 senior technologists at Thoughtworks. The TAB meets twice a year face-to-face and biweekly by phone. Its primary role is to be an advisory group for Thoughtworks CTO, Contributors Rebecca Parsons. The Technology Radar is prepared by the The TAB acts as a broad body that can look at topics that affect technology and technologists at Thoughtworks. With the ongoing global pandemic, we Thoughtworks Technology Advisory Board once again created this volume of the Technology Radar via a virtual event.

Rebecca Martin Fowler Bharani Birgitta Brandon Camilla Cassie Parsons (CTO) (Chief Scientist) Subramaniam Böckeler Byars Crispim Shum

Erik Evan Fausto Hao Ian James Lakshminarasimhan Dörnenburg Bottcher de la Torre Xu Cartwright Lewis Sudarshan

Mike Neal Perla Rachel Scott Shangqi Zhamak Mason Ford Villarreal Laycock Shaw Liu Dehghani

Thoughtworkers are passionate about technology. We build it, research it, test it, open source it, write about it and constantly aim to improve it — for everyone. Our mission is to champion software excellence and revolutionize IT. We create and share the Thoughtworks Technology Radar in support of that mission. The Thoughtworks Technology Advisory Board, a group of senior technology leaders at Thoughtworks, creates the Radar. They meet regularly to discuss the global technology strategy for Thoughtworks and the technology trends that significantly impact our industry.

The Radar captures the output of the Technology Advisory Board’s discussions in a format that provides value to a wide range of stakeholders, from developers to CTOs. The content is intended as a concise summary.

We encourage you to explore these technologies. The Radar is graphical in nature, grouping items into techniques, tools, platforms and languages & frameworks. When Radar items could appear in multiple quadrants, we chose the one that seemed most appropriate. We further group these items in four rings to reflect our current position on them.

For more background on the Radar, see thoughtworks.com/radar/faq.

Moved in/out Radar at No change

Our Radar is forward looking. To make a glance room for new items, we fade items that haven’t moved recently, which isn’t a reflection on their value but rather on The Radar is all about tracking interesting our limited Radar real estate. things, which we refer to as blips. We organize the blips in the Radar using two categorizing elements: quadrants and rings. The quadrants represent different kinds of blips. The rings indicate what stage in an adoption lifecycle we think they should be in. Hold Assess TrialTAdoptAdopt rial Assess Hold A blip is a technology or technique that plays a role in software development. Blips Adopt are things that are “in motion” — that is we find their position in the Radar is changing We feel strongly that the industry should — usually indicating that we’re finding be adopting these items. We use them increasing confidence in them as they move when appropriate in our projects. through the rings. Trial Worth pursuing. It’s important to understand how to build up this capability. Enterprises can try this technology on a project that can handle the risk.

Assess Worth exploring with the goal of understanding how it will affect your enterprise.

Hold Proceed with caution.

Platform Teams Drive Consolidated Convenience Perennially “Too Discerning the Context for Speed to Market over Best in Class Complex to Blip” Architectural Coupling

Increasingly, organizations are adopting a As engineering practices that feature In Radar nomenclature, the final status A topic that recurs virtually every meeting platform team concept: set up a dedicated automation, scale and other modern after discussion for many complex topics (see “Perennially ‘Too Complex to Blip’”) group that creates and supports internal goals become more commonplace with is “TCTB — too complex to blip”: items is the appropriate level of coupling platform capabilities — cloud native, development teams, we see corresponding that defy classification because they in software architecture between continuous delivery, modern observability, developer-facing tool integration on offer a number of pros and cons, a high microservices, components, API gateways, AuthZ/N patterns, service mesh, and many platforms, particularly in the cloud amount of nuance as to the applicability integration hubs, frontends, and so on... so on — then use those capabilities to space. For example, artifact repositories, of the advice or tool or other reasons pretty much everywhere two pieces of accelerate application development, reduce source control, CI/CD pipelines, wikis, and that prevent us from summarizing our software might connect, architects and operational complexity and improve similar tools were usually hand-picked by opinions in a few sentences. Frequently, developers struggle finding the correct time to market. This growing maturity development teams and stitched together these topics go on to articles, podcasts, level of coupling — much common advice is welcome and we first featured this à la carte. Now, delivery platforms such and other non-Radar destinations. Some encourages extreme decoupling, but technique in the Radar in 2017. But with as Azure DevOps and ecosystems such of our richest conversations center that makes building workflows difficult. increasing maturity, we’re also discovering as GitHub have subsumed many of these on these topics: they’re important but Coupling in architecture touches on many antipatterns that organizations should tool categories. While the level of maturity complex, preventing a single succinct important considerations: how things avoid. For example, “one platform to rule varies across platform offerings, the appeal point of view. Numerous topics recur are wired, understanding the inherent them all” may not be optimal, “big platform of a “one-stop shop” for delivery tooling meeting after meeting — and, critically, semantic coupling within each problem up front” may take years to deliver value is undeniable. Overall, it seems that the with several of our client engagements domain, how things call one another or and “build it and they will come” might trade-off lies with having consolidated tool — that eventually fall to TCTB, including how transactionality works (sometimes end up as a wasted effort. Instead, using stacks offer greater developer convenience monorepos, orchestration guidelines for in combination with other tricky features a product-thinking approach can help and less churn, but the set of tools rarely distributed architectures and branching like scalability). Software can’t exist you clarify what each of your internal represents the best possible. models, among others. For those who without some level of coupling outside of platforms should provide, depending on wonder why these important topics don’t singular monolithic systems; arriving at its customers. Companies that put their make it into the Radar, it’s not for lack the right set of trade-offs to determine platform teams behind a ticketing system of awareness or desire on our part. Like the types and levels of coupling becomes like an old-school operations silo find many topics in software development, a critical skill with modern architectures. the same disadvantages of misaligned too many trade-offs exist to allow clear, We do see specific bad practices such as prioritization: slow feedback and response, unambiguous advice. We sometimes do generating code for client libraries and resource allocation contention and other find smaller pieces of the larger topics that good practices such as the judicious use well-known problems caused by the we can offer advice on that do make it in of the BFF patterns. However, general silo. We’ve also seen several new tools the Radar, but the larger topics remain advice in this realm is useless and silver and integration patterns for teams and perpetually too nuanced and unsettled for bullets don’t exist. Invest time and effort technologies emerge, allowing more the Radar. in understanding the factors at play when effective partitioning of both. making these decisions on a case-by-case basis rather than seeking a generic but inadequate solution.

TECHNOLOGY RADAR | 5 © Thoughtworks, Inc. All Rights Reserved. Techniques Tools Adopt Adopt 1. API expand-contract 52. Sentry The Radar 2. Continuous delivery for machine learning (CD4ML) 3. Design systems Trial 4. Platform engineering product teams 53. axe-core 5. Service account rotation approach 54. dbt 55. esbuild Trial 56. Flipper 6. Cloud sandboxes 57. Great Expectations 7. Contextual bandits 58. k6 8. Distroless Docker images 59. MLflow 60. OR-Tools 33 9. Ethical Explorer 10. Hypothesis-driven legacy renovation 61. Playwright 62. Prowler 32 11. Lightweight approach to RFCs 12. Simplest possible ML 63. Pyright 31 13. SPA injection 64. Redash 25 26 68 14. Team cognitive load 65. Terratest 24 69 70 71 66. Tuple 80 15. Tool-managed Xcodeproj 23 67. Why Did You Render 30 16. UI/BFF shared types 72 73 Assess 22 Assess 17. Bounded low-code platforms 68. Buildah and Podman 16 54 74 29 21 15 57 18. Decentralized identity 69. GitHub Actions 58 70. Graal Native Image 12 13 53 56 19. Deployment drift radiator 75 20. Homomorphic encryption 71. HashiCorp Boundary 20 11 55 14 60 21. Hotwire 72. imgcook 28 59 76 73. Longhorn 10 61 22. Import maps for micro frontends 19 77 23. Open Application Model (OAM) 74. Operator Framework 5 63 24. Privacy-focused web analytics 75. Recommender 62 76. Remote - WSL 9 4 25. Remote mob programming 18 78 27 64 26. Secure multiparty computing 77. Spectral 3 78. Yelp detect-secrets 7 8 66 52 65 79 79. Zally 17 6 2 Hold 1 67 27. GitOps 28. Layered platform teams Hold Hold Assess Trial Adopt Adopt Trial Assess Hold 29. Naive password complexity requirements 80. AWS CodePipeline 30. Peer review equals pull request 94 31. SAFe™ 40 34 93 104 32. Separate code and pipeline ownership Languages & 82 81 33. Ticket-driven platform operating models Frameworks 92 41 35 103 Adopt 88 90 91 Platforms 81. Combine 36 42 87 82. LeakCanary 102 Adopt 86 89 43 101 Trial Trial 83. Angular Testing Library 50 38 85 34. AWS Cloud Development Kit 37 83 84. AWS Data Wrangler 44 35. Backstage 85. Blazor 45 84 36. Delta Lake 86. FastAPI 39 100 37. Materialize 87. io-ts 99 38. Snowflake 88. Kotlin Flow 46 98 39. Variable fonts 89. LitElement 48 47 96 97 90. Next.js Assess 91. On-demand modules 49 95 40. Apache Pinot 92. Streamlit 51 41. Bit.dev 93. SWR 42. DataHub 94. TrustKit 43. Feature Store 44. JuiceFS Assess 45. Kafka API without Kafka 95. .NET 5 46. NATS 96. bUnit 47. Opstrace 97. Dagster 48. Pulumi 98. Flutter for Web 49. Redpanda 99. Jotai and Zustand 100. Kotlin Multiplatform Mobile Hold 101. LVGL 50. Azure Machine Learning 102. React Hook Form 51. Homemade infrastructure-as-code (IaC) products 103. River New Moved in/out No change 104. Webpack 5 Module Federation

Hold TECHNOLOGY RADAR Techniques Adopt 1. API expand-contract Techniques 33 2. Continuous delivery for machine learning (CD4ML) 32 3. Design systems API expand-contract 4. Platform engineering product Adopt 31 25 teams 26 68 24 5. Service account rotation approach The API expand-contract pattern, sometimes 69 70 71 80 called parallel change, will be familiar to 23 30 Trial many, especially when used with databases 726. Cloud sandboxes or code; however, we only see low levels of 73 22 7. Contextual bandits adoption with APIs. Specifically, we’re seeing 16 54 8. Distroless74 Docker images complex versioning schemes and breaking 29 21 15 57 58 9. Ethical Explorer changes used in scenarios where a simple 12 13 53 56 10. Hypothesis-driven legacy expand and then contract would suffice. 75 20 renovation For example, first adding to an API while 11 14 55 59 60 11. Lightweight76 approach to RFCs deprecating an existing element, and then 28 10 12.61 Simplest possible ML only later removing the deprecated elements 19 13. SPA injection 77 5 63 once consumers are switched to the newer 62 14. Team cognitive load schema. This approach does require 9 4 18 15. 64Tool-managed78 Xcodeproj some coordination and visibility of the API 27 3 16. UI/BFF shared types consumers, perhaps through a technique 7 8 66 52 65 79 such as consumer-driven contract testing. 17 2 6 1 Assess67 17. Bounded low-code platforms Hold Assess Trial Adopt Adopt 18. TrialDecentralized identityAssess Hold Continuous delivery for 19. Deployment drift radiator machine learning (CD4ML) 20. Homomorphic encryption Adopt 94 40 34 21. Hotwire93 104 82 22. Import maps for micro frontends We see continuous delivery for machine challenge to deliver accessible and usable in product development because they allow81 23. 92Open Application Model (OAM) learning (CD4ML) as a good default products with consistent style.41 This is teams to focus. They can address strategic 24. Privacy-focused web analytics 103 starting point for any ML solution that is particularly true in larger organizations 35 challenges around the product itself without 25. Remote mob programming 88 90 91 being deployed into production. Many with multiple teams working on different reinventing36 the wheel every time a new 26. Secure multiparty computing 42 87 organizations are becoming more reliant on products. Design systems define a collection visual component is needed. 102 ML solutions for both customer offerings of design patterns, component libraries 86 Hold89 101 and internal operations so it makes sound and good design and engineering practices43 27. GitOps business sense to apply the lessons and that ensure consistent digital50 products. 38 28. Layered platform teams Platform37 engineering 85 good practice captured by continuous Built on the corporate style guides of the44 product teams 83 29. Naive password complexity delivery (CD) to ML solutions. past, design systems offer shared libraries requirements 45 Adopt 84 and documents that are easy to find and 39 30. Peer 100review equals pull request use. Generally, guidance is written down as As noted in one of the themes for this 31. SAFe™99 Design systems code and kept under version control so that edition,46 the industry is increasingly gaining 32.98 Separate code and pipeline Adopt the guide is less ambiguous and easier to experience with48 platform engineering ownership 47 96 97 maintain than simple documents. Design product teams that create and support 33. Ticket-driven platform operating As application development becomes systems have become a standard approach internal platforms. These49 platforms are95 models increasingly dynamic and complex, it’s a when working across teams and disciplines 51used by teams across an organization

8 | TECHNOLOGY RADAR © Thoughtworks, Inc. All Rights Reserved. and accelerate application development, maintenance complexity. We’re seeing that two things: the security and the size Techniques reduce operational complexity and improve the tooling to do local simulation of cloud- of the image. Traditionally, we’ve used time to market. With increasing adoption native services limits the confidence in container security scanning tools to detect we’re also clearer on both good and bad developer build and test cycles; therefore, and patch common vulnerabilities and patterns for this approach. When creating a we’re looking to focus on standardizing exposures and small distributions such as Named after “bandits,” or platform, it’s critical to have clearly defined cloud sandboxes over running cloud-native Alpine Linux to address the image size and slot machines, in casinos, customers and products that will benefit components on a developer machine. distribution performance. But with rising this algorithm explores from it rather than building in a vacuum. We This will drive good infrastructure-as-code security threats, eliminating all possible different options to learn caution against layered platform teams that practices as a forcing function and good attack vectors is more important than ever. more about expected simply preserve existing technology silos onboarding processes for provisioning That’s why distroless Docker images are outcomes and balances but apply the “platform team” label as well sandbox environments for developers. There becoming the default choice for deployment as against ticket-driven platform operating are risks associated with this transition, containers. Distroless Docker images by exploiting the options models. We’re still big fans of using concepts as it assumes that developers will have an reduce the footprint and dependencies by that perform well. from Team Topologies as we think about absolute dependency on cloud environment doing away with a full operating system how best to organize platform teams. We availability, and it may slow down the distribution. This technique reduces security (Contextual bandits) consider platform engineering product developer feedback loop. We strongly scan noise and the application attack teams to be a standard approach and a recommend you adopt some lean governance surface. Moreover, fewer vulnerabilities significant enabler for high-performing IT. practices regarding standardization of these need to be patched and as a bonus, these sandbox environments, especially with regard smaller images are more efficient. Google to security, IAM and regional deployments. has published a set of distroless container Service account images for different languages. You can rotation approach create distroless application images using Adopt Contextual bandits the Google build tool Bazel or simply use Trial multistage Dockerfiles. Note that distroless We strongly advise organizations to make containers by default don’t have a shell for sure, when they really need to use cloud Contextual bandits is a type of debugging. However, you can easily find service accounts, that they are rotating reinforcement learning that is well suited debug versions of distroless containers the credentials. Rotation is one of the for problems with exploration/exploitation online, including a BusyBox shell. Distroless three R’s of security. It is far too easy trade-offs. Named after “bandits,” or Docker images is a technique pioneered by for organizations to forget about these slot machines, in casinos, the algorithm Google and, in our experience, is still largely accounts unless an incident occurs. This explores different options to learn more confined to Google-generated images. is leading to accounts with unnecessarily about expected outcomes and balances it We would be more comfortable if there broad permissions remaining in use for long by exploiting the options that perform well. were more than one provider to choose periods alongside a lack of planning for how We’ve successfully used this technique in from. Also, use caution when applying to replace or rotate them. Regularly applying scenarios where we’ve had little data to Trivy or similar vulnerability scanners since a cloud service account rotation approach train and deploy other machine-learning distroless containers are only supported in also provides a chance to exercise the models. The fact that we can add context more recent versions. principle of least privilege. to this explore/exploit trade-off makes it suitable for a wide variety of use cases including A/B testing, recommendations Ethical Explorer Cloud sandboxes and layout optimizations. Trial Trial The group behind Ethical OS — the As the cloud is becoming more and more a Distroless Docker images Omidyar Network, a self-described commodity and being able to spin up cloud Trial social change venture created by eBay sandboxes is easier and available at scale, founder Pierre Omidyar — has released our teams prefer cloud-only (as opposed to When building Docker images for our a new iteration called Ethical Explorer. local) development environments to reduce applications, we’re often concerned with The new Ethical Explorer pack draws

TECHNOLOGY RADAR | 9 © Thoughtworks, Inc. All Rights Reserved. on lessons learned from using Ethical problem. They then conduct iterative, time- be understood and operated, and with each OS and adds further questions for boxed experiments to verify or disprove new tool added to the architecture this tax Techniques product teams to consider. The kit, each hypothesis in order of priority. burden increases. In our experience, teams which can be downloaded for free and The resulting workflow is optimized for often choose complex tools because they folded into cards to trigger discussion, reducing uncertainty rather than following underestimate the power of simpler tools has open-ended question prompts a plan toward a predictable outcome. such as linear regression. Many ML problems When it comes to legacy for several technical “risk zones,” don’t require a GPU or neural networks. For modernization in single- including surveillance (“can someone that reason we advocate for the simplest page apps (SPAs), instead use our product or service to track or Lightweight approach to RFCs possible ML, using simple tools and models of wrapping the legacy identify other users?”), disinformation, Trial and a few hundred lines of Python on the system we instead embed exclusion, algorithmic bias, addiction, compute platform you have at hand. Only the beginning of the data control, bad actors and outsized As organizations drive toward evolutionary reach for the complex tools when you can power. The included field guide has architecture, it’s important to capture demonstrate the need for them. new SPA into the HTML activities and workshops, ideas for decisions around design, architecture, document containing the starting conversations and tips for techniques and teams’ ways of workings. old one and let it slowly gaining organizational buy-in. While we’ve The process of collecting and aggregating SPA injection expand in functionality. a long way to go as an industry to better feedback that will lead to these decisions Trial represent the ethical externalities of our begin with Request for Comments (SPA Injection) digital society, we’ve had some productive (RfCs). RfCs are a technique for collecting The strangler fig pattern is often the default conversations using Ethical Explorer, context, design and architectural ideas strategy for legacy modernization, where the and we’re encouraged by the broadening and collaborating with teams to ultimately new code wraps around the old and slowly awareness of the importance of product come to decisions along with their context absorbs the ability to handle all the needed decisions in addressing societal issues. and consequences. We recommend that functionality. That sort of “outside-in” organizations take a lightweight approach approach works well for a number of legacy to RFCs by using a simple standardized systems, but now that we’ve had enough Hypothesis-driven template across many teams as well as experience with single-page applications legacy renovation version control to capture RfCs. (SPA) for them to become legacy systems Trial themselves, we’re seeing the opposite It’s important to capture these in an audit “inside-out” approach used to replace them. We’re often asked to refresh, update or of these decisions to benefit future team Instead of wrapping the legacy system, we remediate legacy systems that we didn’t members and to capture the technical instead embed the beginning of the new originally build. Sometimes, technical and business evolution of an organization. SPA into the HTML document containing issues need our attention such as Mature organizations have used RfCs the old one and let it slowly expand in improving performance or reliability. One in autonomous teams to drive better functionality. The SPA frameworks don’t common approach to address these issues communication and collaboration especially even need to be the same as long as users is to create “technical stories” using the in cross-team relevant decisions. can tolerate the performance hit of the same format as a user story but with a increased page size (e.g., embedding a new technical outcome rather than a business React app inside an old AngularJS one). SPA one. But these technical tasks are often Simplest possible ML injection allows you to iteratively remove the difficult to estimate, take longer than Trial old SPA until the new one completely takes anticipated or don’t end up having the over. Whereas a strangler fig can be viewed desired outcome. An alternative, more All major cloud providers offer a dazzling as a type of parasite that uses the host tree’s successful method is to apply hypothesis- array of machine-learning (ML) solutions. stable external surface to support itself until driven legacy renovation. Rather than These powerful tools can provide a lot of it takes root and the host itself dies, this working toward a standard backlog, the value, but come at a cost. There is the pure approach is more like injecting an outside team takes ownership of a measurable run cost for these services charged by the agent into the host, relying on functionality technical outcome and collectively cloud provider. In addition, there is a kind of of the original SPA until it can completely establishes a set of hypotheses about the operations tax. These complex tools need to take over.

10 | TECHNOLOGY RADAR © Thoughtworks, Inc. All Rights Reserved. Team cognitive load UI/BFF shared types The problems we see with these platforms Techniques Trial Trial typically relate to an inability to apply good engineering practices such as versioning. A system’s architecture mimics With TypeScript becoming a common Testing too is typically really hard. However, organizational structure and its language for front-end development we noticed some interesting new entrants Team interaction is one of communication. It’s not big news that we and Node.js becoming the preferred BFF to the market — including Amazon the variables that impacts should be intentional about how teams technology, we’re seeing increasing use Honeycode, which makes it easy to create speed and the ease with interact — see, for instance, the Inverse of UI/BFF shared types. In this technique, simple task or event management apps, which teams deliver value Conway Maneuver. Team interaction is one a single set of type definitions is used to and Parabola for IFTTT-like cloud workflows to their customers. The of the variables for how fast and how easily define both the data objects returned by — which is why we’re once again including. Team Topologies author teams can deliver value to their customers. front-end queries and the data served bounded low-code platforms in this volume. to satisfy those queries by the back- Nevertheless, we remain deeply skeptical developed an assessment We were happy to find a way to measure these interactions; we used the Team end server. Ordinarily, we would be about their wider applicability since these for measuring these Topologies author’s assessment which cautious about this practice because of tools, like Japanese Knotweed, have a knack interactions which we call gives you an understanding of how easy the unnecessarily tight coupling it creates of escaping their bounds and tangling team cognitive load. or difficult the teams find it to build, test across process boundaries. However, everything together. That’s why we still and maintain their services. By measuring many teams are finding that the benefits strongly advise caution in their adoption. (Team cognitive load) team cognitive load, we could better advise of this approach outweigh any risks of our clients on how to change their teams’ tight coupling. Since the BFF pattern works structure and evolve their interactions. best when the same team owns both the Decentralized identity UI code and the BFF, often storing both Assess components in the same repository, the UI/ Tool-managed Xcodeproj BFF pair can be viewed as a single cohesive In 2016, Christopher Allen, a key Trial system. When the BFF offers strongly typed contributor to SSL/TLS, inspired us queries, the results can be tailored to with an introduction of 10 principles Many of our developers coding iOS in the specific needs of the frontend rather underpinning a new form of digital identity Xcode often get headaches because the than reusing a single, general-purpose and a path to get there, the path to self- Xcodeproj file changes with every project entity that must serve the needs of many sovereign identity. Self-sovereign identity, change. The Xcodeproj file format is not consumers and contain more fields than also known as decentralized identity, human-readable, hence trying to handle actually required. This reduces the risk is a “lifetime portable identity for any merge conflicts is quite complicated and of accidentally exposing data that the person, organization, or thing that does can lead to productivity loss and risk user shouldn’t see, prevents incorrect not depend on any centralized authority of messing up the entire project — if interpretation of the returned data object and can never be taken away,” according anything goes wrong with the file, Xcode and makes the query more expressive. to the Trust over IP standard. Adopting won’t work properly and developers This practice is particularly useful when and implementing decentralized identity will very likely be blocked. Instead of implemented with io-ts to enforce the run- is gaining momentum and becoming trying to merge and fix the file manually time type safety. attainable. We see its adoption in privacy- or version it, we recommend you use respecting customer health applications, a tool-managed Xcodeproj approach: government healthcare infrastructure Define your Xcode project configuration Bounded low-code platforms and corporate legal identity. If you want in YAML (XcodeGen, Struct), Ruby (Xcake) Assess to rapidly get started with decentralized or Swift (Tuist). These tools generate the identity, you can assess Sovrin Network, Xcodeproj file based on a configuration One of the most nuanced decisions facing Hyperledger Aries and Indy OSS, as well file and the project structure. As a result, companies at the moment is the adoption as decentralized identifiers and verifiable merge conflicts in the Xcodeproj file will of low-code or no-code platforms, that is, credentials standards. We’re watching this be a thing of the past, and when they do platforms that solve very specific problems space closely as we help our clients with happen in the configuration file, they’re in very limited domains. Many vendors their strategic positioning in the new era of much easier to handle. are pushing aggressively into this space. digital trust.

TECHNOLOGY RADAR | 11 © Thoughtworks, Inc. All Rights Reserved. Deployment drift radiator intermediate federated machine learning Import maps for Assess results. Moreover, most HE schemes are micro frontends Techniques considered to be secure against quantum Assess A deployment drift radiator makes version computers, and efforts are underway drift visible for deployed software across to standardize HE. Despite its current When composing an application out of multiple environments. Organizations limitations, namely performance and several micro frontends, some part of Organizations using using automated deployments may require feasibility of the types of computations, HE the system needs to decide which micro automated deployments manual approvals for environments that is worth your attention. frontends to load and where to load sometimes require get closer to production, meaning the them from. So far, we’ve either built manual approvals for code in these environments might well be custom solutions or relied on a broader environments that get lagging several versions behind current Hotwire framework like single-spa. Now there closer to production, development. This technique makes this lag Assess is a new standard, import maps, that leading the code in visible via a simple dashboard showing how helps in both cases. Our first experiences far behind each deployed component is for Hotwire (HTML over the wire) is a technique show that using import maps for micro these environments each environment. This helps to highlight to build web applications. Pages are frontends allows for a neat separation to lag behind current the opportunity cost of completed software constructed out of components, but unlike of concerns. The JavaScript code states development. A not yet in production while drawing modern SPAs the HTML for the components what to import and a small script tag in deployment drift radiator attention to related risks such as security is generated on the server side and then the initial HTML response specifies where makes this lag visible via a fixes not yet deployed. sent “over the wire” to the browser. The to load the frontends from. That HTML simple dashboard. application has only a small amount of is obviously generated on the server

JavaScript code in the browser to stitch the side, which makes it possible to use (Deployment drift radiator) Homomorphic encryption HTML fragments together. Our teams, and some dynamic configuration during its Assess doubtlessly others too, experimented with rendering. In many ways this technique this technique after asynchronous web reminds us of linker/loader paths for Fully homomorphic encryption (HE) requests gained cross-browser support dynamic Unix libraries. At the moment refers to a class of encryption methods around 2005, but for various reasons it never import maps are only supported by that allow computations (such as search gained much traction. Chrome, but with the SystemJS polyfill and arithmetic) to be performed directly they’re ready for wider use. on encrypted data. The result of such a Today, Hotwire uses modern web browser computation remains in encrypted form, and HTTP capabilities to achieve the speed, which at a later point can be decrypted and responsiveness and dynamic nature of Open Application Model (OAM) revealed. Although the HE problem was single-page apps (SPAs). It embraces simpler Assess first proposed in 1978, a solution wasn’t web application design by localizing the logic constructed until 2009. With advances to the server and keeping the client-side code The Open Application Model (OAM) is an in computing power and the availability simple. The team at Basecamp has released attempt to bring some standardization of easy-to-use open-source libraries — a few Hotwire frameworks that power to the space of shaping infrastructure including SEAL, Lattigo, HElib and partially their own application, including Turbo and platforms as products. Using the homomorphic encryption in Python — Stimulus. Turbo includes a set of techniques abstractions of components, application HE is becoming feasible in real-world and frameworks to speed up the application configurations, scopes and traits, applications. The motivating scenarios responsiveness by preventing whole page developers can describe their applications include privacy-preserving use cases, reloading, page preview from cache and in a platform-agnostic way, while platform where computation can be outsourced decomposing the page into fragments with implementers define their platform in to an untrusted party, for example, progressive enhancements on request. terms of workload, trait and scope. Since running computation on encrypted data Stimulus is designed to enhance static HTML we last talked about the OAM, we’ve in the cloud, or enabling a third party to in the browser by connecting JavaScript followed one of its first implementations aggregate homomorphically encrypted objects to the page elements on the HTML. with interest, KubeVela. KubeVela is close

12 | TECHNOLOGY RADAR © Thoughtworks, Inc. All Rights Reserved. to release 1.0, and we’re curious to see if that protects privacy between parties Layered platform teams Techniques implementations like this can substantiate that do not trust each other. It’s aim is to Hold the promise of the OAM idea. safely calculate an agreed-upon problem without a trusted third party, while each The explosion of interest around software participant is required to partake in the platforms has created a lot of value for Secure multiparty Privacy-focused web analytics calculation result and can’t be obtained organizations, but the path to building a computing solves the Assess by other entities. A simple illustration platform-based delivery model is fraught problem of collaborative for MPC is the millionaires’ problem: two with potential dead ends. It’s common in computing that protects Privacy-focused web analytics is a technique millionaires want to understand who is the excitement of new paradigms to see a privacy between parties for gathering web analytics without the richest, but neither want to share resurgence of older techniques rebranded compromising end user privacy by keeping that do not trust each their actual net worth with each other nor with the new vernacular, making it easy to the end users truly anonymous. One trust a third party. The implementation lose sight of the reasons we moved past other without involving a surprising consequence of General Data approaches of MPC vary; scenarios those techniques in the first place. For an third party. Protection Regulation (GDPR) compliance is may include secret sharing, oblivious example of this rebranding, see our blip on the decision taken by many organizations to transfer, garbled circuits or homomorphic traditional ESBs make a comeback as API (Secure multiparty degrade the user experience with complex encryption. Some commercial MPC gateways in the previous Radar. Another computing) cookie consent processes, especially when solutions that have recently appeared example we’re seeing is rehashing the the user doesn’t immediately consent to the (e.g., Antchain Morse) claim to help solve approach of dividing teams by technology “all the cookies” default settings. Privacy- the problems of secret sharing and secure layer but calling them platforms. In the focused web analytics has the dual benefit of machine learning in scenarios such as context of building an application, it used both observing the spirit and letter of GDPR multiparty joint credit investigation and to be common to have a front-end team while also avoiding the need to introduce medical records data exchange. Although separate from the business logic team intrusive cookie consent forms. One these platforms are attractive from a separate from the data team, and we see implementation of this approach is Plausible. marketing perspective, we’ve yet to see analogs to that model when organizations whether they’re really useful. segregate platform capabilities among teams dedicated to a business or data layer. Thanks Remote mob programming to Conway’s Law, we know that organizing Assess GitOps platform capability teams around business Hold capabilities is a more effective model, giving Mob programming is one of those the team end-to-end ownership of the techniques that our teams have found to We suggest approaching GitOps with a capability, including data ownership. This be easier when done remotely. Remote degree of care, especially with regard to helps to avoid the dependency management mob programming is allowing teams to branching strategies. GitOps can be seen headaches of layered platform teams, with quickly “mob” around an issue or piece as a way of implementing infrastructure the front-end team waiting on the business of code without the physical constraints as code that involves continuously logic team waiting on the data team to get of only being able to fit so many people synchronizing and applying infrastructure anything done. around a pairing station. Teams can quickly code from Git into various environments. collaborate on an issue or piece of code When used with a “branch per environment” without having to connect to a big display, infrastructure, changes are promoted from Naive password complexity book a physical meeting room or find a one environment to the next by merging requirements whiteboard. code. While treating code as the single Hold source of truth is clearly a sound approach, we’re seeing branch per environment Password policies are a standard default Secure multiparty computing lead to environmental drift and eventually for many organizations today. However, Assess environment-specific configs as code merges we’re still seeing organizations requiring become problematic or even stop entirely. passwords to include a variety of symbols, Secure multiparty computing (MPC) solves This is very similar to what we’ve seen in the numbers, uppercase and lowercase letters the problem of collaborative computing past with long-lived branches with GitFlow. as well as inclusion of special characters.

TECHNOLOGY RADAR | 13 © Thoughtworks, Inc. All Rights Reserved. These are naive password complexity Framework®), per Gartner’s May 2019 and a lack of development team ownership requirements that lead to a false sense of report, is the most considered and most and involvement in deployments. One Techniques security as users will opt for more insecure used enterprise agile framework, and since cause of this can clearly be the separate passwords because the alternative is we’re seeing more and more enterprises team, another can be the desire to retain difficult to remember and type. According going through organizational changes, we “gatekeeper” processes and roles. Although to NIST recommendations, the primary thought it was time to raise awareness there can be legitimate reasons for using this Some organizations factor in password strength is password on this topic again. We’ve come across approach (e.g., regulatory control), in general seem to think peer review length, and therefore users should choose organizations struggling with SAFe’s over- we find it painful and unhelpful. equals pull request. long passphrases with a maximum standardized, phase-gated processes. We’ve seen this approach requirement of 64 characters (including Those processes create friction in the create significant team spaces). These passphrases are more organizational structure and its operating Ticket-driven platform bottlenecks as well as secure and memorable. model. It can also promote silos in the operating models significantly degrade the organization, preventing platforms from Hold becoming real business capabilities quality of feedback. Peer review equals pull request enablers. The top-down control generates One of the ultimate goals of a platform Hold waste in the value stream and discourages should be to reduce ticket-based processes (Peer review equals pull engineering talent creativity, while limiting to an absolute minimum, as they create request) Some organizations seem to think peer autonomy and experimentation in the queues in the value stream. Sadly, review equals pull request; they’ve taken teams. Rather than measuring effort and we still see organizations not pushing the view that the only way to achieve a peer focusing on standardized ceremonies, forcefully enough toward this important review of code is via a pull request. We’ve we recommend a leaner, value-driven goal, resulting in a ticket-driven platform seen this approach create significant team approach and governance to help eliminate operating model. This is particularly bottlenecks as well as significantly degrade organizational friction such as EDGE, as frustrating when ticket-based processes the quality of feedback as overloaded well as a team cognitive load assessment to are put in front of platforms that are built reviewers begin to simply reject requests. identify types of teams and determine how on top of the self-service and API-driven Although the argument could be made that they should better interact with each other. features of public cloud vendors. It’s hard this is one way to demonstrate code review and not necessary to achieve self-service “regulatory compliance” one of our clients Scaled Agile Framework® and SAFe™ are with very few tickets right from the start, was told this was invalid since there was trademarks of Scaled Agile, Inc. but it needs to be the destination. no evidence the code was actually read by anyone prior to acceptance. Pull requests Over-reliance on bureaucracy and lack are only one way to manage the code review Separate code and of trust are among the causes of this workflow; we urge people to consider other pipeline ownership reluctance to move away from ticket-based approaches, especially where there is a need Hold processes. Baking more automated checks to coach and pass on feedback carefully. and alerts into your platform is one way to Ideally, but especially when teams are help cut the cord from approval processes practicing DevOps, the deployment pipeline with tickets. For example, provide teams SAFe™ and the code being deployed should be with visibility into their run costs and put in Hold owned by the same team. Unfortunately, we automated guardrails to avoid accidental still see organizations where there is separate explosion of costs. Implement security Our positioning regarding “being agile code and pipeline ownership, with the policy as code and use configuration before doing agile” and our opinions deployment pipeline configuration owned scanners or analyzers like Recommender to around this topic shouldn’t come as a by the infrastructure team; this results in help teams do the right thing. surprise; but since SAFe™ (Scaled Agile delays to changes, barriers to improvements

31 25 26 68 24 69 70 71 80 23 30 72 73 22 16 54 74 29 21 15 57 58 12 13 53 56 75 20 11 55 14 60 28 59 76 10 61 19 77 5 63 62 9 4 18 64 78 27 3 7 8 66 52 65 79 17 2 6 1 67 Adopt Hold Assess Trial Adopt Adopt Trial Assess Hold Trial Platforms 34. AWS94 Cloud 40 34 93 104 82 Development Kit 81 35. Backstage AWS Cloud Development Kit 92 41 36. Delta Lake Trial 35 37. Materialize 103 88 90 91 36 38. Snowflake Many of our teams who are already on 42 87 39. Variable fonts102 AWS have found AWS Cloud Development 86 89 Kit (AWS CDK) to be a sensible AWS default 43 Assess 101 for enabling infrastructure provisioning. 50 38 40. Apache Pinot In particular, they like the use of first- 37 85 44 83 41. Bit.dev class programming languages instead of 42. DataHub configuration files which allows them to use 45 84 39 43. Feature100 Store existing tools, test approaches and skills. Like 44. JuiceFS similar tools, care is still needed to ensure 99 46 98 45. Kafka API without Kafka deployments remain easy to understand 46. NATS 48 97 and maintain. The development kit currently 47 96 47. Opstrace supports TypeScript, JavaScript, Python, Java, 49 48. Pulumi C# and .NET. New providers are being added 95 49. Redpanda to the CDK core. We’ve also used both AWS 51 Cloud Development Kit and HashiCorp’s Hold Cloud Development Kit for Terraform to 50. Azure Machine Learning generate Terraform configurations and 51. Homemade enable provisioning with the Terraform infrastructure-as-code platform with success. (IaC) products

Backstage Trial

We continue to see interest in and use of and consistent and centralized technical Lake storage over the direct use of file Backstage grow, alongside the adoption documentation. The plugin architecture storage types such S3 or ADLS. Of course of developer portals, as organizations allows for extensibility and adaptability into this is limited to projects that use storage look to support and streamline their an organization’s infrastructure ecosystem. platforms that support Delta Lake when development environments. As the number using Parquet file formats. Delta Lake of tools and technologies increases, some facilitates concurrent data read/write use form of standardization is becoming Delta Lake cases where file-level transactionality is increasingly important for consistency Trial required. We find Delta Lake’s seamless so that developers are able to focus on integration with Apache Spark batch innovation and product development Delta Lake is an open-source storage and micro-batch APIs greatly helpful, instead of getting bogged down with layer, implemented by Databricks, that particularly features such as time travel — reinventing the wheel. Backstage is an attempts to bring ACID transactions to accessing data at a particular point in time open-source developer portal platform big data processing. In our Databricks- or commit reversion — as well as schema created by Spotify, it’s based upon software enabled data lake or data mesh projects, evolution support on write; though there templates, unifying infrastructure tooling our teams continue to prefer using Delta are some limitations on these features.

16 | TECHNOLOGY RADAR © Thoughtworks, Inc. All Rights Reserved. Materialize a multicloud setup Snowflake is a better Apache Pinot can be fairly complex to Platforms Trial choice. We can also report that we’ve used operate and has many moving parts, but Snowflake successfully with GCP, AWS, if your data volumes are large enough and Materialize is a streaming database that and Azure. you need low-latency query capability, we enables you to do incremental computation recommend you assess Apache Pinot. LinkedIn has evolved without complicated data pipelines. Just WhereHows to DataHub, describe your computations via standard Variable fonts the next-generation SQL views and connect Materialize to the Trial Bit.dev platform that addresses data stream. The underlying differential Assess data discoverability data flow engine performs incremental Variable fonts are a way of avoiding the via an extensible computation to provide consistent and need to find and include separate font Bit.dev is a cloud-hosted collaborative metadata system. correct output with minimal latency. files for different weights and styles. platform for UI components extracted, Unlike traditional databases, there are no Everything is in one font file, and you can modularized and reused with Bit. Web restrictions in defining these materialized use properties to select which style and components have been around for a while, (DataHub) views, and the computations are executed in weight you need. While not new, we still but building a modern front-end application real time. We’ve used Materialize, together see sites and projects that could benefit by assembling small, independent with Spring Cloud Stream and Kafka, to from this simple approach. If you have components extracted from other projects query over streams of events for insights in pages that are including many variations has never been easy. Bit was designed to a distributed event-driven system, and we of the same font, we suggest trying out let you do exactly that: extract a component quite like the setup. variable fonts. from an existing library or project. You can either build your own service on top of Bit for component collaboration or use Bit.dev. Snowflake Apache Pinot Trial Assess DataHub Since we last mentioned Snowflake in Apache Pinot is a distributed OLAP data Assess the Radar, we’ve gained more experience store, built to deliver real-time analytics with it as well as with data mesh as an with low latency. It can ingest from batch Since we first mentioned data discoverability alternative to data warehouses and lakes. data sources (such as Hadoop HDFS, in the Radar, LinkedIn has evolved Snowflake continues to impress with Amazon S3, Azure ADLS or Google Cloud WhereHows to DataHub, the next features like time travel, zero-copy cloning, Storage) as well as stream data sources generation platform that addresses data data sharing and its marketplace. We also (such as Apache Kafka). If the need is discoverability via an extensible metadata haven’t found anything we don’t like about user-facing, low-latency analytics, SQL- system. Instead of crawling and pulling it, all of which has led to our consultants on-Hadoop solutions don’t offer the low metadata, DataHub adopts a push-based generally preferring it over the alternatives. latency that is needed. Modern OLAP model where individual components of the Redshift is moving toward storage and engines like Apache Pinot (or Apache data ecosystem publish metadata via an compute separation, which has been a Druid and Clickhouse among others) API or a stream to the central platform. This strong point of Snowflake, but even with can achieve much lower latency and are push-based integration shifts the ownership Redshift Spectrum it isn’t as convenient and particularly suited in contexts where from the central entity to individual flexible to use, partly because it is bound fast analytics, such as aggregations, are teams making them accountable for their by its Postgres heritage (we do still like needed on immutable data, possibly, metadata. As more and more companies Postgres, by the way). Federated queries with real-time data ingestion. Originally are trying to become data driven, having a can be a reason to go with Redshift. When built by LinkedIn, Apache Pinot entered system that helps with data discovery and it comes to operations, Snowflake is much Apache incubation in late 2018 and has understanding data quality and lineage simpler to run. BigQuery, which is another since added a plugin architecture and SQL is critical, and we recommend you assess alternative, is very easy to operate, but in support among other key capabilities. DataHub in that capacity.

TECHNOLOGY RADAR | 17 © Thoughtworks, Inc. All Rights Reserved. Feature Store concept in scalable persistent messaging, support a continuous streaming flow of Assess a lot of moving parts are required to make data from mobile devices and IoT and Platforms it work, including ZooKeeper, brokers, through a network of interconnected Feature Store is an ML-specific data platform partitions, and mirrors. While these can systems. However, some tricky issues that addresses some of the key challenges be particularly tricky to implement and need to be addressed, not the least of we face today in feature engineering with operate, they do offer great flexibility which is ensuring consumers see only As more businesses three fundamental capabilities: (1) it uses and power when needed, especially at the messages and topics to which they’re turn to events as a way managed data pipelines to remove struggles an industrial enterprise scale. Because allowed access, especially when the network to share data among with pipelines as new data arrives; (2) of the high barrier to entry presented by spans organizational boundaries. NATS 2.0 microservices, collect catalogs and stores feature data to promote the full Kafka ecosystem, we welcome the introduced a security and access control analytics or feed data discoverability and collaboration of features recent explosion of platforms offering the framework that supports multitenant lakes, Apache Kafka Kafka API without Kafka. Recent entries clusters where accounts restrict a user’s across models; and (3) consistently serves has become a favorite feature data during training and interference. such as Kafka on Pulsar and Redpanda access to queues and topics. Written in offer alternative architectures, and Azure Go, NATS has primarily been embraced platform. Because of Since Uber revealed their Michelangelo Event Hubs for Kafka provides some by the Go language community. Although the high barrier to platform, many organizations and startups compatibility with Kafka producer and clients exist for pretty much all widely used entry presented by the have built their own versions of a feature consumer APIs. Some features of Kafka, programming languages, the Go client is full Kafka ecosystem, store; examples include Hopsworks, Feast like the streams client library, are not by far the most popular. However, some we welcome the recent and Tecton. We see potential in Feature compatible with these alternative brokers, of our developers have found that all the explosion of platforms so there are still reasons to choose Kafka language client libraries tend to reflect Store and recommend you carefully offering the Kafka API assess it. over alternative brokers. It remains to the Go origins of codebase. Increasing be seen, however, if developers actually bandwidth and processing power on small, without Kafka. adopt this strategy or if it is merely an wireless devices means that the volume of JuiceFS attempt by competitors to lure users data businesses must consume in real time (Kafka API without Kafka) Assess away from the Kafka platform. Ultimately, will only increase. Assess NATS as a possible perhaps Kafka’s most enduring impact platform for streaming that data within and JuiceFS is an open-source, distributed could be the convenient protocol and API among businesses. POSIX file system built on top of Redis provided to clients. and an object store service (for example, Amazon S3). If you’re building new Opstrace applications, then our recommendation NATS Assess has always been to interact directly with Assess the object store without going through Opstrace is an open-source observability another abstraction layer. However, JuiceFS NATS is a fast, secure message queueing platform intended to be deployed in can be an option if you’re migrating legacy system with an unusually wide range of the user’s own network. If we don’t use applications that depend on traditional features and potential deployment targets. commercial solutions like Datadog (for POSIX file systems to the cloud. At first glance, you would be forgiven example, because of cost or data residency for asking why the world needs another concerns), the only solution is to build message queueing system. Message queues your own platform composed of open- Kafka API without Kafka have been around in various forms for source tools. This can take a lot of effort Assess nearly as long as businesses have been — Opstrace is intended to fill this gap. It using computers and have undergone years uses open-source APIs and interfaces such As more businesses turn to events as a of refinement and optimization for various as Prometheus and Grafana and adds way to share data among microservices, tasks. But NATS has several interesting additional features on top like TLS and collect analytics or feed data lakes, Apache characteristics and is unique in its ability authentication. At the heart of Opstrace Kafka has become a favorite platform to scale from embedded controllers to runs a Cortex cluster to provide the scalable to support an event-driven architectural global, cloud-hosted superclusters. We’re Prometheus API as well as a Loki cluster style. Although Kafka was a revolutionary particularly intrigued by NATS’s intent to for the logs. It’s fairly new and still misses

18 | TECHNOLOGY RADAR © Thoughtworks, Inc. All Rights Reserved. features when compared to solutions like having to deal with the complexities of a Homemade infrastructure-as- Platforms Datadog or SignalFX. Still, it’s a promising Kafka installation. For example, Redpanda code (IaC) products addition to this space and worth keeping simplifies operations by shipping as a single Hold an eye on. binary and avoiding the need for an external dependency such as ZooKeeper. Instead, it Sometimes organizations Products supported by companies or implements the Raft protocol and performs communities are in constant evolution, build frameworks or Pulumi comprehensive tests to validate it’s been at least the ones that get traction in the abstractions on top Assess implemented correctly. One of Redpanda’s industry. Sometimes organizations tend to of existing external capabilities (available for enterprise build frameworks or abstractions on top of products to cover very We’ve seen interest in Pulumi slowly customers only) is inline WebAssembly the existing external products to cover very specific needs, thinking but steadily rising. Pulumi fills a gaping (WASM) transformations, using an specific needs, thinking that the adaptation embedded WASM engine. This allows that the adaptation hole in the infrastructure coding world will provide more benefits than the existing where Terraform maintains a firm hold. developers to create event transformers in will provide them more ones. We’re seeing organizations trying to While Terraform is a tried-and-true their language of choice and compile it to create homemade infrastructure-as-code benefits. However, they standby, its declarative nature suffers WASM. Redpanda also offers much reduced (IaC) products on top of the existing ones; underestimate the from inadequate abstraction facilities and tail latencies and increased throughput due they underestimate the required effort to required effort to keep limited testability. Terraform is adequate to a series of optimizations. Redpanda is an keep those solutions evolving according those solutions evolving when the infrastructure is entirely static, exciting alternative to Kafka and is to their needs, and after a short period of according to their needs, but dynamic infrastructure definitions worth assessing. time, they realize that the original version and soon realize that the call for a real programming language. is in much better shape than their own; Pulumi distinguishes itself by allowing original version is in there are even cases where the abstraction configurations to be written in TypeScript/ Azure Machine Learning on top of the external product reduces the much better shape JavaScript, Python and Go — no markup Hold original capabilities. Although we’ve seen than their own. language or templating required. Pulumi is success stories of organizations building tightly focused on cloud-native architectures We’ve observed before that the cloud homemade solutions, we want to caution (Homemade infrastructure-as- — including containers, serverless functions providers push more and more services about this approach as the effort required code (IaC) products) and data services — and provides good onto the market. We’ve also documented to do so isn’t negligible, and a long-term support for Kubernetes. Recently, AWS our concerns that sometimes the services product vision is required to have the CDK has mounted a challenge, but Pulumi are made available when they’re not quite expected outcomes. remains the only cloud-neutral tool in ready for prime time. Unfortunately, in our this area. We’re anticipating wider Pulumi experience, Azure Machine Learning falls adoption in the future and looking forward into the latter category. One of several to a viable tool and knowledge ecosystem recent entrants in the field of bounded emerging to support it. low-code platforms, Azure ML promises more convenience for data scientists. Ultimately, however, it doesn’t live up to its Redpanda promise; in fact, it still feels easier for our Assess data scientists to work in Python. Despite significant efforts, we struggled to make it Redpanda is a streaming platform that scale and lack of adequate documentation provides a Kafka-compatible API, allowing it proved to be another issue which is why we to benefit from the Kafka ecosystem without moved it to the Hold ring.

Tools 33 Trial 53. axe-core 32 54. dbt 55. esbuild Sentry 31 25 Adopt 26 68 56. Flipper 24 69 70 71 57. Great Expectations 80 Sentry has become the default23 choice 58. k6 for many of our30 teams when it comes 59. MLflow 72 to front-end error reporting. The 73 60. OR-Tools convenience of features like22 the grouping 61. Playwright 16 54 74 62. Prowler of errors29 or defining21 patterns for 15 57 discarding errors with certain parameters 58 63. Pyright 12 13 53 56 helps deal with the flood of errors coming 75 64. Redash from many end user20 devices.11 Integrating 55 65. Terratest 14 60 Sentry28 in your CD pipeline allows you to 59 76 66. Tuple 61 upload source maps for10 more efficient 67. Why Did You Render 19 77 error debugging, and it helps easily 5 63 62 trace back which errors occurred in Assess 9 4 which version18 of the software. We also 64 78 68. Buildah and Podman 27 3 appreciate that while Sentry is primarily a 69. GitHub Actions 7 8 66 SaaS offering, its source code is publicly 52 65 79 70. Graal Native Image 17 2 available and it can6 be used for free for 1 67 71. HashiCorp Boundary smaller use cases and self-hosting. 72. imgcook Hold Assess Trial Adopt Adopt Trial Assess Hold 73. Longhorn 74. Operator Framework axe-core 94 75. Recommender 40 34 93 76. Remote - WSL Trial 104 82 77. Spectral 81 Making the web inclusive requires 92 78. Yelp detect-secrets 41 serious attention to ensure accessibility Extending the functionality of axe-core is building the pipelines. It does this while 79. Zally 35 103 is considered and validated at all stages the commercially available axe DevTools88 90, 91encouraging good engineering practices 36 Hold of software42 delivery. Many of the popular including functionality that guides87 team such as versioning, automated testing accessibility testing tools are designed for members through exploratory testing for a and deployment.102 SQL continues to be the 80. AWS CodePipeline 86 89 testing after a web43 application is complete; majority of accessibility issues. lingua101 franca of the data world (including as a result, issues are detected late and databases, warehouses, query engines, data often50 are harder to fix, accumulating 38 85 lakes and analytical platforms) and most of 37 83 as debt. In our recent44 internal work on dbt these systems support it to some extent. Thoughtworks websites, we45 included the Trial 84 This allows dbt to be used against these 39 100 open-source accessibility (a11y) testing systems for transformations by just building engine axe-core as part of our build Since we last wrote about dbt, we’ve used 99 adaptors. The number of native connectors processes. It provided team members46 it in a few projects and like what we’ve 98 has grown to include Snowflake, BigQuery, 48 with early feedback on adherence to 47 seen. For example, we like96 that dbt97 makes Redshift and Postgres, as has the range of accessibility rules, even during early the transformation part of ELT pipelines community plugins. We see tools like dbt increments. Not every issue can be found more49 accessible to95 consumers of the data helping data platforms become more “self through automated inspection,51 though. as opposed to just the data engineers service” capable.

21 | TECHNOLOGY RADAR © Thoughtworks, Inc. All Rights Reserved. esbuild run in a build pipeline, Great Expectations continuously evolve a machine-learning Tools Trial makes assertions during execution of a model includes a series of experiments (a data pipeline. We like its simplicity and collection of runs), tracking the performance We’ve always been keen to find tools that ease of use — the rules stored in JSON of these experiments (a collection of can be modified by our data domain metrics) and tracking and tweaking models OR-Tools is an open- can shorten the software development feedback cycle; esbuild is such an example. experts without necessarily needing data (projects). MLflow facilitates this workflow source software suite for As the front-end codebase grows larger, we engineering skills. nicely by supporting existing open standards solving combinatorial usually face a packaging time of minutes. and integrates well with many other tools optimization problems. As a JavaScript bundler optimized for in the ecosystem. MLflow as a managed These optimization speed, esbuild can reduce this time by a k6 service by Databricks on the cloud, available problems have a very factor of 10 to 100. It is written in Golang Trial in AWS and Azure, is rapidly maturing, and we’ve used it successfully in our projects. large set of possible and uses a more efficient approach in the We’ve had a bit more experience We find MLflow a great tool for model solutions, and tools like process of parsing, printing and source map generation which significantly surpasses performance testing with k6 since we first management and tracking, supporting both this are quite helpful in build tools such as Webpack and Parcel covered it in the Radar, and with good UI-based and API-based interaction models. seeking the best solution. in building time. esbuild may not be as results. Our teams have enjoyed the focus Our only growing concern is that MLflow is comprehensive as those tools in JavaScript on the developer experience and flexibility attempting to deliver too many conflating (OR-Tools) syntax transformation; however, this doesn’t of the tool. Although it’s easy to get started concerns as a single platform, such as stop many of our teams from switching to with k6 all on its own, it really shines with model serving and scoring. esbuild as their default. its ease of integration into a developer ecosystem. For example, using the Datadog adapter, one team was quickly able to OR-Tools Flipper visualize performance in a distributed Trial Trial system and identify significant concerns before releasing the system to production. OR-Tools is an open-source software suite Flipper is an extensible mobile application Another team, with the commercial version for solving combinatorial optimization debugger. Out of the box it supports of k6, was able to use the Azure pipelines problems. These optimization problems profiling, interactive layout inspection, log marketplace extension to wire performance have a very large set of possible solutions, viewer and a network inspector for iOS, tests into their CD pipeline and get Azure and tools like OR-Tools are quite helpful in Android and React Native applications. DevOps reporting with little effort. Since seeking the best solution. You can model Compared to other debugging tools k6 supports thresholds that allow for the problem in any one of the supported for mobile apps, we find Flipper to be automated testing assertions out of the languages — Python, Java, C# or C++ — and lightweight, feature rich and easy to set up. box, it’s relatively easy to add a stage to choose the solvers from several supported your pipeline that detects performance open-source or commercial solvers. We’ve degradation of new changes, adding a successfully used OR-Tools in multiple Great Expectations powerful feedback mechanism optimization projects with integer and Trial for developers. mixed-integer programming.

We wrote about Great Expectations in the previous edition of the Radar. We MLflow Playwright continue to like it and have moved it to Trial Trial Trial in this edition. Great Expectations is a framework that enables you to craft built- MLflow is an open-source tool for machine- Playwright allows you to write Web UI in controls that flag anomalies or quality learning experiment tracking and lifecycle tests for Chromium and Firefox as well as issues in data pipelines. Just as unit tests management. The workflow to develop and WebKit, all through the same API. The tool

TECHNOLOGY RADAR | 22 © Thoughtworks, Inc. All Rights Reserved. has gained some attention for its support provides some type inference and guards as-code tools such as Terraform, you can of all the major browser engines which it that understand conditional code flow create real infrastructure components (such Tools achieves by including patched versions of constructs. Designed with large codebases as servers, firewalls, or load balancers) Firefox and Webkit. We continue to hear in mind, Pyright is fast, and its watch mode to deploy applications on them and then positive experience reports with Playwright, checks happen incrementally as files are validate the expected behavior using in particular its stability. Teams have also changed to further shorten the feedback Terratest. At the end of the test, Terratest Tuple is a relatively new found it easy to migrate from Puppeteer, cycle. Pyright can be used directly on the can undeploy the apps and clean up remote pair programming which has a very similar API. command line, but integrations for VS resources. This makes it largely useful for tool, designed to fill Code, Emacs, vim, Sublime, and possibly end-to-end tests of your infrastructure in a the gap Slack left in other editors are available, too. In our real environment. the marketplace after experience, Pyright is preferable to Prowler abandoning Screenhero. Trial alternatives like mypy. Tuple (Tuple) We welcome the increased availability Trial and maturity of infrastructure Redash configuration scanning tools: Prowler Trial Tuple is a relatively new tool optimized for helps teams scan their AWS infrastructure remote paired programming, designed to fill setups and improve security based on Adopting a “you build it, you run it” DevOps the gap Slack left in the marketplace after the results. Although Prowler has been philosophy means teams have increased abandoning Screenhero. Although it still around for a while, it has evolved a lot attention on both technical and business exhibits some growing pains — platform over the past few years, and we’ve found metrics that can be extracted from the availability is limited to Mac OS for now it very valuable to enable teams to take systems they deploy. Often we find that (with Linux support coming soon), and it responsibility for proper security with a analytics tooling is difficult to access for has some UI quirks to work through — short feedback loop. Prowler categorizes most developers, so the work to capture we’ve had good experience using it within AWS CIS benchmarking checks into and present metrics is left to other teams those constraints. Unlike general-purpose different groups (Identity and Access — long after features are shipped to end video- and screen-sharing tools like Zoom, Management, Logging, Monitoring, users. Our teams have found Redash to be Tuple supports dual control with two Networking, CIS Level 1, CIS Level 2, EKS- very useful for querying product metrics mouse cursors, and unlike options such as CIS), and it includes many checks that and creating dashboards in a way that Visual Studio Live Share, it isn’t tied to an help you gain insights into your PCI DSS can be self-served by general developers, IDE. Tuple supports voice and video calls, and GDPR compliance. shortening feedback cycles and focusing clipboard sharing, and lower latency than the whole team on the business outcomes. general-purpose tools; and its ability to let you draw and erase in your pair’s screen Pyright with ease makes Tuple a very intuitive and Trial Terratest developer-friendly tool. Trial While duck typing is certainly seen as a feature by many Python programmers, Terratest caught our attention in the past Why Did You Render sometimes — especially for larger as an interesting option for infrastructure Trial codebases — type checking can be testing. Since then, our teams have been useful, too. For that reason a number using it, and they’re very excited about it When working with React, we often of type annotations are proposed as because of its stability and the experience it encounter situations where our page is very Python Enhancement Proposals (PEPs), provides. Terratest is a Golang library that slow because some components are re- and Pyright is a type checker that works makes it easier to write automated tests for rendering when they shouldn’t be. Why Did with these annotations. In addition, it infrastructure code. Using infrastructure- You Render is a library that helps detect why

23 | TECHNOLOGY RADAR © Thoughtworks, Inc. All Rights Reserved. a component is re-rendering. It does this by having the build tool integrated directly executables on multiple operating systems Tools monkey patching React. We’ve used it in a into the source code repository. An including Mac OS, Windows and multiple few of our projects to debug performance enthusiastic community has emerged distributions of Linux. Since it requires issues with great effect. around this feature and that means a a closed-world assumption, where all wide range of user-contributed tools and code is known at compile time, additional imgcook is a SaaS product workflows are available to get started. configuration is needed for features such from Alibaba, which can Buildah and Podman Tools vendors are also getting on board as reflection or dynamic class loading where intelligently transform Assess via the GitHub Marketplace. However, types can’t be deduced at build time from various design files— we still recommend you proceed with the code alone. Sketch, PSD, even Even though Docker has become the caution. Although code and Git history static images—into sensible default for containerization, can be exported into alternative hosts, a development workflow based on front-end code. we’re seeing new players in this space HashiCorp Boundary that are catching our attention. That is the GitHub Actions can’t. Also, use your best Assess case for Buildah and Podman, which are judgment to determine when a project is (imgcook) complementary projects to build images large or complex enough to warrant an HashiCorp Boundary combines the secure (Buildah) and run containers (Podman) independently supported pipeline tool. networking and identity management using a rootless approach in multiple But for getting up and running quickly on capabilities needed for brokering access Linux distributions. Podman introduces smaller projects, it’s worth considering to your hosts and services in one place a daemonless engine for managing and GitHub Actions and the ecosystem that is and across a mix of cloud and on-premise running containers which is an interesting growing around them. resources if needed. Key management approach in comparison to what Docker can be done by integrating the key does. The fact that Podman can use either management service of your choice, be Open Container Initiative (OCI) images built Graal Native Image it from a cloud vendor or something like by Buildah or Docker images makes this tool Assess HashiCorp Vault. HashiCorp Boundary even more attractive and easy to use. supports a growing number of identity Graal Native Image is a technology that providers and can be integrated with parts compiles Java code into an operating of your service landscape to help define GitHub Actions system’s native binary — in the form of permissions, not just on host but also on a Assess a statically linked executable or a shared service level. For example, it enables you to library. A native image is optimized to control fine-grained access to a Kubernetes CI servers and build tools are some of the reduce the memory footprint and startup cluster, and dynamically pulling in service oldest and most widely used in our kit. time of an application. Our teams have catalogs from various sources is on the They run the gamut from simple cloud- successfully used Graal native images, roadmap. All of this stays out of the way hosted services to complex, code-defined executed as small Docker containers, in the of the engineering end users who get the pipeline servers that support fleets of serverless architecture where reducing start shell experience they’re used to, securely build machines. Given our experience time matters. Although designed for use connected through Boundary’s network and the wide range of options already with programming languages such as Go management layer. available, we were initially skeptical or Rust that natively compile and require when GitHub Actions were introduced smaller binary sizes and shorter start times, as another mechanism to manage the Graal Native Image can be equally useful to imgcook build and integration workflow. But the teams that have other requirements and Assess opportunity for developers to start small want to use JVM-based languages. and easily customize behavior means Remember the research project pix2code that GitHub Actions are moving toward Graal Native Image Builder, native-image, that showed how to automatically generate the default category for smaller projects. supports JVM-based languages — such as code from GUI screenshots? Now there is It’s hard to argue with the convenience of Java, Scala, Clojure and Kotlin — and builds a productized version of this technique —

TECHNOLOGY RADAR | 24 © Thoughtworks, Inc. All Rights Reserved. imgcook is a SaaS product from Alibaba that ReadWriteMany (RWX) you can even mount recommendations on how to optimize them can intelligently transform various design the same volume for read and write access based on your actual usage. The service Tools files (Sketch/PSD/static images) into front- across many nodes. Choosing the right consists of a range of “recommenders” end code. Alibaba needs to customize a storage system for Kubernetes is a nontrivial in areas such as security, compute usage large number of campaign pages during task, and we recommend you assess or cost savings. For example, the IAM the Double Eleven shopping festival. These Longhorn based on your needs. Recommender helps you better implement Recommender is a are usually one-time pages that need to the principle of least privilege by pointing service on Google Cloud be developed quickly. Through the deep- out permissions that are never actually used that analyzes cloud learning method, the UX’s design is initially Operator Framework and therefore are potentially too broad. resources and provides processed into front-end code and then Assess a recommendation for adjusted by the developer. Our team is optimization based on evaluating this tech: although the image Operator Framework is a set of open-source Remote - WSL actual usage. processing takes place on the server side tools that simplifies building and managing Assess while the main interface is on the web, the lifecycle of Kubernetes operators. The imgcook provides tools that could integrate Kubernetes operator pattern, originally Over the past few years Windows (Recommender) with the software design and development introduced by CoreOS, is an approach to Subsystem for Linux (WSL) has come up lifecycle. imgcook can generate static code encapsulate the knowledge of operating a few times in our discussions. Although as well as some data-binding component an application using Kubernetes native we liked what we saw, including the code if you define a DSL. The technology capabilities; it includes resources to be improvements in WSL 2, it never made it is not perfect yet; designers need to managed and controller code that ensures into the Radar. In this edition we want to refer to certain specifications to improve the resources are matching their target state. highlight an extension for Visual Studio the accuracy of code generation (which This approach has been used to extend Code that greatly improves the experience still needs to be adjusted by developers Kubernetes to manage many applications, working with WSL. Although Windows- afterward). We’ve always been cautious particularly the stateful ones, natively. based editors could always access files about magic code generation, because Operator Framework has three components: on a WSL file system, they were unaware the generated code is usually difficult to Operator SDK, which simplifies building, of the isolated Linux environment. With maintain in the long run, and imgcook is testing and packaging Kubernetes operators; the Remote - WSL extension, Visual Studio no exception. But if you limit the usage to a Operator lifecycle manager to install, manage Code becomes aware of WSL, allowing specific context, such as one-time campaign and upgrade the operators; and a catalog to developers to launch a Linux shell. This pages, it’s worth a try. publish and share third-party operators. Our also enables debugging of binaries running teams have found Operator SDK particularly inside WSL from Windows. Jetbrains’ IntelliJ powerful in rapidly developing Kubernetes- too has seen steady improvement in its Longhorn native applications. support for WSL. Assess

Longhorn is a distributed block storage Recommender Spectral system for Kubernetes. There are many Assess Assess persistent storage options for Kubernetes; unlike most, however, Longhorn is built The number of services offered by the big One of the patterns we’ve seen repeat from the ground up to provide incremental cloud providers keeps growing, but so does itself in this publication is that static snapshots and backups, thereby easing the convenience and maturity of tools that error- and style-checking tools emerge the pain of running a replicated storage help you use them securely and efficiently. quickly after a new language gains for non–cloud-hosted Kubernetes. With Recommender is a service on Google Cloud popularity. These tools are generically the recent experimental support for that analyzes your resources and gives you known as linters — after the classic and

25 | TECHNOLOGY RADAR © Thoughtworks, Inc. All Rights Reserved. beloved Unix utility lint, which statically Yelp detect-secrets box, it will validate against a rule set Tools analyzes C code. We like these tools Assess developed for Zalando’s API style guide, because they catch errors early, before but it also supports a Kotlin extension code even gets compiled. The latest Yelp detect-secrets is a Python module for mechanism to develop custom rules. Zally instance of this pattern is Spectral, a detecting secrets within a codebase; it scans includes a web UI that provides an intuitive Zally is a minimalist linter for YAML and JSON. Although files within a directory looking for secrets. interface for understanding style violations OpenAPI linter that helps Spectral is a generic tool for these It can be used as a Git pre-commit hook or and includes a CLI that makes it easy to ensure an API conforms to formats, its main target is OpenAPI (the to perform a scan in multiple places within plug into your CD pipeline. the team’s API style guide. evolution of Swagger) and AsyncAPI. the CI/CD pipeline. It comes with a default Spectral ships with a comprehensive set configuration that makes it very easy to (Zally) of out-of-the-box rules for these specs use but can be modified to suit your needs. AWS CodePipeline that can save developers headaches You can also install custom plugins to add Hold when designing and implementing APIs to its default heuristic searches. Compared or event-driven collaboration. These to similar offerings, we found that this tool Based on the experiences of multiple rules check for proper API parameter detects more types of secrets with its out-of- Thoughtworks teams we suggest specifications or the existence of a the-box configuration. approaching AWS CodePipeline with caution. license statement in the spec, among Specifically, we’ve found that once teams other things. While this tool is a welcome move beyond simple pipelines, this tool can addition to the API development Zally become hard to work with. While it may workflow, it does raise the question of Assess seem like a “quick win” when first starting whether a non-executable specification out with AWS, we suggest taking a step back should be so complex as to require As the API specification ecosystem matures, and checking whether AWS CodePipeline will an error-checking technique designed we’re seeing more tools built to automate meet your longer-term needs, for example, for programming languages. Perhaps style checks. Zally is a minimalist OpenAPI pipeline fan-out and fan-in or more complex developers should be writing code linter that helps to ensure an API conforms deployment and testing scenarios featuring instead of specs? to the team’s API style guide. Out of the nontrivial dependencies and triggers.

31 25 26 68 24 69 70 71 80 23 30 72 73 22 16 54 74 29 21 15 57 58 12 13 53 56 75 20 11 55 14 60 28 59 76 10 61 19 77 5 63 62 9 4 18 64 78 27 3 7 8 66 52 65 79 17 2 6 1 67 Adopt Hold Assess Trial Adopt Adopt Trial Assess Hold 81. Combine 82. LeakCanary Languages & 94 40 34 93 104 82 Trial 81 92 83. Angular Testing Library Frameworks41 84. AWS Data Wrangler 35 103 85. Blazor 88 90 91 36 86. FastAPI 42 87 102 87. io-ts Combine 86 89 88. Kotlin Flow Adopt 43 101 89. LitElement 90. Next.js 50 38 85 A long time ago we placed ReactiveX — a 37 91. On-demand modules 44 83 family of open-source frameworks for 92. Streamlit 45 84 reactive programming — into the Adopt 39 100 93. SWR ring of the Radar. In 2017, we mentioned 94. TrustKit the addition of RxSwift, which brought 99 46 98 reactive programming to iOS development Assess 48 97 using Swift. Since then, Apple has 47 96 95. .NET 5 introduced its own take on reactive 49 96. bUnit programming in the form of the Combine 95 97. Dagster 51 framework. Combine has become our 98. Flutter for Web default choice for apps that support iOS 99. Jotai and Zustand 13 as an acceptable deployment target. It’s 100. Kotlin Multiplatform Mobile easier to learn than RxSwift and integrates 101. LVGL really well with SwiftUI. If you’re planning 102. React Hook Form to convert an existing application from 103. River RxSwift to Combine or work with both in 104. Webpack 5 Module Federation the same project, you might want to look at RxCombine. Hold Angular Testing Library short in documentation, Angular Testing Trial Library does provide good sample tests LeakCanary that helped us in getting started faster for Adopt As we continue developing web various cases. We’ve had great success applications in JavaScript, we continue with this testing library in our Angular Our mobile teams now view LeakCanary enjoying the Testing Library approach projects and advise you to trial this solid as a good default choice for Android of testing applications; and carry on testing approach. development. It detects annoying memory exploring and gaining experience with its leaks in Android apps, is extremely simple packages — beyond that of React Testing to hook up and provides notifications with Library. Angular Testing Library brings all AWS Data Wrangler a clear trace-back to the cause of the leak. the benefits of its family when testing UI Trial LeakCanary can save you tedious hours components in a user-centric way, pushing troubleshooting out-of-memory errors on for more maintainable tests focused AWS Data Wrangler is an open-source multiple devices, and we recommend you primarily on behavior rather than testing library that extends the capabilities of add it to your toolkit. UI implementation details. Although it falls Pandas to AWS by connecting data frames

28 | TECHNOLOGY RADAR © Thoughtworks, Inc. All Rights Reserved. to AWS data-related services. In addition build solutions, not just for data science an implementation of Reactive Streams Languages & to Pandas, this library leverages Apache but for back-end services too. In these on top of coroutines. Unlike RxJava, Arrow and Boto3 to expose several APIs scenarios, we’re having good experiences flows are a native Kotlin API similar to Frameworks to load, transform and save data from with FastAPI — a modern, fast (high- the familiar sequence API with methods data lakes and data warehouses. An performance), web framework for building that include map and filter. Like important limitation is that you can’t do APIs with Python 3.6 or later. Additionally, sequences, flows are cold, meaning that large distributed data pipelines with this this framework and its ecosystem include the values of the sequence are only FastAPI is a modern, library. However, you can leverage the features such as API documentation using constructed when needed. All of this high-performance web native data services — like Athena, Redshift OpenAPI that allow our teams to focus on makes writing multithreaded code much and Timestream — to do the heavy lifting the business functionalities and quickly simpler and easier to understand than framework for building and pull data in order to express complex create REST APIs, which makes FastAPI a other approaches. The toList method, APIs with Python 3.6 transformations that are well suited good alternative to existing solutions in predictably, converts a flow into a list which and later. for data frames. We’ve used AWS Data this space. is a common pattern in tests. Wrangler in production and like that it (FastAPI) lets you focus on writing transformations without spending too much time on the io-ts LitElement connectivity to AWS data services. Trial Trial

We’ve really enjoyed using TypeScript Steady progress has been made since we Blazor for a while now and love the safety that first wrote about Web Components in 2014. Trial the strong typing provides. However, LitElement, part of the Polymer Project, is getting data into the bounds of the type a simple library that you can use to create Although JavaScript and its ecosystem system — from, for example, a call to a lightweight web components. It’s really is dominant in the web UI development back-end service — can lead to run-time just a base class that removes the need for space, new opportunities are opening errors. One library that helps solve this a lot of the common boilerplate, making up with the emergence of WebAssembly. problem is io-ts. It bridges the gap between writing web components a lot easier. Blazor continues to demand our attention; compile-time type-checking and run-time We’ve had success using it on projects, and it’s producing good results with our teams consumption of external data by providing as we see the technology maturing and building interactive rich user interfaces encode and decode functions. It can also the library being well liked, LitElement is using C# on top of WebAssembly. The fact be used as a custom type guard. As we becoming more commonly used in our Web that our teams can use C# on the frontend gain more experience with io-ts in our Components-based projects. too allows them to share code and reuse work, our initially positive impressions are existing libraries. That, along with the confirmed, and we still like the elegance of existing tooling for debugging and testing, its approach. Next.js such as bUnit, make this open-source Trial framework worth trying. Kotlin Flow We’ve had a bit more experience using Trial Next.js for React codebases since the FastAPI last time we wrote about it. Next.js is an Trial The introduction of coroutines to Kotlin opinionated, zero-configuration framework opened the door for several innovations that includes simplified routing, automatic We’re seeing more teams adopting — Kotlin Flow is one of them, directly compilation and bundling with Webpack and Python as the preferred language to integrated into the coroutines library. It’s Babel, fast hot reloading for a convenient

TECHNOLOGY RADAR | 29 © Thoughtworks, Inc. All Rights Reserved. developer workflow among other features. Bokeh. We’re using it in a few projects and .NET 5 It provides server-side rendering by default, like how we can put together interactive Assess Languages & improves search engine optimization visualizations with very little effort. and the initial load time and supports We don’t call out every new .NET version Frameworks incremental static generation. We’ve had in the Radar, but .NET 5 represents positive experience reports from teams SWR a significant step forward in bringing using Next.js and, given its large community, Trial .NET Core and .NET Framework into a continue to be excited about the evolution single platform. Organizations should Streamlit is an open- of the framework. When used in appropriate circumstances, start to develop a strategy to migrate source application our teams have found that the React their development environments — a framework in Python Hooks library SWR can result in cleaner fragmented mix of frameworks depending used by data scientists for On-demand modules code and much improved performance. on the deployment target — to a single Trial SWR implements the stale-while-revalidate version of .NET 5 or 6 when it becomes building interactive data HTTP caching strategy, first returning data available. The advantage of this approach visualizations. On-demand modules for Android is a from cache (stale), then sending the fetch will be a common development platform framework that allows tailored APKs request (revalidate) and finally refreshing regardless of the intended environment: (Streamlit) containing only required functionality to the values with the up-to-date response. We Windows, Linux, cross-platform mobile be downloaded and installed for a suitably caution teams to only use the SWR caching devices (via Xamarin) or the browser structured app. This could be worth trialing strategy when an application is supposed to (using Blazor). While polyglot development for larger apps where download speed return stale data. Note that HTTP requires will remain the preferred approach for might be an issue, or if a user is likely only to that caches respond to a request with the companies with the engineering culture to use some functionality on initial installation. most up-to-date response; only in carefully support it, others will find it more efficient It can also simplify the handling of multiple considered circumstances is a stale response to standardize on a single platform for devices without requiring different APKs. A allowed to be returned. .NET development. For now, we want to similar framework is available for iOS. keep this in the Assess ring to see how well the final unified framework performs TrustKit in .NET 6. Streamlit Trial Trial SSL public key pinning is tricky. If you bUnit Streamlit is an open-source application select the wrong policy or don’t have a Assess framework in Python used by data scientists backup pin, your application will stop for building interactive data applications. working unexpectedly. This is where bUnit is a testing library for Blazor that Tuning machine-learning models takes TrustKit is useful — it’s an open-source makes it easy to create tests for Blazor time; instead of going back and forth on the framework that makes SSL public key components in existing unit testing main application (the one that uses these pinning easier for iOS applications. There frameworks such as NUnit, xUnit or MSUnit. models), we’ve found value in quickly building is an equivalent framework for Android as It provides a facade around the component standalone prototypes in Streamlit and well. Picking the correct pinning strategy allowing it to be run and tested within the gathering feedback during experimentation is a nuanced topic, and you can find more familiar unit test paradigm, thus allowing cycles. Streamlit stands out from competitors details about it in the TrustKit Getting very fast feedback and testing of the such as Dash because of its focus on rapid Started guide. We’ve used TrustKit in component in isolation. If you’re developing prototyping and support for a wide range of several projects in production, and it has for Blazor, we recommend that you add visualization libraries, including Plotly and worked out well. bUnit to your list of tools to try out.

30 | TECHNOLOGY RADAR © Thoughtworks, Inc. All Rights Reserved. Dagster WebAssembly and WebGL to render Skia mobile cross-platform applications more Languages & Assess paint commands to the browser canvas. A enjoyable and efficient. With KMM you few of our teams have started using Flutter write code once for business logic and the Frameworks Dagster is an open-source data for Web and like the initial results. app core in Kotlin and then share it with orchestration framework for machine both Android and iOS applications. Write learning, analytics and plain ETL data platform-specific code only when necessary, pipelines. Unlike other task-driven Jotai and Zustand for example, to take advantage of native UI These state management frameworks, Dagster is aware of data Assess elements; and the specific code is kept in libraries for React aim to flowing through the pipeline and can different views for each platform. Although be small and simple to use. provide type-safety. With this unified view of In the previous Radar, we commented on still in Alpha, Kotlin Multiplatform Mobile is evolving rapidly. We’ll certainly keep an eye Perhaps not by complete pipelines and assets produced, Dagster can the beginning of a phase of experimentation schedule and orchestrate Pandas, Spark, with state management in React applications. on it, and you should too. coincidence, both names SQL or anything else that Python can invoke. We moved Redux back into the Trial ring, are translations of the The framework is relatively new, and we documenting that it is no longer our default word state into Japanese recommend that you assess its capabilities choice, and we mentioned Facebook’s Recoil. LVGL and German, respectively. for your data pipelines. In this volume we want to highlight Jotai Assess and Zustand: Both are state management (Jotai and Zustand) libraries for React; both aim to be small and With the increasing popularity of smart Flutter for Web simple to use; and, perhaps not by complete home and wearable devices, demand for Assess coincidence, both names are translations of intuitive graphical user interfaces (GUIs) the word state into Japanese and German, is increasing. However, if you’re engaged So far, Flutter has primarily supported native respectively. Beyond these similarities, in embedded device development, rather iOS and Android applications. However, the however, they differ in their design. Jotai’s than Android/iOS, GUI development may Flutter team’s vision is to support building design is closer to that of Recoil in that state take a lot of effort. As an open-source applications on every platform. Flutter for consists of atoms stored within the React embedded graphics library, LVGL has Web is one step in that direction — it allows component tree, whereas Zustand stores the become increasingly popular. LVGL has us to build apps for iOS, Android and the state outside of React in a single state object, been adapted to mainstream embedded browser from the same codebase. It has much like the approach taken by Redux. The platforms such as NXP, STM32, PIC, Arduino, been available for over a year now on the authors of Jotai provide a helpful checklist to and ESP32. It has a very small memory “Beta” channel, but with the recent Flutter decide when to use which. footprint: 64 kB flash and 8 kB RAM is 2.0 release, Flutter for Web has hit the enough to make it work, and it can run stable milestone. In the initial release of smoothly on various Cortex-M0 low-power web support, the Flutter team is focusing on Kotlin Multiplatform Mobile MCUs. LVGL supports input types such progressive web apps, single-page apps and Assess as touchscreen, mouse and buttons and expanding existing mobile apps to the web. contains more than 30 controls, including The application and framework code (all Following the trend of cross-platform mobile TileView suitable for smart watches. The MIT in Dart) are compiled to JavaScript instead development, Kotlin Multiplatform Mobile license it chose doesn’t restrict enterprise of ARM machine code, which is used for (KMM) is a new entry in this space. KMM is and commercial use. Our teams’ feedback mobile applications. Flutter’s web engine an SDK provided by JetBrains that leverages on this tool has been positive and one offers a choice of two renderers: an HTML the multiplatform capabilities in Kotlin and of our projects using LVGL is already in renderer, which uses HTML, CSS, Canvas includes tools and features designed to production, more specifically in small batch and SVG, and a CanvasKit renderer that uses make the end-to-end experience of building manufacturing.

Building forms for the web remains one At the heart of many approaches to The release of the Webpack 5 Module Frameworks of the perennial challenges of front-end machine learning lies the creation of a Federation feature has been highly development, in particular with React. Many model from a set of training data. Once anticipated by developers of micro frontend of our teams working with React have been a model is created, it can be used over architectures. The feature introduces a using Formik to make this easier, but some and over again. However, the world isn’t more standardized way to optimize how Machine-learning models are now assessing React Hook Form as a stationary, and often the model needs to module dependencies and shared code are often need to change potential alternative. React Hooks already change as new data becomes available. managed and loaded. Module federation as new data becomes existed when React Hook Form was created, Simply re-running the model creation allows for the specification of shared so it could use them as a first-class concept: step can be slow and costly. Incremental modules, which helps with the deduplication available. Incremental the framework is registering and tracking learning addresses this issue, making it of dependencies across micro frontends by learning makes it possible form elements as uncontrolled components possible to learn from streams of data loading code used by multiple modules only to learn from streams of via a hook, thereby significantly reducing incrementally to react to change faster. once. It also lets you distinguish between data incrementally to react the need for re-rendering. It’s also quite As a bonus the compute and memory local and remote modules, where the to change faster. In our lightweight in size and in the amount of requirements are lower and predictable. remote modules are not actually part of implementations we’ve had boilerplate code needed. In our implementations we’ve had good the build itself but loaded asynchronously. good experience with River, experience with the River framework, but Compared to build-time dependencies like so far we’ve added checks, sometimes npm packages, this can significantly simplify a Python library for online manual, after updates to the model. the deployment of a module update with machine learning. many downstream dependencies. Be aware, though, that this requires you to bundle (River) all of your micro frontends with Webpack, as opposed to approaches such as import maps, which might eventually become part of the W3C standard.

32 | TECHNOLOGY RADAR © Thoughtworks, Inc. All Rights Reserved. We are a software consultancy and Want to stay up to date with all community of passionate purpose-led Radar-related news and insights? individuals, 9,000+ people strong across Follow us on your favorite social channel or 48 offices in 17 countries. Over our 27+ become a subscriber. year history, we have helped our clients solve complex business problems where technology is the differentiator. subscribe now When the only constant is change, we prepare you for the unpredictable. thoughtworks.com/radar #TWTechRadar