Skip to content

Blog

Managing Multi-Language Projects with Bazel

caption

In today’s software development landscape, it’s rare to encounter a project built with just one programming language or platform. Modern applications often require integrating multiple technologies to meet diverse requirements. This complexity is both a challenge and an opportunity, demanding robust tools to manage dependencies, builds, and integrations seamlessly. Bazel, a powerful build system, is one such tool that has proven invaluable for multi-language projects.

Recently, I decided to extend my Maelstrom challenges with a C++-based test to explore how Bazel can simplify managing multi-language dependencies and streamline development workflows.

Why Bazel for Multi-Language Projects?

Bazel’s design philosophy emphasizes performance and scalability, making it an excellent choice for projects that involve multiple languages. With its support for Bazel modules, adding dependencies is as simple as declaring them in a MODULE.bazel file. For example, integrating the popular logging library spdlog is straightforward:

Bazel and Rust: A Perfect Match for Scalable Development

caption

Bazel never fails to impress, and its support for Rust demonstrates its versatility and commitment to modern development. Two distinct dependency management modes—Cargo—based and pure Bazel—allow developers to tailor workflows to their projects' needs. This adaptability is particularly valuable for integrating Rust applications into monorepos or scaling complex systems. I decided to explore how Bazel supports Rust, including managing dependencies, migrating from Cargo.toml to BUILD.bazel, and streamlining integration testing.

Harnessing Cargo-Based Dependency Management

Bazel’s ability to integrate with Cargo, Rust’s native package manager, is a standout feature. This approach preserves compatibility with the Rust ecosystem while allowing projects to benefit from Bazel’s powerful build features. By using rules_rust, a Bazel module can seamlessly import dependencies defined in Cargo.toml and Cargo.lock into its build graph.

Returning to Rust: A Journey Through Tooling, Performance

When I started tackling the Maelstrom challenges, my initial thought was to use C++. It’s a language I know inside out, and its performance is hard to beat. However, as I contemplated setting up the project, I realized I couldn’t justify fighting with the C++ pipeline for free. Crafting a proper CMake or Bazel configuration might be worthwhile for large-scale projects or when compensated, but for personal experiments? It’s an unnecessary headache.

Why Go is My Default Choice

For most non-performance critical scenarios, Go is my default, no-brainer choice. It has a clean build system, excellent tooling, and a developer experience that doesn’t make me dread the setup process. Go’s simplicity allows me (and any level team) to focus on solving the problem rather than wrestling with the environment. Yet, this time, I decided to take a different path.

Minimal CI for Go library with GitHub actions

caption

Continuous Integration (CI) has become an essential part of modern software development, and for good reason. It ensures code quality, speeds up development, and catches potential issues early. However, you can get started without an elaborate CI setup. Even a minimal CI setup can significantly improve your workflow. Here's why every project should have at least minimal CI and how to implement it effectively using GitHub Actions.

What Constitutes Minimal CI?

For a project to benefit from CI without excessive complexity, it should include the following essential components:

  1. Project Compilation: Verify that the codebase is always in a buildable state.
  2. Unit Test Execution: Ensure the core functionality works as expected.
  3. Static Code Analysis: Catch bugs and enforce coding standards before they become an issue.

GFSM: A Simple and Fast Finite State Machine for Go

caption

Design patterns are not widely used in Go, as they can often lead to unnecessary complexity in the codebase. However, the Finite State Machine (FSM) is an exception that proves to be incredibly useful. When I set out to design GFSM, I aimed to create a fast and straightforward FSM implementation for Go.

I initially sought a quick and straightforward FSM solution for Go, but I couldn't find anything that met my needs. Drawing inspiration from the speed-focused and minimalistic principles of C++ while remaining true to Go's idioms, I developed GFSM to fill this gap. The outcome is GFSM—a library that distinguishes itself from alternatives like looplab/fsm by prioritizing speed and simplicity.

Whether orchestrating microservices, handling distributed systems, or designing embedded systems, GFSM brings the reliability and efficiency needed to keep things running smoothly.

Reflections on Reading “Introduction to Reliable and Secure Distributed Programming”

caption

I’ve had Introduction to Reliable and Secure Distributed Programming sitting on my bookshelf for years, silently whispering, “Read me when you have time.” Of course, “later” always seemed like the right time. As software developers, we often prioritize practical, hands-on books that help solve immediate problems—topics like Kubernetes, Kafka or mastering another layer of C++ intricacies. But in hindsight, neglecting foundational theory is a mistake.

I began my journey with distributed systems over a decade ago. Back then, I wish someone had handed me this book and insisted I dive in immediately. It’s the kind of resource that can help set the foundation for anyone venturing into the complexities of distributed computing. Instead, I learned through trial, error, and practical exposure, which, while valuable, left gaps that only became apparent when I finally picked up this book.

The book doesn’t hold back—it dives deeply into the theoretical underpinnings of distributed systems. Despite my experience, I found several topics both fascinating and challenging. The discussions on randomized consensus with coin and hierarchical consensus stood out to me, offering insights that are as practical as they are thought-provoking. These are concepts that, while grounded in theory, can influence how we design systems in real-world applications.

Sabbatical and Distributed Systems

caption

View from the rooftop of the apartment, where we settled down. I love running up on the left side of the river—quite a nice trail run.

Working at a unicorn company for four years was an exhilarating experience, though incredibly exhausting. Tackling significant projects that few have ever attempted means Google can’t offer much help. Surrounded by highly talented colleagues, many of whom you may never encounter again in one place, all while facing a relentless pace and high expectations from leadership, it eventually takes a toll. After a while, you just need to pause, rest, and reflect. That’s why I left Motional—to take several months to recharge, spend time with family, and reconnect with my hobbies.

The irony is, after more than 20 years in IT, I’ve become somewhat of a workaholic, so proper rest eludes me. Just last weekend, I found myself speaking at the DevFest ’24 conference, sharing insights on best practices in project development with the local IT community. The event exceeded all my expectations. It’s been incredibly gratifying to see how much Kyrgyzstan’s IT sector has grown since I left in 2003. I sincerely hope this growth continues and that, eventually, talented engineers will want to stay here or return to the country instead of trying themselves outside, as I did. However, I’m also acutely aware that I’m now overqualified for most roles in the Kyrgyz job market, which is a bit bittersweet. So, my journey will likely inevitably take me elsewhere after my sabbatical.

It’s funny (or maybe a bit sad, depending on how you look at it), but my workaholic side insists that taking it easy isn’t enough. Now, with plenty of free time, I’ve decided to tackle something I’ve always wanted to dive deeper into: Distributed Systems theory. Yes, the theory itself—not just the practical implementation that most developers focus on. So, I have a Plan.

TIL: how to debug randomly hanging Python applications

Usually, if a Python-based application hangs, you either read logs or grab one of the PBD-based solutions, attaching to the application, and uses the Python console for investigation. The approach is straightforward; for example, you installed pdb-attach, and add a few lines to your application:

import pdb_attach
pdb_attach.listen(50000)

and expect that "magic" will just works:

> python -m pdb_attach <PID> 50000
(Pdb) YOU HAVE PDB SESSION HERE

But sometimes, magic is broken, and my theory is (I didn't search for proof) that this is due to GIL. So, sometimes, no PDB prompt after you have attached to the application with PDB. In my case, the application hang in the multiprocessing.Process call where I used a gRPC server. The gRPC server didn't react to the termination request, the process cannot stop, and like aggravating circumstances, all these are a part of PyTest that hang 1 of 20 executions.

This is a general PDB-based debuggers issue, which means all other tools like pyrasite-shell and PyTest PBD integration also don't work. The only option here is GDB for Python, which is surprisingly amazing! First of all, you need to install Python extension for GDB.

sudo apt-get install python3.9-dbg

Then you can connect to your Python application which is a regular Python process with GDB, and explore the call-stack!

> gdb

(GDB) attach <PID>
(GDB) py-bt

If you use not APT-based Linux, search for proper instruction here.