Published

This was my first time ever streaming on twitch.tv/waylonwalker. I am excited to get going. I have been streaming early in the morning while I am still waking up, so still a bit groggy as I go. https://youtu.be/Y07UBr9Ccjs Kedro Spaceflights # It all started with kedro/issues/606, Yetu called out for users of kedro to record themselves doing a walk through of their tutorials. I wanted to do this, but was really stuck at the fact that recording or editing somewhat polished vide is quite time consuming for me. Notes #

With the latest releases of kedro, it is now possible to run kedro pipelines from within scripts. While I would not start a project with this technique, it will be a good tool to keep in my back pocket when I want to sprinkle in a bit of kedro goodness in existing projects. New to Kedro # What is Kedro If your just learning about kedro check out this post walking through it No More Rabbit Hole of Errors # as of 0.17.2 I’ve tried to do this in kedro and it turned into a rabbit hole of errors. First kedro needed a directory, if you tried to fake one in it would then ask for logging setup. These errors just kept coming to the point it wasnt worth doing and I might as well use a proper template for real projects and stick to simple function calls for things that are not a kedro project. Kedro in a script # To get kedro running, you will need a pipeline, catalog, and runner at a minimum. Those who have used kedro before the pipeline will look very similar to what you are familiar with, but…

Kedro can have a chatty logger. While this is super nice in production so see everything that happened during a pipeline run. This can be troublesome while trying to implement a cli extension with clean output. Silence a Python log # First, how does one silence a python log? Python loggers can be retrieved by the module’s function. Then their log level can be changed. Much of kedro’s chattiness comes from INFO level logs. I don’t want to hear about anything for my current use case unless it’s essential, i.e., a failure. In this case, I set the log levels to ERROR as most errors should stop execution anyways. python logging levels # Get or Create a logger # Getting a python logger is straightforward if we know the name of the logger. The following block will grab the logger object for the logger currently registered under the name passed in. 🔥 If a logger doesn’t exist under the passed in name, it will create one for you. Set Level # Once we get the logger, we need to silence it by sett…

Running multiple processes using the same diskcache object can cause issues with locks. As I was trying to setup a rich Live display for markata I ran into issues where each part could not nun simultaneusly. As I had followed the instructions from discache it was not directly aparant to me, so I had to make a simple example to experiment and play with at a small scale. Minimum reproducible error # Minimum reporducible error is one of my superpowers in development. I do this very often to sus out what is really happening. My day to day work is processing data with python, I keep a number of very small data sets handy to break and fix. This helps separate complexities of the project and the problem. Let’s break it # Markata has a lot going on. It’s a plugins all the way down static site generator built in python. Trying to find the root cause through the layers of plugin and cli modules can be a pain, but in this case building a very simple minimum reporducible error was much easier. 📝 i…

In 2021 I changed the way I navigate between tmux sessions big time. Now I can create, kill, switch with ease, and generally keep work separated into logical groups. Update # Since making this post, I have made ~20 other posts in short form that all have a YouTube video to go along with them you can find them all on my tmux-playlist. Chris Toomey’s Tmux Course # I took Chris’s tmux course in December 2020 and it was fantastic. Even as a seasoned tmux user, I learned quite a bit. Before the course, I was proficient in navigating within each of my tmux sessions but rarely started more than one session. A few months later, I have adopted a lot of what I learned from Chris and made it my own. I am now keeping projects to their own session and can move between them fluidly with just a few keystrokes. For high-traffic projects, I have them bound to a single keystroke for instant switching. This change has been a game-changer from the mess of windows I used to have and the nightmare it was to…

Trim branches no longer on origin # Find branches already merged #