Published

Inspired by discourse’s link expansion I am rolling out expansions for one line links on the blog waylonwalker [1]. I was able to find a gatsby plugin gatsby-remark-embedder [2] that expands one line links for social cards for popular platforms like twitter and YouTube through a repose from Kyle Mathews to my tweet. https://twitter.com/kylemathews/status/1329817928666005504 Use Cases # [3] This covers a couple of use cases I have with very little effort. - Twitter - YouTube install # [4] npm i gatsby-remark-embedder gatsby-plugin-twitter This was super quick and simple to setup, the only thing that was extra was to install the gatsby-plugin-twitter plugin as well as the gatsby-remark-embedder. enable # [5] // In your gatsby-config.js module.exports = { // Find the 'plugins' array plugins: [ `gatsby-plugin-twitter`, { resolve: `gatsby-transformer-remark`, options: { plugins: [ { resolve: `gatsby-remark-embedder`, options: { customTransformers: [ // Your custom t...

I wanted a super simple way to cross-link blog posts that require as little effort as possible, yet still looks good in vanilla markdown in GitHub. I have been using a snippet that puts HTML [1] into the markdown. While this works, it’s more manual/difficult for me does not look the best, and does not read well as Goals for new card # [2] The new card should be fully automated to expand with title, description, and cover image. Bonus if I am able to attach a comment behind it. - fully automated - card expansion - Title - description - cover image Old Card # [3] If you can call it a card 🤣. This card was just an image wrapped in an anchor tag and a paragraph tag. I found this was the most consistent way to get an image narrower and centered in both GitHub and dev.to. <p style='text-align: center'> <a href='https://waylonwalker.com/notes/eight-years-cat/'> <img style='width:500px; max-width:80%; margin: auto;' src="https://dropper.wayl.one/file/99f80283-2f27-4664-8c4a-d1...

grepr # [1] grepr() {grep -iRl "$1" | xargs sed -i "s/$1/$2/g"} ```bash grepr() {grep -iRl "$1" | xargs sed -i "s/$1/$2/g"} grepd # [2] grepd() {grep -iRl "$1" | xargs sed -i "/^$1/d"} CocSearch # [3] :CocSearch published: false -g *.md References: [1]: #grepr [2]: #grepd [3]: #cocsearch

- customize for the job - Why are you a good fit? - What will you bring to the role? - Give real outcomes - give real experience - Stop tech vomiting - if you link to GitHub - Make a profile readme - Guide me to your best work - have some activity - if you link to LinkedIn - Provide some benefit that is not on your resume - Have a logical flow of experience (dont make me hunt for past experience) - Keep it under 2 pages - Who you know. - Reference real experience - Deployed 12 data pipelines with over 500 nodes to process 200GB of data at a Fortune 100 company - vs - Knowledge of Data Engineering methodology with python EC2 - Dont be so fluffy

There are many reasons that you should be using kedro. If you are on a team of Data Scientists/Data Engineers processing DataFrames from many data sources should be considering a pipeline framework. Kedro is a great option that provides many benefits for teams to collaborate, develop, and deploy data pipelines What is Kedro [1] Starter Template # [2] Kedro makes it super easy to get started with their cli that utilizes cookiecutter under the hood. conda create -n my-new-project -y python=3.8 kedro new kedro install kedro run Create New Kedro Project [3] read more about how to start your first kedro project here Collaboration # [4] Kedro provides many tools that help teams collaborate on a single codebase. While writing monolithic scripts it can be easy to pin yourself in a corner where it is difficult to have multiple people making changes to the notebook/script at the same time. Kedro helps guide your team to break your project down into small pieces that different members o...

Reasons to Kedro # [1] - collaboration - Sharable catalog - small nodes over monolithic notebooks - catalog - easily load anything without needing to run - No need to write read/write code - pipeline - No need to keep execution order in your head - easily run a slice of a pipeline - plugins - pip install - make your own - hooks - flexible expandable cli Reasons Not to Kedro # [2] - Already utilizing another DAG framework - Data is not in a widely supported format - Micro short-lived project - Large Project / Deadline - Use a lower profile project to learn first - Team not willing to change - Need minimal dependencies - God Project - kedro owns everything?? References: [1]: #reasons-to-kedro [2]: #reasons-not-to-kedro

Latest Post # [1] latest [2] STOP LEAVING Browser Tabs open and save them here! - https://nesbitt.io/2026/03/04/package-managers-need-to-cool-down.html - https://mariozechner.at/posts/2026-03-25-thoughts-on-slowing-the-fuck-down/ - https://danielmiessler.com/blog/ai-stops-being-artificially-cheap --- - jbrancha til [3] - The Video Course Launch that Made Me Think [4] - photo prism [5] - box python library [6] - kedro on hn [7] - How can a Data Scientist refactor Jupyter notebooks towards production-quality code? [8] - Sourcing vs executing in Bash [9] - Should We Follow The Open-Closed Principle? [10] - Create multi-dimensional arrays in pure Python: The Correct Way [11] - Beware of These 9 Red Flags in a Developer Interview [12] - How to Overcome Impostor Syndrome as a Developer [13] - lazy load youtube videos [14] - lite youtube embeds [15] - full subtitle youtube search [16] --- - Jungle Scout - Kedro Case Study [17] - Kedro Sessions [18] - Julia Evans - A...

Kedro 0.16.6 [1] is out! Let’s take a look through the release notes Deployment Docs # [2] This is really exciting to see more deployment options coming from the kedro team. It really shows the power of the framework. The power of some of these orchestrations options is incredible. - Argo [3] - Prefect [4] - Kubeflow [5] - Batch [6] - SageMaker [7] Most of them hinge on a sweet combination of the kedro cli, docker image, and the pipeline knowing your nodes dependencies. Argo, Prefect, and Kubeflow have an interesting technique where they translate the pipeline and its dependencies from kedro to their language. Batch uses the aws cli to submit jobs, one node per job, and listen for them to complete. It will submit all nodes with completed dependencies at once, meaning that we can get some massive parallelization. I did a quick and dirty test of one of these by simulating the technique in a bash script and saw a 40 hr pipeline finish in about 1 hour. I am excited to get thi...

I started making stories as kind of a brain dump a few times per day and posting them to [LinkedIn](https://www.linkedin.com/in/waylonwalker/(https://www.linkedin.com/in/waylonwalker/). Here are the last 11 days of stories. I store all the stories on my website with the hopes of doing something with them on my own platform eventually. For now it makes it easy to make these posts. cd static/stories ls | xargs -I {} echo '' Stories 10-10-2020 - 10-21-2020 # [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] References: [1]: #stories-10-10-2020---10-21-2020 [2]: https://waylonwalker.com/stories/TIL-kedro-sorts-nodes.png [3]: https://waylonwalker.com/stories/disable-base-pip.png [4]: https://waylonwalker.com/stories/discovered-social-cards.png [5]: https://waylonwalker.com/stories/find-kedro-de1-contributor.png [6]: https://waylonwalker.com/stories/hacktoberfest-2020-kedro-538-tests-pass.png [7]: https://waylo...